QL / PASCAL

[pjw]

Pardon my ignorance, but why is it so hard to make these compilers (FPC, C68,..) produce "pure" Qdos code, ie code that is non-self-modifying and position independent? AFAICS there is nothing inherent in the languages themselves that prevent it. Could it be fixed?

Another question borne of my ignorance: Could (some of, at least) those massive libraries that seem to be included with even the simplest programs be made to be pre-loaded as shareable toolkits a la Qlib_run, ptrmen_cde, etc? Wouldnt that be desirable?

When youre done tut-tuting and shaking your wise heads, please give me a kind, clear answer even I can understand

[tofro]

Pardon my ignorance, but why is it so hard to make these compilers (FPC, C68,..) produce "pure" Qdos code, ie code that is non-self-modifying and position independent? AFAICS there is nothing inherent in the languages themselves that prevent it. Could it be fixed?

Another question borne of my ignorance: Could (some of, at least) those massive libraries that seem to be included with even the simplest programs be made to be pre-loaded as shareable toolkits a la Qlib_run, ptrmen_cde, etc? Wouldnt that be desirable?

When youre done tut-tuting and shaking your wise heads, please give me a kind, clear answer even I can understand

Per,

the simple reason why we want (and have, like in C68 and FreePascal) programs that need to self modify (and relocate) and not pure position independant code and data is size vs. performance. As you will know, PC and other address register-relative addressing only works +/- 32kBytes away, because the register offsets on a 680xx (with xx < 20) are only 16 bits wide.

You could work around this using jump islands (jump across larger address ranges with several PC relative branches in-between) or emulate PC-relativity with a base address register set to "here" and do run-time relocation in that address register on the fly, leaving the code in RAM unmodified. Obviously, you're buying "purity" for significantly worse performance and more complexity (and, the compilers ported from platforms that don't have such limitations and thus have absolutely no mechanisms to generate these work-arounds). Compilers for segmented DOS have jumped through hoops to address a similar problem in the x86 world and only got it acceptably(?) right only after years.

You could easily create full PC-relative, non-relocating programs by targetting only platforms >= 68020, but this doesn't really look like something we want.

You could also easily create full PC-relative, non-relocating programs by limiting yourself to no more than 32kBytes of code and 32kBytes of data, which is also quite a bit of a serious limitation.

68k Macs and 68k PalmOS handhelds used to have similar problems. MacOS solved this by limiting a single code resource to no more than 32kBytes, larger programs mangled together from several such resources, with inter-code-resource jumps to be managed by the programmer themselves, PalmOS did something similar, and you could easily receive a compiler/linker error basically saying "PROCEDURE too far away for a BRA, move it closer", which could get you in a serious mess re-arranging your code.

All in all, the downsides of "non-pure programs" seem to be much less inconvenient (to me, at least) than having to accept the alternative limitations.

On your other question: C68 has actually tried that approach, which basically boils down to shared libraries on other systems, towards the end of its active development. You can create so-called RLLs that represent shared libraries, are loaded on demand and stay resident for other programs to use. There's never been much uptake on this, although I think it somewhat works with the latest compiler and library patches. I don't know of a single program in the public that uses RLLs.

[mk79]

Pardon my ignorance, but why is it so hard to make these compilers (FPC, C68,..) produce "pure" Qdos code, ie code that is non-self-modifying and position independent? AFAICS there is nothing inherent in the languages themselves that prevent it. Could it be fixed?

Additionally to Tofro's answer, QD for example is written in assembler and also has to do relocation fixups on startup because the executable is so big. So this is not a language thing but a result of the executable size.

What's more interesting is if the data space is accessed absolutely or relatively, which is the pre-requisite for THINGisation of the code. I understand there is limited support for this in FreePascal but not the default behaviour. And again this would limit your static data to 32kb, which could be a problem for bigger programs.

Another question borne of my ignorance: Could (some of, at least) those massive libraries that seem to be included with even the simplest programs be made to be pre-loaded as shareable toolkits a la Qlib_run, ptrmen_cde, etc? Wouldnt that be desirable?

ProWesS is the only thing I know that uses a DLL approach. So yeah, in principle it's possible, but there is not an established infrastructure for it (yet).

[pjw]

Thanks Tobias, you are a great resource here, y'know!

While I was aware of the limitations of relative addressing (its been discussed here too, after all. And then theres those pesky doubly offset vectors..) I guess the main reason for my ignorance is that Ive never (knowingly) encountered it as a problem. I vaguely remember that with Turbo you needed a different setting if the object code were to exceed a certain size, but with Q-Liberator this, to my knowledge, isnt even mentioned.

Does this mean there is some limit to the size of S*BASIC or Qliberated programs? If not, how can Turbo and Qlib circumvent this with apparent ease, while c68, FPC, & al cant?

I havent written no more than some small toolkits in assembler these past decades, so I may have lost touch, but doesnt this sort of thing work?

xref.l faraway
..
add.l faraway,pc
we're now faraway

or

lea.l faraway,a2
jmp.l (a2)

Thanks Marcel, for your further clarification! Yet Im still somewhat at sea, as you see.

Its not quite clear yet whether this is just a digression or a new topic, so apologies for hogging the thread for now..

[mk79]

Does this mean there is some limit to the size of S*BASIC or Qliberated programs? If not, how can Turbo and Qlib circumvent this with apparent ease, while c68, FPC, & al cant?

Simple, QLiberator does not compile to machine code, so machine code limits do no apply. Turbo does more in this direction but I guess jumps are also done in some abstract way.

I havent written no more than some small toolkits in assembler these past decades, so I may have lost touch, but doesnt this sort of thing work?

xref.l faraway
..
add.l faraway,pc
we're now faraway

No, that instruction does not exist.

lea.l faraway,a2
jmp.l (a2)

You just moved the problem from the JMP to the LEA. LEA does not have more addressing modes available than other instructions, so now the LEA target must either be absolute (so needs fixup, which is what C68 and FreePascal does) or +/-32kb from the current PC.

Marcel

[tofro]

Does this mean there is some limit to the size of S*BASIC or Qliberated programs? If not, how can Turbo and Qlib circumvent this with apparent ease, while c68, FPC, & al cant?

Simple, QLiberator does not compile to machine code, so machine code limits do no apply. Turbo does more in this direction but I guess jumps are also done in some abstract way.

I havent written no more than some small toolkits in assembler these past decades, so I may have lost touch, but doesnt this sort of thing work?

xref.l faraway
..
add.l faraway,pc
we're now faraway

No, that instruction does not exist.

lea.l faraway,a2
jmp.l (a2)

You just moved the problem from the JMP to the LEA. LEA does not have more addressing modes available than other instructions, so now the LEA target must either be absolute (so needs fixup, which is what C68 and FreePascal does) or +/-32kb from the current PC.

Marcel

What works is something like the below:

Code: Select all

* This macro jumps (in a "PC-relative" way) to a 
* subroutine which would normally be not reachable 
* by a bsr. Use in large programs you dont want to 
* make 'impure'. 
FARJSR          MACRO   target,register   
                lea     .lab[.L],[register]           
                add.l   ([register]),[register]           
                jsr     ([register])           
                bra.s   .across[.L]  
.lab[.L]        dc.l    [target]-* 
.across[.L]               
				ENDM
		
* Use like in
				FARJSR faraway_label,a2
		 

(which looks a lot more complex than a simple bsr) but wastes a register and is not exactly fast.
The linker puts the (long) distance between the target and "here" into .lab[.L] (the brackets ar just some MACRO magic), the code loads an address register with "here" and adds the distance, then jumps there.

Before I'd use this in a compiled program, (for all jumps, because at compile time we can't know how far away a label will be after linking) I'd rather live with an impure one.

[janbredenbeek]

I havent written no more than some small toolkits in assembler these past decades, so I may have lost touch, but doesnt this sort of thing work?

xref.l faraway
..
add.l faraway,pc
we're now faraway

No, that instruction does not exist.

I have never hit this problem myself (the farthest reference in QBOX was just under 32K) but you would have to set up a register (say A4) to point to the base of the program (near the start, so PC-relative addressing could be used) and then do something like

Code: Select all

move.l #faraway-base,d0
jsr (a4,d0.l)

.. which will cost you at least one register and a lot of overhead, unless you can optimise jumps which are less than 32K away.
Incidentally, the GST QC compiler indeed used A4 as a program base pointer, but with added offset of 32K so programs could be up to 64K in size with position-independent references from -32768(a4) to +32767 (a4).

Jan

[pjw]

Thanks guys.

It appears Ive spent the past thirtysome years in a 32k bubble without knowing it! (Or if I ever did know, Ive even forgotten when I forgot!)

I shant infringe any further on this thread, but should I ever hit any such limits in my own meagre programming efforts I'll know where to turn for advice Thanks again for taking the trouble to explain!

[NormanDunbar]

In case anyone is thinking of having a bash at writing the SYSUTILS.PP functions (in rtl/sinclairql) then I'm already in there messing about.

I've added code to make the following work, hopefully, and I'll be testing shortly:

• FileOpen()
• FileClose()
• FileSeek()
• FileTruncate()
• DeleteFile() (naming standard went awry there! )
• FileCreate()
• FileRead()
• FileWrite()
• RenameFile() (another standards slip-up!)

Once tested, I'll post a patch file, as usual.

Cheers,
Norm.

[Chain-Q]

In case anyone is thinking of having a bash at writing the SYSUTILS.PP functions (in rtl/sinclairql) then I'm already in there messing about.
I've added code to make the following work, hopefully, and I'll be testing shortly:
(...)
Once tested, I'll post a patch file, as usual.

Sounds great. There are some pre-made sysutils tests in directory tests/test/unit/sysutils. Not sure all can work as-is on a QL, but they're at least a good starting point, to make sure the code behaves as applications expect it.