Q_Liberator malaise

[tofro]

... well producing the Q60 from a perfect hardware design in 2002, I had to use a bugged and bearly functional operating system.

In retrospect QDOS Classic by Mark Swift mag of been better. But I preferred to use SMSQ/E.

The MOVEP isxue has nothing to do with the 68040, 68060 Cache problem, that was not another bodge done by the SMQ/E programmi g team. Which I was keep out of the group and had no input, till recent times.

The problem looks to me lack of understanding of the Cache intructions, but the use and definition is clearly outlined in the SMSQ/E Reference Manual. Quite strange.

I will do an analysis of the setup of the Q60 caching in the SMSQ/E source code.

There's not much an operating system like SMSQ/E can do in order to facilitate the use of self-modifying code (which everyone seems to suspect QLiberator does - I don't know).
The CPU needs to be told that what might just have been written to the data cache needs to land in main memory immediately, (so, write-through) and the instruction cache might need a re-load of that memory if the change would affect its current contents. Otherwise, the CPU might always risk to execute "old" code. Doing this in general in the OS effectively disables the caches completely and slows down all programs (significantly), even those that wouldn't need it because they are properly programmed.

This can only be solved properly when the actual application (that "knows" program code has just been modified) flushes the data cache and re-loads the instruction cache if needed. That would mean QLiberator runtimes needs to be explored and fixed with cache flush and re-load instructions depending on the CPU we're running on. Modifying the OS to "fix" the problem is the wrong place. You can make SMSQ/E execute QLiberated programs, but at the cost of slowing down everything.

In theory, you could hit the same problem even with the 68020 and upwards - but the '020 cache is so small that it is unlikely.

[Derek_Stewart]

Sorry for the spelling mistakes.

So the problem area is Qliberator?

Is there a central repository for the Qliberator source code?

Does Turbo compiled programmes have the same problem?

[tofro]

Sorry for the spelling mistakes.

So the problem area is Qliberator?

Does Turbo compiled programmes have the same problem?

It really is and no, they don't (at least afaik).

[Derek_Stewart]

Where is Qliberator source code located?

[tofro]

Where is Qliberator source code located?

I don't know if the QLib runtimes have ever been fully re-engineered.

Source code as so far analyzed should be there: https://dilwyn.theqlforum.com/qlib/index.html

And I guess what needs to be fully understood is the runtime system (usr/qlib_run_asm)

[Peter]

There's not much an operating system like SMSQ/E can do in order to facilitate the use of self-modifying code (which everyone seems to suspect QLiberator does - I don't know).

A timing issue or something related to hardware access would also explain the difference between writethrough and copyback.
The first step should be to reproduce the problem with relatively simple code instead of complex programs that use toolkits.

[tofro]

There's not much an operating system like SMSQ/E can do in order to facilitate the use of self-modifying code (which everyone seems to suspect QLiberator does - I don't know).

A timing issue or something related to hardware access would also explain the difference between writethrough and copyback.
The first step should be to reproduce the problem with relatively simple code instead of complex programs that use toolkits.

That's why I wrote "everyone seems to assume..." and "I don't know" above. I'm missing proof that "the problem" (that I myself have never experienced, because I don't use QLiberated code often) is caused by self-modifying code and/or cache modes at all. Before that proof is there from a minimal, reproducible example, there's not much point in rummaging about with the OS/QLib code (Unfortunately, my Q40 is in storage in favour of some of the "smaller boxes" I got from you recently, so no quick chance to test.)

[dilwyn]

Where is Qliberator source code located?

I don't know if the QLib runtimes have ever been fully re-engineered.
Source code as so far analyzed should be there: https://dilwyn.theqlforum.com/qlib/index.html
And I guess what needs to be fully understood is the runtime system (usr/qlib_run_asm)

Small update of QLib 3.46 tonight. Not a new version, just tidying up and fixing a few typos etc.

Make sure that you scroll down the page to v3.46 for the latest version.

[Peter]

I'm missing proof that "the problem" (that I myself have never experienced, because I don't use QLiberated code often) is caused by self-modifying code and/or cache modes at all.

In the early days of the Q60, I never used QLiberated programs, always had copyback cache on, without problems.
Only in the recent years, I came to like Dilwyn's software, which unfortunately also means QLiberated and crashes on Q60.

Still I would have much less interest in a solution, if it was just for the historic Q60 and my personal software taste. My larger problem is that SMSQ/E defaults to copyback off, and that will be the typical user experience if I design a new 68060 machine. I can hardly motivate myself to the insane effort of designing a very fast hardware (in 68K measures) already knowing that the machine will be crippled to half speed.

I wish SMSQ/E default had remained copyback on, as it was in the beginning! That way, the fact that only QLiberated programs misbehave would have created much more pressure towards a fix, or to use Turbo. QLiberator would have required a hint that one needs to disable copyback cache. And people would have experienced the massive speed difference once they do. It would not have been my policy to let the OS conceal a single, poorly behaving BASIC compiler.

[tofro]

Just out of interest, I dug a bit into the source code for the QLib runtimes.

First, (excuse the rant) the code is about as "clean" as the average public toilet in Frankfurt Main Station. I found stuff like "jump to 3 vectors before a QDOS vector address" or "jump to 6 bytes behind that vector" and similar. Assuming that the disassembly has been properly done, it's a bit of a miracle that code works on multiple platforms at all.

Self-modifying code in shared code (this is what lib_run is) can only be reached by an address register (jumping to a fixed data address would collide with multiple QLiberated programs running in parallel). So I searched for jmp(ax) and jsr(ax) instructions only.

There are a number of highly suspicious instructions that look to me like jumps into code prepared before thus hinting to self-modified code.

An example:

Code: Select all

L049E   lea     $0C(a5,d3.l),a1
        move.l  -$0008(a1),d3
          beq.s L04BC
        movem.l d0-d7/a0-a6,-(a7)
        jsr     $00(a1,d3.l)

Apparently, there's only a few places (like 3 or 4) in qlib_run that aren't obviously jmp or jsr instructions over a QDOS vector or a program-internal jump table. Rather than completely understanding the code (which I think would be quite a a bit of an exercise), one could, as an experiment and provided we have reproducible example code, try what happens if these jumps are decorated with '040+ cache flush and invalidate instructions.