Page 15 of 18
Re: QLiberator decompiler
Posted: Fri Feb 07, 2020 3:29 pm
by EmmBee
HAOUI wrote:
IMHO, What we need instead, is PARNAM$() or PARTYP$() working for compiled $$external. I already wrote similar functions as assembler extensions that work for $$external (see sources and binary in attached archive). I called them PARNAMEX$() and PARSTREX$(). I am not totally happy with these extensions but it does work. I think it could be better written with some more information. Also, it could be easily amended to support interpreted programms and replace tk2 original functions.
Any feedback, suggestion welcomed.
Alain
paramex.zip
I've just tested this out. It works well. Just what we need. Well done!
If it can be amended to work in Basic as well, that would be great.
It would be a total replacement for the existing similar commands in TK2.
Re: QLiberator decompiler
Posted: Sat Feb 08, 2020 11:01 am
by Martin_Head
I know how to decompile a compiled SuperBASIC program. But I don't know a great deal about how QLiberator actually works.
To use a name with a Procedure, you could use the same method as the operating system. It looks at the name table entry, If it finds a 'name' it converts it into a string. It does this by manipulating the the name table entry. I can't remember the exact details, but I have done it for SuperBASIC extensions in the past, by copying a short routine from the SMSQ/E source code.
You could do this on the SuperBASIC side, or perhaps in the QLib object itself.
I'm pretty sure that QLiberator externals, are just added to the underlying SuperBASIC using BP.INIT like normal SuperBASIC extensions.
I was having a little look yesterday at my QLIB dumps. On the Externals list page of my De-Lib Decompiler Technical Notes, The Externals list seems to be the data for a BP.INIT in the initialization area of the _OBJ file. If you study the Externals list code , it does look like BP.INIT data. Where the offset to the start of the command, is that offset to the NOP. That NOP($4E71) is followed by a MOVE.L #...,D4. That value being a pointer to the start of the code for the Procedure/Function.
I don't know where, or how QLiberator generates this code, or if the NOP serves a purpose. But it may be possible to use those two bytes as a short branch to a subroutine to do the name table manipulation. Assuming that QLib uses a name table the same as QDOS.
Re: QLiberator decompiler
Posted: Sat Feb 08, 2020 2:29 pm
by HAOUI
EmmBee Wrotes
I've just tested this out. It works well. Just what we need. Well done!
If it can be amended to work in Basic as well, that would be great.
It would be a total replacement for the existing similar commands in TK2.
Attached archive contains mimics of TK2 PARUSE(), PARTYP(), PARAM$() and PARSTR$() commands which can work in interpreted programs and QLIB compiled $$external. They use same parameters in both cases.
Mainly useful to fetch NAME type actual parameters passed without quotes to compiled $$external.
Under S*BASIC interpreters, they work strictly as original TK2 commands.
When used in COMPILED $$external PROCedure :
PARUSE(),PARTYP() work as expected
PARNAM$(),PARSTR$() fetch actuals parameters passed from any S*BASIC caller
Included sources, binaries and rudimentary readme.
Alain
Re: QLiberator decompiler
Posted: Sat Feb 08, 2020 3:44 pm
by HAOUI
Thank you Martin_Head for your feedback. Appreciated.
My understanding now is :
1) Indeed, $$external is registred by BP.INIT as machine-code PROC but as it is not and can't be executed, QLIB as you say insert a call to a RUNTIME machine-code at the start of external which creates a job and executes external as a normal and independant QLIBerated program
2) QLIBerated programs use the same Name Table as QDOS but not the same Return Table. This is why PARNAM$ and PARSTR$ TK2 commands couldn't never work and may easily crash system.
3) The created job starts with its own Name Table. Manual states that it is a copy of the caller NT but this is not completely true. Unfortunately this NT doesn't contain the passed parameters (they still available on the NT of the caller until return from job). So, using of classical assembler methods you mentionned can't work, either.
4) By-pass I found is to access the NT of the caller and search wanted parameters. It does work but I don't like very much this method, because I have to enter SuperVisor mode and we can't know the number of actually entered parameters when command was issued (QDOS doesn't create an entry in its Return Table when calls "machine-code" exts).
As I said before, giving that external copies back altered parameters, I guess that somebody (RUNTIME or Job, it can't be SB or QDOS) holds some pointers to this area (but where ?). Without more info, I am afraid to have to deassemble QLIB RUNTIMEs to understand /improve (too old now for this hack..).
Alain
Re: QLiberator decompiler
Posted: Mon Feb 10, 2020 10:52 am
by Martin_Head
I don't think the QLib runtime module has anything to do with externals. I think the externals are handled by the _OBJ file itself.
I had a very quick look at your assembler code, where you are examining the name table entries with A3 and A5. This is the routine I was thinking about for converting a 'name' into a 'string'
Code: Select all
; ***************************************************************
; UT_GTNM1 Take a supplied string, or name parameter and
; put it on the maths stack as a string. Missing parameters
; cause an empty string to be placed on the maths stack.
; On exit BV_RIP is cleared
; Based on code from SMSQE copyright 1985 Tony Tebby
;
; Entry A1 points to top of Maths stack
; A3 Points to first parameter in name table
; A5 Points to last paramter in name table
; Exit A1 Points to string on Maths stack
; D0 Error return
; D1-D3,A2 Smashed
; ***************************************************************
ut_gtnm1 cmp.l a3,a5 ;is there at least one param?
bgt.s ut_gtnam ;yes, there is
; Return an empty string on the maths stack
ut_gtnul bsr ut_ckri6 ;make room for string on maths stack - 6 bytes
sub.l #2,a1 ;total length of string 2 bytes (length word)
clr.w (a6,a1.l) ;set string length to zero bytes
bra.s utgn_ok ;exit without error
; Get the name type word from the name table entry
ut_gtnam moveq #$f,d0 ;extract type of name
move.w (a6,a3.l),d1 ;get the type word from the name table entry
cmp.w #$0300,d1 ;internal substring array?
beq.s ut_gtnst ;..yes, get it
and.b d1,d0 ;..no, mask out everything but last 4 bits
beq.s ut_gtnul ;return an empty string if a null parameter
tst.w 2(a6,a3.l) ;is there a name?
bmi.s ut_gtnst ;..no, it's $FFFF. Get it as a normal string
subq.b #1,d0 ;is it a string?
bne.s ut_gnmpt ;..no, get it as a name instead
; Get a normal string parameter onto the maths stack
ut_gtnst bsr.s ut_gnst1 ;get one string (no check)
bne.s utgn_rts ;problem? Exit with error
moveq #3,d1 ;round up to word, including char count
add.w (a6,a1.l),d1 ;get length of string on maths stack
bclr #0,d1
add.l d1,bv_rip(a6) ;clear maths stack, but leave A1 pointing at the string
bra.s utgn_ok ;exit without error
; Get a name parameter onto the maths stack
ut_gnmpt moveq #0,d1
move.w 2(a6,a3.l),d1 ;get the pointer to the real entry - in the name table
blt.s ut_gtnul ;... expression is no good - in case a $FFFF slips through
lsl.l #3,d1 ;in multiples of 8 bytes - length of name table entries
add.l bv_ntbas(a6),d1 ;add to base of BASIC name table - to find entry in name table
ut_ntnam moveq #0,d3
move.w 2(a6,d1.l),d3 ;get the entry from name table
add.l bv_nlbas(a6),d3 ;and add to start of name list, to find entry in name list
; Put the name pointed at by D3 onto the maths stack
ut_nmtos moveq #3,d1 ;get the length of the name as a long word
add.b (a6,d3.l),d1 ;get length of entry from name list
bclr #0,d1 ; rounded up (+2)
bsr.s ut_chkri ;make room for D1 bytes on the maths stack
add.l d1,d3 ;move to end of string (ish)
ut_nm_lp subq.l #1,d3 ;adjust pointers
subq.l #1,a1
move.b -1(a6,d3.l),0(a6,a1.l) ;copy bytes of name onto maths stack
subq.w #1,d1 ;decrement loop count
bgt.s ut_nm_lp
clr.b (a6,a1.l) ;clear top byte of string length word
utgn_ok moveq #0,d0
utgn_rts rts
; Reserve D1 bytes on the maths stack
ut_ckri6 movem.l d1/d2/d3/a2,-(a7) ;save registers
moveq #6,d1 ;we want 6 free bytes on the stack
bra.s utc_do
ut_chkri movem.l d1/d2/d3/a2,-(a7) ;save registers
utc_do move.w bv_chrix,a2 ;reserve space
jsr (a2)
movem.l (a7)+,d1/d2/d3/a2 ;and restore them
ut_setri move.l bv_rip(a6),a1
rts
; Get a normal string parameter onto the maths stack
; Returns with D3=no of strings returned, and D0 an error code
ut_gnst1 move.l a5,-(a7) ;save name table top pointer
lea 8(a3),a5 ;set name table to have just one entry
movem.l d4/d6/a0/a2,-(a7) ;save registers
move.w ca_gtstr,a2 ;get a string
jsr (a2) ;do the operation
movem.l (a7)+,d4/d6/a0/a2 ;restore registers
move.l (a7)+,a5 ;restore name table top pointer
rts
I pulled this from my MDI driver source. It allows me to have a SuperBASIC extension that can use a command like - MOUNT_MDI 1,dos1_imagefile_img
Re: QLiberator decompiler
Posted: Tue Feb 11, 2020 12:57 pm
by HAOUI
I understood what you meant and made several similar assembler routines in the past for extensions, but this has no chance to work for externals. As I said before, parameters passed to the external are not copied into the name table of the created job for external. That's why I tried this time to access the name table of the caller, no other choice I am afraid.
Also, I do confirm, when external is called, QLIB RUNTIME (embeded or rte) is started first.
Thanks,
Alain
Re: QLiberator decompiler
Posted: Fri Feb 14, 2020 5:03 pm
by RalfR
EmmBee wrote:Ralf, about sedit_63 - have you renamed the vars, yet?
Just found the source today. A test.win where I have copied a lot of files without directories.
Always good to have a backup in the office! Now I have to put all the files together.
Re: QLiberator decompiler
Posted: Sat Feb 15, 2020 12:07 pm
by EmmBee
Ralf R. wrote:EmmBee wrote:Ralf, about sedit_63 - have you renamed the vars, yet?
Just found the source today. A test.win where I have copied a lot of files without directories.
Always good to have a backup in the office! Now I have to put all the files together.
Would be good if the original authors of QLiberator could suddenly find their sources!
Re: QLiberator decompiler
Posted: Sun Feb 16, 2020 9:52 am
by RalfR
EmmBee wrote:Would be good if the original authors of QLiberator could suddenly find their sources!
I am pretty sure, that TT has the sources, I think he was much involved in that project.
Re: QLiberator decompiler
Posted: Mon Feb 17, 2020 8:33 pm
by HAOUI
Ralf R. wrote:EmmBee wrote:Would be good if the original authors of QLiberator could suddenly find their sources!
This is the best thing that could happen, especially for RUNTIMEs which could not be decompiled. Deassembling 11 Kbytes of pure binary code may give thousands and thousands of assembler instructions, uncommented/undocumented. A real nightmare. IMHO, any improvement on the compiler side may need some work on RUNTIMEs also.
Alain