IP Network driver

[tofro]

Can you check the channel ID in your I/O routine?

In my own tests of how to possibly "stack" a device driver, I have experienced the odd case that trying to read (using an I/O trap within the I/O routine of the "upper" driver) from the "bottom" end channel actually ended up in calling the "top" end driver's read routine recursively (oddly with the channel ID of the "bottom" channel), eventually ending up in a supervisor stack overflow. This is what I meant with "all sorts of odd behavior" above.

(And this was exactly where I stopped trying that stacking )

About 9 iterations would quite well fit with the 100 available supervisor stack bytes.

Tobias

[Martin_Head]

Can you check the channel ID in your I/O routine?

In my own tests of how to possibly "stack" a device driver, I have experienced the odd case that trying to read (using an I/O trap within the I/O routine of the "upper" driver) from the "bottom" end channel actually ended up in calling the "top" end driver's read routine recursively (oddly with the channel ID of the "bottom" channel), eventually ending up in a supervisor stack overflow. This is what I meant with "all sorts of odd behavior" above.

(And this was exactly where I stopped trying that stacking )

About 9 iterations would quite well fit with the 100 available supervisor stack bytes.

Tobias

Interesting...

I kind of fits in with some of the behaviour I have seen when the driver fails during my debugging. It always stops at the same place, It displays a message saying its about to enter a subroutine, but if I have a message at the start of that subroutine, I does not get displayed. So a BSR fails, which suggests a stack issue.

I also put a debugging message in, to show when a data packet has been read and processed (just before the I/O routine would exit normally). And this never gets shown.

The last thing I did yesterday was to patch the routine that does the calls to the actual IP driver to read a packet, so that it skips the calls and just returns an error. I must have tried it a couple of dozen times, and it never failed at all. Which points to an issue with calling the IP drivers.

I have been thinking about your earlier suggestion of calling the IP driver routines directly.

You say you've had problems calling I/O routines while in an I/O routine. I've done this before in my Microdrive, and Floppy disk image file drives and not had any problems.

Martin Head

[tofro]

Martin,

in the case I describe, my i/O routine had exactly one entry point (they normally have )and one exit, both heavily plastered with ut_mtext calls for logging.
What I have seen was repeated entry messages and not a single exit message before a crash. There was no direct call to the subroutine entry anywhere, only some trap#2/3 towards the lower layer. That lead me to the conclusion that trap#2/3s will/might go haywire when executed in an I/O routine.

As I was only trying to use the stacking for debugging, I found another way around that. But you are right, your image drivers must be doing something similar, and they seem to work. Where exactly the difference is, and what cases would lead to the observed behaviour, is beyond me at the moment, however.

Regards,
Tobias

[Martin_Head]

What you normally should be able to do from your driver is calling the below device driver directly instead of trap - ing into the OS to make that call. You'd need to find the open, close and I/O entry points from the device driver linkage block, set up the registers according to their specific requirement (it's all in the docs), and directly JSR to those addresses. I have never tried this myself, because in my driver I could find another, simpler, work-around, but I think that has the highest chance of working.

Gave this a try yesterday, and first impressions look good. Did a dozen LBYTES without crashing, Also LOAD and COPY looked good. I think I will now rip all the debugging stuff out of the source code, and do some tiding up, as it's a bit of a mess at the moment, with bits commented out all over the place.

On a slightly different note, can I ask...

1. On the original driver there are checksums on the data sent. At the moment I have disabled them while I was tracing the routines. Should I re-enable them, or are they unnecessary. Shouldn't the TCP connection handle data errors?

2. The original driver would send an acknowledgement after every packet received, so the sender knew it got through OK. At the moment I have this turned off. Should I reinstate it? Again wont TCP ensure that all the data gets to the receiver, and in the right order?

3. This is tied in with 2 above. If I don't send acknowledgements to say the data packet has been received, then data packets will queue up waiting to be read. How big is the buffer TCP uses store unread data? What happens when it fills? Will data get lost? Or will the sender be told to hold until there is space available?

Thanks in advance

Martin Head

[tofro]

Martin,

that sounds good!

With regards to question (1) and (2) - You shouldn't need to care about checksums or acks, at least not if none of your upper layers ask for them - That's all handled on the TCP level for you, it would just slow down transfer rates.

(3) is a bit harder, maybe. Windows apparently uses a maximum of 64kB buffers per socket (Which should be ample for QL comms....), and I guess we can safely assume that limit applies to the QPC stack as well.

What happens if the sender sends data faster than the receiver can digest them: The receiving end will apply backpressure to the sender when buffers on the receiving end tend to become full -
What happens on the sending side depends on how you opened the socket there:

Standard behaviour is: The write or send will simply block (which is generally a bad thing in a device driver...) until congestion goes away and send buffers become available again.

If you set the socket to SO_NONBLOCK (i.e. non-blocking I/O) via setsockopt() (excuse my C), the write would return with E_WOULDBLOCK to tell you buffers are full, try again later.
You would definitely want to go with non-blocking sockets (if you could, see below).

Unfortunately, I can't find the SO_NONBLOCK option in the uqlx specification. Maybe (?) the timeout you hand in in the IP_SEND call helps (although the docs say "should be -1") - I wonder if non-blocking I/O is supported by the QPC stack at all.

Data will normally not be lost (TCP cares for that) in that case. As long as the receiver remains available, the transfer will just slow down.

Tobias

[Martin_Head]

The driver is coming along nicely, and is getting close to being releasable.

However I have hit a bit of a problem with remote file access using FSERVE

I can for example:-

DIR n7_flp1_
COPY n7_flp1_program_bas TO ram1_test
VIEW n7_flp1_program_bas
OPEN_IN#4,n7_flp1_program_bas:INPUT#4,a$:PRINT a$
SAVE n7_ram1_test

They all work fine, But if I try to LOAD or MERGE I get problems.

MERGE opens a file, copies a few data packets across, then gives a "invalid syntax error", then closes the file.

LOAD will open a file, copy a few data packets across, then do another open, then copy some data packets across, then do a close, copy some more data packets, then do another close. Or it may just crash the system.

With both LOAD and MERGE, if I try to LIST afterwards nothing seems to make it to BASIC

However if I try the driver in Qemulator, with QPC2 acting as the file server. LOAD and MERGE seem to work OK.

Also I can SAVE neto_7 on one computer, and LOAD neti_3 on another, OK

So what is so special about LOAD and MERGE in SMSQ/E on a directory device, rather than a serial device. I have not been able to locate the LOAD and MERGE commands in the SMSQ/E source code to try to follow what is happening (Can anyone tell me where they are?)

The SMSQ/E sources documentation refers to LOAD and MERGE using the "Home directory thing" What is this and what does it do? Is it relevant to a device driver? If the device driver needs to handle it, then why doesn't it (the network driver is the one from SMSQ/E).

I'm guessing that one of my driver routines is altering a register that the LOAD and SAVE routine is using, but which one? and in which routine, open, or I/O?

Martin Head

[tofro]

The SMSQ/E sources documentation refers to LOAD and MERGE using the "Home directory thing" What is this and what does it do? Is it relevant to a device driver? If the device driver needs to handle it, then why doesn't it (the network driver is the one from SMSQ/E).

Martin,

nice to hear that your driver is coming along.

The "Home directory thing" is a piece of code expanding the SMSQ/E operating system. What it basically does is maintaining a "Home Directory" and possibly, a "Home file name" on a per-job base.
That is, it remembers the directory where the job was loaded from and supplies this to the application, if being asked for. It also remembers where the job was last loading a file from supplies this back to the application later on.

SBASIC load and merge tell the Home Thing to remember the file name a program was loaded from . Thus, when you just type "SAVE" without supplying a file name, the file is being saved back to where it was originally loaded from. Actually a nifty feature, but should not affect your driver in any way.

And: Most of SAVE and LOAD is in sbsext_ext_basic_asm, with helper routines in sbsext_ut.
And the only difference between QDOS and SMSQ/E that comes to mind atm is that SMSQ/E reads the file header on LOAD in order to ensure you only load plain files. QDOS doesn't do this, if I remember right.

Tobias

[Martin_Head]

Thanks for the response

And the only difference between QDOS and SMSQ/E that comes to mind atm is that SMSQ/E reads the file header on LOAD in order to ensure you only load plain files. QDOS doesn't do this, if I remember right.

That may answer what I thought was strange with LOAD doing two opens and two closes. It opens the directory to read the header, then opens the actual file to load.

I thought that maybe I will try, on entering the I/O routine, save all the registers except D0,D1 & A1 on the stack and restore them on exit to make sure that none are causing problems for LOAD/MERGE. And if it works, de-protect them one at a time until the problem recurs.

Martin Head

[Martin_Head]

The driver is now basically working on QPC2, there may still be the odd bug floating about, I need to do some more testing.

Anyway, why I am here. I was wondering if one or two people would be willing to test the driver on networks other than my simple cabled Windows for workgroups (or whatever they call it nowadays) network. I would like to know if the driver can handle networks with routers, and WiFi etc. Could it even handle a Virtual Private Network over the internet? Also test it on other emulators that support the IP drivers. uQLx, SMSQemulator? I have had a quick play with Qemulator, and it works up to a point. The file server FSERVE works, but it pretty much halts Basic, and upsets Qemulator (if you go to one of the menus under the title bar, the menus disappear and the title bar says - Not responding). I don't know if this is an issue with the driver, or with the IP drivers on Qemulator.

I can post the driver here, or email it. Whichever you want.

Martin Head

[swensont]

Martin,

I can give the driver a try. I have both Q-emulator and SMSQmulator running on my Linux laptop. I have both Ethernet and wireless connections to test. I also have Wireshark installed to do any TCP/IP capture and analysis.

I will need some instructions on setting up the driver and testing it out. I''m usually on the forum chat staring about 1 pm Pacific on most weekdays (about 9 pm GMT).

Tim Swenson