SlipKnot-like Browser for QL - Mimir

[bwinkel67]

For my summer project I took a crack at writing a web browser. The goal is to have it hooked via WiFI modem to the internet and grab live pages the way SlipKnot worked. I've gotten my Simulant WiFi modem to connect to freeshell.org (i.e. sdf.org), a free UnixBSD server that gives telnet access and has tools like lynx.

For my immediate goal I wanted to get a prototype running on a BBQL. It currently only parses Lynx dump files, since a majority of the work getting through complex pages is done by Lynx. This also reduces the size of files transferred to the QL. Is that better than using lynx in a terminal program? Besides making the process easier (just exec a file and browse), I hope to eventually be able to display images.

The current version is buggy and limited. I still haven't implemented paging down (I only spent this week on it) so all it can show is the first page and if you have links off the first page I have no idea what it'll do. You use left and right arrows to move links, hit enter to follow a link, and alt-left to go back to the previous page. You can also type/edit the URL line to try different files. Right now my demo has full pathnames, but I'll change that and grab the device of the URL being entered and then expect everything to be there. It's useful for local pages (i.e. you could replace a readme/PDF with a clickable help system).

Give it a try. In Q-Emulator, attach the MDV file to mdv2 and type "exec mdv2_mimir" and when it asks for URL, get started with "mdv2_test_htm" (I realize these aren't HTML files so I need to find a better suffix). I plan to finish the front-end this summer but the back-end (WiFi modem connection to server) will have to wait for winter or next summer.

MIMIR.zip

(40.8 KiB) Downloaded 12 times

[bwinkel67]

Oh, Mimir is the North god of knowledge...

Btw, I did look at what's out there and found two versions of Lynx for the QL but both require excessive memory (at least 640K if not 2MB). I also found a few HTML parsers, the most interesting being QMosaic. Unfortunately I have not been able to get it to work on any emulator but it looks pretty cool (it parses HTML 1 I believe) and is only 19K in size. I would love to see how fast that runs and if I could potentially move to HTML in some form to get a bit more formatting on the QL side (at least font sizing, etc since we're dealing with only 4 colors).

If I could get QMosaic to work, I bet it would work similarly though my executable is only about 5K, so very small. Of course I don't have the code for QMosaic and don't even know what it was written in. My browser is written in C and once I get it to a good state I'll create a GitHub repository. I'll eventually look to port it to other platforms. Always wanted to code in C on an Amiga so that might be next...but only after I get this fully working on the QL.

[bwinkel67]

Someone asked for navigation clarification. I edited the original post but will articulate it here in a list.

Once you've mounted MIMIR.MDV on mdv2_, and exec'ed the mimir executable:

• to get started, type mdv2_test_htm and hit enter
• to go to next or previous link use -> and <-
• to follow a link, hit enter
• to go to previous page, hit ALT <-

[bwinkel67]

I'm getting some more time on this project again after summer travel.

I've been thinking on how I should do the connection to the server. I wasn't looking forward to having to somehow internally script things so the program would put out a command, then have to wait and look for a string like "Login", then enter the username, then wait for a string like "Password", then enter the password, then maybe look for something else until you are logged in. It would be different for any platform the system had to connect to and of course I'd have to make it configurable so a user could script it for their setup.

But that sounds like soooo much work. My solution, when putting it in CONNECT mode, just give a simple terminal program where the user does the login themselves, i.e. ATDTfreeshell.org for me (replace freeshell.org with whatever) and proceed to log into the server. Then, once you get to the command prompt, just get back into BROWSE mode (i.e. hit one of the function keys) and go. I think that will make things really simple. Plus, I wrote a simple terminal program in the 90s so I have the code for it already.

[bwinkel67]

Finally had some time for coding. The browser is almost fully functional for local files. I still need to untie the device so that it's not bound to a single device. The idea is that I parse a typed-in URL and if it starts with mdv1, mdv2, flp1, win1, etc I make that the default directory. Then, all my file links (i.e non http ones) would simple have the name without the device.

The only other caveat is that currently it has hard coded maximums for number of lines per webpage (60) and it doesn't do any boundary checks. Still, it otherwise works pretty well and only 6K in size. I'm sure it has bugs but so far I haven't hit upon them from the set of test pages.

To run it in Q-emulator, attach the MDV to mdv2_ and start up the executable via: exec mdv2_mimir

Here is an updated set of commands:

• To get started, type a file path...I recommend to run the demo, i.e. type: mdv2_test_htm
  - The last link on the test_htm page (info_htm) has multiple screens to try out.
• Use left-arrow and right-arrow to move through the links on the current page.
  - Currently it doesn't jump to the next page if the next link is there...something to add in the future.
• Hit ENTER to follow a link
• Use the down-arrow to go to the next screen of the current page
• Use the up-arrow to go back to the previous screen of the current page
• Use ALT left-arrow to go back to previous webpage
• Use ESC to clear out the text from the URL prompt
• Use Ctrl-Q to quit the browser.

MIMIR.zip

(43.42 KiB) Downloaded 10 times

[Edit: I think the next phase is to integrate a simple terminal emulator (I have code) so I can attach a wifi modem to a Telnet server...summer is quickly ticking down ]

[bwinkel67]

Well, luckily my new browser doesn't rely on dial-up:

AOL is ending its dial-up service, disconnecting from the 1990's internet era

• https://www.cbsnews.com/news/aol-dial-u ... wtab-en-us

[tofro]

Well, luckily my new browser doesn't rely on dial-up:

AOL is ending its dial-up service, disconnecting from the 1990's internet era

• https://www.cbsnews.com/news/aol-dial-u ... wtab-en-us

I didn't even know that AOL is still a thing. Over here, they closed down 15 years ago.

[bwinkel67]

Small but relevant change. All the local file links in the various _htm files no longer have the device name. The browser now uses the device name of the last one that was typed in (i.e. mdv, ram, flp, win) and sets that as default. Anytime a new full path is used the default is changed and added to any new file. If it's a URL (i.e. http:// or https://) no device is added -- I basically look for a '/' in the 6th position to see if it's a URL so it's a bit of a hack but works.

So now you can copy all the _htm files to either a flp or ram, etc... and then run it from there and it should work fine. If you look at test_htm you'll see the links are now just the file names, so easier to create a linked help system for instance.

Code: Select all

                        Mike's Test Page

This page will test my browser. It will show serveral links and then
link to other pages. Here is a list of links:

  o [1]News - list of mostly US news broadcast statins
  o [2]Readme - part of Digital C SE readme file
  o [3]keytest_bas - prints key code for key pressed
  o [4]Info - list of websites

References

  1. news_htm
  2. readme_txt
  3. keytest_bas
  4. info_htm

Ok, enough getting the browsing part to work. Next is the Telnet connection and then eventually figuring out how to transfer files over and load into browser automatically when http is used.

MIMIR.zip

(44.06 KiB) Downloaded 9 times

[bwinkel67]

Code snippet to parse devices. I don't have strstr() available so I came up with this as a quick reference using strchr().

Code: Select all

char dev[12] = "ramdvflpwin";
char def[5] = "";
char lname[MAXLINE] = "";


...


   char str[4], name[MAXLINE];

   if (lname[4] == '_')  /* is it a device? */
   {
      for (i=0; i<3; i++)
         str[i] = tolower(lname[i]);
      str[3]='\0'; 

      /* checks only legal position totals for dev = r a m d v f l p w i n */
      val = strchr(dev, str[0]) + strchr(dev, str[1]) + strchr(dev, str[2]) - 3*dev;
      if (val == 3 || val == 9 || val == 18 || val == 27)
      {
         strncpy(def, lname, 5);
         name[0] = '\0';
      }
      else
         strcpy(name, def);
   }
   else if (lname[6] != '/')  /* no http:// or https:// and no dev, so default */
      strcpy(name, def);

   strcat(name, lname);

[Edit: I should see if I can replace the if/else with a switch for more efficinet code:]

Code: Select all

      switch(val)
      {
         case 3: case 9: case 18: case 27:
            strncpy(def, lname, 5);
            name[0] = '\0';
            break;
         default:
            strcpy(name, def);
      }

[tofro]

You are certainly aware that

Code: Select all

if (name [4]=='_')

is not a very good test for a device name, especially not when considering default directories, the fact that QDOS device names don't need to be three characters long, that there are network file names, and that people can be (and have been, in the past) very creative with device names (at least 'dsk' and 'sdc' come to mind as examples). QPAC2 "Files" seems to have found a mechanism to extract the names of all directory device drivers in a system and displays them in the file selector - I do, however, have no idea what dirty trick it uses to achieve that....