Jr Cartridge blank for eprom

[alanh]

I thought Cartridge BASIC added a file to DOS's 'vocabulary' - a sort of non-traditional ROM BIOS extension that supplied a file name and entry point rather than being directly executed.

-Alan

[Brutman]

It looks like DOS executes the BASIC.COM program which then tries to wrapper the ROM BASIC. I think that happens on a PC or an XT too. On a Jr though, attempting to run the built in Cassette BASIC in this way results in the "Cartridge Required" message. I'm not sure if there was a technical reason for that - probably more to do with getting people to buy the cartridge. The "Cartridge Required" message is coming from the system ROM.

And yes, if you look at the beginning of the BASIC cartridge you will see a ROM signature, initialization code, and the 'BASIC' command that DOS knows to look for. Cartridges can also include BASIC code with a slightly different signature.

[Shaos]

In the beginning of BASIC cartridge I think they have both names - BASIC and BASICA, but anyway - it's not working with JR-IDE and DOS 3.20 - BASIC immediately exited back to DOS and BASICA simply hung (with BASIC cartridge inserted)

Actually it is not a big problem for me because now I have TurboC 2.01 that is running pretty well

P.S. I didn't try TurboC++ yet, because its box states that 286 is required...

[Shaos]

As for "optimal way", that's called "optimal parsing", which is a way to guarantee the most efficient matches (offset+length) of the source material, at the cost of speed. Storer and Szymanski described it in their 1984 paper that described LZSS, if you want to look it up out of curiosity; their solution was to parse the data BACKWARDS which initially seems wacko but makes sense if you think about it. LZ4, ZX7, MegaLZ, and exomizer all have an optimal parsing option. Don't know if RNC Pro Pack does, but it still performs competitively.

I found "RNC Pro Pack" and Exomizer. It looks like Exomizer can't handle big files, so I tried RNC only (both methods) plus added a few retro DOS archivers results:

Code: Select all

147,456 JRCARTS7.IMG - original
130,281 JRCARTS8.IMG - my 00-00-00 compression
103,307 JRCARTS7.PP2 - RNC Pro Pack (method 2)
101,009 JRCARTS7.ZX7
100,180 JRCARTS7.IMG.hrm - Russian Hrum
 99,974 JRCARTS7.IMG.mlz - Russian MegaLZ
 98,006 JRCARTS7.LZH - created by LHA v2.13
 80,016 JRCARTS7.IMG.bz2
 79,098 JRCARTS7.LZ4
 72,959 JRCARTS7.PP1 - RNC Pro Pack (method 1)
 71,950 JRCARTS7.zip - modern ZIP from Debian
 71,925 JRCARTS7.ARJ - created by ARJ v2.41a
 71,865 JRCARTS7.AIN - another Russian archiver from 90s
 71,855 JRCARTS7.ZIP - old PKZIP v2.06
 71,808 JRCARTS7.IMG.gz
 70,041 JRCARTS7.HA - HA 0.98 (Harri Hirvola)
 67,437 JRCARTS7.IMG.hst - Russian Hrust
 66,970 JRCARTS7.RAR - old RAR v1
 65,765 JRCARTS7.RAR - modern RAR v5
 62,543 JRCARTS7.7z - modern 7-zip
 62,508 JRCARTS7.IMG.xz - modern LZMA2


[Brutman]

I've been working on a nice little LZW compressor/decompressor that would be interesting to try out on your image. It's never going to get as good as ZIP (because I don't want to invest the time), but it also works much faster too.

(I'm going to use it mostly for text compression. It's basically equivalent to the old Unix compress program.)

[Shaos]

I've been working on a nice little LZW compressor/decompressor that would be interesting to try out on your image. It's never going to get as good as ZIP (because I don't want to invest the time), but it also works much faster too.

(I'm going to use it mostly for text compression. It's basically equivalent to the old Unix compress program.)

I sent you original binary by e-mail

I'm also working on LZ-like compression utility
Most of the work was done in October 2013 and since that time I can't find a night or two to finish it...

[Shaos]

I finished version 0 of my LZ77-like compression (byte-oriented) that I designed in October 2013:

https://github.com/shaos/shaff

and it put me closer to RNC Pro Pack 2 and ZX7 in terms of compression ratio:

Code: Select all

147,456 JRCARTS7.IMG - original
130,281 JRCARTS8.IMG - my 00-00-00 compression
112,715 JRCARTS7.IMGFF - my byte-oriented LZ77-like compression (SHAFF0) <<<<<
103,307 JRCARTS7.PP2 - RNC Pro Pack (method 2)
101,009 JRCARTS7.ZX7
100,180 JRCARTS7.IMG.hrm - Russian Hrum
 99,974 JRCARTS7.IMG.mlz - Russian MegaLZ
.....


it means now I've got 18KB for menu program!

[Shaos]

SHAFF0 format is very simple:

Code: Select all

XX - any byte other than #FF is a single data byte
#FF #00 - single byte #FF
#FF 0xxxxxxx LENGTH (distance -1...-127)
#FF 10xxxxxx LENGTH (distance -128...-190 and -191 means last distance longer or equal to -191)
#FF 11xxxxxx xxxxxxxx LENGTH (directly encoded distance from -191 to -16383)
special case without LENGTH:
#FF #C0 #00 - end of block (instead of distance -16384)


SHAFF0 decoder in C:

Code: Select all

     i = 0;
     while(i<=cursize)
     {
       character = fgetc(f);
       if(character==0xFF)
       {
         character = fgetc(f);
         if(character==0) /* Literal 0xFF */
            buf[i++] = 0xFF;
         else /* Reference */
         {
           /* Decode offset */
           if((character&0xC0)==0xC0)
           {
              offset = (((short)character)<<8)|((short)fgetc(f));
              lastoffset = offset;
              if(offset==-16384)
              {
                 /* Marker of the block end */
                 if(i!=cursize)
                 {
                    printf("ERROR: Something wrong with size (%i)...\n",i);
                 }
                 break;
              }
           }
           else if(character==191)
              offset = lastoffset;
           else
              offset = -character;
           /* Decode length */
           character = fgetc(f);
           if((character&0xC0)==0)
              length = (character<<8)|fgetc(f);
           else if((character&0xC0)==64)
              length = character + 68; /* 132-64 */
           else
              length = character - 124; /* 4-128 */
           j = i + offset;
           if(j < 0)
           {
              printf("ERROR: Something wrong with offset (%i)...\n",offset);
           }
           else
           {
              do
              {
                if(j>=cursize)
                {
                  printf("ERROR: Something wrong with size (%i)....\n",j);
                }
                else buf[i++] = buf[j++];
              } while(--length);
           }
         }
       }
       else /* Literal */
          buf[i++] = character;
     }


[Trixter]

Cute :-) Your decomp inner loop would look something like this:

Code: Select all

12    0866:0100 AC            LODSB
04    0866:0101 3CFF          CMP     AL,FF
16/04 0866:0103 742B          JZ      0130
11    0866:0105 AA            STOSB
18    0866:0106 E2F8          LOOP    0100


61 cycles if code encountered, 49 if literal, 8 bytes.

LENGTH is 1 or 2 bytes:
1xxxxxxx - for 4...131
01xxxxxx - for 132..195
00xxxxxx xxxxxxxx - for up to 16383

If you care about decomp speed, you might want to put codes at the end instead of the beginning. For example, 1xxxxxxx means this is necessary:

Code: Select all

cmp al,10000000b
je handle_1
...
handle_1:
and al,01111111b
add al,4


But if you change your codes to be at the end, you can do this:

Code: Select all

shr al,1
jc handle_1
...
handle_1:
add al,4


This is both smaller and faster on 8086. It shifts the bits into carry and jumps based on carry bit, and when you're finished you don't need to adjust the value as it is already in the correct format.

EDIT: I just thought of another optimization: Don't use FF for the code; instead, use the value that shows up the least. Scan the entire file before compressing to determine which value shows up the least, then use that as the code. Otherwise, data that has a lot of FFs in it (like most 8-bit programming!) won't compress very well...

[Brutman]

I'm not going to win this competition anytime soon ... my general LZW algorithm took 147456 bytes of input and produced 132797 bytes of output. Which is reasonable considering the binaries look like random data, but not great when simple heuristics that take advantage of the structure of binaries can beat it.

My output codes are fixed at 12 bits long, so I have to overcome a 50% penalty right from the start. If I had the variable bit rate part done I would expect most of the output codes to be 9 bits for this file size, thus taking another 25% off of my output size.