GPS project

[Brutman]

Ok, good news and bad news ...

First, the good news: I have an SNTP server that is using a serially attached GPS as the time source. That makes it a stratum-1 time server. The timestamps that it is serving are within 1ms of time.google.com or time.nist.gov, and there is little jitter or variability.

Now the bad news: It's running on a 40Mhz 80386 based system.

The code runs on a Jr just fine, but the machine is not fast enough to get to 1 or 2 ms of timing resolution and serve requests over TCP/IP. The time offset on the Jr is around 13ms, which is a lot of time. However, the machine takes around 6ms just to respond to a ping packet so I should have realized that getting to 1 or 2 ms of timing resolution with an SNTP server was not going to be possible. (This is using a real Ethernet card too, so I can't blame my usual Xircom parallel-port attached adapter.)

My new, more realistic goal for the Jr is to serve timestamps that are within 10ms of the real thing. To do this I'm going to:

• continue to speedup the 8253 interrupt, but instead of speeding it up 64x I can get by with just 16x speedup. This will reduce the system load a little.
• Scrub out all 32 bit multiplications or divisions - they require a call to the C runtime which is probably very expensive.
• Try to convert as many of the 16 bit multiplications and divides into shift operations, or partial shift operations.
• Run some micro-benchmarks to find out where the time is being spent.

And once again, this is not a practical project. ;-0

[Trixter]

Another thought: This is meant to turn the PCjr into a time server, which is an embedded systems device, so you may want to consider completely screwing over the machine: Disable all interrupts other than the ones you need via the PIC, maybe even consider disabling the NMI (although you'll lose keyboard if you do that), don't bother calling INT 08h to maintain DOS time, etc. Stop the PCjr and DOS from stealing time away from your time server.

Might also be worth optimizing and testing on a 5150, just so you can rule out PCjr weirdness. Once you are satisfied on 5150, you can then test on PCjr and see how much worse it is (if at all) which can guide you on what to hobble on the PCjr as per my previous paragraph.

[Brutman]

I'm willing to suffer for my art, but not like that ...

Another reference point: I'm testing a PS/2 Model 25 right now. Specs are an 8086 CPU running at 8Mhz, which has a 16 bit path to memory. Like the Jr it's results are slower and variable; it's offset is between 5 and 11ms from the good time servers. The time to respond to a ping is around 3.3ms. This machine is close enough were some light optimization work can get it under 10ms consistently. But it's also about 2x faster than a PCjr by virtue of the clock rate, the bigger prefetch buffer on the 8086, and the path to memory.

[Brutman]

I take back everything I said about the PCjr.

The short story: I made a call to the C runtime at a bad time, and wow, it was expensive. I need to look at the Open Watcom code and do some benchmarks, but the call to time( ) takes like 19ms on a PCjr!

The call was made on the send response path, but I had already set the timestamp in the response packet. (The additional time() call was not for the NTP protocol, but to update the ARP cache.) The extra 19ms looked like network delay to NTP, and because it only happened on the response packet it also threw off the time offset calculation because that expects the network paths to be symmetrical.

I'll scrub out any usage of time() calls now that I know they are evil on a PCjr. The corrected code now shows no more than 2.5ms of difference between a public time server on the internet and the PCjr with the GPS, and the network round trip time matches the ping time for the machine. (Ping time is around 5.5ms.)

[Trixter]

I'm surprised you were making any C runtime calls at all, to be honest. This is an "embedded device" function; you don't want to call any code you didn't write.

Glad you found it though!

[Brutman]

It's not a demo Jim. ;-0 There are like 4600 lines of code in the files that directly contribute to the executable (which is still under 50KB). Sending and receiving packets, memory management for buffers, ARP, IP, UDP, and SNTP specifics are all there. The C runtime is needed for time conversion functions, 32 bit math, etc. Nobody wants to rewrite all of that.

To your point though, I am careful about not calling unpredictable things on critical paths. The C runtime isn't all bad. The string functions, memcpy, and a lot of the routines are safe to call because their runtime is predictable and bounded, assuming you know the inputs. On critical paths I don't call anything that I can't estimate ahead of time. Another big reason not to use the C runtime is side effects. For example, I avoid allocating and deallocating memory because I don't want to fragment the DOS heap.

My failure here was my estimate for how much the time() call takes. I was expecting a few hundred cycles, which is acceptable on this project. It seems to be taking much more than that, which I have not been able to figure out yet. I need to do a benchmark to confirm that the time() call really is that heavy, and then I'll start picking it apart to see where things went wrong. The code in question was updating the ARP cache to ensure there was no ARP miss when sending out the UDP packet for the response. Spending a few extra cycles up front to avoid an ARP query was definitely worth the investment. In the interim I've moved the time() call to a better place where it doesn't affect the calculation. A few hundred cycles of slop I was willing to accept, but not milliseconds worth.

mTCP is not exactly standards compliant in a lot of places, and for ARP I record the time of each cache entry update but I don't actually age things out of the ARP cache. I use the time for cache eviction, so I'll change to use a monotonically increasing counter that will let me detect the oldest entry without knowing the precise time.


Mike

[Brutman]

Ok, picking part the Open Watcom 1.9 C runtime ...

A call to time( ) breaks down to:

• Two calls to "TinyGetDate"
• One call to TinyGetTime
• One call to mktime

Running each of these in a loop 1000 times results in:

• time(): 269 BIOS ticks (14.7ms each call)
• mktime(): 106 BIOS ticks (5.8ms each call)
• DOS Int 21, function 2A (get date): 51 BIOS ticks (2.8ms each call)
• DOS Int 21, function 2C (get time): 51 BIOS ticks (2.8ms each call)

So two calls to TinyGetDate, one to TinyGetTime, and one to mktime add up to 14.2ms. There is some additional code in the path, but its only probably a few hundred more cycles so it's nothing to get alarmed about.

Overall though, that's a lot of code to run. 14.2ms is close to 68,000 clock cycles. Even for just getting the current time (hour, minute and seconds) it takes ~13,500 clock cycles, which seems like a lot.


Mike

[Trixter]

It's not a demo Jim. ;-0

I know, but it is an SNTP server. I would have thought any time-related functionality would be your own code.

[Brutman]

Like I said - it was for the ARP cache, and it predates anything I was doing for NTP. All of the NTP specific timestamp generation is my code. I have to use some of the C runtime, but that's not on the critical path.

Any ideas on why the DOS 21 interrupts for getting the date or the time are so slow? Is DOS doing some unrelated housekeeping each time Int 21 is invoked?

[Trixter]

DOS does housekeeping during practically every function ;)

DOS was written for size, not speed. So a quick look at https://github.com/microsoft/MS-DOS/blo ... e/TIME.ASM shows that there is is a monolithic DATE16 that grabs both date and time. That calls READTIME, which calls SETREAD, which looking at yet more source appears to be reading from a device driver for the time because it invokes a DEVIOCALL2. But I got lost in the weeds trying to find the source of the time device driver, what reads/writes TIMEBUF, and other stuff so I stopped. But needless to say, it's not a simple read-and-calc from 0040:046c.

I believe the TIME device driver is something supplied by the OEM, using the skeleton sources provided in the OEM distribution disks (ie. the OEM Adaptation Kit). More info here: https://www.os2museum.com/wp/dos-2-11-from-scratch/
But I couldn't find more after 30 minutes so I gave up.