A Serial Chart Recorder
for
Precision Frequency Applications

By Murray Greenman ZL1BPU

A program which plots data from a serial data stream,
specifically telemetry from a micro-based frequency standard.


M. Greenman ZL1BPU 15 June 2005

DESCRIPTION

The venerable Rustrak recorder RECORD2F is a PC program which simulates the functions of an old-fashioned chart recorder, specifically the "rustrak™" type recorder, which samples analog data periodically by tapping a meter needle on a chart. For improved display, in this program the chart runs sideways instead of down, and the input data is a serial data stream. Just for nostalic reasons, RECORD2F still goes "clunk" whenever it starts a new sample!

However, the program is rather more sophisticated than a simple analog chart recorder, as it is very flexible in what it will plot, and also has useful statistical capabilities. The setup is stored in a file - and that includes what data to plot, the chart title, and what colours to plot. Some parameters such as chart speed are also adjustable while the program is running. It records two chart channels and also has two independent meter displays. Finally it calculates the on-going OFFSET and P-P VARIANCE of the oscillator it is monitoring.

RECORD2F uses an ASCII data stream, and telemetry in HEX-ASCII form (that is, each byte represented by a pair of ASCII characters 0 - 9 and A - F). All that it needs is for the same data to always be in the same place in the telemetry sentence. A more general-purpose version (without OFFSET and VARIANCE calculations) is also available.

The program is a DOS executable, but will run full-screen on most Windows computers from WINDOWS™ 3.1 through to the latest WINXP and WIN2000. It requires a serial port that is supported by the operating system as a true COM port. (It probably won't work with USB serial adaptors). It may also run in a DOS window on some newer computers, although the graphics may be slightly distorted due to screen support limitations. Some display controllers do not support the dual scan modes required for "DOS BOX" graphics operation. The program requires little overhead, and will operate on a 486/25 DOS PC, will operate from a DOS floppy, but does require a VGA display with at least 640 x 480 resolution (mono LCD works fine as well, provided suitable colours are chosen).

PURPOSE

The purpose of the program is to provide both real time and long term performance display and analysis for ZL1BPU frequency standards, precision transmitters, clocks and frequency comparators. Devices of this type operate from a master reference which is measured and controlled, and this reference also clocks the micro controlling it.

The devices typically transmit a telemetry frame every second, containing time, master oscillator PHASE sampled at the edge of an external GPS or TV sync frequency reference, and the FEEDBACK voltage used to control the master oscillator (the input and output of the control algorithm). The RECORD2F program shows these PHASE and FEEDBACK terms from the frequency controller, and calculates from the PHASE the OFFSET and P-P VARIANCE of the oscillator.

Just to make things very clear, the frequency control algorithm used by these devices resides in the device, and operates in an autonomous manner. All the RECORD2F program does is monitor the performance of the master oscillator and its control system.

SETUP

The setup file RECORD.SET is used to tell the program some specific parameters to use. This is best done before the program is run, although the program can call DOS EDIT to make any changes to the current setup from the program. You need to set these parameters in the file using a text editor. Open RECORD.SET with a text editor (DOS EDIT, Windows NOTEPAD etc), and change the first eight lines to suit your purposes. Do not add any lines or change the order of the data.

Here is an example:


1 600 13,2,"PHASE",1 17,5,"FBACK",1 11,4,"PHASE",1 16,3,"FBACK",1 "ZL1BPU GPSCLOCK" 10.000
After these eight lines may be some lines of text which describe what the settings do. These are ignored by the program. The information in the eight lines shown is as follows:
  1. COM port to use (a number 1 - 4)
  2. Default chart speed, in seconds per sample (a number 1 - 65000)
  3. Chart channel 1 (PHASE) data pointer, data name, on/off status
  4. Chart channel 2 (FBACK) data pointer, data name, on/off status
  5. Meter channel 3 (PHASE) data pointer, data name, on/off status
  6. Meter channel 4 (FBACK) data pointer, data name, on/off status
  7. Chart Title (text to display at top of chart)
  8. Operating frequency of reference in MHz (used for OFFSET calculations)
The COM port setting is obvious. You need to check first that the COM port is working with the serial data source, and what the port number is. This is easily done using a Terminal program. The RECORD2F software always works at 9600bps (9600-N-8-1).

The chart speed can be any convenient value. By setting a default, you have the opportunity to use recording rates that are not provided by the program function keys. The program offers 1, 2, 5, 10 and 60 seconds per sample. Note that all data is recorded on the screen (i.e. every sample, typically received at 1Hz rate), but the chart only advances at the set speed. Thus at slower rates, there are multiple dots recorded for each time sample step. The fastest sample rate (chart speed) is one sample per second.

The data pointers are explained later. The data name can be any text up to six characters long, and must be in quotes. All values on the lines 3 - 6 must be comma delimited. The frequency is the nominal carrier frequency of the master oscillator in MHz, and is used for relative offset and variance calculations. (This is the frequency at which the micro operates, and may or may not be the actual crystal frequency in your design).

DATA FORMAT

The way the software works makes the program fairly universal. The graphing routine is 8-bit, i.e. the graph has a range of 256 pixels. In the setup file are numbers which tell the graphing routine which text characters in the telemetry data to use for each graph channel, and what their names are. So long as the data supplied is in a constant format, it can plot anything. There is one major difference in this version however - the data used to calculate OFFSET and VARIANCE is 16-bit, and is sampled from a fixed location in the data stream (starting at character 11).

RECORD2F expects data in small text frames, up to 30 or so characters long, and terminated by CR and LF. Typically, there would be one frame per second, although this is not limited by the software, which will handle both slower and faster (to a maximum of about 10 frames/sec, depending on the speed of the computer). Each frame results in data recorded on the graph, so:

Time will tick by without data, and the samples will simply be recorded as straight lines (at the last received value). The graph moves sideways, in order to maximize the visible chart length. Time and the sample recording point move from left to right, and when the screen is filled, the chart moves slightly right to left, allowing the chart to continue. When the graph slips off the left end, the oldest data samples are lost.

The data frame format is not fixed, but must:

In this version of the application the PHASE is expected to be 16-bit and to start at character 11. Frames without CR and LF will be rejected. Pointers set beyond the data will result in no data plotted. If the number at the end of a channel definition line in the setup file is "1", the channel will be plotted or metered. If anything else, it will not be plotted or metered. Here is a typical data frame format:
hh:mm:ss %PPPPFFFFxxxxxxxx<CR><LF>
^
where the UTC time is sent first, followed by a space and a device identifier (shown as "%" above). Then follows the PHASE (PPPP) and the FEEDBACK (FFFF), both 16-bit values (i.e. between 0 and 65535 or 0x0000 and 0xFFFF in hex notation). The following data indicated as "xxxxxxxx" depends on the device, and could represent anything. Here we just concentrate on the PHASE and FEEDBACK, and also only plot the lowest 8 bits of each. That's like dividing each value by 256 and only plotting the remainder, which gives best sensitivity. RECORD2F is a dual channel plotter, and uses different colours to indicate the two channels. It also has two colour coded meters. All these colours are user programmable.

This same format is used by all the ZL1BPU Frequency References, Frequency Comparator, SuperClock and GPSClock. The values sent at the start of the frame "hh:mm:ss" is of course the UTC time (if set). Time is represented in packed BCD, (two numeric digits 0-9, representing "00" to "99") so is both "human readable" and RECORD2F readable. There may be further data shown as "xxxxxxxx" which should be ignored by RECORD2F. The "^" marks the first character in the frame.

DISPLAY DETAILS

RECORD is designed to graph 8-bit data, but can do so with 12-bit, 16-bit (or more) data as well. The data is sent as HEX-ASCII. In this "human readable" format, alphanumeric characters are used, where 0-9 and A-F represent hexadecimal values, so "0" to "F" means 0 - 15 decimal, "00" to "FF" can represent 0 - 255 decimal, and "0000" to "FFFF" can represent 0 - 65535 decimal, and so on.

As each frame of data is received, it is displayed verbatum in the bottom right corner of the screen. It is then interpreted, using the pointers for channel numbers 1 to 4 (chart and meters) to decide which data to display. The selected data for channel 1 and channel 2 (PHASE and FBACK) are then reprinted (highlighted), using the colour chosen for that channel. In the example below, you'll see "09:33:29 %0CC508E9A011".

screen shot
The RECORDER display
Click on image for a full-screen view

The chosen data is converted to binary (representing 0 - 255 decimal), and plotted on the chart. The vertical position represents the binary value (PHASE in Hz, FBACK in ~1mV steps), and the horizontal position the current sample point, which moves to the right at the sample rate.

The main chart is 256 dots high, so there are 256 vertical locations to plot in, which is why the lowest 8 bits are usually plotted (28 = 256). The chart is about 600 dots long, so will record 600 sample events, as the dot position changes with each sample (each time the program emits a "clunk"). Furthermore, when the plotting point gets to the right-hand side, the whole plot slides along to the left by 10 positions, and plotting continues. You lose the oldest 10 plotted points each time. A new sample every minute means about 10 hours of chart space.

The vertical scale marks shown on the left are from 0 to 255, marked every 10, and labelled every 50. The units depend on the parameter you are plotting. For example, with a 10MHz reference, the resolution is 100ns (1/f), so each dot, which represents one cycle, also represents 100ns of change of PHASE. The horizontal scale is time, marked as the chart progresses, in 10 seconds, 100 seconds and 1000 second marks of different heights. When the plot is very slow, the faster marks merge to become a grey band.

Below the main chart is a small "trend graph" which plots each sample of the data at full speed, a new horizontal position for every data sentence. This plot is only 64 dots high, and the data wraps around when it exceeds 63 or goes below zero. In other words, only the lowest six bits of the data are plotted. The trend graph is 256 samples long, and simply wraps around to the beginning - it does not scroll along like the main chart. However, there is a single dot marker under the graph so you can see where it is plotting.

There are two major advantages to the technique of plotting the least significant eight bits of data. First, the data can never overflow the graph, since it only ever represents 0 - 255, and the graph width is 0 - 255. Second, by choosing the pointer value wisely, data can be represented with "gain". For example, imagine the data "PPPP" in the above example is represented as "AAaa" to indicate the weight of the data. If the setup has a pointer value of 11, the values interpreted will be the most significant bits ("AA"). If the pointer is 12, the data bits interpreted are the next two ("Aa"), so the graph ignores the most significant data and plots with a gain of 16! If the pointer was set to 13, only the least significant bits ("aa") would be plotted, with a gain of 256. As the plotted data rolls off the top of the graph, it rolls back onto the bottom of the graph, with nothing lost.

Since the recorder has two channels, it is possible to plot a single channel at two gain settings - for example, setting CH1=11 and CH2=13 would plot "AA" and "aa" in the above example. The vertical scale to the left of the graph is marked in arbitrary 0 - 500 units, convenient for data representing 0-5V, typical of microcontroller A-D converters.

If you look closely at the screen example, you'll see that the recorded data is sometimes a thick line, sometimes thin. This is because sometimes in each sample period there are several values recorded. This might indicate that the data should be recorded faster, or it might indicate noise on the data.

STATISTICS

Under the main chart on the right the chart sample interval and elapsed time since last reset are displayed. Below these are three sets of statistical information, unique to the RECORD2F version of the program. These are also colour coded, although the colours are fixed, as is their data source. The first line, "PHASE", shows the reset phase (16 bit), the current phase (16 bit) and the phase difference since reset. (At 10MHz this difference can be read as 100ns units). This text is coloured the same as the PHASE plot.

The second line shows the OFFSET, i.e. the frequency error, which is calculated by dividing the 16 bit phase difference since reset by the time elapsed and the operating frequency:

ERROR = (CURRENT PHASE - STARTING PHASE) (ELAPSED TIME x REFERENCE FREQUENCY)
The value is a very small number, a unit-less figure of merit (i.e. Hz per Hz), and is the usual way reference accuracy is quoted. The value can be positive or negative, indicating whether the reference is high or low of the true frequency. Depending on the performance of your reference, you may see figures as small as 1e-12 or more (1 part in 1012). In the ZL1BPU GPS disciplined references, this is an indication of the ageing rate of the reference crystal. The best reference crystals age at perhaps 1e-10 per day, and the GPS or TV referenced control system must compensate for this - in fact this is one of its main purposes.

A very small error occurs because of the lag in compensating for such slow changes in the reference. The performance which results has a small steady offset (rather than an ever increasing offset with age if uncompensated), and the size of offset is reduced by about two orders. If there was no ageing, the error in a GPS disciplined reference would be vanishingly small.

The OFFSET is calculated in real time, and will generally continue to improve as plotting time progresses. After a few hours recording it will stabilize. A quick indication of the offset can be deduced in a matter of a few minutes. OFFSET text is coloured pale blue. Note - if you start the recording while the oscillator is still stabilizing, the statistics won't look too good, remembering that the start point for offset calculations is the initial PHASE measurement. Wait until the oscillator has been stabilized and start again.

When used with non-locked references (for example while assessing a free-running oscillator or calibrating a Rubidium Standard), OFFSET is an accurate indication of the frequency error. See this example of a free-running 5MHz oscillator. The green phase trace is constantly moving upwards, but both the OFFSET and VARIANCE are constant. The calculations are made using the full 16-bit data, so will not be upset when the PHASE trace wraps around the screen. RECORD2F isn't much use unless the oscillator is able to achieve 1 part in 10-7 or better. Note - Be aware that if the PHASE rolls over from zero to maximum or vice-versa, the calculation will be invalid. You should reset the plot and start again. If this becomes a problem, perhaps your oscillator isn't very good!

The final figure displayed is the peak-to-peak VARIANCE. This is a measurement of the stability of the reference, and is achieved by remembering the highest and lowest 16-bit PHASE measurements since reset. The difference between these represents the p-p error. The value displayed is a unit-less figure of merit (i.e. Hz per Hz), calculated in the same way as the OFFSET. In order to isolate VARIANCE from OFFSET (because of course a change in phase of any type would move the minima and maxima apart), the maximum and minimum error values are repetitively calculated as a difference from the mean of the last 100 16-bit PHASE samples. The p-p error is the difference between the maximum and minimum. Note - The integration period of the mean value is affected by the chart speed, not the data frame rate. At one sample per second, the average is over 100 seconds; at one sample per hour, it will obviously be over 3600 seconds.

VARIANCE is scalar (there is no sign to the VARIANCE value, since the maximum is always bigger than the minimum!). The program cannot calculate the Allan Variance, since this requires all the data to be stored for post-processing to determine the standard deviation, and can't be plotted in real time. (The Allen Variance would typically be about 1/3 of the p-p value shown). Since this P-P VARIANCE information is available in real time, it is a handy measurement. Most good references show numerically similar VARIANCE and OFFSET. If one or the other is markedly worse, look at that aspect of the oscillator design. VARIANCE typically shows a valley-shaped curve with a minimum around the 10 - 100 second time interval. RECORD2F cannot give variance figures over short periods. This text is coloured pink.

Both the OFFSET and VARIANCE are plotted on the main chart, using the same colours as their text reports, pale blue and pink respectively. They are plotted in such a way that the more important values (PHASE first, FBACK second) overwrite them on the screen. The 100 sample mean is also plotted - as a dotted line in the same colour as the main PHASE plot. (You can see it clearly in the first example below). If the reference is really stable, you may never see this plot as it will be under the PHASE plot.

OTHER STUFF

A legend showing the colours of the plots and meters is also indicated on the screen under the plot. Under the trend graph is a pair of "meters" which can be set to record the same data or different data to the main chart. It is handy to set these up to record the most significant 8 bits of each data channel, so you can get a good idea of how far through the range the data is. The meters have two pointers - one is set to the data value at reset (coloured black) and the other (in the assigned channel colour) follows the data. The meters are cleared and redrawn at each sample, but if the data rate is greater than the sample rate, the meter pointers will "move" and "blur" in a most realistic way, as the pointers are drawn after every telemetry frame. When the graph is reset or the sample rate changed, the current data value is saved, and subsequently displayed on the meters as a second (black) pointer.

OPERATION

The program is operated by typing its name (RECORD2F) at the DOS prompt, or double-clicking the executable in Windows Explorer or My Computer. It needs the RECORD.SET setup file to reside in the same folder as the executable.

At the very bottom of the screen on the left is a legend showing what the Function Keys do. Pressing one of the keys F1 to F5 will change the sample rate to one of the built-in settings. At the same time, a vertical blue line will be shown on the chart, and the statistics will be reset. The next sample at the new setting will not occur until the time has elapsed for the next sample at the OLD sample rate. If you have a very long default time set, it will be quicker to edit the setup and start again.

Pressing F6 will cause a reset without changing the current sample rate. With F7 you can add a text comment to the chart. To save the chart to the clip-board, press ALT and PRINT SCREEN at the same time. Then change to a graphics program (Paint, Paintshop Pro, etc) and paste the picture from the clip-board. You can then clip out the chart and save it. This is how the example screen-shots were made.

F10 calls up the DOS editor to make changes to the setup file. Once the changes are made and saved, the program restarts. It appears that DOS EDIT changes the VGA colour pallette and does not restore it, so you may find that on restarting the colours are strange. If this happens, quite and start again.

Oh yes - to exit the program at any time, just press ESC!

Some examples of RECORD2F plots -

Except as noted, all references were locked to a modest Rockwell Jupiter GPS module using 1pps (100ns). On a Navman Jupiter-T GPS (specialized timing version, 25ns), the "VNG-in-a-Box" achieves 10-12 at 104sec p-p Variance and almost 10-13 at 105sec.

© Murray Greenman ZL1BPU 2002-2005