Audio via COM port (for Spectrum Lab)


Contents


1. Introduction

To connect external A/D conversion hardware without the need to write an extra 'driver' for the PC, Spectrum Lab supports reading 'audio via the serial port' (aka 'COM port') in various formats.

Most of the formats are detected automatically because the microcontroller (which drives the A/D converter) embeds 'info headers' in the serial data stream, which SL uses to detect the number of channels, bits per sample, sampling rate, and possibly the precise (GPS-based) timestamp of the digitized samples.

For developers: The default streaming format is simple, actually it can be the same as for streaming non-compressed audio over a raw TCP/IP connection. The audio header structures are specified here (as "C" sourcecode, compatible with Paul Nicholson's VLF Tools). If another -even simpler- sample format shall be used, the format must be specified in the 'params' field. More on that in the next chapter...


2. Configuring audio via COM port

Open SL's Audio Settings tab. On the panel titled 'Input Device / Pipe / Driver', expand the list of input audio devices. It usually contains several normal audio devices (soundcards, USB audio devices, possibly ASIO), a few specialized drivers for SDR, and -if such interfaces exist on your system- a few COM ports. Under Windows, COM ports are listed by their numbers only, e.g. 'COM1', 'COM2' (which used to be the onboard RS-232 interface in the good old days), 'COM3 (which is typically used for the first USB-to-RS232-adapter enumerated by Windows, etc etc.


Audio-via-COM, step 1 : select the COM port in SL's "Input Device" combo


A COM port may be shown as 'occupied' in the combo box, which is not necessarily an error:
The port may already be occupied by Spectrum Lab itself (when already selected, and audio processing started.
Otherwise (e.g. "COM8 (occupied)") the serial port may be already opened by another application.

(the "occupied" indicator described above was removed, because some crappy Bluetooth driver
seriously slowed down the COM port enumeration process, thus SL now cannot check in advance if
a COM port is occupied anymore)
Because COM ports can only be used exclusively, you will need to find the application which has occupied that port and close it, to make the port available for Spectrum Lab.

Besides the COM port number, SL needs to know much more about the format of the digitized samples entering the COM port. That additional information must be provided in the 'params' field right below the list of 'Input Devices / Streams / Drivers'. This doesn't have anything to do with the Radio Control Port settings on SL's TRX Interface settings !


Audio-via-COM, step 2 : describe serial port settings in the "params" field

The first part of the string entered in the 'params' field describes the serial data format, using the conventional notation like "115200,8-N-1" for "115200 bits per second, 8 data bits, no parity, one stop bit".
Flow control is not supported, and not required on today's computers (which are always fast enough to receive a few hundred kilobits per second without a receive buffer overflow).
The following parts in the 'params' field are optional. When not specified, as already mentioned, SL expects a stream of samples compatible with Paul Nicholson's VLF Tools (see header structure in a separate document). For anything else (e.g. simple PIC microcontroller sending samples), you must specify the format of the audio samples as shown in the table below.
format-tokenexplanation
U8each sample is an 8-bit unsigned integer (0..254; value 255 should be reserved as sync pattern)
S8each sample is an 8-bit signed integer (+/-127; -128 is reserved as sync pattern)
U16each sample is a 16-bit unsigned integer (0..65535), little endian (least significant byte first)
S16each sample is a 16-bit signed integer, little endian (+/-32767; -32768 is reserved as sync pattern)
U16_BEeach sample is a 16-bit unsigned integer (0..65535), big endian (most significant byte first)
S16_BEeach sample is a 16-bit signed integer (+/-32767), big endian
U24each sample is a 24-bit unsigned integer (0..16777215), little endian
S24each sample is a 24-bit signed integer (+/-8388607), little endian
U24_BEeach sample is a 24-bit unsigned integer (0..16777215), big endian
S24_BEeach sample is a 24-bit signed integer (+/-8388607), big endian
U32each sample is a 32-bit unsigned integer (0..4294967295), little endian
S32each sample is a 32-bit signed integer (+/-2147483647), little endian
U32_BEeach sample is a 32-bit unsigned integer (0..4294967295), big endian
S32_BEeach sample is a 32-bit signed integer (+/-2147483647), big endian
IQ12DL4YHF's 12-bit I+Q sample format for PIC12F675, described below
GPSDO33- or 5-byte frames from DL4YHF's PIC-based GPSDO, Version 1

If the number of channels (in each sample point) isn't implicitly defined by the sample format (for example, "IQ12" implies that each sample point consists of two values digitized at the same time: One for 'I', one for 'Q'), the number of channels must be appended to the format specifier, again separated by another comma. See second example at the end of this chapter.



3. Frame synchronisation (for external ADC connected via serial port)

Since the microcontroller usually starts transmitting at an arbitrary time, and because single bytes can always get lost 'on the wire' (especially if the 'wire' is a simple UHF ASK/FSK radio link), the receiver needs some sort of frame synchronisation (unless the stream consists of a single channel, with a single byte per sample). This can be achieved by reserving a certain bit pattern which does not occurr in the 'normal' digitized data. For signed integer values, a suitable bit pattern is the most negative number N (which the sender can easily replace by N+1, in the unlikely case that 'N' occurs in a digitized sample). The only situation where no sync pattern is required is in streams with only 8-bit samples (format U8 or S8), and one channel.

For 16-bit signed integers,
the default sync pattern is 0x8000 (hex). The sender (microcontroller) must replace any sample value 0x8000 (= -32768 decimal) with the next negative number, which is 0x8001 (= -32767 decimal). A similar pattern is also used to mark any 'header' in a non-compressed audio stream via TCP/IP.

For 16-bit unsigned integers,
the default sync pattern is 0xFFFF (hex'). The sender (microcontroller) must replace any sample value of 0xFFFF ('largest possible 16-bit value) by 0xFFFE (= 65534 decimal), which doesn't hurt much because the ADC's clipping point must be avoided (with some 'headroom') anyway.

The sender only needs to insert the 'sync pattern' occasionally (once every few hundred milliseconds). If it starts sending before launching Spectrum Lab, SL may receive garbage for a few hundred milliseconds because it doesn't know to which part of the frame the first received byte belongs. After recognizing the first sync pattern, the received samples will be properly reassembled, and the 'garbage' (usually a strong noise) disappears.

If the sender uses frame synchronisation (as explained above), append the string ',SYNC' (without quotes) after the format-token in SL's 'Params'-field as shown in the previous chapter. SL will then discard all received bytes until the first successfull reception of the complete sync pattern, and automatically remove the sync pattern from the input samples.


4. Configuration examples (for external ADC connected via serial port)

With DL4YHF's PIC based A/D converter for the serial port, each sample point (with two channels) consists of four bytes. The first byte in each sample point (aka 'frame' in the PIC firmware description) is the header byte. It usually contains 0xFF (255 decimal) which looks like a long 'stop bit' to the receiving UART, and is used for synchronisation purposes. The second byte in each frame contains the eight least significant digits of the 'I'-channel. The third byte contains the eight least significant bits of the 'Q'-channel. The fourth byte in each frame contains the four most significant digits of the 'I'-channel in bits 3..0, and the four most significant bits of the 'Q'-channel in bits 7..4 .
The PIC sends those four-byte-frames with 115200 bits per second, 8 bits per UART-frame, no parity, one stop bit. Thus, the complete format description entered in the 'params' field would be:

  params:   115200,8-N-1,IQ12
          (with baudrate,serial data format,sample frame format)

In this special case (PIC-based ADC with serial interface), the ADC doesn't report its own sampling rate, so the 'Nominal Sample Rate' field in SL's Audio Settings must be set to that value.
Otherwise (with a wrong sampling rate) the frequencies displayed on SL's main frequency scale would be wrong.
For example, the PIC-based serial ADC mentioned above delivers 2500 I/Q samples per second, thus the displayable baseband frequency range is -1250 Hz ... +1250 Hz.
This may be different for other 'external' A/D-converters on the serial port !

Example to receive a serial stream with 9600 baud, 16 bits per sample (signed integer), single channel, sync pattern 0x8000:

  params:   9600,8-N-1,S16,SYNC,1
          (with baudrate,serial data format,sample frame format with 'sync' option, number of channels in the stream)


5. Testing audio-via-COM-port with a Python script

To test interfacing audio-via-COM-port without a microcontroller, you can emulate two COM ports on a single machine using com0com (a 'Null-modem emulator that can create a simple pair of COM ports on your PC). In June 2018, the author used com0com V3.0.0.0, available from sourceforge with a signed driver. The name of the downloaded file was "com0com-3.0.0.0-i386-and-x64-signed.zip", and with a bit of luck you can find it somewhere (beware, there's a confusing number of versions floating around, some without a signed driver which caused a lot of headache. So make sure you pick the right one).

Next, pick a serial port library for your favourite Python flavour. For the simple example presented below, I used pySerial. Finding an easily downloadable installer on pypi.org/project/pyserial was difficult (there was only a python 'wheel' package from 2017-07-23).
In this case, the downloaded whl-thing (wheel package) had a funny long name, so the command to install pySerial from it was:

c:\tools\pyserial>pip install pyserial-3.4-py2.py3-none-any.whl
 →  Processing .\pyserial-3.4-py2.py3-none-any.whl
    Installing collected packages: pyserial
    Successfully installed pyserial-3.4
(If the 'pip' command isn't recognized, install pip and try again. However, most of today's Python installer should have it.)

Processing binary data isn't one of the strongest points of Python. There was the 'bytearray', and the 'struct' module. Both could be used to assemble a block of samples, but much less elegant than simply using the C structs listed in Spectrum Lab's documentation about non-compressed web streams. Anyway, the 'struct' module in Python appeared overly complex, a 'bytearray' would require assembling each 16-bit value from two bytes, NumPy would be overkill for this purpose, so let's try an 'array' (imported module) to send samples from a Python script to Spectrum Lab via COM. The script itself is located in SL's Goodies/Audio_via_COM_from_Python folder.

Last modified : 2018-06-16