Sending and receiving webstreams with Spectrum Lab

This chapter describes how to send (transmit) or analyse (receive) web audio streams using Spectrum Lab.

Since 2011, this is possible without the AudioIO-plugin (DLL), because native support for Ogg / Vorbis is integrated in Spectrum Lab.

Analysing web streams
1. Notes on the URL format; Examples
Sending streams to the web
1. Configuration of the output stream
2. Unattended operation
Logging web streams (compressed audio data) as disk files
VLF 'Natural Radio' audio streams
1. Links to VLF Natural Radio audio streams
2. Setting up a timestamped VLF 'Natural Radio' audio stream
Operation as audio stream server
Internals

1. Analysing web streams

To open an internet audio stream for playback and analysis, select 'File' (in the main menu), Audio Files & Stream, Analyse and Play Stream / URL .

An input box opens with the previously visited stream URLs. Pick one from the list, or enter the complete URL ^(*) of the audio stream in the edit field.

To store the URL for the next session, set the checkmark 'save URL history'. If you are concerned about privacy, leave it unchecked.

Next, click "Ok". The program will try to establish a connection with the Internet (or, depending on the URL, with an audio stream server in your LAN).
This may take a second or two.

1.2 Notes on the URL format

The URL must contain the complete address of the audio stream resource.

^(*) Note on the URL:: In some cases, SL is able to resolve the stream URL if the given URL is not an Ogg resource, but an M3U redirector (aka "playlist").
But some websites respond with an error when the HTTP 'GET' request doesn't originate from a webbrowser, or doesn't carry a bunch of red tape (which SL doesn't support), or doesn't use HTTP Version 1.1 (SL only uses HTTP/1.0).

Examples :: rawtcp://67.207.139.49:4401 ( "VLF1" from Todmorden, on Paul Nicholson's Live VLF Natural Radio server; with GPS-based timestamps)
rawtcp://67.207.139.49:4406 ( "VLF6" from Spenge near Bielefeld, raw stream with GPS-based timestamps)
rawtcp://67.207.139.49:4416 ( stereo combination of VLF1 from Todmorden (L) and VLF6 from Spenge (R), with timestamps)
rawtcp://67.207.139.49:4427 ( stereo combination of VLF1 from Todmorden (L) and VLF6 from Marlton (R), with timestamps)
http://dradio.ic.llnwd.net/stream/dradio_dkultur_m_a.ogg ( Deutschlandradio Kultur from Germany )

If it doesn't work, copy the URL into the address bar of your favourite web browser (if the web browser isn't able to play the audio, chances are low that this program will be able to play it). See next chapter for details about the supported audio stream formats, and the supported network protocols.

Often, the 'real' stream resource is hidden by an awful lot of Javascript or other stuff. In that case, playing around with the web browser can often reveal the 'real' stream URL. Spectrum Lab makes no attempt to execute Javascript, only to find out the real URL. In fact, it cannot execute browser scripts.

2. Sending audio streams to a remote server

This chapter describes how Spectrum Lab can be configured to send one audio stream to a remote server. In this case, SL acts as client, i.e. initiates the connection, and then starts sending live audio to the remote server. This function was first used to feed live data from VLF receivers to the Live VLF Natural Radio server (with high-speed internet connection) operated by Paul Nicholson.
In contrast to the server, the streaming client as described below only requires a moderate DSL connection. For example, an monophone Ogg/Vorbis stream with 32000 samples/second, and Vorbis 'Quality' set between 0.4 and 0.5 required about 64 ... 75 kBit/second.

In SL's file menu, select Audio Stream Output. This opens a control panel for the audio stream output.

The control on this control panel are:

Configuration file for the outbound stream: That file contains all the details which the program must know to send the stream. It's a plain text file which you have to write yourself, and store in a safe place.
(you will need some info about the remote server for this - see next chapter)
The stream configuration file is not a part of the Spectrum Lab configuration (only the name of the file, but not the file itself). This eliminates the risk of passing passwords and other 'private' details along with a normal configuration file.
Automatically reconnect, Start output stream when Spectrum Lab starts:: Used for 'unattended operation'.
Send Channels:: Defines which of the input audio channels you want to send (left / right audio channel, or both), and whether to send GPS-based timestamps in an Ogg stream or not.
Save audio logfiles:: With this option, a "copy" of the outbound audio stream (Ogg/Vorbis) will be saved on your harddisk. The dotted button near the input field opens a file selector.

2.1 Configuration of the output stream

The configuration file may look like this:


; Configuration file for an OUTBOUND Ogg/Vorbis audio stream .
; Loaded by Spectrum Lab shortly before starting to send a stream.

; server: <protocol>://<host>:<port> .  protocol: rawtcp or http .
server = rawtcp://127.0.0.1:1234

; password: some audio streaming servers may ask for it
password =

; quality: see Vorbis documentation; 0.5 seems to be a good default value
quality = 0.5

Empty lines, or lines beginning with a semicolon, are ignored by the parser.
At the moment, the three keywords shown in the example ('server', 'password', 'quality') are the only recognized keywords in the stream configuration file. Everything else is silently ignored.

Non server-specific options are defined in the dialog box shown above. Only those data entered in the configuration dialog will be saved in a *.usr or *.ini file by Spectrum Lab.
The stream configuration file will never be modified by SL - it will only be read but never written (again, it's your task to write it with a plain text editor).

2.2 Notes on unattended operation

For long term operation, without regular supervision, you should..

Set the option 'Automatically reconnect when connection lost' on the control panel shown above.
To let SL start streaming without your intervention as soon as it is launched, set the option 'Start output stream when Spectrum Lab starts' on the same panel.
Turn off annoying screen savers, automatic hibernation, etc.
Stop windows from rebooting your PC without your permission !
Don't forget the settings in the 'windows security center' or 'windows action center' (..what a name..).
(in german: "Updates herunterladen, aber Installationszeitpunkt manuell festlegen"). If you turn off automatic updates in the 'security center' (or whatever MS decided to call that thing today), check for system updates yourself.
Otherwise, you may find your audio stream broken down for a few days, just because windows decided to reboot your PC after installing a security update.
Alternative (not recommended):
Leave automatic updates enabled (i.e. let windows update your system whenever it thinks necessary), and find out how to let windows launch Spectrum Lab automatically after each reboot.
Under Windows XP, this was possible by placing a link to the program in the 'Autostart' folder.
But you will also have to find out how to disable the login procedure at system boot, etc... especially under "Vista" and later versions you're on your own at this point.
If the purpose is to feed timestamped VLF streams to Paul Nicholson's Live VLF Natural Radio server, consider using Paul's VLF Receiver Toolkit under Linux.

3. Logging web streams

This is possible with received (inbound) as well as transmitted (outbound) streams. The stream data in Ogg/Vorbis audio format (plus the optional timestamp channel) can be logged as a file on the harddisk or similar.
Because the storage format of an Ogg file is exactly the same as an Ogg stream, the stream data are written without any conversion, so this causes no significant additional CPU load. Also, logging the audio data this way saves a lot of disk space compared to the wave file format.
At the moment, SL does *not* generate a different name for the logfile (with a sequence number, or date+time in the filename) but this may change in future versions.

4. VLF radio streams

At the time of this writing, the following live VLF radio streams had been announced in the VLF group:

4.1 GPS-timestamped VLF radio streams

The following VLF live streams can only be played with the VLF Receiver Toolkit under Linux, or Spectrum Lab under Windows (since V2.77 b11). Note that 'rawtcp' is not an official protocol name, thus your browser will not know "what to do" with the above pseudo-URLs. But you can copy and paste them into URL input box shown in chapter 1.

rawtcp://67.207.139.49:4401: "VLF1" from Todmorden, on Paul Nicholson's Live VLF Natural Radio server;
raw Ogg stream with GPS-based timestamps
rawtcp://67.207.139.49:4406: "VLF6" from Spenge near Bielefeld, raw stream with GPS-based timestamps
rawtcp://67.207.139.49:4416: stereo combination of VLF1 from Todmorden (L) and VLF6 from Spenge (R), with timestamps
rawtcp://67.207.139.49:4427: stereo combination of VLF1 from Todmorden (L) and VLF6 from Marlton (R), with timestamps

More timestamped VLF streams will hopefully be available soon, since they may provide valuable information for research - see discussion and announcements about ATD (arrival time difference calculations), origins and spacial distribution of Whistler entry points, etc; in the VLF group.
As a motivation for prospective VLF receiver operators, the next chapter describes how to configure Spectrum Lab for sending such (GPS-) timestamped audio streams.

The following VLF live streams (links copied from abelian.org/vlf/ ) can also be played with any audio player. They do not contain GPS based timestamps so don't expect precise time markers on the spectrogram display, while playing these in Spectrum Lab:

http://67.207.143.181/vlf1.m3u: VLF1 from Todmorden (UK), here: without timestamps
http://67.207.143.181/vlf3.m3u: VLF1 from Sheffield (UK)
http://67.207.143.181/vlf6.m3u: VLF1 from Spenge (near Bielefeld, Germany), here: without timestamps
http://67.207.143.181/vlf26.m3u: VLF26 from Marlton (New Jersey), here: without timestamps
http://67.207.143.181/vlf15.m3u: VLF15 from Cumiana (Italy)
http://67.207.143.181/vlf16.m3u: Stereo combination of Todmorden (VLF1, L) and Spenge (VLF6, R), here: without timestamps
http://67.207.143.181/vlf27.m3u: Stereo combination of Todmorden (VLF1, L) and Marlton (VLF26, R), here: without timestamps

4.2 Setting up a timestamped VLF 'Natural Radio' audio stream

Prior to 2011, some of the VLF streams listed above used the combination of Spectrum Lab + Winamp + Oddcast as described here. Since the implementation of Ogg/Vorbis in Spectrum Lab, plus Paul Nicholson's extension to send GPS-based timestamps in his VLF receiver toolkit, the transition from the old (non-timestamped) MP3 based streams to the Ogg/Vorbis streams (with or without timestamps) has become fairly simple.

First of all, if possible at your receiver's location, consider sending a timestamped audio stream. All you need in addition to your hardware (VLF receiver, a soundcard with hopefully a stereo 'Line' input) is a suitable, despite cheap, GPS receiver. The author of Spectrum Lab had good success with a Garmin GPS 18 LVC (emphasis on 'LVC', since the Garmin GPS 18 "USB" doesn't deliver the important PPS output). If you are interested in the principle of GPS pulse synchronisation, also look here (later).

Here is a typical VLF receiver setup (hardware), with a GPS receiver delivering precise UTC time and date (through its serial NMEA data output) and the sync pulse ("PPS", typically one pulse per second):

Timestamped VLF Receiver Hardware

R1 and R3 form a voltage divider for the NMEA data, to reduce the amplitude to an acceptable level for the soundcard.

R2 and R3 forms a different divider for the 1-PPS (sync signal with one pulse per second). Important: The divided voltage of the 1-PPS voltage must be higher than the voltage of the NMEA (as seen by the soundcard), otherwise the software has difficulties to separate them. Also, the input of the soundcard must not be overloaded / clipped on any channel.

Capacitor "C" is not really required - it's only there to emphasize that a 'DC' coupling between the GPS receiver and the input to the soundcard is not necessary.

Fortunately, in all GPS receivers tested so far (Garmin GPS 18 LVC, and a Jupiter unit) had no overlap between PPS and NMEA.

A Garmin GPS 18 LVC with a relatively new firmware (anno 2011) produced this combined waveform on the 'R' audio input:

GPS waveforms at the soundcard input

A suitable configuration to send a timestamped VLF stream with the hardware shown above is contained in the Spectrum Lab installation.

To load it, select File (in SL's menu), 'Load Settings From', and pick the file 'Timestamped_VLF_Stream_Sender.usr' from the configurations directory.

5. Operation as audio stream server

For ad-hoc tests, and only for a limited number of clients, SL can also operate as server.
At the time of this writing (February 2012), the built-in audio stream server was still 'under development'. When implemented, there will be an extra control panel for the server in SL's 'file' menu.

6. Internals

6.1 Supported formats

At the moment, only Ogg / Vorbis audio is supported. Optionally, the Ogg container may contain (sic!) an extra logic stream for the GPS-based timestamps. These streams are compatible with Paul Nicholson's VLF receiver toolkit, which is available as C sourcecode along with a detailed documentation at abelian.org/vlfrx-tools/.

MP3 streams, as well as MP3 files, are not supported to avoid hassle with Fraunhofer's software patents.

The "Content Type" (in the HTTP response from the remote server) must be application/ogg or audio/ogg.
Only for raw TCP connections the program will recognize ogg streams from the first four bytes ("OggS") .

6.2 Supported protocols

At the moment, the reader for compressed audio streams only supports HTTP and 'raw' TCP/IP connections. These protocols are recognized by the begin of the URL (or pseudo-URL):

http://: Hypertext transfer protocol. This is the common protocol (also for audio streams) used in the internet.
rawtcp://: Uses a raw TCP/IP connection, not HTTP. Used for software tests; and to feed audio into VLF Natural Radio stream servers. The token "rawtcp" is not an officially recognized protocol; it's only used inside Spectrum Lab to leave no doubts about the protocol.

6.3 Stream Test Application

During program development, a simple test utility was written as replacement for a remote audio stream server (audio sink for Ogg/Vorbis encoded webstreams).
The sourcecode and executable (a windows console application) included in the 'Goodies' subdirectory after installing Spectrum Lab.
The "C" sourcecode contains a short description.
When launching the executable (TCP_Dummy_Sink_for_AudioStreams.exe) without parameters, it will open TCP port 12345 and listen for connections. Only one connection can be handled at a time. After being connected (for example, from Spectrum Lab), the utility will write all received data into a file, beginning with the file "test000.ogg". This name was chosen because in most cases, Ogg/Vorbis encoded streams will be sent by Spectrum Lab. For each new accepted connection (after the previous one has been disconnected), the file sequence number is increased, a new file will be written (test001.ogg, test002.ogg, etc etc). If the connection works as it should (and nothing gets lost "on the way"), the contents of the received file (written by the "TCP Dummy Sink" utility) will be exactly the same as the stream audio logfile (optionally written by Spectrum Lab, while sending the stream).