README for the folder "server_pages"

Last modified  :  2020-06-18 by WB (DL4YHF)

Contents
  1. Introduction
  2. Location of SL's HTTP server pages
  3. Special filenames in SL's HTTP server pages folder
    1. _spectrum.jpg, _spectrum.bmp
    2. _watercolours.jpg
    3. _rdfcircle.jpg
    4. _audiostream.ogg
    5. _ifunc.html (remote function call using IFRAMEs) / _iproc.html (remote procedure call)
    6. ipblocklist.txt or ci-badguys.txt (list of IP addresses blocked by the 'sysop')
    7. sl_auto_blacklist.txt (automatically blacklisted 'malicious' IP addresses)
    8. index.wrx (optional HTML template file 'inspired by OpenWebRX')
  4. Spectrum Lab's built-in web server and OpenWebRX
    1. Web RX Server configuration file

1. Introduction

This folder contains a few sample pages in HTML format, which can be loaded by the built-in HTTP server in Spectrum Lab. In 2006, this started as an experiment to control SpecLab through a web browser via local area network without the need to install a special web server (or even PHP).

Many years later (2019 to 2020), the HTTP server in Spectrum Lab was drilled up to host not only the original 'server pages' (which relied on now outdated techniques like 'IFRAMES'), but it can also host the html pages, javascript and CSS originating from HA7ILM's
OpenWebRX project.

The main purpose of this folder is that you can put YOUR OWN HTML pages in it, to develop your own browser-based 'application', using Spectrum Lab in the background (without Spectrum Lab's own user interface).

Some reserved keywords may be embedded in the HTML code in this directory, which will then processed directly inside SL whenever an application tries to load the page (via HTTP protocol).

Note: The firewall build inside windows XP usually prevents an application to act as a server for incoming HTTP connects. No problem as long as you don't enable this tiny built-in web server inside Spectrum Lab. But as soon as you start the server for the first time, the Security Center will prompt you with a message that it has just blocked something. Convince it NOT to block, and the HTTP server will work. Note: You don't need a "server edition" of windows for this. The whole server is built inside SpecLab.

To open any webpage saved in this directory, you must enter the IP address of the machine on which SpecLab is running, since there will be no DNS name service running at that time. Example:
Your PC is configured with a fixed IP address in the local network, for example 192.168.0.22 (you can find that address in the HTTP server control panel built inside SpecLab, even if the server is not running yet).

To test the server locally, 127.0.0.1 should work too. So for a local test, enter 127.0.0.1 in the address bar of your web browser. Smart browsers expand the adress automatically to something like


 http://127.0.0.1/ .  

(some browsers do this without showing you the full address)
 
Most of the links in this documents actually include the above 'local, numeric IP address' because when loading the pages from the 'server_pages' folder as a file won't work, because in that case SL's built-in HTTP server has no chance to process the data, and generate the requested data on-the-fly.

The web browser will then automatically request a page named index.html from this server, using port number 80 by default. If that port is already occupied by another server application, use a different port number, which you must add to the IP address in the browser as in this example (where port number 1234 is used) :

http://127.0.0.1:1234/

Most browsers are smart enough to request a file named "index.html" from the server, if no file is specified. For that reason, a file named "index.html" should always be located in the SERVER PAGES folder. When SpecLab was installed, a simple example is already placed there ( written by SL's author, who didn't know much about Javascript when implementing the HTTP server )

If you are familiar with scripting languages like Perl: Look for sample applications in the "Spectrum Lab Users" forum, for example:

http://tech.groups.yahoo.com/group/SpectrumLabUsers/message/471

... to be continued ...


What the HTTP server can NOT do ...

For safety reasons, the server cannot "break out" of this directory. Any files which are placed outside this directory are not accessable through the built-in server. Relative filenames or links containing ".." are not allowed for simplicity.
But beware, the simple HTTP server was not designed to tolerate a huge load from multiple simultaneous connections. It was tested only once as a real WEB SERVER ("on the internet"), but it will mostly be used only in a local area network !
If you need a grown-up WEB server, install Apache ;) For this tiny server, use simple HTML pages only. Javascript or even Java is ok, because they run on the client side (in your browser, not on the server).


2. Location of SL's HTTP server pages

The pages saved in this directory are 'ordinary' HTML documents. Just HTML - no PHP, Perl, or similar ! But more or less 'dynamic' contents are possible :
Certain filenames beginning with an underscore have a special meaning, and will cause Spectrum Lab to do some special processing on-the-fly (see list below), like generating a graphic file with the current spectrogram.
You can create your own HTML documents (or even templates) in your own 'server' directory - the directory can be freely selected, unless a paranoid "operating system" prevents accessing it.

Because Spectrum Lab needs to create some of the 'files' in the HTTP server folder itself, it must have write-permission for this directory. For that reason, it is highly recommended to install Spectrum Lab (including it's "server pages") in a folder outside the windows 'programs' folder (do you remember that folder with the ever-changing name, "C:\Program Files", "C:\Programme", "C:\Programme (x86)", or whereever they wanted us to place our 'programs' in ? ... ).

To experiment with different sets of "server pages" (including HTML, CSS, Javascript modules) in Spectrum Lab, the path to the 'server pages' is configurable as explained in the
Spectrum Lab documentation (online).

3. Special filenames in the SERVER PAGES folder

The filenames which trigger "special actions" when requested from SL's integrated HTTP server begin with an underscore.
All these special names are intended to be used by HTML pages written by yourself - besides the examples inside SL's own 'server_pages' folder.
Note that the links below only work properly if SL's integrated HTTP server is running, because the 'resources' requested this way are generated on-the-fly.
Not all of them will be 'real' files on the harddisk, because some of these resources are actually just blocks of memory (in RAM) which the built-in server can send just as if they were files.

Besides the 'automatically generated content files' listed below, there are a few other files with special filenames located in the server pages for the
web receiver configuration (sl_webrx_config.txt), an optional IP address blocklist (ipblocklist.txt or ci-badguys.txt, deposited there by the 'sysop'), and an automatically generated blacklist (sl_auto_blacklist.txt) in which SL remembers the IP addresses of those 'bad guys' that really have been knocking on the door.

 

 2.1 _spectrum.jpg   or   _spectrum.bmp
  Lets SpecLab generate an up-to-date image of the current 
  spectrogram and save it, shortly before the HTTP server
  can send the file. Example in server_pages\index.html !
  The size and contents of the image depend on the settings
  in Spectrum Lab of course. The image may show the
  spectrum graph and/or spectrogram (waterfall) with frequency
  scale and -possibly- the amplitude bar too.
  It does NOT include the waterfall colour legend, and the
  "info box" which is overlaid on otherl screenshots .
  Instead, for freedom of the layout of the remote control
  "user interface" written in HTML, the colour legend etc 
  can be requested from the HTTP server as separate image.

  Note: If one HTTP client requests a new _spectrum.jpg
        while another client still receives an older image,
        the integrated HTTP server will automatically 
        create an image with a different name for the second
        request (but only if the second request is made
        before the first request is complete).
    In that case, you will find files like 
       "_spectrum.jpg", "spectru2.jpg", ... "spectru9.jpg"
        in the server directory. There will be a maximum
        of ten different temporary images in this directory.
        They can all be deleted safely after SL terminated.


 2.2 _watercolours.jpg   or   _watercolours.bmp
  Produces a small image with the colours and dB-scale for
  the waterfall, using the current contrast- and brightness
  control values. 
  Without a query string (after the filename), the image
  is simply copied from the brightness/contrast panel on
  SL's main window. 
  Alternatively, a few parameters can be set in the query-
  string, for example:
   _watercolours.jpg?h=20&w=100&c=40&b=70
  h=height in pixels, w=width in pixels,
  c=contrast value (0...255),
  b=brightness value (0...255).
  Note: Requesting this image for a certain contrast and
  brightness does NOT affect the contrast+brightness of 
  the waterfall itself. 
  TO set a new contrast+brightness for the main display,
  use the interpreter functions 'water.ctr'+'water.brt',
  sent as a command to SpecLab's interpreter (see chapter
  "Invoking commands and functions in SL's interpreter).
  

 2.3 _rdfcircle.jpg   or   _rdfcircle.bmp
  Produces an image of the Color Direction Finder Circle,
  like the "colour disk" on the RDF tab in the main window.


 2.4 _audiostream.ogg
  Compressed Vorbis audio stream, in an Ogg stream container.
  Highly experimental, and at the moment (2013-06), only supports 
  ONE SINGLE remote listener (client).


 2.5 ipblocklist.txt   or   ci-badguys.txt
  These are IP address blocklists, which you -as the 'sysop'-
  can deposit in the 'Server Pages' folder to prevent malicious
  users from 'entering' by their IP address. This is not very
  efficient, and a lot of effort to keep these lists up-to-date,
  so unless one of the 'bad guys' steels a lot of bandwidth by
  continously knocking at the door, you can block him this way.
  For details, see the Spectrum Lab user manual (available online).


 2.6 sl_auto_blacklist.txt
  This file also contains a list of 'blocked' / 'blacklisted'
  IP addresses, but it's not intended to be placed there by
  the sysop. Instead, SL writes this file itself (shortly before
  closing) to remember the most recent IP addresses of potentially
  malicious visitors between two SL sessions.
  Again, for details, consult the Spectrum Lab user manual.


 2.7 index.wrx 
  This is the optional HTML template file for an OpenWebRX-alike server.
  It is not contained in the Spectrum Lab installer
   (because OpenWebRX was, and will always be, available under 
    the GNU Affero GPL v3 license - which Spectrum Lab is not).
  You can download a modified variant and 'install' (i.e. unpack)
  it, along with some other files originating from OpenWebRX's
  "htdocs" folder, as described here (online).

2.5 Invoking commands for SpecLab's interpreter

(via HTTP / Javascript)
Please note the big difference between interpreter COMMANDS and interpreter FUNCTIONS in SpecLab : To invoke them from a HTML document (typically from javascript), different pseudo-html-pages can be requested from SpecLab via HTTP, with parameters passed in the so-called Query String (do a websearch on 'HTTP Query String' for details):

_ifunc.html

Invokes an interpreter FUNCTION, and sends the result back to the caller using a piece of javascript (through IFRAMES).
Example:
_ifunc.html?pk_a=peak_a(500,800)&pk_f=peak_f(500,800)

The query part in this example contains TWO calls for SpecLab internal command interpreter, to calculate the peak amplitude and -frequency between 500 and 800 Hz. For a start, let's call these functions from the web browser 'manually' by entering the complete URI (with query string), and see what happens ...

http://127.0.0.1/_ifunc.html?pk_a=peak_a(500,800)&pk_f=peak_f(500,800)
Note
For historical reasons ("old style" without the XMLHttpRequest object),
the call will usually be made through an invisible IFRAME.
From SL's point of view, it doesn't matter which of the various 'AJAX'-methods is used.
Examples which use the XMLHttpRequest object and JSON are included in the 'server pages'.
    To call functions in SL from Javascript,
    include the module "comms.js" and use the
    the function CallIFunc(key,expression) .

On reception of the above request, SpecLab sends the following response back. To see the response in the browser, open a "View Page Source"-window. It will show something like this:
   
  <html>
  <head>
  <script type="text/javascript">
    window.parent.HandleIFunc('pk_a','-53.015');
    window.parent.HandleIFunc('pk_f','719.705');
  </script>
  </head></html>
The 'response' shown above is in fact an HTML document with a piece of machine-generated javascript, which will be executed by your browser. It calls a javascript function called 'HandleIFunc' inside the parent window (we'll come back to that later), to handle the results of the function call to SL's command/function interpreter.
Note
The principle used here is a rather antique kind of "Remote Procedure Call".
A nice introduction, actually the inspiration for '_ifunc.html'
(before the XMLHttpRequest object was available) was available at
developer.apple.com/internet/webcontent/iframe.html, but -unfortunately-
like many others, that webpage has vanished into air.
For new applications, use XMLHttpRequest instead of IFRAMES anyway.

Here, we won't go into the details of using IFRAMES to communicate between client (=your webrowser) and Server (=spectrum lab in this case).
If you want to use IFRAMES in your javascript application, look at "server_pages/comms.js" (after installing SL), and study the examples mentioned below.

A simple example for a remote function call is in the file controls.html (in SL's server_pages).
There is also a javascript module named 'comms.js' which does most of the communications between the HTML document (running in a browser window) and Spectrum Lab.
It's up to you how and where to implement HandleIFunc in your remote control application.
All you need to know (from SL's point of view) is:

Here is a simple example for the implementation of HandleIFunc, copied from server_pages/controls.html:
   
  function HandleIFunc(key,result) // called from server via IFRAME
  {
    // copy the result (delivered by SpecLab)
    // into the field identified by the key :
    document.getElementById(key).value = result;
  }

_iproc.html

Almost like "_ifunc.html", but "_iproc.html" calls a PROCEDURE in Spectrum Lab's internal interpreter.

(Remember: FUNCTIONS in SL return a value, PROCEDURES don't). But even though a PROCEDURE doesn't return a value, your javascript will be "called back" (in the HTTP response), as soon as SL has processed the call. This may take a few milliseconds, or even seconds, depending on the callee. Like _ifunc.html, _iproc.html was designed to be used in IFRAMES.
The response frame should look like this:
   
  <html>
  <head>
  <script type="text/javascript">
    window.parent.HandleIProc('');
  </script>
  </head></html>
Note
If the remote procedure call is ok, the argument send to window.parent.HandleIProc() is an empty string, which means "all is ok" / "no error" . Otherwise, the string will be an human-readable error message from Spectrum Lab (in a string), which you can show in some kind of "error history" in your script if you like; or just ignore the response.

Since the built-in HTTP is used as a 'real' web server by some users, invocation of '_iproc.html' and '_ifunc.html' can be disabled on the HTTP Server control panel for security reasons.
See the SL help files (ext_comms.htm) for details about this ancient kind of 'remote procedure calls' through the HTTP server .


Reading the spectum buffer in JSON format (added 2013-05)

The contents of Spectrum Lab's 'FFT buffer' can be read via the integrated HTTP server. To simplify processing those data in a web browser (client side), the JSON format was chosen. Unlike XML, JSON is simple enough to be parsed in most other programming languages (besides javascript which typically runs in a web browser). A simplistic example for accessing the spectrum buffer from an piece of Javascript is in the file
  fft_expt.html (located in SL's server_pages directory) :
  
  
  


var http_request_for_FFT = new XMLHttpRequest();

   ....


// Spectrum-Lab specific stuff to read an array of spectra / FFTs via XMLHttpRequest (bare-bone javascript):
function RequestFFTArray()  // calls an INTERPRETER FUNCTION in SpecLab
{
  // Encode the parameters to retrieve a bunch of spectra in the query part of the 'address' (URI):
  var url = '_fft_expt.json' // see README_server_pages for details on '_fft_expt.json' !
       + '?f1=' + encodeURIComponent( document.getElementById("freq1").value )   // start frequency
       + '&f2=' + encodeURIComponent( document.getElementById("freq2").value )   // end frequency
       + '&i1=' + encodeURIComponent( document.getElementById("index1").value )  // start index (FFT counter)
       + '&i2=' + encodeURIComponent( document.getElementById("index2").value ); // end index (FFT counter)
  http_request_for_FFT.open("GET", url, true);
  http_request_for_FFT.onreadystatechange = ProcessFFTResponse;
  http_request_for_FFT.send(null);
} // end RequestFFTArray()

The above function is called periodically (in the simple demo, click on 'Start' for this). It requests a file named '_fft_expt.json' from Spectrum Lab's integrated webserver, along with a few optional parameters :

  Syntax for the REQUEST / URI with 'query string' :

   _fft_expt.json?f1=StartFrequency&f2=EndFrequency&i1=StartIndex&i2=EndIndex
                     |____________|    |__________|    |________|    |______|
                         |                      |         |             |
                         frequency range in Hertz      FFT sequence counters, optional
  Parameters:
    f1 = start frequency in Hertz, for the first frequency bin contained in the response   
    f2 = end frequency in Hertz, for the last frequency bin contained in the response 
    i1 = FFT sequence number (counter) for the first FFT (spectrum) in the response.
          To begin with the oldest spectrum available in SL's buffer, use i1=oldest.
    i2 = FFT sequence number (counter) for the last FFT (spectrum) in the response
          To export all spectra up to the latest (newest), use i2=newest.
  Examples: (copy as address into a web browser) :
    (A)     http://127.0.0.1/_fft_expt.json
    (B)     http://127.0.0.1/_fft_expt.json?f1=0&f2=1500
    (C)     http://127.0.0.1/_fft_expt.json?f1=0&f2=1500&i1=oldest&i2=latest
 (A) If the frequency range is omitted, this function retrieves ALL frequency bins.
 (B) If the FFT counter range is omitted, this function only retrieves
     the LATEST (newest) available spectrum (FFT).
 (C) With i1=oldest and i2=latest, the FFT export function actually returns
     all spectra available in SL's buffer, which can be quite a lot.
     Use it only when necessary, to avoid unnecessary network traffic.

Since the FFT sequence counters are contained in the JSON response (in each array element, as explained further below), the FFT sequence counter delivered in a previous call, incremented by one, can be used as parameter 'i1' in the following call. This way, the server only sends the 'new' FFTs.

The response itself contains the FFT sequence counters, which allows the caller only to request the NEW FFT(s) in the next call. In the simple example (fft_expt.html), a JSON parser is invoked to convert the server response into a Javascript object. In other programming languages, similar parsers may exist, but you may as well write a specialized parser yourself since JSON is a very simple format.

Example of a JSON-formatted FFT (spectrum), copied and pasted
from the text box on the FFT-export-test-page, manually 'beautified' :


[ { "utime":1369079131.357303,
    "index":3560124,
    "f1":187.500006,
    "bw":375.000013,
    "type":"real",
    "unit":"dB",
    "nbins":8,
    "cpb":1,
    "bins":[
             -68.960938, -62.971455, -68.468681, -80.667328, 
             -77.757141, -78.784439, -82.008812, -84.861069
           ] 
  }
]



  Note: Whitespaces, which include carriage return + new line,
        have no syntactic meaning in JSON (outside strings).

The response (JSON) is an array of objects. Each object contains 
several name:value pairs, with the following meaning:


    "utime" : timestamp of data acquision in UNIX format, which is
              'seconds elapsed since 1970-01-01 00:00:00 UTC' 
    "index" : FFT index (sequence counter) .
    "f1"    : center frequency of the first bin in Hertz.
    "bw"    : bin width in Hertz. Depends on sampling rate and FFT size.
    "type"  : type of the data in each frequency bin:
              "real" for an amplitude spectrum (most simple case) .
              Depends on the configuration in spectrum lab, menu:
              Options.. FFT settings.. FFT output.. Type .
              "real" for output type 'Normal, amplitude only'.
    "unit"  : Physical unit of the output. Can be modified under
              Options.. FFT settings.. FFT output.. Unit .
    "nbins" : Number of frequency bins in the EXPORTED SPECTRUM. 
              This may be less than the total number of bins,
              depending on the requested frequency range,
              and the width of each frequency bin ("bw").
    "cpb"   : Components per bin. 1 for "real" data,
              2 for complex data (I+Q), 
              3 for radio directio spectra (amplitude, azimuth, elevation), 
              etc. For details, see notes below.
    "bins"  : An array with the exported frequncy bins.
              Format depends on the spectrum data type;
              usually floating point numbers, but
              (for example, radio direction spectra)
              some components may actually be integer.


In the simple example (fft_expt.html), the result is processed as soon as the server response has arrived, by calling 'ProcessFFTResponse' (javascript) :
function ProcessFFTResponse()  // processes the FFT array, after the JSON response arrived from SpecLab
{
  var my_JSON_object = {};
  if (http_request_for_FFT.readyState==4 && http_request_for_FFT.status==200) 
   { // Parse the response (JSON). The result should be an ARRAY OF SPECTRA,
     // as specified in ?/server_pages/README_server_pages.html , 
     // chapter 'Reading the spectum buffer in JSON format' .
     my_JSON_object = JSON.parse(http_request_for_FFT.responseText);
     if(  my_JSON_object )
      { UpdateSpectrumGraph( my_JSON_object );
      }
   }
  
  // For debugging, copy the complete server response (JSON formatted) into a 'text area'.
  document.getElementById("text_dump").value = http_request_for_FFT.responseText; // or JSON.stringify(my_JSON_object); 
  
  
} // end ProcessFFTResponse()

... to be continued ...

4. Spectrum Lab's built-in web server and OpenWebRX

When Spectrum Lab's tiny HTTP server was written in 2006, developing 'highly interactive' browser based applications was a complete mess (remember those 'IFRAMEs' to emulate AJAX-like behaviour - yucc).
Even getting simple things done in javascript, and getting that messy javascript running in all 'major browsers' of that era (2006-2007) was such a painful experience that the author of SL gave up any plan of a fluent running, completely browser-based 'remote' user interface for Spectrum Lab.

Looking back at this (in 2019), much has improved since 2006: With the advent of HTML5, CSS, web audio and 'WebSockets', many of the most annoying shortcomings have been solved. There are stunning examples of what can be done on the *client* side (which means inside the web browser, not in the web server), like smoothly scrolling waterfall displays, digital audio processing even in Javascript (since at least some of today's browsers compile Javascript, instead of interpreting it on-the-fly).

One of those examples is HA7ILM's OpenWebRX, which uses C++ and Python on the server side, and HTML / CSS / Javascript on the client side. Since Andras (HA7ILM) has made all sources for his OpenWebRX available on Github with a
  GNU AFFERO GENERAL PUBLIC LICENSE,
it would be foolish and a complete waste of time to re-invent the wheel for the HTML/Javascript/CSS part that runs in any (well.. almost any) major browser.
To play a bit with this, without buying a KiwiSDR or similar suitable hardware, Spectrum Lab's ancient built-in web server was drilled up to support a small subset of what the SERVER-side part of OpenWebRX (e.g. the Phyton part of OpenWebRX) would implement in a 'real' OpenWebRX installation under Linux.

Details about using OpenWebRX (instead of Spectrum Lab's antique 'server pages', HTML files written long before we had HTML5, web audio, and 'WebSockets' for Javascript) are in the HTML documentation located in SL's server pages themselves :
  See SpecLab/server_pages/README_server_pages.html .

OpenWebRX itself was available on Github, see

github.com/simonyiszk/openwebrx

Also don't miss Andras' BSc thesis about OpenWebRX, which (in 2019) could be downloaded from

sdr.hu/openwebrx

If the remote client asks Spectrum Lab's own web server for the root document, and there's only a file named 'index.wrx' but no 'index.html' as in Spectrum Lab's own (ancient) 'server pages' directory (configurable - it may be anywhere on your local harddisk), it will read the template file (file with extension '.wrx') and replace some of the special tags as specified in the BSc thesis (anything beginning with "%[" and ending with "]" after the token), e.g.
%[RX_TITLE]
%[WS_URL]
%[CLIENT_ID]
Note
Spectrum Lab recognizes only a small subset of what OpenWebRX may use in the special tags of 'index.wrx'. At the time of this writing (01/2019), implementing support for %[RX_PHOTO_HEIGHT], %[RX_PHOTO_TITLE], %[RX_PHOTO_DESC] was considered not worth the effort, since Borland's C++Builder V6 (the environment used to develop Spectrum Lab) didn't even support JPEG files, etc etc. If you want to customize these, edit the file index.wrx after unpacking it from openrx_master.zip.
Other tokens like %[RX_ANT], %[RX_QRA], etc are replaced by dummy strings.

If the OpenWebRX template file (index.wrx) contains tags or tokens that the tiny web server in Spectrum Lab doesn't understand, you may see an error message or warning in the web server's Event Log.

4.1 Web RX Server configuration file

Keywords in the web template (e.g. "%[RX_TITLE]" in index.wrx) and their replacement strings are saved as sl_webrx_config.txt, in the customizeable folder with the 'servable' web pages. The format is straightforward (INI-file syntax), and only intended to be parsed by Spectrum Lab itself (shortly before starting the web server).

For details about the OpenWebRX-based 'Web Receiver' (server side), see the Spectrum Lab documentation (online).



See also (in Spectrum Lab's help files) :
  Communication between Spectrum Lab and other programs,
 
Spectrum Lab's tiny OpenWebRX-compatible server,
 
Spectrum Lab's built-in interpreter,
  Sending and receiving audio streams with Spectrum Lab.