xdemorse User Manual

  1. Introduction
  2. Features
  3. Compilation
  4. Command line options
  5. Operation
  6. Bugs and annoyances
  7. Version history
  8. Copying

1. Introduction
xdemorse is a X/GTK+-3 application for decoding Morse code signals into text. xdemorse detects the "dihs" and "dahs" that make a Morse code character via the computer's sound card, which can be connected to a radio receiver tuned to a CW Morse code transmission or to a tone generator. The input signal is processed by a Goertzel tone detector algorithm which produces "mark" or "space" (signal/no signal) outputs and the resulting stream of Morse code "elements" is decoded into an ASCII character for printing to the Text viewer.

2. Features

Goertzel tone detector
xdemorse was designed from the beginning to be simple and efficient, decoding Morse code using mostly integer arithmetic and simple processing methods. It uses the computer's sound card to read the "dit" and "dah" elements that make the Morse coded characters by implementing an integer Goertzel tone detector algorithm, which measures the level of sound signals from a radio receiver or tone generator. This Goertzel tone detector is centered on a nominal frequency of 500 Hz, which is lower than the more common 700-800 Hz BFO setting, because my PSK31 applications have to work with a 500 Hz audio tone due to various limitations. I have chosen to run my Morse decoding applications on the same BFO setting, to avoid regular changes to my receiver.

The Mark/Space (key-down - key-up) condition is detected by saving the output of the Goertzel tone detector into a ring buffer, for a period of two Morse code Unit Elements, e.g. for the duration of two "dits". The change from Mark (tone) to Space (no tone) and vice-versa, is decided by summing the saved tone detector outputs, over one Unit Element, into a "leading edge" sum and then summing the following outputs into a "trailing edge" sum, again over one Unit Element. The ratio of the leading edge to the trailing edge sums rises and falls, depending on whether the leading edge or trailing edge sum is the sum of Mark tone or Space tone outputs from the Goertzel detector. This ratio is compared to a reference threshold to determine the Mark or Space condition of the tone from the Receiver. The sum of the leading and trailing edge sums is also used as an indication of signal strength and is used as a kind of "squelch" to minimize spurious character decodes.

Error and noise tolerance
xdemorse has a certain level of tolerance towards operator errors (bad "fist") regarding deviation from the standard duration of the various elements that make up the Morse code. Below is a table of the basic Morse code elements and the range of duration that xdemorse can deal with:

   Morse Element           Duration       xdemorse
-------------------------------------------------
       dit                  1 units   1/2-2 units
       dah                  3 units     2-4 units
Inter-elem space            1 units   1/2-2 units
Inter-char space            3 units     2-4 units
Inter-word space          > 5 units     > 4 units

The Morse code characters, punctuation marks and special signals that xdemorse can currently recognize are in the file doc/Morse-code.txt. Extra characters etc can be added to the collection by inserting the hex equivalent of the decoded Morse code character in the xdemorse.h header file before compilation. Please see detailed explanation in that file.

"Waterfall" display with CAT
xdemorse has a "Waterfall" (audio spectrum) display derived from an integer-arithmetic DFT of the receiver's audio output. A vertical white line indicates the center frequency of the tone detector. When tuning a signal manually, its trace should be centered on this line for best results.

xdemorse has built-in CAT capability but only for the Yaesu FT847 or FT857 and Elecraft K2 or K3. This is an unfortunate limitation but an attempt to convert xdemorse to using the "hamlib" library was unsuccessful for various reasons. Also, starting with version 3.4, xdemorse now has support for the Microtelecom Perseus SDR Receiver, including frequency entry from a spin button. Also, the waterfall display can be used to tune in a signal by clicking with the left mouse button near its trace. xdemorse will scan a small section of the display either side of the mouse pointer, looking for the strongest trace and then tune the receiver so that the signal is centered on the red line. Since the FT847/FT857 has only a 10 Hz resolution when tuned by the CAT, there can be an error of +-5 Hz in the tuning of a signal. Other errors resulting from the finite resolution of the DFT can increase the tuning discrepancy and since xdemorse has a sharp tone detector, it may be necessary to fine tune the receiver manually for best results. The K3 vfo has a resolution of 1Hz so that tuning in this case is more accurate.

Sound-card set-up
xdemorse automatically sets up the sound card and tries to set the input capture level to the level specified in the .xdemorse/xdemorserc config file in the user's home directory. If it fails it will give an appropriate warning and the mixer level for capture must be set up manually using a mixer application like 'aumix'.

For proper operation the level of the input audio signal must be adjusted from the receiver or the tone generator, using the 'scope' display at the bottom left of the main window. If the audio signal is taken from an outlet that provides a fixed audio level, then the Capture level entry in .xdemorse/xdemorserc will need to be edited by trial and error, until the signal plotted in the scope display is about 80% of the height of the scope frame.

3. Compilation
Please note that I use Void Linux AMD64 which is a "bleeding edge" type distribution, so there may be compilation and/or run time difficulties if you are using a relatively old distro. This is mostly true of the basic dependencies like GTK+-3 and Glade-2, and there can also be sound card incompatibility problems at run time. Support for the Perseus Receiver is optional and is automatically included during compilation, only if the libperseus-sdr library is installed on the system.

To compile the package, first run the included "autogen.sh" script in the package's top directory, to produce a fresh build environment. Then the "configure" script can be run with optional parameters to override the default settings and compiler flags.

Running "make" in the package's top directory should produce the executable binary in src/. Running "make install" will install files under the default /usr/local/ prefix. The xdemorse binary is installed into /usr/local/bin, the desktop file is installed in /usr/local/share/applications/ and the application icon in /usr/local/share/pixmaps/. Documentation is installed in /usr/local/share/doc/xdemorse/. From version 3.3 the xdemorse working directory is installed in /usr/local/share/examples/ and it is copied to home-directory/.xdemorse when the program is started. The ~/.xdemorse/xdemorserc file must edited by the user as required.

If you have problems identifying your sound card mixer devices (capture source, capture volume etc), running the "amixer" tool from the alsa-utils package will produce a detailed list of all available devices and functions of the sound card. There is also this hypertext documentation file which you can copy to a location of your choice.

4. Command line options:

xdemorse [-hv]
-h: Print this usage information and exit.
-v: Print version number and exit.

5. Operation
Before running xdemorse, it may be necessary to customize some entries in the .xdemorse/xdemorserc configuration text file in your home directory. There are a few items that may need editing: The ALSA sound card name (default is hw:0), the DSP sampling rate (default is 48000), the ALSA "channel" name (default is FRONT_LEFT), the capture source (default is Capture), the capture volume control (default is Capture) and the capture volume setting (default is 80%). If -- is set in the Capture volume entry, then xdemorse will not try to set the capture volume. For CAT control, the serial port device for the Yaesu or Elecraft transceivers must be specified (default is /dev/ttyUSB0). The Perseus SDR Receiver is auto-detected. The type of transceiver or receiver must also be specified: Yaesu FT847 or FT857, Elecraft K2 or K3 or Perseus SDR. The NONE setting disables CAT control. If the Perseus SDR receiver is selected, the next setting is the frequency correction factor for the Perseus's frequency reference. This is usually less than +/- 10.0 units (ppm).

Before xdemorse is used to decode Morse signals from the transceivers, the input level to the sound card must be set up correctly. For this, the radio receiver or tone generator must be connected to the "line" input of the sound card with a suitable cable and connectors, with either the left or right channel connected to the receiver's o/p if in stereo mode, or if in mono mode normally the left channel input. The receiver's output level is adjusted so that the maximum signal input level, as shown in the 'scope' display at the bottom left of the main window, is about 80% of the scope's window height. The receiver should be tuned to a strong, clean CW transmission, with the AGC setting in the fast position and IF/AF bandwidth in the narrowest setting. Please note that the receiver should be tuned accurately to the incoming signal so that the BFO note is 500Hz. This can be done by slowly tuning the receiver, with a tuning step of 1Hz if available, so that the INPUT SIGNAL display in the scope is at a maximum. The waterfall display is also very useful for tuning, since a signal trace tuned to be exactly under the red middle line of the waterfall will result in a 500 Hz BFO frequency.

Starting with version 3.5, xdemorse now has a spin button that can be used to set the frequency of the receiver down to 1Hz (if supported by the rig). Clicking the +/- buttons changes frequency in steps of 100 Hz and using the scroll wheel by 1Hz. It is also possible to type in the required frequency (in Hz!). Additionally, xdemorse now has a combo box button that can be used to specify the receiver to be used with xdemorse. The "Receive" button is reset when a new receiver is selected as above.

By default xdemorse starts with Morse code speed set to 20 wpm and the detector thresholds set at 50 for the Level ("squelch") and at 2.0 for the Ratio (mark/space detector). These are shown in the three spin-wheel widgets at the right of the scope window. As xdemorse begins to decode Morse coded signals, it estimates the sending speed and adapts to it, indicating the actual speed in the 'Speed' spin-wheel, if the "Auto" check button is enabled. If the Morse transmission is a lot faster or slower than the indicated speed, it may not be decoded correctly and manual changing of the speed setting will be needed to fix the problem. Auto adaptation to incoming Morse speed can be disabled with the "Auto" check button.

To decode Morse code from a communications receiver, set the BFO so that the tone of the CW signal is near to 500 Hz and then start xdemorse. Assuming that keying is not too bad (operator's 'fist' is reasonably good) then decoded characters should appear in the 'Receive Window' of xdemorse. Some changes to the speed and/or squelch and ratio settings may be helpful and experience will show the best setting for the latter. A maximum tuning step of 1 Hz should be selected on digital receivers and tuning should be slow. A narrow CW filter is an advantage although very narrow (less than ~200 Hz) IF or dsp/AF filters tend to ring and cause spurious characters, mainly E's to be printed on the screen.

Please note that the 'scope' display of xdemorse has three functions, it can display the input signal waveform, the output from the Mark/Space (signal/no-signal) Ratio detector or the Level (squelch) value. The first function is the default and this and the other displays can be selected from the four radio buttons at the right of the Scope display.

Generally xdemorse has a fairly wide tolerance of sending errors (bad fists or noise) but in practice it has proved to be difficult to correctly decode CW signals on the amateur bands, with the main problem appearing to be bad keying habits. Sometimes inter-character spacing is too small so two or more characters are taken as one and therefore cannot be decoded correctly. Characters not recognized by xdemorse are indicated by an asterisk * on the screen. Word spacing also seems to be inconsistent with many hams resulting in fragmented or fused words.

Finally a list of all characters, punctuation marks and special signals recognized by xdemorse are listed in the doc/Morse-code.txt file. Additional characters can be entered in two tables in the xdemorse.h header file before compilation, following the instructions in there.

6. Bugs and annoyances
I have fixed whatever bugs I came across testing xdemorse but there may be some hiding, waiting for the right conditions to appear.

The Morse code decoding algorithm may need further improvements to make it more tolerant to errors and noise and the mark/space detection process may need to be improved to reduce spurious output due to noise interference.

7. Version history

Version 0.1: First beta release of this basic Morse decoding X/GTK+-3 application, which is based on the console Morse decoding tool 'demorse'.

Version 0.2: Added a simple "Oscilloscope" display of the incoming waveform or the Detector output.

Version 0.3: Added a simple DFT-based "Waterfall" display of the receiver's audio spectrum. The Waterfall display incorporates CAT functions for the FT847 or FT857 transceiver allowing netting of the receiver by clicking on the trace of a signal.

Version 0.4: Replaced the original software synchronous tone detector with a Goertzel tone detector, since it has slightly better performance.

Version 0.5: Changed Mark/Space tone detection from threshold-ing the signal level to detecting the mark->space and space->mark transition edge. Seems to give more reliable decoding.

Version 0.6: Added a display function for the output of the Goertzel signal detector. This is displayed int the "scope" window at the left lower corner of the xdemorse window.

Version 0.7: Changed the detector scheme to a Mark/Space transition (edge) detector which works by counting the number of consecutive signal samples (steps) that change by more than 5% in the same direction (up or down). If the number of steps is more than a given threshold (default 60% of the Morse element duration) then this is taken to indicate a rising or falling edge. This seems to work better than the previous slope detection scheme.

Version 0.8 Added a parser to read a runtime configuration file from the user's home directory. This file (~/.xdemorse/xdemorserc) must be edited to match the user's setup. The command line options for specifying the various parameters xdemorse needs have been removed. Also its no longer necessary to make any changes to the xdemorse.h header.

Version 0.9 After a bug report from Juha Vierinen regarding seg faulting of xnec2c, my graphical adaptation of NEC2, I changed all "sprintf" commands to "snprintf" to avoid buffer overruns. Following on the above changes, I revised all similar situations in xdemorse source code and changed all "sprintf" commands to "snprintf" just in case. While going through the xdemorse source code, I also fixed some minor bugs like typos and tidied error messages and other aspects of the GUI.

Version 1.0 After a bug report from Pino Zollo ZP4KFX, I modified the waterfall display function to avoid a divide by zero condition that caused a SIGFPE crash.

Version 1.1 After a bug report from Pino Zollo ZP4KFX regarding GUI problems with my Hellschreiber program xfhell, I made some changes to the GUI code in most of my GTK2 applications.

Version 1.2 After a bug report from Pino Zollo ZP4KFX, regarding failure of "xdemorse" to start the Morse decoding loop after reading its configuration file, I have modified the functions that read this file so that more detailed error messages are printed if entries are malformed.
I added 3 new entries to the ~/.xdemorse/xdemorserc config file to allow a wider range of max and min Morse speeds and to define the initial Morse speed (WPM) at start-up. I also added a "Receive" button in the GUI to start and stop xdemorse as needed.

Version 1.3 Made the page size of spin buttons 0 to make setting of spin button values compatible with GTK 2.4.

Version 2.0-beta After many reports of difficulties and/or failure to set up and run xdemorse, due to poor or non-existent support for some sound cards in ALSA's OSS compatibility layer, I carried out major changes in xdemorse to use the ALSA sound API instead of the original OSS API. Because of very sparse or lacking documentation for the ALSA sound API, I had great difficulty getting xdemorse to work under ALSA, especially the Mixer interface. I am releasing xdemorse as version 2.0-beta for public testing, hoping it will be more successful with modern sound cards. Please don't blame me if it is not, even simple sound card programming with ALSA can be very difficult and/or tricky and buggy, since there are few if any tutorials available, especially for the Mixer interface.

I have also made several changes to the transceiver CAT code to deal with error conditions better and to be enabled by the "Receive" button, so that CAT is enabled only when xdemorse is actually receiving Morse signals. Please note that all the above changes inevitably resulted in regression bugs which I fixed, together with a few that were already there from the previous version. However, it is unlikely that I was able to catch all the bugs, so please do not hold me responsible for any problems xdemorse may create while in use on your system.

Version 2.1 Increased the height of the 'scope' and waterfall displays to improve resolution. Also separated the 'Auto Speed' check-button into its own frame. Again changed the Mark/Space detection scheme to measuring the average value of the Goertzel detector's slope.

Version 2.2 Added CAT support for the Elecraft K3 transceiver. It is likely that this will work with the K2 as well, although I am not able to verify this. Improved the Integer DFT functions to improve the detection of signals, as well as to present a color coded waterfall display. Also modified the "scope" display, to present a colored signal display.

Version 2.3: Replaced the GDK drawing primitives with the equivalents from the Cairo drawing library (e.g. gdk_draw_line() with cairo_line_to()). This gives a nicer anti-aliased display of the signal detector.
Changed the waterfall display style so that signals are shown with a color coding on a black background. Added a check button to control the drawing of the white center line in the waterfall. The signal "scope" display is now in bright green on dark green background.

Version 2.4: I have updated the basic files of the GNU Autotools build system, to be compatible with the current version of these tools at the time of writing (February 2013).

Version 2.5: I have fixed a number of defects reported by the source code auditing tools of the Coverity Scan website. These were mainly related to programming practices and did not affect the operation of this application.

Version 2.6: I have made changes to the Mark/Space detector in the hope of improving the low signal/noise ratio performance of xdemorse. I believe xdemorse is now at least a little more effective than earlier versions in weak signal reception.

Version 2.7: I have modified the installation script to comply with Debian practices, as requested by João Eriberto. Documentation is installed in /usr/share/doc/xdemorse/ and the runtime config file in /usr/share/doc/xdemorse/examples/xdemorserc.examples. The desktop file is installed in /usr/share/applications/ and the menu icon in /usr/share/pixmaps/.

Version 2.8: After a bug report form Richard Lawn, regarding faulty CAT control of his FT847 by xsatcom, I borrowed back my old FT847 and after a lot of changes to the xsatcom CAT code, I fixed whatever bugs I found and incorporated some of those bug fixes into xdemorse as needed. These bugs were a result of adding support to my ham radio applications for the Elecraft K3, without testing them on an FT847 as I no longer had my old rig.

Version 2.8: I made some changes to the Strlcat() function and its usage in the xdemorse code, to improve safe handling of string concatenation operations. Hopefully this has not broken the handling of various strings in xdemorse! ;-)

Version 2.9: I made some changes to the DFT code used to produce the Waterfall display, e.g. to decimate the sampling rate by simple "box" averaging instead of skipping samples.

Version 3.0: I fixed a bug in the CAT code that performs the click-to-tune function (tuning of the Transceiver by clicking on a signal trace in the Waterfall).

Version 3.0.1: After a suggestion by Wolfgang, DL2KI, I modified xdemorse's source code so that UTF-8 characters can be included in the decoding tables, which makes it possible to decode and display Extended Latin characters, as in German.

Version 3.1: I updated the GUI code to the latest GTK+-3.22. This made it necessary to make minor changes to the User Interface but functionality remains the same. An advantage of this version is that the GUI specification is in a Glade-3 XML file in the ~/.xdemorse/ folder (xdemorse.glade), and this can be edited in Glade-3 if some cosmetic changes are desired.

Version 3.2: I carried out some source code clean-up and fixed a possible memory leak through the use of GtkBuilder objects.

Version 3.3: I edited xdemorse.glade to obtain necessary margins and spacing of widgets using more efficient settings. Also moved the "Xdemorse" Control frame to the left, since where it was on the right, hovering the pointer over the +/- buttons of the Spin buttons caused erratic redraws of the "Scope" display.

Version 3.4: I migrated the internationalization system from "intltoolize" to GNU GetText, as the former is now considered obsolete. Also replaced the autogen.sh script with a newer version.

Version 3.4.1: I edited Makefile.am so that make install will now place relevant application files (documentation, pixmaps, desktop file, glade UI file etc) into the right places.

Version 3.5: I added optional support for the Microtelecom Perseus SDR receiver and provided a combo box button to allow selection of the receiver to use. I also added a spin button which can be used to tune the receiver to the required frequency.

Version 3.6: I replaced the integer DFT functions with an open-source integer FFT function I found on the internet. This I had to modify to fit the needs of this application and this change resulted in a worthwhile reduction of processor workload.

Version 3.6.1: I removed a stray printf() statement that I was using for debugging IFFT. It got away from testing!

Version 3.6.2: I have separated the pseudo-colorization code for the waterfall into a separate function and edited it to improve the colorization algorithm.

Version 3.6.3: After a bug report by Robert Kennedy VA2FB, I changed the installation scripts to install files under /usr/local/share by default (or other prefix specified to the configure script) and for xdemorse to create its working directory at startup, if it is not there already. Relevant messages are printed to the terminal since with no Glade GUI description file, a graphical interface cannot be created.

Version 3.6.4: Fixed minor issues reported by linters and made minor changes to the installation scripts in Makefile.am

Version 3.6.5: Applied minor patches supplied by Ricardo Mones, maintainer of xdemorse for Debian.

Version 3.6.6: Made changes to the Goertzel Tone Detector's parameters to narrow its bandwidth and improve detection. This, however, has made it necessary to tune in the CW station more accurately.

Version 3.6.7: Reduced the Block size parameter of the Goertzel detector to 1/2 of the original because it seemed to effect the Morse decoder's performance.

8. Copying This software package is released under the GNU Public License. Please see the COPYING file for more details.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details:

August 16 2002.