Improving Open-AMBE for D-Star

 

 

Overview:

The AMBE vocoder (speech coder) problem started when D-Star was developed and introduced to ham radio in the late 90's early 2000's. It's since been compounded by DMR, Yaesu Fusion, and other digital radios. The proposed solution has been to replace this with an open source codec.   However, none of these digital modes have a means to specify or differentiate the audio codec should they ever want to move away from AMBE.   And there still isn't much of a working implementation of Codec 2 for VHF/UHF radios.  There is a working implementation for HF. Codec2/FreeDV is a good conception, but it hasn't come to fruition in the advent of the AMBE patents expiring and other digital radios also employing AMBE and flooding the market.  Besides we still don't have radios with open firmware where we could load our own apps or alterative codec.

Keen observers have noticed a number of good radio initiatives that sadly haven't come to be.  The; Connect Systems CS7000, Wireless Holdings DV4 Mobile, the HT of the Future (aka: Algoram, Katena, Whitebox), the Newradio initiative in Germany/Austria, and so on.  There is a new initiative, the M17 Project, but its too early to tell if that will reach production.  

Manufacturers are a fairly conservative bunch. They don't want to invest in anything unless they know it's going to sell.  We all understand.  Fortunately, much of the innovation now is purely in the form of software, which is much easier to mass-produce than hardware. So all you need the manufacturers for is to make general purpose SDR hardware, which is an easier sell than some new special mode.  It's also important for leaders to recognize the works of the dedicated software developers.  And when the hardware manufacturers see the potential from their work, to work out an agreement so the software folks receive more than just a thank you.

For many hams these days the AMBE concern is not well understood.  First you have to take a larger look at the point of amateur radio.  To learn about radio, propagation, and the electromagnetic spectrum in general.  To understand how it works, and maybe even build or modify your own equipment.  In order to learn we must be able to inspect; to tinker, or at the very least have access to a specification we can build from.  Closed source applications such as; Dplus, Hamvoip, Peanut, and others don't contribute to the advancement of the radio art because they retard people from learning the technical operations.

So while D-Star was interesting as it was a new thing to play and learn about when it came about, there was this "black box" aspect where you just buy and are a passive consumer.  This didn't sit well with many. Why even bother with a hobby of sharing and learning, if thereís a big wall saying  ďthis far, no further!Ē, that you are not permitted to create your own radio for. Or permitted to understand.

The "HT of the Future" talk that Bruce Perens also gave some time ago, did also stress the need to have open radio firmware.  Let's take a quick look at why this would be good for the hobby.

The Linksys WRT54G WiFi router of the early 2000's was a good example of the good that can come from open firmware/open source.  The history here was the original factory firmware was discovered to be based on Linux components, which are covered by the GPL.  This required the manufacturers to release the source code.  With the code in hand, developers learned exactly how to talk to the hardware inside and how to code any features the hardware could support. It has spawned a handful of open source firmware projects for the WRT54G that extend its capabilities, and reliability, far beyond what is expected from a cheap consumer-grade router. In short, due to open source, one can load a third party firmware on the router and give a $60 consumer home-grade router all the functionality of a $600 Cisco professional router. 

There have only been a few radio firmware reverse engineering efforts.  The most well known is the MD380 project by Travis, KK4VCZ and his group.  The hobby can use a lot more of this and a lot more people like Travis.   We haven't yet figured out how to re-write a radio's firmware to create that elusive digital radio that can do more than one digital mode.  But that day may still come.  Software Defined Radio was likely a foreign concept to many 20 plus years ago when this problem was first brought to our awareness. The USRP, HackRF, HamShield, RTL-SDR, are known to many now, and having to have a hardware dongle to do the speech coding with those is illogical. 

The Need:

At this point, the AMBE used in D-Star is out of patent. Since there was never a published specification for it like its other variants, hams have thus far not been able to create a software version with similar performance.  (Good sounding software AMBE does exist for other modes)

Software AMBE is extremely handy for folks running cloud based conference bridges like DV Switch, or XLX Reflectors that allow hams to cross communicate between digital modes, promoting interoperability and cutting down on digital fragmentation .  It also can be used for internet connected apps (DroidStar) that connect to the RF gateways without need for hardware (MMDVM hotspots & HT's).

The original reverse engineering thread: https://forums.radioreference.com/threads/decoding-d-star-any-success.215282/

Its been said elsewhere that the suspected key to getting it better output is with the quantization tables:

// decode V/UV parameters
// load b1 from ambe_d 
//TODO: use correct table (i.e. 0x0000 0x0005 0x0050 0x0055 etc)

This likely can't happen at the very least unless the coefficients for encoding the specific D-Star variant of AMBE  are exposed in a software implementation somewhere like the one in the MD380 firmware.  The good news is it appears Travis and his team have begun working on that.  (reach out to BitBangingBytes, and or Abalamahalamatandra if you'd like to help with this)

In an ideal situation, the D-Star spec would have required publication of this like the APCO specifications did.  After the fact, I don't foresee any reason for the patent holder to release such details.  Another ideal scenario is if the FCC rules actually emphasized or required digital transparency, then not only would this enable those who want to learn and code/build, it would likely force working source code for AMBE to be published tomorrow.

It seems sad that various national amateur radio organizations + have not stepped up to support open source efforts.  So until then, please get a hold of me if you have the skills and desire to work on this for the benefit of the ham radio community and or join the OpenDV group.  I will ensure you are recognized for your efforts. - Steve, KB9MWR


Arm Chair Lawyer Notes:  

There are a number of people in the ham community that feel the need to play the part of an "arm chair lawyer."  I see this need to discourage from everything from tower climbing, FCC rules interpretation, to areas like this, concerning software patents.  Bruce Perens' who brought the potential patent issues years back to the ham communities awareness has actually created a negative stigma.  People (will) think these patents are still in effect well beyond their expiration.  Bruce's comments were well intended and more targeted for actual/potential software developers.  The fact is there is intellectual property cases that is in the courts for many end consumer electronic devices, mostly thanks to China, and patent trolling/hoarding. Outside of the ham word, end users likely don't concern themselves or give this any thought.  

--From Bruce Perens's AMBE DCC talk--

You've Already Paid Once...

Multi Band Excitation (MBE) was initially conceived in the mid-1980ís, at the Massachusetts Institute of Technology (M.I.T.)  Those speech models were developed by Griffin and Lim. The AMBE technology was developed under a United States Government grant though the Rome Air Development Center at Griffiss Air Force Base of the United States Air Force.  You paid for this technology with your taxes.  Publicly funded research should be open.

Doctrine of Laches: If you have a patent, and you know of an infringement but you wait for the market to get larger before you bring suit, that is reason your lawsuit to be denied. The usual duration for a Laches case is 5 years, although Laches cases have been win with as little a 1- year delay

Key patents were filed in April 2003, and could potentially be invalidated by the prior art of David Rowe, who wrote his thesis in 1997, and some of his work is from earlier than that.

Also, DVSI made use of the AMBE codec in commerce before some of their patent applications, potentially invalidating their own patents. There's one ham and intellectual property expert who feels that due to the date of first use in commerce, the important patent claims will expire in 2016.

Bruce Perens supposedly has a report from a patent analyst. It would be nice if he could share that.  There were about more than a dozen patents related to this stuff. However all but a one are expired, and that remaining runs till 2028. It's nearly impossible to tell what patents pertain to what, unless you are some sort of technical-codec lawyer with a high IQ.  But the short version is when it comes to patents, the rule of thumb is; Expiry: the longer of 17 years from issue date or 20 years from filing date.  All the patents for the version of AMBE used in D-Star expired back in 2017 per Bruce Perens (source).  The remaining patent concerns AMBE+2 used in DMR, Fusion, and NXDN. It was applied for in 2003, but was tardily granted granted in 2013, thus making that remaining patent for those modes expire in 2028.

The open source version of AMBE for D-Star, DMR, Fusion have existed since May 2010 as mbelib. All of them with the exception of D-Star were figured out from public TIA documents. (The APCO spec required a technical publication of the vocoder details). Some smart folks did there best a while later in 2013 to prove a point through a reverse engineering and guess-and-test, as the D-Star codec is completely undocumented.  But sadly it still sounds like it could use work when compared against the audio processed by the chip.

"If anybody wants to continue the research / work, I suggest you look at the osmocom GMR code that Sylvain  Munaut worked on. Those phones use a similar codec - I believe with longer frames for the satellite latency. Initially he used the mbelib code, enhanced it for things like tone support, but he later rewrote the synthesis code completely. See OsmocomGMR for his  presentations and source code." (per a author)

DSD+ or dsdplus - Is a closed source Windows fork of DSD that came about in December 2013. It caused the original DSD author to give up his development. (These DSD+ guys have gall, as you can pay for fast lane access to new features.)

If you really have to be arm chair lawyer like, here is the AMBE talk: K6BP "AMBE Exposed" from the 2014 DCC

I am not a lawyer, but I don't see anyone bothering you (haven't heard of it thus far) unless you are making a profit selling something that has makeshift AMBE under the hood. I even think Bruce eludes to this when he mentions doctrine of latches, etc.  However making use of mbelib /open source AMBE for anything other than casual home receiving is problematic until all the applicable patents are expired.  Obviously for most U.N. member states; non-commercial/research usage of patented technology is covered by exceptions on the definition of "patent infringement." Actual practice may differ. Ref

And on that note; In May 2018 Chinese AMBE (Team6160) started to appear on the market.  It's a WT3080 AMBE 3000 Compatible Device (it does use the 460800 baud rate). Apparently, this chip is loaded with the bootleg code that is used in some Chinese DMR radios.   (see Nanjing Wutong Microelectronics Center aka: http://www.indusic.com)


For Developers:

https://github.com/szechyjs/mbelib/commit/2656ec2202e15468f92dca0c226919166928c52e - This is the initial DSTAR AMBE implementation.  
https://github.com/nostar/dudestar
- DudeStar / DroidStar
https://github.com/f4exb/dsdcc - dsd rewrite - C++ library with a single decoder object
https://github.com/LX3JL/xlxd
https://github.com/szechyjs/dsd
https://github.com/szechyjs/mbelib
https://github.com/LouisErigHerve/dsd
http://git.osmocom.org/op25/tree/op25/gr-op25_repeater/lib - Op25 has encode and decode support for AMBE (D-Star, DMR and YSF) and IMBE (P25)
https://github.com/balint256/op25/tree/master/op25/gr-op25_repeater/lib/imbe_vocoder - Pavel's IMBE Encoder/Decoder Fixed-Point implementation 
http://git.osmocom.org/osmo-gmr/tree/src/codec 
https://github.com/on1arf/voice-ann
https://github.com/dl1bff/ircDDB-mheard/blob/master/ircDDB-mheard.c
- Line 383 - handles the bit interleaving and FEC processing
https://github.com/travisgoodspeed/md380tools/  - The MD380 Emulator (capable of AMBE encoding and decoding)
https://github.com/kawks/dstar-n800
https://github.com/g4klx/AMBETools

TIA-102.BABA - TIA Standard - Project 25 Vocoder Description
TIA-102.BABA-1 - APCO Project 25 Half-Rate Vocoder Addendum
TIA-102.BABC - TIA/EIA Standard - Project 25 - Vocoder Reference Test

GMR-1 Speech Codec
DV Dongle Technical Reference Manual
Decoding AMBE+2 in MD380 Firmware in Linux
AMBE_Exposed
md380tools - support group

Comparison Samples:

The following are created from the supplied DSDCCX discriminator output samples. The samples were taken at the output of the discriminator of the radio. (S16LE 48 kS/s single channel (mono) samples.)  The sample for each mode was passed thru mbelib, and a separate pass thru AMBE dongle to create the following side by side comparison files.

Syntax used:
sox -t s16 -r 48k -c 1 $INPUTSAMPLE -t s16 -r 48k -c 1 - | dsdccx -T3 -i - -fa -o - | sox -q -t s16 -r 8k -c 1 - $OUTPUT_mbelib.wav
sox -t s16 -r 48k -c 1 $INPUTSAMPLE -t s16 -r 48k -c 1 - | dsdccx -D /dev/ttyUSB0 -T3 -i - -fa -o - | sox -q -t s16 -r 8k -c 1 - $OUTPUT_dongle.wav
#-fa (DMR), -fy (YSF), -fd (D-Star)

dmr_it_8_dongle.wav
dmr_it_8_mbelib.wav

dstar_f1zil_1_dongle.wav
dstar_f1zil_1_mbelib.wav
dstar_f1zil_2_dongle.wav
dstar_f1zil_2_mbelib.wav

ysf_f5zoo_dongle.wav
ysf_f5zoo_mbelib.wav


+ Nov 1 2002, I had communications with the ARRL on the topic of bringing the ham populous up to date on the State of Amateur Digital Voice.  And that they should be pointing out there is ample experimentation and building in the open source software communities. And how thanks to these folks Amateur Radio is in fact vibrant and relevant. And that story just isnít being told.  Lets see if they ever address this via their own poduim (QST, social media etc).. Also brought up was reinstating the "Future Systems Committee", akin to the RSGBs "Emerging Technology Coordination Committee," to help foster communications between manufactures and the community etc,

A condensed version of this AMBE call for help may be found in the Autumn 2020 issue (#147) of TAPRís quarterly newsletter