Modifications for Retevis RT3 / Tytera MD-380 (?)

by Wolfgang "Wolf" Büscher, DL4YHF
Last updated 2017-03-05 (Morse output and software-controlled brightness).

Main website: www.qsl.net/dl4yhf
This page: /RT3/index.html

RT3 modifications

Disclaimer

Proceed at your own risk ! The modifications on this page may not work with your radio, and the findings described here may only apply to a single unit. The author won't be liable for anything at all, blah, blah.. (consider this as the usual disclaimer. It also applies to the description of firmware modifications, and to any link you may find on this webpage).

The Retevis RT3 is a monoband handheld transceiver for DMR (digital voice) and analog FM. It is almost identical to the Tytera MD-380. You certainly already know the radio (since you visited this page), where to find the programming software, and how to prepare a suitable "codeplug" (configuration) to operate via repeaters in your area.
If you don't, ask the big brother for "Retevis RT3" or "Tytera MD-380". Don't miss ham-dmr.de, the codeplugs by DK4WN, the descriptions by DG9VH, the introduction You and your MD-380 by AC2SN (known as ladyada), the programming software at Retevis and VA3XPR, and (for experienced users and developers) the 'Experimental' firmware by KK4VCZ.

Contents

1. Hardware Modifications

2. Codeplugs

3. Software / Firmware Modifications

  1. Save the 'factory calibration' with the Retevis / Tytera CPS
  2. Upgrading from Firmware "D002.xxx" to "D003.020"
  3. Installing KK4VCZ's 'Experimental' firmware
  4. Building KK4VCZ's MD380 Tools on windows
  5. Downloading firmware with the MD380-tools
  6. Updating 'USERS.CSV' and downloading it
  7. More firmware experiments ..digging in deeper
  8. Morse output as an aid for visually impaired operators

See also (related subjects on the author's website):
      MD380 firmware overview
      MD380 hardware details and measurements

"8.4 V" Battery Charger: Modified to avoid overcharging the LiIon battery

The RT3 was shipped with several accessories, including a charger for the radio's 7.4 V LiIon battery. That charger has a bi-colour LED on the front side, which should indicate 'red' while charging, and 'green' when fully charged.
According to the label, charging current should be around 400 to 450 mA. Thus a 2000 mAh LiIon accumulator (2 cells in series, thus really a "battery") should have been charged within 5 hours. My expectation was the LED would switch from red to green after this time.
But it didn't. In fact, the LED never turned green. After removing the radio from the charger, and checking the voltage directly at the battery terminals (see next chapter), almost 8.6 Volts were measured. Ooops ! That's a bit much for two LiIon cells in series !
In the interest of lifetime, the voltage for a single cell should never exceed 4.2 Volts. More voltage will not "fill in" significant extra charge, but reduce the battery's life time. So the charger was modified as described below, to achieve a 'healthier' max output voltage of 8.4 V (for two cells in series).
The principle is easy. The charger shipped with the RT3 uses an MC34063 in a switching-mode step-down converter configuration.
This IC uses an internal reference voltage of 1.25 V, which is compared against the charger's output voltage ("8.4" V) divided down to 1.25 V. The voltage divider consists of resistors R3A (33 kOhm) paralleled with R3B (1.8 kOhm), resulting in 1.707 kOhm, and R4 (10 kOhm) in series.

                                      
                                       ____
                      R4         ,----|____|----|
                      ____       |  R3A : 33 kOhm
      +"8.4" V  -----|____|------*
     (battery                    |     ____
      voltage)       10 kOhm     *----|____|----|
                                 |  R3B : 1.8 kOhm   
                                \|/
                              1.25 V (to MC34063 control input, pin 5)
                     
The regulator's theoretic output voltage with the original components (shown above) is

    1.25 V * ( 1 + 10 kOhm / 1.707 kOhm ) = 8.57 V .

This 'surprisingly high' voltage for a 2-cell LiIon battery pack was confirmed by measuring it at the battery after removing it from the charger !
Almost 8.6 V is quite a lot for two LiIon cells in series, especially if the radio was kept in the charger for longer than necessary. So how to reduce the charger's maximum output voltage ? The photo below shows the charger's PCB, component side, with the originally populated resistors (R3A, R3B, R4 labelled in white), and a few voltages measured when the battery was fully charged (but the LED was still 'RED' when it should have been 'GREEN').

   
    (Click on image to magnify)

A simple method to bring down the maximum output voltage to 8.4 V is shunting R4 (10 kOhm 'original') with a 390 kOhm resistor.
The 'paralleled' resistance is
    1 / ( 1/10kOhm + 1/390kOhm) = 9.75 kOhm,
so with the modified voltage divider, the theoretic output voltage should be:

    1.25 V * ( 1 + 9.75 kOhm / 1.707 kOhm ) = 8.39 V .

That's 4.19 Volts per LiIon cell - ok. And indeed, with this modification, the battery stopped 'drawing current' when fully charged, and the LED turned green after a couple of hours (btw, that's what the LM358 is for, visible in the right half of the photo above). Voila ! An even simpler alternative (instead of 'shunting' R4) would be to remove the 33 kOhm resistor (R3A in the photo shown above).
The theoretic output voltage would then be:

    1.25 V * ( 1 + 10 kOhm [R4] / 1.8 kOhm [R3B] ) = 8.19 V .

The two-cell LiIon battery would possibly not be "fully" charged very quickly with this voltage, but forgetting to remove the radio from the charger would do even less harm then.
As an indicator for a "charged" battery, if you also don't trust the on-screen battery indicator symbol: Six hours after a full charge, without load, the battery voltage was 8.10 V. It dropped by 4 mV after turning the radio on (circa 70 mA load current), thus the internal resistance of 'new' battery (as shipped from the manufacturer, 2016-12) was about 0.06 Ohms - will keep an eye on this as time passes.

Exposed 'hot' LiIon battery contacts


The LiIon battery's charger contacts can 'provide current' during normal operation !

Both the LiIon battery, and the charger appear to have THREE CONNECTORS, but the center connector isn't connected anywhere (at least, not inside the cheap charger).

There doesn't appear to be a 'smart' protection circuit inside the battery to protect it from excessive loads.
  THIS APPLIES NOT ONLY TO THE CONNECTORS THAT 'FEED THE RADIO' (not exposed),
  BUT ALSO TO THE CONNECTORS FOR THE CHARGER,
  which are exposed when operating the radio !

Remember this when dropping the RT3 (or MD-380) into your pocket along with the bundle of keys, or other metal objects
- a short across the battery's outer, exposed contacts will possibly ruin it, or even worse.


I didn't want to try how many amps can be drawn from the exposed contacts on the backside of the transceiver. But as you can see on the right, it's enough to power an ancient filament lamp.



Dimmable LCD- and keyboard backlight

Please see the notes at the end of this chapter -
    the backlight intensity is adjustable via software now !

This modification was once done 'in hardware' as described below. If for some reason you need to stick with Tytera's original firmware, it may still be useful for you - thus the following chapter was not removed.

With Tytera's firmware, the backlight was turned off completely after a programmable timeout. That's not nice, because when this purely transmissive display isn't illuminated from the background, it's completely unreadable.
A 'permanently' lit display, with reduced intensity but still easily visible, was initially realized with a simple 220 Ohm resistor connected between collector and emitter of the backlight-switching transistor on the display-/"key"-board.
For reasons only the developers of the original firmware will know, the LCD (TFT) itself is not powered down. So it makes perfect sense to keep the backlight on, "deeply dimmed". The modification shown below is made on the display board, not on the CPU/RF board:

           
    Left: Display PCB.   Middle: Original backlight switch.  Right: Modified backlight for "continuous ambient light".
    Click on any image to magnify.

The blue arrow points to the transistor's collector. The thin wire on the right photo connects the upper end of the 220 Ohm (SMD) transistor to the 3.3 V supply for the display. When the backlight is on, there are 3.3 Volts on the collector. When off, 0 Volts. The forward voltage of the white LEDs is somewhere around 2.9 V, so the 220 Ohm resistor feeds as continuous current of less than 2 mA to the LEDs, which was enough to read everything on the display in a darkened room (or outdoors). Even the keys were very dimly illuminated - barely visible but enough to operate them in complete darkness - nice ! The additional power consumption from the battery is neglectable.

Note for developers: The backlight is controlled via pin PC6 by the CPU (STM32F405). This pin can possibly be reconfigured as PWM (pulse width modulator) instead of GPIO, but at the time of this writing it was unknown if Timer8 or Timer3 are still available for this purpose. Having the backlight completely software-controllable (with different intensity settings for "idle" and "active") would be is great...

Update on the display backlight (2017-01-08)
There are at least two methods for a software-based 'permanent background illumiation' now !
One by Clay, KK6RWR, on github.com/claywalker0929/md380tools,
and one by myself, which can be downloaded from here (right-click and "save as",
then unpack and flash 'experiment.bin' with Tytera's Upgrade.exe or the MD380-tools. ONLY FOR RADIOS WITHOUT GPS SO FAR, i.e. RT3 / MD380.) .
For some reason, the dimmed backlight only worked when the 'Intro Screen' (menu 'Utilities'..'Radio Settings') was set to 'Char String', not 'Picture' !

A short, bilingual description about the backlight settings is in the zipped archive, and here /
Eine kurze, zweisprachige Beschreibung der Beleuchtungfunktion gibt's im Archiv, oder auch hier .
Aus noch zu 'erforschenden' GrЧnden funktioniert die gedimmte Beleuchtung nur, wenn der 'Intro Screen' (im MenЧ 'Utilities'..'Radio Settings') auf 'Char String' (nicht auf 'Picture') gesetzt ist !


The firmware in dl4yhf_md380_firmware_experiment.zip/Firmware_XYZ.bin has a few new items in the setup menu:
Press the 'Menu' key, select 'Utilities', 'MD380Tools'.
scroll down to 'Backlight', then press 'Confirm', and set 'Level Low' / 'Level High' to one of the following:
  0 = backlight completely off after the configurable time (only for 'Level Low', without activity)
  1 = lowest possible setting, display still readable at night
    ...
  9 = maximum possible brightness (don't use this when running on batteries !)
This sets the brightness used when the radio is idle (no activity for X seconds).
The value of 'X' (in seconds) can be configured in Tytera's 'Radio Settings' menu (not under 'MD380Tools').

'Level High' cannot be set to 0 (zero brightness), because without backlight you can't navigate through the menu because the screen is unreadable.
Usually, you will set the 'High' setting higher than the 'Low' setting, but that's up to you.

Last modifications of this temporary branch (not a permanent fork) of the MD380-Tools:
2017-02-11:
Dimmed backlight merged into md380tools-master (thanks Travis!), the temporary branch (below) was deleted. It uses the UART to produce PWM, to keep the CPU load as low as possible. See current measurements here.

2017-01-10:
A temporary branch of the MD380-Tools ('DL4YHF-Backlight') was online at GitHub .
This should fix the problem 'no backlight on power-on'. It happened when the backlight intensity configuration, stored in SPI flash, contained 'zero'.
In the first 'alpha' variant this could be fixed by entering the setup menu with the aid of a strong flashlight(!), and setting the 'Backlt High' parameter to something nonzero.
A new binary (experiment.bin) is only in the zip file on this page (see above), not in the Github repo.

2017-01-12:
On some radios, the UART-based 220 Hz PWM didn't seem to work (backlight only on or off but not dimmable).
To find the reason, dl4yhf_weak_backlight_experiment.zip now also contains an older development step (experiment_PWM_via_GPIO.bin) that only generates a 74 Hz PWM via software.
Please test and post your results (either 'positive' or 'negative') on the md380tools group, topic 'Permanent "weak backlight" via PWM' !

This 'temporary' branch will be merged with Travis' master when 'beta testing' is finished.

Codeplugs

You will find hundreds of codeplugs for your DMR radio.. so ask the big brother for one in your area.
Below is "mine" (well, not really .. it was copied from DK4WN, and tailored for this area).
Only meaningful for visitors from East Westfalia, Germany, in the area of Herford, Spenge, Enger, Bielefeld; thus the rest of this chapter in German:

Der folgende Codeplug basiert auf einem "VFO-Сhnlichen" Codeplug von DK4WN.
HinzugefЧgt wurden einige Analog-Relais (DB0BI, DB0OWL, DB0KB) und beide Zeitschlitze von DB0HFD.
Diese Frequenzen sind zwar ohnehin (mit der Frequenz als Kanalnamen) bereits vorhanden, aber so werden sie in der "Home-Zone" (OWL) im Klartext angezeigt. ZusСtzlich zur Zone "OWL" gibt's auch eine Scan-Liste "OWL", in der nur die oben aufgefЧhrten, aus "jeder Lebenslage" mit der Gummiwurst zu arbeitenden Repeater enthalten sind.
In der Zone "OWL" bedeuten.. Bitte nicht vergessen, den EIGENEN DMR-ID in der CPS (Customer Programming Software von Retevis, "CPS_DMR.exe") einzugeben !


(zum VergrШ▀ern in's Bild klicken)
Wenn im GerСt bereits die 'experimentelle' Firmware geladen ist, geht das auch ohne PC unter
  "Menu".."Utilities".."MD380Tools".."Edit DMR-ID".


RT3 Firmware modifications


3.1 Save the 'factory calibration' with the Retevis / Tytera CPS

Each radio is 'calibrated' at the factory, including (but not limited to) modulator drive levels, and I/Q modulator and demodulator balance. This calibration is important to keep the emission of unwanted signals as low as possible. Calibration data are stored in a non-volatile memory. The entire set of 'individual calibration data' can be read out using Retevis' or Tytera's original CPS, and stored in a safe place. Don't use the calibration data from a friend's radio if you lost yours !.
The 'Test Mode' must be enabled in the CPS by editing 'settings.ini', located in the CPS directory. Modify the line
  testmode=0
to
  testmode=1
then restart the CPS. When connected to the radio via USB, press Control-T. This opens a tabular display with all calibration values (click on image to see mine, but use yours, and don't modify anything unless you know exactly what you're doing):
Click on 'Read' to update the data (read from radio), then 'Save Test Data' and store your calibration as a file in a safe place. On that occasion, also save two screenshots from the 'Test Mode' table (scroll down to the end to see the rest of the I/Q modulator calibration data, e.g. 'Analog Send I Range', 'Analog Send Q Range'). In the author's radio, there were even different values for different frequencies.
If the radio's calibration gets lost (which hopefully never happens, when playing with the experimental firmware), you can hopefully load this file again, and write it back into the radio with the original CPS. As a ham radio operator, you are responsible for the emissions of your radio, and any responsibility of the manufacturer has already ended when we opened the radio or modified the firmware !

Related subject: Inspecting C5000 register values with the MD380 tools

3.2 Upgrading from Firmware "D002.xxx" to "D003.020"


Besides saving the 'factory calibration', before playing with the 'experimental' firmware (see next chapter) and alternative programming software, it's necessary to find out what's originally used in the radio (and which "Vocoder" (*) is used in the hardware).
The rig shown above (bought on Ebay in december 2016) showed the following (select 'Menu' .. 'Utilities' .. 'Radio Info' .. 'Versions') :

    Radio Info   
 Firmware Ver. : 
     D002.034    
 CP Ver. :       
       V01.32    
According to the list at VA3XPR, a radio with this firmware is compatible with the 'TYT Firmware' D003.020 (for radios with firmware version D002.XXX and D003.XXX). That's the recommended 'normal' firmware for this radio, seen in December 2016 on the VA3XPR website.
Before 'diving in too deep' (with Travis Goodspeed's 'Experimental' firmware), the old Retevis RT3 firmware Version "D002.034" was replaced by the Tytera MD-380 firmware (downloaded as 'TYT-MD-380-Firmware-v3.20-old-vocoder.zip' from the VA3XPR site). The zipped file not only contains the firmware image, but also a windows executable to actually perform the download. There are a few notes which you must read before updating the firmware (several users have already 'bricked' their RT3's or MD380's by flashing a firmware that wasn't compatible with the target hardware, and/or the vocoder (*) used in their radio). Quoted from 'updating.doc' in the zipped archive:

 MD-380 has two versions, new vocoder and old vocoder, so please check version first in the radio,
 check menu-utilities-radio info-versions and if you find the version is D002 and D003,
 it should (be) the old vocoder and the latest firmware is MD380-D3.20,
 while if the version is D013, the firmware should be MD380-D13.20.
 
From the 'Versions' screen shown above, it was obvious that the author's Retevis RT3 contains the 'old vocoder' (*). This may be important later, when checking out Travis' 'Experimental' firmware.
When launching 'Upgrade.exe' (unzipped from TYT-MD-380-Firmware-v3.20-old-vocoder.zip), we're prompted with this scary dialog:

Umm. What next... "Open Update File" or "Open Code File" ? The answer is in "Operation.doc". That document also explains which buttons to press (on the radio) during power-on to switch it into bootloader-mode:
>
> Press side key 1 and PTT key to turn on the radio, the indicator blinks and open upgraded software .
>
Indeed, the indicator LED near the volume knob now flashes RED / GREEN. So proceed as explained by the screenshots in "Operation.doc" (the button to click next is "Open Update file", not "Open Code file").
In this case, the only selectable file was "TYT-TFT-MD380-D3.20.bin", so there's not much doubt about which to use.
Next, click "Download Update File", and keep fingers crossed that all work as planned...
Despite the alarming window title with "Keine RЧckmeldung" ('program doesn't respond'), the author was finally greeted by this:


Guess this message means something like "Success!" in Chinese.
Sigh... take a deep breath in, and restart the radio (power-off, and power-on again).
Select 'Menu' .. 'Utilities' .. 'Radio Info' .. 'Versions' again, et voila:

    Radio Info   
 Firmware Ver. : 
     D003.020    
 CP Ver. :       
       V01.32    
A quick test on the local DMR repeater confirmed the "Tytera" firmware (D003.020) co-operates with the RT3's "old" vocoder (*).
The RT3 was ready for the next step: Install Travis Goodspeed's 'Experimental' firmware... but that's another story.

3.3 Installing KK4VCZ's 'Experimental' firmware

If you want the latest version, you need to compile it yourself. The master is available at github.com/travisgoodspeed/md380tools. Quite new 'compiled binaries' used to be at github.com/roelandjansen/md380tools/releases. For german users, the introduction by DG9VH appeared to be the best startpoint.
For others, there's a step-by-step guide at learn.adafruit.com/tytera-md-380-dmr/updating-md-380-firmware.
Last not least, there's an active group of developers at groups.google.com/forum/#!forum/md380tools.


   i looked at the building steps and ran home screaming for mommy

Patient and brave users (familiar with Linux on a PC or Cygwin on a windows machine) will start with the MD380 USB Tools's README. But being impatient, lazy, or both, instead of installing the entire toolchain (to install the prerequisites and compile the 'Experimental' firmware, pulled from github), I first tried an already compiled binary found at
www.darc-saar.de/moziloCMS/index.php?cat=Relais&file=md380tools.zip.
That archive contained a file named 'firmware-2016-08-12.bin', which was flashed into the RT3 using the same firmware-"upgrader" as in the previous chapter:


The scary error message "Keine RЧckmeldung" (program doesn't respond) appeared a few seconds later again, and the progress bar stopped when only "half done", but don't panic ... after some time, the already familiar 'Ey1/4..|!' / "OK" - message from 'Upgrade.exe' popped up again.
After restarting the RT3, the 'special Saar edition' of KK4VCZ's 'Experimental' firmware identified itself as follows:
(it's still under 'Menu' .. 'Utilities' .. 'Radio Info' .. 'Versions'):

    Radio Info    
 MD380Tools Ver.: 
     2016-08-12   
 CP Ver. :        
       V01.32     

The 'experimental' firmware obviously replaced the old 'firmware version' with its own compilation date (in the congenial ISO8601 format) - very nice !
Again, it could be confirmed (on the local DMR repeater) that the RT3's "old vocoder" (*) is still supported by this firmware.
(*) Old vs new "Vocoder" ?
It later turned out that what the Retevis firmware notes call "vocoder" isn't an AMBE 'vocoder' chip, since there is no Vocoder hardware (chip) in this radio. The vocoder, which converts human voice into an amazingly low number of 'bits per second' (and back), is part of the firmware that runs in the STM32F405 CPU. Travis even managed to run this software vocoder on a Linux machine (with Cortex emulator), but that's another story.
So what the firmware compatibility notes (by Retevis) call "new vocoder" and "old vocoder" must be something else..
maybe just a 'copy protection' for the patent-encumbered AMBE-core (?).
See also: Travis' clarification about "which firmware to download" at github.com/travisgoodspeed/md380tools/wiki/Download-Firmware .
> If you install the wrong one, you will get a blank white screen at startup.

What was still missing was the DMR user database (users.cvs), loaded into the radio's serial flash chip (W25Q128FV, 16 MByte).
At this stage, it's time to 'start playing' with a more up-to-date version of the patched/experimental firmware. So install the MD380 Tools on a PC...


3.4 Building the MD380 Tools on a windoze PC (yet another way to get there..)


Since no Linux was installed at the 'workhorse' (PC), I decided to follow the steps explained by Limor, AC2SN (known amongst microcontroller enthusiasts as Ladyada) - see
  learn.adafruit.com/tytera-md-380-dmr/updating-md-380-firmware.
I later used another toolchain due to strange problems with 'Cygwin'.. details on that below.
Here just a short list of what you will need, and where to find the prerequisites:
To make sure the scripts, the Unix-like commands, and the cross-compiler for Cortex / ARM can be properly invoked, check the 'Path' environment variable. It should contain something like the following (but often, it contains a lot more):

C:\cygwin\bin;C:\tools\gcc_arm\bin
(On a german PC, the windoze "Path" variable can be checked and modified in the windoze system control:
  "Start".."Systemsteuerung".."System".."Erweiterte Systemeinstellungen".."Erweitert".."Umgebungsvariablen". Aaargh !!)

So back to this chapter's subject: To build (compile, link, controlled by make) the MD380 Tools, enter the following command in your favourite Unix-like shell. I assume everything is installed in the directories mentioned above.
We're going to let Git(!) clone the md380tools repository to our harddisk. This avoids a few headaches later, when modifying the firmware for our own purpose.
Wolf@YHF6 ~
$ cd c:/tools

Wolf@YHF6 /c/tools
$ git clone https://github.com/travisgoodspeed/md380tools.git  # Let's try this ONCE..

git: bash: git: command not found      -> Git doesn't seem to be installed;
                                        let's check if MinGW/MSYS knows "where it is" :
Wolf@YHF6 ~
$ where git
INFORMATION: Es konnten keine Dateien mit dem angegebenen
Muster gefunden werden.  # Before the next step, (re)installed Git for Windows v2.10.2 . Then try again:

Wolf@YHF6 ~
$ where git
c:\tools\Git\cmd\git.exe
c:\tools\Git\mingw64\bin\git.exe  

Wolf@YHF6 /c/tools
$ git clone https://github.com/travisgoodspeed/md380tools.git  # only do this ONCE !
Cloning into 'md380tools'...      # no sign of life for several minutes from MinGW

Wolf@YHF6 /c/tools
$ cd md380tools           # this is the directory created and filled by 'git clone'

Wolf@YHF6 /c/tools/md380tools
$ make clean dist  # downloads a lot from the internet, so only do this when 'necessary' (once)
 ...(over 1000 lines of text emitted by 'make' here)...
Wolf@YHF6 /c/tools/md380tools
$
The shell window will be filled with a lot of text. Watch for 'error' or 'fatal' messages.
Some of the errors seen during the first attempts to "make clean dist", and how to fix some of them:
/usr/bin/env: python2: No such file or directory
Reason: The makefile tries to invoke python2 which didn't exist.
First tried to fix this with an 'alias':
alias python2="python"
This didn't work when "python2" was invoked from make, so
c:\tools\Python27\python.exe was simply duplicated as c:\tools\Python27\python2.exe .

zip -r dist/md380tools-`date "+%Y-%m-%d"`.zip dist/md380tools-`date "+%Y-%m-%d"`
/bin/sh: zip: command not found
make: *** [dist] Error 127
Fix: Install the 'zip' command for shell: For MinGW, let its installer install
package 'msys-zip' (ZIP compression utility).
For unknown reasons, this package (zip-3.0-1-msys-1.0.14-bin.tar.lzma) disappeared from the MinGW Installer's list some day. After re-installing everything:
$ where zip → C:\MinGW\msys\1.0\bin\zip.exe . Ok again.

make[2]: Entering directory '/cygdrive/c/tools/md380tools/applet'
fatal: Not a git repository (or any parent up to mount point /cygdrive/c) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Ignore this (and similar Git-related errors) for now, since we're not going to create a new Git repo from our modified files yet.
We'll possibly get back to Git-related trouble in future.

"make" -C applet clean
fatal: Not a git repository (or any of the parent directories): .git
Ignore this, or pull the repository via 'git clone' (see next Git-related 'fatal' message).

"make" -C applet FW=D13_020 all
fatal: bad revision 'HEAD'
More git-related trouble. Should not happen after pulling the md380tools via 'git clone' as explained further above.
Ignore this as long as you don't want to commit or "fork" your own modifications.
You can fix this yourself after studying this book ;o)

DEBUG: reading "bin/S013.020.bin" (...)
WARNING: Funky header: (...) Trying anyway.
DL4YHF: No idea what this means, and I didn't feel like finding out myself.
Complete output from 'make clean dist' saved as 'make_clean_dist_output_2016_12_30_after_GIT_CLONE.txt' .

Within a minute or two, "make clean dist" produced a new file 'md380tools-YYYY-MM-DD.zip' in the 'dist' folder (YYYY-MM-DD = date in ISO-format).
The "fresh" compiled and linked firmware image will be in applet/experiment.bin .
To test it, download that file into the radio as already described ("Upgrade.exe", turn radio on with PTT and the function key above pressed, connect USB, "Open Update File", "C:\tools\md380tools\applet\experiment.bin", "Download Update File") ...
If we haven't messed up the compilation, the radio reboots (after turning off/on) with the screen contained in the md380tools.
The date of compilation is shown on the 'welcome' screen after power-on. Most changes are in the 'Utility' menu, under 'MD380Tools'.

Curious people will like to see the C sources (interesting parts are in applet/src) - see 'More firmware experiments', and subsequent chapters.


3.5 Downloading new firmware with the MD380-tools

Being fed up with switching the USB-drivers back and forth (as explained here), I tried to download the freshly compiled firmware via md380tools.
The radio must be turned on in bootloader-mode for this purpose (LED blinking red/green).
In this mode, the 'TestLibUsb' utility identified the radio as follows:
bus/device  idVendor/idProduct
bus-0/\\.\libusb0-0001--0x0483-0xdf11     0483/DF11
- Manufacturer : AnyRoad Technology       (!)
- Product      : Digital Radio in USB mode
To update the firmware without rebuilding it, use this command (also works in window's "cmd.exe"):

Wolf@YHF6 /c/tools
$ python md380_dfu.py upgrade dist/md380tools-2017-01-05/firmware-2017-01-05-NoGPS.bin
  (or, later, when experimenting with own firmware extensions, after 'make clean image_D13' ):

$ python md380_dfu.py upgrade applet/experiment.bin
Beginning firmware upgrade.
Writing firmware:
0% complete ......
100% complete, now safe to disconnect and/or reboot radio
Important: Pick the suitable firmware for your radio, from the recent 'dist' folder. These files begin with a certain signature (something like "OutSecurityBin  JST51", can be seen with a hex file viewer). Anything without this signature will not work.
These *.bin files have been created from the 'merged' original firmware and the patched parts as roughly described below:
After flashing, turn the radio off and on again. If it doesn't come up with the boot screen within a second, there's something wrong, quickly turn it off again (who knows what the CPU is doing when 'running in the wild'). Then, try again as explained above (this time with the last tested firmware).
During the author's own additions to the firmware, an RT3 didn't want to start a dozen times, but when activating the bootloader the LED always began to blink RED / GREEN again (i.e. "bootloader still alive"), and the radio could always be 'unbricked'.

3.6 Updating 'USERS.CSV' (the DMR-ID database) and downloading it


The 'experimental' firmware has a feature to show the callsign and name of the transmitting station on the radio's screen.
Note:
This is completely different from looking up the DMR-ID in the normal 'Contacts' list, as most DMR radios do.
The 'Contacts' list, stored along with the 'Codeplug' data, would be limited to a few hundred entries.
The UHF versions of the MD380/RT3/etc have plenty of space in their serial Flash (16 MByte) to store the entire database of registered DMR users (over 49000 entries at the time of this writing). The firmware looks up the received DMR-ID in its Flash memory, and shows the callsign and operator name on the display beside the numeric ID.
The process of updating the database from the internet, and downloading it into Flash memory may be cumbersome - especially under Windows..

3.6.1 Updating USERS.CSV under windows

Details on how to get this job done under Windows were difficult to find (in 2016-12 on github/../README.md), but after collecting bits and pieces of information, it worked (details further below).
Note:
You must have the MD380 Tools installed on your system, including a functional Python / PyUSB / libusb to "talk to the radio", as explained in the previous chapter. The patched firmware can be downloaded with the "Upgrade.exe" utility by Tytera, but not the DMR user database.
Ed N6YN posted the following fix in the google group:
With either patched firmware v2.032 or D13.020 installed in your radio...
run Zadig, and install libusb-win32 (v1.2.6.0) or (libusb-win32 from:
https://sourceforge.net/projects/libusb-win32/ as described in MD380tools / Windows-instructions.txt)

Once the driver is installed you will see that if you issue a
   ./md380-tool spiflashid
command, you will received either a # 1 or #2 from the above responses.

Now, execute the "testlibusb-win.exe" file, then click on the Refresh button at the bottom of the popup window.
(Note this file is located in the libusb-win32-bin-1.2.6.0\bin\x86 folder).
Once you click on the Refresh button you will find that if you again issue a
   ./md380-tool spiflashid
command in the MINGW32 window,
you will receive the proper response from the radio:
      SPI Flash ID: EF 40 18
      W25Q128FV 16MByte
Now you can issue a 'make flashdb' and your CSV file will be properly installed into the SPI Flash.
(remember to perform a "make clean all" in your md380tools/db folder, to update the "users.csv" file)
We will use this trick a couple of times later, when running the MD380-tools under window. The above (testlibusb-win.exe) will then be invoked from the commmand shell to 'tickle the USB interface' before accessing it from Python.

Details about installing libusb_win32, PyUSB, and run "Python with USB" in a command shell on windows, are in an extra page on this site
- see Wrong SPI Flash ID, or "how to transfer the DMR User Database into the radio on Windows".


Wolf@YHF6 ~
$ cd c:/tools/md380tools
         # before the next step, turn radio on and connect USB, and let 'TestLibUsb' run (under windows)
Wolf@YHF6 /c/tools/md380tools  
$ make flashdb
"make" -C db stripped.csv
make[1]: Entering directory `/c/tools/md380tools/db'
touch custom.csv
python2 get_special_IDs.py
Fetching list of special IDs from BM master servers.
  .....
./md380-tool spiflashwrite user.bin 0x100000
SPI Flash ID: ef 40 18
W25Q128FV 16MByte
erase 2662838 bytes @ 0x100000
flashing 2662838 bytes @ 0x100000
reboot radio now

Wolf@YHF6 /c/tools/md380tools
$

Because the above script in the MD380 Tool's makefile downloads this database from multiple sources in the internet, it's much easier to keep this list up-to-date than editing your 'Contacts' list (which is the usual method on other DMR radios to display callsign and name besides the DMR-IR).


3.6.2 Using a customized USERS.CSV / user.bin

In a few cases, new registered users appeared 'quite late' in the automatically generated database. To add them 'manually', load the file 'stripped.csv' (located in md380tool/db) into a good text editor like Notepad++ (NOT the normal 'windows notepad', which destroys the Unix line ending format, or doesn't display such files properly). Insert new items, but keep the long list sorted by DMR-ID. Then save the modified database under a new name 'my_user_db.csv', to prevent it from being overwritten when rebuilding the md380tools the next time.
Next, invoke the following commands, which are based on the automated 'make flashdb' (which we don't use here because we don't want to use the automatically downloaded database but "our own"):

Wolf@YHF6 ~
$ cd c:/tools/md380tools

Wolf@YHF6 /c/tools/md380tools
$ wc -c < db/my_user_db.csv > user.bin

Wolf@YHF6 /c/tools/md380tools
$ cat db/my_user_db.csv >> user.bin

Wolf@YHF6 /c/tools/md380tools
$
This creates a new file 'user.bin', from our manually edited database. It contains the size of database (in bytes) as a decimal number in the first line (that's what "wc" does: count characters), followed by the database itself (appended with the "cat" command). Download 'user.bin' into the radio's serial Flash memory as already explained.
Remember to switch to a suitable libusb driver first, and 'tickle' the USB port by running testlibusb-win.exe before launching 'spiflashwrite' - if that's still necessary with the current md380tools:
Wolf@YHF6 /c/tools/md380tools
$ c:/tools/libusb-win32/bin/amd64/testlibusb-win.exe  # DL4YHF tickle USB, must recognize "Patched MD380"
                                                      # before we continue with the next command !

Wolf@YHF6 /c/tools/md380tools
$ python md380_tool.py spiflashwrite user.bin 0x100000
SPI Flash ID: ef 40 18                                # anything else here may indicate a problem
W25Q128FV 16MByte
erase 2618720 bytes @ 0x100000
flashing 2618720 bytes @ 0x100000
reboot radio now

Wolf@YHF6 /c/tools/md380tools
$

See also: How to switch back to the default USB driver (to use the original Retevis / Tytera CPS again).

3.7 More firmware experiments (2016-12-30)

As already mentioned somewhere, the 'Babun' shell could successfully run all the scripts to build the patched firmware, but not download the DMR database into the radio, because PyUSB didn't manage to load any of the libusb-DLLs ("No backend available", details here).
Only the simple command shell from MinGW32 could immediately 'talk to the patched firmware' via USB. So for most of the 'firmware experiments' described in this chapter, I used the MinGW32 shell (besides an IDE for embedded software development which I am familar with from the QRL, but that's not required to compile and link the modified firmware).
If you already installed MinGW32 as suggested here (into C:\MinGW), start the shell as command (on 'german' windows: "Start", "AusfЧhren"):
  C:\MinGW\msys\1.0\msys.bat
(launches the Linux-like shell). If that doesn't work, check if 'MSYS' is installed via MinGW Installation Manager:
  C:\MinGW\libexec\mingw-get\guimain.exe
(install "MSYS", along with the Bash package (Bourne Again Shell) and its documentation).
The Bash-like MSYS/MinGW32 shell identifies itself as 'MINGW32:', followed by the current working directory in the window title. It helps to increase the 'window buffer size' to 220 characters per line and 2000 lines, so the shell window can be scrolled back far enough to look for errors and warnings.
In the following chapter(s), text with a black background was copied from the shell.
Commands entered there are sometimes marked in cyan colour (manually, only in this HTML file).
Blue characters are comments or notes, describing what the next step is supposed to do.

Wolf@YHF6 ~
$ cd c:/tools/md380tools

Wolf@YHF6 /c/tools/md380tools
$ make clean
Note: We don't use 'make clean dist' here, since we don't want to make a new 'distribution'. Furthermore, 'make' (precisely: md380tools/Makefile) is smart enough to not download the original firmware images if they already exist.
To compile all sources (including those which have not been edited, use 'make all'.
Again, scroll back in your favourite command shell, and watch for errors from the C compiler.

   .....
../../md380-gfx --firmware=patched.img --gfx=0x80f9ca8-poc-gs.ppm relocate
DEBUG: reading "patched.img"
0x80f9ca8 160?щ40
[[253, 255, 252, 0], [216, 218, 215, 0], [26, 27, 25, 0], [195, 197, 194, 0], [0,
], [99, 101, 98, 0], [68, 70, 67, 0], [114, 116, 113, 0], [148, 150, 147, 0]]
?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?└?   
The above output when converting "patched.img" with the PoC-or-GTFO logo looked bad, and very different from Babun's "zsh":

../../md380-gfx --firmware=patched.img --gfx=0x80f9ca8-poc-gs.ppm relocate
DEBUG: reading "patched.img"
0x80f9ca8 160О40
[[253, 255, 252, 0], [216, 218, 215, 0], [26, 27, 25, 0], [195, 197, 194, 0], [0, 2, 0, 0],
иииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииии
иииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииии
иииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииии
иииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииииии
иииииииииииииииииииииииииииииииииииииииииииииииииииииииииииии122иии322иииииииииииииииииииии
иииииииииииииииииииииииииииииииииииииииииии131иииииииииииииии3445ии644иииииииииииииииии115и
иии777772222289aииииииииииииииииииииииииb7444448aии47ииииииии3445ии644иииииииииииии39444444
иииcd444dccc824441ииииииииииииииииииии52442e15e44e948ииииииии3445ии644ииииииииииииe444865a7
иииии7445иииииa4446ииииииииииииииииии1444fиииии574448ииииииии3445ии644иииииииииииd4475иииии
иииии7445ииииииf442иииииииииииииииии5444aиииииии54448ииииииии3445ии644ииииииииииe448иииииии
иииии7445иииииии4443ииииииииииииииии944bиииииииии3448ииииииии3445ии644ииииии
Something to worry about ? No. "md380-gfx", which is now implemented in applet/ppm2h.py, spits out a few UTF-8 characters that 'Babun' understands, but not MinGW32's default shell. To fix this cosmetic aspect, replace MinGW's default shell by the UTF-8 capable 'mintty' as explained here. Then, in the menu under "Options".."Text", set Character Set to UTF-8.

As a C developer, for a few quick experiments, it's tempting to simply modify a few sourcefiles (mostly in 'applet/src', some in 'playground/src'), rebuild everything, and try if it works (don't expect you can debug on the target - your search for a standard JTAG- or SWD- connector in the RT3/MD380 will be in vain..). But consider this: The MD380 Tools are a 'River of Constant Change' - you may inadvertedly overwrite 'your' *.c file when unzipping the next version from Github ! You don't want to commit+push a half-cooked egg on Github. There are advanced ways to get around this (locally, using Git). Anyway I (that's DL4YHF) decided to give modified C-modules a new name, or at least move them to a directory that doesn't conflict with the existing MD380tools directory structure.
The firmware building process is quite different from 'classic' software development. Instead of letting the linker 'decide' on which address a function, variable, stack, heap etc shall be located (possibly aided by a scatter-loading file, as my favourite development system likes to call it), the patched RT3 / MD380 firmware must co-operate with the original firmware, which still resides at fixed addresses (this especially applies to the HR_C5000 chip interface and the software-based AMBE-compatible vocoder).



To get familiar with the firmware modules (written in C), a short overview was written in plain text. This was later converted into HTML, with links to the C source.
Right-click on the following link, and 'save target as' md380_fw.html in the 'docs' directory of your md380tools. Don't just "follow the link" below and save the page after it was loaded into the browser - at least Firefox stupidly converted the relative links inside the file to absolute ones (directing to the URL from where the page was downloaded), which renders the purpose of the 'links to the C files' (stored on your harddisk) useless.

    md380_fw.html

(in December 2016, the 'master' of the above file was located at http://www.qsl.net/dl4yhf/RT3/md380_fw.html )


3.8 Morse output for visually impaired operators (2017-03)

At the time of this writing (2017-03-10), the Morse output is far from being perfect. For example, it fails to retrieve certain menu items, and sometimes interferes with message beeps generated by the original firmware.

But for visually impaired hams, it may already be useful as-is.
The "narrator" (in German: "Vorleser") reports..
As most firmware extensions, Morse generator and narrator are configured in the MD380Tools menu:
      Menu      
  Contacts
  Scan
   ...
  Utilities     
    Utilities   
  Radio Settings
  Radio Info
  Program Radio
  MD380Tools    
   MD380Tools   
   ...
  Side Buttons
   ...
  Morse output  
 Morse output   
  Mode          
  Speed [WPM]
  CW pitch [Hz]
  CW volume

MD380Tools menu with configuration of the Morse 'narrator'
      Mode      
  off
  short
  verbose      
  test/future
    
   Speed [WPM]  
  15
  18            
  22
  30
    
  CW pitch [Hz] 
  400
  650           
  800
   ?
    
   CW volume    
  low           
  medium
  high
  (automatic?)

When switching from one menu to a submenu (via the green 'Confirm' key), the narrator doesn't repeat the menu titles (shown with a gray background above), because the title of a submenu is the same text as in the item from where it was invoked (parent, with a few exceptions).

The following modes have been implemented so far:

off
The narrator doesn't start talking automatically. But it can still be activated "on request", if one of the programmable Side Buttons has been set to 'Morse Narrator'. Pressing the 'Narrator'-button will announce the current channel (name), or menu title followed by the currently selected menu text. So if you didn't get everything on the first time, press the button again to repeat the announcement.
How to configure a sidebutton to start the 'Narrator' will be explained later.
short
Output text as short as possible.
When turning the channel encoder, reports the channel number.

verbose
Reports more details.
When turning the channel encoder, reports the channel name.

test/future (or "DevOnly")
Sends out some debugging info in Morse code. Only intended for development.
Many of the front buttons were already occupied by the experimental/optional Netmon feature. At the time of this writing, the two side buttons (above and below the PTT) were configurable for approximately 25 different functions. Thus, there is no 'fixed' key available to start a longer Morse output on request.
Solution: A 26th function was added to the 'programmable' side buttons. To request a 'long report' from the narrator (e.g. with channel name, zone, battery voltage, and who-knows-what-else), one of the side button functions must be configured as shown in the example below:
   MD380Tools   
  Set Talkgroup
  Mic bargraph
  Experimental
  Side Buttons  
  Side Buttons   
  Top Pressed    
  Bottom Pressed
  Top Held
  Bottom Held
  Top Pressed   
  1750 Hz Tone
   ...
  Morse Narrator
  Morse Repeat

Configuration of a Side Button to start telling a 'long story' in Morse code.
Not required for normal messages like channel name and menu announcement !

The 'story' told when pressing the side button configured this way depends on the current radio state, and the configuration of the morse output.
In verbose mode:

On the main screen (i.e. not in a menu):
Channel name, zone name, battery voltage. Example:
DB0HFD-2   zone OWL   vbat 8.2 V
In a menu:
Menu title (text with gray background), currently selected item. Example:
utilities   radio settings
In any of the Netmon screens:
Reads out the entire screen (console) in Morse code.
This may be a 'really long story' - only for CW freaks, or for code practice.


Any message ('long story' requested via sidebutton or 'automatic report' of channel or menu item) can be stopped by a very short press of the PTT key. This also prevents sending out an over-modulated Morse code message over a repeater by accident.
Another programmable side-button function (Morse Repeat) repeats the previous automatically reported message, e.g. channel name or current menu item.

Unfortunately, the 'beep' (tone) generated by the CPU isn't routed via the analog volume control pot to the speaker output. Thus the output volume cannot be controlled directly via the potentiometer. At the moment, three different volume levels can be selected from the menu. They are realized by different PWM duty cycles, which makes the tone rich in harmonics ("squeaky" like the sidetone in some older electronic keyers). A clean sinewave may be possible in a future version, at the expense of a higher CPU load (over 10000 interrupts per second).
A voice output will not be possible with this radio - it's RAM is too short to synthesize audio ("text to speech"), so take this as an opportunity to learn Morse code. For fun, start at 15 or 18 WPM, and after some practice, try 22 or 30 WPM (words per minute).

A preliminary version (with compiled binary for MD380 / RT3 without GPS only) is in the zipped archive with the 'weak backlight experiment'.
The principle (in the modified firmware) is here.

After the problems listed at the begin of this chapter have been fixed, the 'Morse narrator' described here will eventually find its way into the main branch of Travis' MD380-Tools.