From rec.radio.amateur.digital.misc 936 Path: ucbvax!dog.ee.lbl.gov!agate!library.ucla.edu!europa.eng.gtefsd.com!uunet!Germany.EU.net!thoth.mch.sni.de!news.sni.de!kassel!natter!schro From: schro@natter.sni.de (W.F.Schroeder) Newsgroups: rec.radio.amateur.digital.misc Subject: Re: RTTY/SITOR/AMTOR/FEC help! Message-ID: Date: 4 Jan 94 09:06:33 GMT References: <1994Jan1.003107.1@ntuvax.ntu.ac.sg> Sender: news@kassel.wi.pdb.sni.de (Ralf Malisch) Reply-To: schroeder.pad@sni.de Organization: Siemens Nixdorf Informationssysteme AG, Paderborn, Germany Lines: 687 asirene@ntuvax.ntu.ac.sg writes: >Hi, > What is the difference between Baudot, Sitor, AMTOR, FEC? >Are their formats sufficiently similar for the same hardware to >decode or are there some major differences which need separate >hardware? Or are the differences merely in the transport layer, >which can be resolved in software without needing to knock out the >hardware? Thanks for any advise on the abov. >73s de 9VG Daniel Hope this helps ... ----------------------------------------------------------------------------- PACTOR - An introduction to the protocol (Level 1). =================================================== (as at 5th November 1990) Objectives. ----------- PACTOR (PT, latin "the Mediator") was developed by radio amateurs for radio amateurs in the course of experiments in radio communcation; commercial use is not planned. PACTOR should overcome the current shortage of a reliable and quick means of telecommunication on short wave, especially for the bands below 10 MHz. The design of PT is most suitable for the following points: - Error-free data transmission (in practice: proportion of errors less than 10 ** -5) - True binary data transmission (e.g. including the ASCII character set) - Good use of the current channel capacity (automatic adjustment of baud rate, complete bit synchronous transmission, Memory-ARQ, no special request frames) - Good tolerance of interference while maintaining the integrity of the link. - Simple means of operation, optimum half-duplex working with quick switchover and mutually acknowledged QRT - Flexible polarity (arbitrary Mark and Space definition) - Simple hardware construction (Level 1 also suitable for portable and emergency use with ordinary amateur radio rigs) - Required bandwidth maximum 600 Hz - Complete transparency for third parties (eg. supervision of official authorities) - Compatibility when upgrading level definition and the supervisor subsystem The following developments should be tested with the introduction of more efficient hardware: * More efficient character compression (eg Markov encoding) * Adaptive error correcting coding * Modified Memory-ARQ * Modem circuitry with signal processing for bit-level synchronous demodulation and automatic frequency drift correction, and for testing new modulation techniques * Higher data rates for frequencies above 10 MHz (automatic switchover capability between 200 and 400 baud) Protocol basics. ---------------- The PACTOR mode is similar to AMTOR, which is very good for ordinary shortwave working. This uses half duplex ARQ; packets (blocks) of data carrying the information are acknowledged with short 'receipt' signals (controls) by the receiving station. When errors occur the receiver can request the repetition of a packet with relevant control signals. * Synchronous operation. To use the available channel capacity well, synchronised transmission is necessary. The synchronisation must be maintained throughout the whole of the period of connection. * Packet length. A long series of tests has shown that for operation in heavily fluctuating conditions, it is not good policy to adjust the data packet length automatically the the actual conditions. Any modification of the time period leads, especially in bad conditions, to overlapping of data blocks and controls; accordingly this method was rejected as inefficient. The question of the most suitable packet length is answered by measuring various shortwave channels with an FSK transmission. The data obtained was transmitted by a computer simulation on ARQ networks. The optimum packet length on shortwave had a duration on average of about one second. The cycle time (packet length + acknowledgment time) in any case should clearly be shorter than the most frequent fading period. Short noise impulses are also included in the computer simulation; without taking these 'bursts' into account the packet lengths could get considerably bigger. Alternatively the desire for fewer error packets could be aided by the use of error-correcting redundancy as in digital audio techniques. The benefits of these are widespread but are only noticable where the quality of the channel is known and is almost constant. Shortening of packet lengths complies with the rest of the operating techniques (short break-in times). * Packet integrity. Data blocks have CRC checking. The use of error-correcting codes is not planned at the moment, but may be built into future upward compatible versions in combination with more efficient hardware. * Using corrupted packets. In order to use information received in corrupted packets, PACTOR uses a Memory-ARQ process (see below), which produces a clear increase in transmission rates in bad conditions. * Adjusting the baud rate. The throughput of information in actual contacts is limited on the one hand by the available bandwidth (for the given noise level) and on the other by the digital FSK and by apparent phase distortions due to overmodulation. Good results were obtained on shortwave using 200 baud especially during the daytime. In the evening increased distortion of transmission occurs, requiring the reduction of sending speeds to 100 baud. PACTOR provides automatic control for selecting the most efficient baud rate usable. The protocol in detail. ----------------------- Basic principles: MASTER = the station which initiates the link. SLAVE = the called station. TX = transmitter of packets containing information. RX = receiver of packets containing information. BK = break-in, put out by the receiver to switch direction of sending. The following is based on data organised into bytes, since this conforms with the architecture of micro computers. The least significant bit is sent first. The polarity of the FSK transmission is set one way at the initiation of the connection. With each new packet or control signal the polarity is reversed. * Timing. PACTOR works on a synchronised system with a definite time sequence. Total cycle time : 1.25 sec. Packet duration : 0.96 sec. Slot for control signal transmission : 0.29 sec. Control signal duration : 0.12 sec. This leaves an idle time of 170 msec for switchover procedures and settling time, allowing for as in AMTOR a maximum DX distance of about 20,000 km (slight difference compared with CQ-DL 11/90). * Cycle and modes of operation. 1) Internal (standard mode): crystal controlled with 15 ppm accuracy or better. The SLAVE timing is synchronised to the MASTER timing. 2) External (planned for special applications): both stations derive their timing from a central source (eg 50 Hz mains, TV scan frequency, DCF 77). * Control signals and packet format. The control signals (CS) are sent by RX to acknowledge received data. Repetition of the same CS indicates 'REQUEST', ie the request for a block to be repeated. CS3 is used to start the BK-packet, CS4 requests a change in baud rate (see below). The CS has a standard length of 12 bits and is always sent at 100 baud. The bit patterns are chosen to produce optimum Hamming distances and efficient symmetrical characteristics. Code Function Bit pattern (hex) CS1 Acknowledge 4D5 CS2 Acknowledge AB2 CS3 Break-in 34B CS4 Speed change D2C All fields, except special cases, comprise one of the three types Header, Data and Admin (status + CRC). * Structure of packets. 100 baud: /Header/Data field 64 bits/Status byte/CRC1/CRC2/ 200 baud: /Header/Data field 160 bits/Status bute/CRC1/CRC2/ Details: 1) Header: bit pattern 55 (hex) The header byte supplies additional information which is used for synchronisation, for REQUEST recognition in Memory-ARQ as well as in monitor mode. In each pattern carrying new information the bit pattern is inverted. 2) Data field: The data field can contain any digital information; the format of the codes is specified in the status byte. At the present time the choice is between 8 bit ASCII and 7 bit ASCII (Huffman compressed). The ASCII code RS (1E hex) serves as IDLE character in both cases. Huffman coding is especially suited to German and English plain text. The resulting average character length is about 4 or 5 bits. Each packet represents one unit on information: if a character does not fit completely into one packet, it is sent in the next data packet. There is a code table in the appendix. 3) Status byte format: bit 0: packet count (LSB) bit 1: packet count (MSB) Format bit 3 bit 2 bit 2: data format (LSB) --> --------------------------- bit 3: data format (MSB) --> ASCII 8 bit 0 0 bit 4: not yet defined Huffman code 0 1 bit 5: not yet defined not defined 1 0 bit 6: BK request not defined 1 1 bit 7: QRT bit Bits 0 and 1 are used as a packet count; succesive packets with the same value are identified by RX as REQUEST packets. Using a modulus 4 count facilitates the handling of instances of unrecognised control signals (in practice very unlikely). 4) CRC bytes. The CRC is calculated according to the CCITT standard, for the whole packet starting with the data field (ie without Header or, for CS3, without BK field). * Initiating a PACTOR connection. MASTER sends a synchronising packet in the following format: /Header/S L A V C A L L /S L A V C A / (Header = 55 hex) /<-----100 baud------->/<-200 baud-->/ This packet contains the SLAVE callsign (max 8 bytes in ASCII) in a 100 baud and 200 baud bit pattern. Shorter callsigns are padded with 0F (hex). The SLAVE station scans the bit stream in the synchronising packet for its own callsign (both polarities are tried) and after succesful synchronisation answers by sending CS. The actual synchronisation is dealt with by the header byte and the eligible number of bytes in the first callsign. The 200 baud pattern is used as a check on the quality of the channel. If it is received without error, it is answered with CS1, if not then with CS4 which leads to an instant reduction of data rate to 100 baud. Similarly the MASTER searches between two packets following CS in the receiver's window using both polarities. As soon as the first correct CS is received and synchronised, the first normal data packet is sent with Header = AA (hex) and packet count = 1. * Level definition in the initialisation of the link. During the initiation of the connection the maximum possible system level for the conditions is unknown, so each connection starts on the lowest level (1). The first characters transmitted contain information about the highest usable software level number, followed by MASTER callsign, finishing with (eg 1DF4KV ). If the SLAVE station discovers a discrepancy (system level lower than its own), it notifies the MASTER immediately by means of BK and supervisory information (see below), otherwise the link continues at the level specified by the MASTER. The BK situation is only acted upon at the termination of the MASTER system information (). * Changing direction of transmission. After a correctly received packet the RX station can request a reversal of direction of transmission with a BK control signal (CS3). The CS3 becomes the head of the first data packet sent by the new TX. The CS3 covers the first three bytes of the packet at 200 baud, and the first two at 100 baud; so in this instance 4 bits remain unused. The CRC is computed as in a normal data packet, starting from the beginning of the data field. The packet count is set to 0. The next packet after a BK packet gets a header of 55 (hex). If the TX receives a CS3, he switches immediately to RX mode and reads in the rest of the packet completely. If he receives the packet with no errors he answers with CS1, or if not with CS3, in case an immediate 're-break' is required. A repetition of the BK packet is requested with CS2. The answering CS to a BK packet starts a CS-length (0.12 sec) delay before the end of the current TX block. By setting bit 6 of the status byte, the TX can also request a BK himself; this status causes the RX to acknowledge with CS3. * Speed decrease 200 to 100 baud. The RX can always send a CS4 after a corrupt packet, this is always interpreted on the TX side as 'REJECT'; the unacknowledged packet just sent is rejected and the information contained in it is re-sent in the 100 baud mode. The packet count of the rejected 200 baud packet is retained in the first 100 baud packet. If the RX recieves a 100 baud block with the same packet count as a previous 200 baud packet, this means doubly received information; as many characters (except IDLE) as arrived in the last 200 baud packet must be ignored in the copy. After a 'speed down' CS4, the RX and TX set the header to 55 (hex) to reinstate the correct Memory-ARQ function (see below). Other occurences: 'REJECT' of a QRT packet: -> transmission of a 100 baud QRT packet. 'REJECT' of a BK packet: -> transmission of a normal 100 baud packet with Header = 55 (hex) instead of the CS3 header. CS4 serves as 'REQUEST' CS for the first 100 baud block, acknowledgment follows by means of a CS1 or CS3, after which a CS4 signal meaning 'speed up' (see below) can be sent again. * Speed increase 100 to 200 baud. The RX can acknowledge with CS4 after each correctly received 100 baud packet (except 'REQUEST' packets), which forces the TX to switch to 200 baud. The RX switches immediately after CS4 to 200 baud receiving. (should the TX not get the CS4, then this is not combined with additional loss of information; even though the RX has already adjusted to the 'false' speed. The next packet would be a 'Request' packet, therefore irrelevant). CS4s in succession (without the correct CS interspersed) is interpreted as 'Request', therefore ignored. After successfully increasing the speed, at least a second 200 baud packet must be received before the next CS4 ('speed down'). Possible rejection: after the CS4, if the RX obtains an error free 200 baud packet at latest by the end of a predetermined number of cycles, then he answers with an OK CS (different from the last CS before the CS4) and then switches back to 100 baud. The TX must for his part test the CS following the CS4 for OK, and if found switch back to 100 baud without delay and repeat the 200 baud information (or in the case of a QRT, construct a 100 baud QRT block). * Supervisor concept. In order to guarantee compatibility with future versions of software, it must be already possible to transmit system information (SI) in the level 1 protocol. To this end a supervisor character (SB) is defined, which shifts from normal text mode to the supervisor level. The character following the SB contains the supervisor function code (SIC) followed by some parameters. A terminates the SI. Preliminary SI definition: SB: ASCII = 1C (hex) SIC: A: level number follows ('1' = level 1) B: Mycall follows (could for example be introduced into IDLE packets, to enable identification for any third parties monitoring) a . . : to put out control characters 01 to 0f (hex); enables complete data transparency. The complete SI runs like normal text over the PT system, to avoid complications with switching speed. SIs must be able to be inserted immediately as required into the running text, after the last normal character being processed, to avoid errors when repeating messages. * Ending a PACTOR connection. Terminating an ARQ link introduces all sorts of problems because information without real acknowledgment must be sent, or acknowledgment of an acknowledgment would be necessary again and again for proper closedown of contact. PACTOR uses special QRT packets to minimise the danger of an uncontrolled QRT: 100 baud: /Header/G I S L L A C /Status byte/CRC1/CRC2/ 200 baud: /Header/X/G I S L L A C /Header/XXX/Status byte/CRC1/CRC2/ /<-----100 baud------>/ (maximum of 7 significant bytes allowed in callsign) (X = arbitrary) The QRT packet is contructed according to the current speed and mode; in doing so the callsign of the other station must be sent from right to left. The callsign is always sent in a 100 baud bit pattern; the QRT bit in the status byte must be set. When a QRT packet is received the RX sends another acknowledgment control character and then goes into standby mode. The QRT packet must not be passed on to any decoding logic. Since it is very likely in noisy conditions that the first acknowledgment CS may not be received correctly from the TX, the RX, who has gone into standby mode, must still answer with the correct CS any further QRT packets in the scan time frame. The RX looks for any in the incoming bit stream after the byte reversed callsign in the QRT packet and synchronises with it if need be. Each QRT packet received leads to an acknowledgment CS being sent, in which the polarity of the CS is set to that of the QRT packet. The RX must of course remember the previous relationship (before standby) betweed CS polarity and block polarity; otherwise the CS frame may be lost. * Memory ARQ. Normal ARQ systems dispose of information contained in corrupt packets. Such a procedure leads to the complete failure of a link very swiftly in even low noise conditions, since receiving an error-free packet becomes extremely unlikely. Using Memory ARQ, copies of the same packet (not yet received error-free) have the received levels (the good signal plus the noise) aggregated for each corresponding bit individually, whereby the effective signal-noise ratio improves ideally by a factor of n, where n is the number of aggregated packets. For example for the case where n=2 and with constant signal and noise levels: (N = noise level, S = signal level, <..> = average) Total signal = 2*S + N1 + N2; Power of the aggregated signals: <(2*S + N1 + N2) ** 2> = 4*S**2 + + + ; = 0 (insignificant); = = noise power; --> S/N ratio doubles where n=2, and so on for any n. This supports the theory that the bit confidence relies on the effective signal-noise ratio --> special case of the law of conservation of energy. When individual bit length and bit energy are undefinable, Memory-ARQ uses the combined signal energies. Memory-ARQ lowers the noise level threshold for a link by 10-15 dB. In reality, fluctuating channels must be avoided as packets with bad noise worsen the levels of all aggregated packets considerably. To help this the average noise over the whole packet can be subtracted from these individual aggregations. To minimise the quantification of noise, the use of an 8-bit A-D converter is recommended for observation of the signal levels received. In PACTOR the following points are derived from Memory-ARQ: Aggregated packets where the CRC tallies are treated like normal error-free packets. (As the use of a 16 bit CRC guarantees a higher level of integrity, Memory-ARQ brings no disadvantages of any sort.) Old 'Request' packets sent by TX because of noise in a OK-CS must not be counted in the aggregation. To aid this there is a 'similarity' test, carried out in PACTOR during the scutiny of the header byte. In consecutive packets (carrying new information) the header is inverted, so the RX can recognise without the CRC (in the event of corruption) whether this is a 'Request' packet. Each correct packet (with the CRC ok) leads to the cancellation of the Memory-ARQ aggregates. With CS3 packets, obviously the 'similarity' test must be carried out for the CS3 header. PT-Memory-ARQ is also capable, due" to the constantly changing polarity, of averaging constant noise levels, the carrier, or other unsymmetries on the channel. Appendix -------- PT Huffman code --------------- Huffman coding is relatively indifferent to differences between real and theoretical alphabet character frequencies, so that similar good results are obtained in German and English plain text. The compresssion factor attained with ASCII amounts to about 1.7 (resulting in 4.5 bits per character). Substantial improvements could be achieved by considering the statistical relationships between the individual characters (Markov encoding). Code in order of frequency, LSB (sent first) on the left: Char ASCII Huffman space 32 10 e 101 011 n 110 0101 i 105 1101 r 114 1110 t 116 00000 s 115 00100 d 100 00111 a 97 01000 u 117 11111 l 108 000010 h 104 000100 g 103 000111 m 109 001011 13 001100 10 001101 o 111 010010 c 99 010011 b 98 0000110 f 102 0000111 w 119 0001100 D 68 0001101 k 107 0010101 z 122 1100010 . 46 1100100 , 44 1100101 S 83 1111011 A 65 00101001 E 69 11000000 P 112 11000010 v 118 11000011 0 48 11000111 F 70 11001100 B 66 11001111 C 67 11110001 I 73 11110010 T 84 11110100 O 79 000101000 P 80 000101100 1 49 001010000 R 82 110000010 ( 40 110011011 ) 41 110011100 L 76 110011101 N 78 111100000 Z 90 111100110 M 77 111101010 9 57 0001010010 W 87 0001010100 5 53 0001010101 y 121 0001010110 2 50 0001011010 3 51 0001011011 4 52 0001011100 6 54 0001011101 7 55 0001011110 8 56 0001011111 H 72 0010100010 J 74 1100000110 U 85 1100000111 V 86 1100011000 28 1100011001 x 120 1100011010 K 75 1100110100 ? 63 1100110101 = 61 1111000010 q 113 1111010110 Q 81 1111010111 j 106 00010100110 G 71 00010100111 - 45 00010101111 : 58 00101000111 ! 33 11110011101 / 47 11110011110 * 42 001010001100 " 34 110001101100 % 37 110001101101 ' 39 110001101110 _ 95 111100001100 & 38 111100111001 + 43 111100111110 > 62 111100111111 @ 64 0001010111000 $ 36 0001010111001 < 60 0001010111010 X 88 0001010111011 # 35 0010100011011 Y 89 00101000110101 ; 59 11110000110100 92 11110000110101 [ 91 001010001101000 ] 93 001010001101001 127 110001101111000 ~ 126 110001101111001 } 125 110001101111010 124 110001101111011 { 123 110001101111100 ' 96 110001101111101 ^ 94 110001101111110 31 110001101111111 29 111100001101100 27 111100001101101 25 111100001101110 24 111100001101111 23 111100001110000 22 111100001110001 21 111100001110010 20 111100001110011 19 111100001110100 18 111100001110101 17 111100001110110 16 111100001110111 30 111100001111000 15 111100001111001 14 111100001111010 12 111100001111011 11 111100001111100 9 111100001111101 8 111100001111110 7 111100001111111 6 111100111000000 5 111100111000001 4 111100111000010 3 111100111000011 2 111100111000100 1 111100111000101 0 111100111000110 26 111100111000111 -----------------------------------------------------------------------------