Over a year ago discussions about possible improvements to the M17 Specifications started in the M17 Discord and moved to several Github issues. The goals were to:
- Allow for expansion: The existing V2 specification has no room for new payload definitions or additional kinds of META content.
- Make the spec easier to implement and understand.
- Remove features that haven’t been adopted.
The changes include:
- A redesign of the TYPE field, putting the various subfields into byte-aligned nibbles. The Payload Contents becomes a 4-bit selector allowing for existing modes (Data, 3200 bps Codec2 Voice and Packet), a reserved version detection pattern, so V3-aware software can properly decode V2 traffic and a bunch of values available for future expansion, e.g. new codec or streaming data modes.
- Decoupling the META contents into its own 4-bit field, separating it from encryption subtype and allowing room for new kinds of META content.
- Improve the META Text Data by changing to a different control byte which increases the length of a message from 52 to 195 bytes.
- Dropping 1600 bps Codec2 Voice+Data mode. This mode has never seen any significant uptake, so removing it simplifies the spec.
So the proposed V3 meets the goals set out for it, but are there any downsides? There is one big one: Compatibility. V2 to V3 is forward-compatible, that is, V3 radios, hotspots, repeaters and reflectors will be able to receive transmissions from V2 devices and software. But it’s not backward-compatible, so V2 devices won’t recognize V3 transmissions, in fact they’ll misparse them. What happens then depends on the contents of the V3 data and the parsing code, but it’s likely to vary depending on the details of both. So the behavior of a V2 radio when receiving V3 is effectively undefined. (There may even be a case where it works!) Indeed, that’s why it’s called V3 – the major version change indicates that it’s not compatible. A lot of thought has gone into mitigating this problem. If we could update hotspots, repeaters and reflectors before radios are updated it’s possible to reduce many of the interoperability problems. But fundamentally, a radio running V2 firmware won’t be able to receive a V3 transmission. Even if the network infrastructure can translate between formats, there’s no way for RF infrastructure (repeaters, hotspots) to do the same. Devices running V3 will still receive V2 transmissions, but if they transmit using V3, any un-upgraded devices won’t hear them. So there needs to be a “flag day” when all the devices in an area switch from V2 to V3. That’s not too bad if it’s just a radio or two and a hotspot, but it would be harder to coordinate in areas with M17 repeaters.
I’ve been concerned that a breaking change might cause problems for the M17 community. So, working with an AI, I came up with a proposal for a backward-compatible change to V2 that accomplishes many (but not all) of the goals of V3. It keeps the layout of the TYPE field in the current spec, but repurposes reserved bits as escapes to allow for expansion.
The idea behind this approach is that the existing spec has some reserved values and bits that can be repurposed to allow for new functionality. Specifically, Data Type (bits 1-2 in the TYPE field, see Table 3.4 in the spec) has the 002 value reserved and Encryption Subtype (bits 5-6 in TYPE, see Table 3.8 in the spec) has the 112 value reserved. So those reserved values can be used as escapes, telling software that is aware of the new spec to look for a new payload (in the case of Data Type) or META type (in the case of Encryption Subtype). On the other hand, software that isn’t aware sees a Reserved value and should ignore it – we plan to audit existing implementations to confirm this.
| Bits | Field | Rule |
|---|---|---|
| 0 | Packet/Stream | Unchanged |
| 1-2 | Data Type | 01 = Data, 10 = Voice (Codec2 3200); 00 = escape: payload type in bits 12-15; 11 = Voice+Data (deprecated) |
| 3-4 | Encryption Type | Unchanged |
| 5-6 | Encryption Subtype | Unchanged when encrypted. Unencrypted: 00/01/10 = Text/GNSS/ECD (unchanged); 11 = escape: META is self-typed (first META byte = type ID) |
| 7-10 | CAN | Unchanged |
| 11 | Signed | Unchanged |
| 12-15 | Extended Payload Type | Payload type ID when Data Type = 00; otherwise 0000 |
The new Extended Payload Type is stored in bits 12-15 of TYPE, which had been Reserved. So we now have 15 values available for identifying new payloads, with value 00002 reserved. As in the proposed V3, the new payloads can be used for new codecs, data modes or whatever we can come up with.
The existing spec defines three uses of the 14 byte META field: Text Data (002), GNSS Data (012), and Extended Callsign Data (102). (These are only available when encryption is not being used because META contains encryption-related data in that case.) In order to provide for additional uses of the META field, we use the previously-reserved 112 value as an escape. In this case, it indicates that the META area is self-typed: the first byte of the 14 byte META area defines its type, allowing for up to 255 new types (with value 002 reserved as a sentinel).
The V3 spec changes META Text Data to allow for up to 195 byte messages. With this approach, the old Text Data would be unchanged for compatibility, but we can define a new Long Text Data type with similar functionality.
So what are the benefits of doing it this way? We get the same capabilities that V3 offers (new payloads/codecs, META types) but with backward compatibility. All existing features are identical, so radios and infrastructure will continue to work as they always have. New features should fail safe on old radios, i.e. the radio should just stay mute when it sees additional features.
This new proposal is still Version 2 of the M17 protocol. There’s no need for any version signaling because all existing cases are bit-for-bit the same – only new functionality uses previously-reserved values and fields, which older software already knows to ignore.
How about downsides?
- We lose the benefits of V3’s cleanup of TYPE. Fields continue to be non-nibble-aligned, with conditional fields, escapes and dual-meaning subtype bits.
- New extended META payloads lose one byte from 14 down to 13, because the extended META type is stored in META itself.
- Legacy formats (Voice+Data, the three original META types) live alongside new ones, so there are two code paths to maintain. We can deprecate things like Voice+Data, but code still needs to be able to deal with them.
- We’re assuming old code will ignore transmissions that repurpose the reserved values and fields. We need to audit the existing code to verify that’s the case and fix it if it isn’t.
So I see the trade-off as backward compatibility (old infrastructure continues to work, no “flag day” for switching to V3) versus a cleaner spec and simpler code for the future. I think either choice is reasonable, but overall I’d choose compatibility. But it’s not up to me! Interested members of the M17 community are encouraged to comment on the Github issue and let us know what you think, so we can make a choice that works for all of us.
