OTR: Encrypted Instant Messaging

Yes, for once, I have not just some app for secure messaging, this time, it’s a protocol. OTR, or Off-the-Record Messaging, is a protocol for establishing end-to-end encrypted messaging between two participants over a standard instant messaging channel like IRC or XMPP (Jabber). It also allows for deniable authentication, where during the conversation you can be assured that only you two are talking, but after the conversation, there is no way for an outside third party to 100% prove you talked, since it’s theoretically possible for an attacker to have forged the communications record. It’s complicated, but not too hard to wrap your head around.

OTR has three major versions in use, and a 4th version in development. Most IRC / XMPP clients usually have some sort of OTR support (usually, as an add-on, in the case of Irssi or WeeChat), At a high-level overview, OTR will construct a secured session with a key exchange, and then use per-message keys to provide forward secrecy. Each user has a long-lived public key pair that associates an ID (like nick@network for IRC) with a key fingerprint. These keys are persistent for your user, which are used to generate per-session keys, which generate per-message keys. It’s crypto, this happens a lot. In addition, you can also make use of authentication (making sure the other side is who they say they are) by either ’trusting’ their key’s fingerprint, or using what’s called the Socialist Millionaires Protocol (SMP), which I’ll explain later. Most OTR-capable clients will disable logging of chat transcripts when an OTR session is established, hence the name, Off The Record. Technically, version 1 of the protocol is a little insecure, so most clients should strive to support v2 at the minimum, or v2 + v3.

Remember that OTR is meant to be transport-agnostic, meaning it can theoretically take place over any instant messaging platform that allows sufficiently long messages to be sent. Yes, the common implementation is within IRC and XMPP clients, but it’s not limited to them. Also, OTR is limited to one-on-one communications, it’s not equipped to properly handle multi-recipient (group) conversations, or as long as I’m talking about limitations, offline conversations. Meaning, I cannot start an OTR session with someone who’s offline, and I cannot reliably send messages to someone who’s offline either.

Compared to PGP

PGP (Pretty Good Privacy) is another system for providing end-to-end encrypted and verified communications, and one that’s actually still used within some specific circles. PGP is meant for emails, and also works on public-key cryptography. For a quick explanation of that, what that means is that each user has a key pair, the ‘public’ side and the ‘private’ side. Only they know their private key, and their public key is free to share. When a message is encrypted with one key in the pair, only the other can decrypt it. What this results in is that if I encrypt a message with someone’s public key, only they should be able to read it. If I encrypt a hash digest of my message with my private key (called the message ‘signature’), then they can prove only I was the one able to send it. For more explanation, I’ve already talked about PGP.

Anyways, moving on, it may seem odd that I’m comparing an instant messaging encryption protocol to an email encryption protocol, but it’s not that far out, even OTR itself does it, claiming itself as the better option. And, well, I can kinda see why. The thing about PGP is that 50% of its functionality is built around message signatures. “If I can verify this signature from X, then X and only X could have sent this message.” While secure, this also means that there’s no deniability, you can’t later plausibly state that you never sent the message, because it’s got a signature from your key pair, which only you should be holding. OTR solves this through what’s called deniable authentication. How this works is a little complicated, but basically, messages in OTR are verified through a Message Authentication Code, or MAC. These MACs, once used, are then shared, meaning they’re public knowledge after the fact. This, with a few more features, means that as a message is being sent, it can be proven as real. After the message was sent, a third party can neither confirm nor deny the real identity of the sender, since an attacker could, in theory, forge a perfectly valid message, meaning that the participants can plausibly deny having any involvement with the communications they allegedly participated in.

Initiation Conversation

If you’ve ever been on something like IRC / XMPP and seen messages starting with ?OTR from people in a channel… they have a misconfigured OTR capable client. If that came in a direct message context, that means that, yes, they are OTR capable and basically requesting to start an OTR session if possible. The rest of this string, fun fact, shows the compatible versions. Just ?OTR? means v1, whereas ?OTRv23? means version 2 and version 3. A client supporting all current versions would prefix the message with ?OTR?v23?. There’s also the ‘hidden’ way of doing this, called ‘whitespace tagging.’ This involves sending not just a little prefix string like that, but a specific sequence of spaces and non-printable ASCII characters, which are either going to be printed as spaces, or not at all. This might indent the message a little in your client if the tag is not recognized, but it’s not as immediately apparent and/or annoying as seeing ?OTRv23? prefixed to everything they say.

Once both clients agree to a conversation, an authenticated key exchange takes place. This is a pretty standard Diffie-Hellman key exchange, allowing for both sides to compute a shared secret without actually sending enough information over the wire for a third party to also compute that secret, and then can later authenticate the other user within that session if desired. Once in a conversation, DH exchanges are almost constantly used to generate new symmetric keys for each message, providing some level of forward secrecy, where a compromise of one message key does not compromise the other messages.

Socialist Millionaires’ Protocol (SMP)

While you can authenticate a user by trusting the fingerprint of their main user key, probably by sharing it through some other, trusted, out-of-band way, OTR also comes with an in-band approach, the Socialist Millionaires’ Protocol. The name comes from the Socialist millionaire problem, a crypto problem where two participants want to confirm if some secret held on both sides is identical, while not disclosing any information about the secret itself. OTR’s SMP works as a simple question/response mechanism, where one user will ask the other a question, and provide their client with the answer. If the other client provides the same answer, after a few back-and-forths with client-to-client messages, we can therefore mark the fingerprint as trusted. Ideally, this is kinda like a security question on a password reset: You want it to be something that your intended recipient would know, and only they would know.

OTR Version 4

While version 3 is well and good, version 4 is currently in development, with a few changes, some of which are… pretty big.

• Use of Elliptic Curve Cryptography (ECC)
• Support for non-interactive conversations (a.k.a., where one side is offline, something that previous versions did not support at all)
• Use of Stronger cryptographic methods
  • Use of Deniable Authenticated Key Exchange (DAKE) instead of a normal AKE
    • Use of “DAKE with Zero Knowledge” (DAKEZ) for interactive conversations, and “Extended Zero-knowledge Diffie-Hellman” (XZDH) for non-interactive conversations
  • Use of a Double Ratchet Algorithm (similar to how Signal works) for key management
  • Replace SHA-1 and SHA-2 with SHAKE-256
  • Switch from AES for the per-message symmetric keys to ChaCha20

Now this may not make a lot of sense for most people, but suffice it to say the security of an OTR session’s contents will improve greatly with v4. Additionally, the added support for non-interactive conversations means that one does not have to wait for the receiver to come online before sending them something. There’s also one additional change, which is support for out-of-order networks. What this means is that OTRv4 can work in cases where it’s not guaranteed that messages are received in the order that they were sent. This doesn’t happen too often, but the added support just means that it’s that much more flexible, if required.

OTRv4 supports non-interactive conversation initiation by use of a so-called ‘prekey server’ that can store just enough information for the sender to construct their half of an XZDH exchange, and then start sending messages as per usual. What this kinda means is that for someone to send messages to you while you’re away, you will have needed to upload your prekey data to a prekey server, but they’re meant to be untrusted, and the prekey data itself isn’t enough to break the sessions that would be created with it. The part that makes a DAKE a, well, DAKE is that the exchange itself could be forged by someone, meaning that, once again, after-the-fact, there’s no concrete proof that two participants exchanged keys (and then messages), but during such, both can be assured that the other side is genuine. Even I’m not 100% on the exact details of deniable exchanges and deniable authentication like that, but, well, there it is.

OTR is actually a surprisingly complicated protocol, with each version adding a little more complexity on top of the last, but, for accomplishing what it set out to, it seems it passes that bar easily. A nice option for people who want to talk securely over other established channels, like on Libera, instead of requiring, say, trading phone numbers (Signal, Telegram), or setting up a Matrix¹ account. Now, if you’d like to play around with this yourself, you’re more than welcome to attempt to try it with me, I’m usually found as teknikal_domain on Libera Chat. The WeeChat plugin (/script install otr.py, with the extra pip dependencies) only supports v2, meaning that if it can’t establish, I might need to switch clients to Irssi, which I believe handles both. Unfortunately for me, there’s no Emacs ERC plugin for OTR at the moment, and I am nowhere near qualified enough to write the code to handle that in elisp (or, I’m not really qualified to implement any version of OTR at all, at least not yet).

And if you’d like to learn more, you can check out the homepage, or the version 2 and version 3 protocol specifications, if you’re a nerd like me.