The Extensible Messaging and Presence Protocol (XMPP) promised to make all instant messaging services as easy to use and interoperable as email. After a few short lived attempts (Google Talk, Windows Live Messenger, AIM, etc.) it largely failed to deliver on that promise and IM is now awash with custom protocols and well meaning, but ultimately foolish, attempts to reinvent the wheel. It’s common to see a phone with 3 or more messaging clients installed, and even open source advocates who should know better encourage this distressing trend by promoting messaging solutions that only support proprietary protocols.

In this post I won’t attempt to convince you that this is an intolerable situation, or that interop is good for accessibility or network reach and size. I won’t even attempt to convince you that XMPP is the right protocol and architecture for the job. Instead, we’ll ask the question: Where did XMPP go wrong, and is it possible to salvage the dream of a federated network of chat services?

XML

Let’s start with the technical aspects of the protocol. Under the hood, XMPP makes heavy use of XML, everybody’s favorite technology to despise. For the most part, the haters are right (brace yourself for strong opinions): XML is over-engineered garbage that should generally be avoided in new projects. Ultimately though, it’s also very widespread, has a plethora of libraries in any well established programming language, and can operate in a very efficient streaming mode that already exists in many of those libraries (meaning that application authors don’t have to bear the burden of streaming, namespacing or otherwise identifying payloads, or state tracking). This makes XML a good fit for realtime, asynchronous, stateful protocols like XMPP (incidentally, these are the same properties that make XMPP a good fit for instant messaging). When you’re a developer you sometimes have to use the best tool for the job because it’s proven and has widespread support, even if it’s not the most fun technology to work on. It’s important for protocol designers to take as much work off protocol library developers shoulders as possible. This is one of the most important design considerations they have to reason about, and this sometimes means using existing tools that fit the task at hand even if they don’t love them.

Despite this, many developers will tell you not to use XMPP because it involves XML, even though as an application developer you may never have to look at or think about XML (in practice you do with some libraries, and don’t with others, but that’s easily remedied). This sort of bad reputation (deserved or no) is almost impossible to recover from, and is probably XMPP’s greatest barrier to adoption in the open source and web development communities.

The only way I see this being overcome is if one or two large services with lots of clout force the matter and embrace the protocol (like Google Talk, except without refusing to implement basic features like server-to-server TLS). To make this a reality, we’d have to use a combination of user pressure for interoperability, and better marketing by the XSF (or whatever body pops up to advocate for XMPP). So let’s talk about marketing and what problems we can solve there to make the minor technical warts less of an issue.

Marketing

The second area in which XMPP suffers is marketing and branding. The community around XMPP (and I suspect I’m guilty of this myself) likes to compare the protocol to commercial chat services. It’s even common to hear someone positioning the entire Jabber¹ network as an alternative to services such as Slack and Skype, or confuse the network (Jabber) and the protocol (XMPP) even though they would not confuse, for example, Email and SMTP.

Naturally, an ad hoc conglomerate of servers and clients run by one or two developers with no real coordination or collaboration doesn’t compete very well with a service run by a rich corporation that can hire server and client developers, operations, and support. Luckily, it doesn’t have to compete as long as we don’t position it that way. It’s a bit like arguing that a small email server run by a friend is a competitor to Gmail: it’s valuable to have users self-hosting SMTP servers, but it’s certainly not going to gain widespread adoption (nor should it).

Instead of targeting the self-hosting crowd, or advertising the public network, the XMPP community should be pushing for the larger players to support XMPP for interop with third party clients or servers. Whether that means allowing third party clients in a limited way (as HipChat and Slack did until recently), and/or supporting full federation with other services (as Google Talk did) doesn’t much matter and may be different for each service. Supporting third party clients that speak XMPP is valuable so that users with machines that can’t support the official clients, or with special accessibility needs can use a client suited to their use case (without someone having to reverse engineer the protocol and develop new client for every service they use). Along the same lines, supporting federation even if you only support first party clients is still valuable because users with special needs can use networks with clients that support their needs and still communicate with other federated services with limited functionality or with text based fall backs (similar to how you can send money between Gmail accounts, but if you use Fastmail like me you get a link to a web service to transfer the money). Unfortunately, convincing chat services that this can provide value requires average users of those services to put pressure on the service operators to interoperate, or some solid value proposition for why interoperating provides more value than locking users in and relying on the network effect to bring in their friends.

Once the services agree that they should interoperate, it would be up to the XMPP community to swoop in and position the protocol as the most widely supported protocol (in terms of libraries, clients, a large existing network of users, etc.) for the job. However, the average user doesn’t care about interoperability and will consistently vote (with their wallets and signups) against their own interests by supporting the walled gardens. The sort of “hearts and minds” campaign that’s required to change this is, I think, beyond the XMPP community, which likes to try and convince people with technical arguments using words like “federation” and “extensibility” that most people simply don’t care about even if they do care about the end result. I suspect that even a unified XMPP community with a sustained marketing push couldn’t put together the massive amount of work required to convince most people to push back against having half a dozen chat apps on their machines. Many people are simply too used to the pain, and have grown desensitized to it. Even if we could somehow convince them to do so and the major players decided to be more interoperable, the community no longer has the working relationship with large chat providers to convince them not to start from scratch and make up yet another protocol using the latest trendy technology. So, if we can’t convince the average chat user to push back against having 6 chat clients installed, and we can’t convince the big companies that it provides more value to interoperate than not, what small things can we do to make this possible in the future?

What Can I Do?

If, like me, you still think that XMPP is our best (albeit faint) hope for a ubiquitous instant messaging network, this post shouldn’t make you despair. There are ways you can help at any level. If you don’t mind dealing with the nitty-gritty details and working on fixing the warts of the protocol, consider volunteering with the XSF. There’s lots of work to be done from marketing, to event planning, to editing or writing standards. You can get started by joining the chat room at xsf@muc.xmpp.org.

If you’re a developer and don’t want to deal with things at the level of community engagement or standards, consider working on existing clients or servers to make them more user friendly, or to support more of the basic features from the compliance suites². This probably would have the biggest impact on changing user perception of XMPP. Many people try to use Pidgin or some other unmaintained XMPP-specific client, and don’t realize it’s about 10 years out of date, or is making compromises to support multiple protocols, and then blame the problems on XMPP. A new client that I’m particularly excited about for all the major platforms (although it’s still in the very early stages and is using a language that probably signifcantly limits its contributor pool) is Dino, and if you’re an Android developer, Conversations is very nice. If you’re a fan of macOS and iOS, I hear Monal is very nice these days, although I haven’t used it in some time myself. Each of these has its own development chat room to help get you up to speed.

If you don’t want to work on XMPP directly but run an open source community, consider using open protocols (XMPP or IRC) for your group chat. It’s easy to setup a nice web client for your users, and some networks like Freenode (which uses IRC) or Conversations (which uses XMPP) have them already. The minor inconvenience or lack of polish is far outweighed by the benefit you provide in the long term to your users who can’t use the official clients, or who may not be able to use a commercial service for legal or political reasons (eg. due to U.S. export laws). You don’t have to advertise the fact that you’re using XMPP or IRC, just be a user and help build the network.

Individual users can also help by simply using open protocols: consider signing up for one of the many existing services such as Conversations or jabber.at. If you have your own domain but don’t want to run your own server, you can even get hosting using your own custom domain from conversations.im (I am not affiliated with Conversations in any way, but I do use their custom domain hosting and highly recommend it). If you’re not sure what to do once you have an account, search for a chat room or ask your friends and family to join too. I write a lot of Go and can often be found in the English language Go room I created. If a room doesn’t exist for your favorite topic on a popular server or you can’t find it with the search (which is limited to a handful of servers), create one and start inviting people!

You can also use other services that use XMPP; for example, if you’re looking for a good VOIP provider, jmp.chat lets you receive SMS and MMS messages via any XMPP client, and voice calls via any SIP client (such as the one built into Android’s dialer).

It’s also helpful to dispel XMPP myths when you hear someone repeating them as fact. Any protocol has warts, and one that’s 20 years old ends up having a lot of legacy ones repeated as if they were still fact today.

Finally, if you run a commercial chat service or are considering starting one: reach out and let’s chat about how federating with other services (via any protocol) can give you a competitive advantage. My JID is the same as my email address, so take your pick of networks message me at sam@samwhited.com.

It may not sound like much, but just encouraging the use of open protocols like XMPP and IRC makes it more likely that in the future the news will spread and we’ll be able to start moving (slowly) back towards a more interoperable world.