Skip to content

XMPP Migration

Jump to bottom Edit New page
devhell edited this page Sep 9, 2015 · 24 revisions

Schedule

Switching the server software has been done with a downtime of 6 hours, 44 minutes and 40 seconds on September the 1st from 00:00:47 UTC to 06:45:27 UTC.

A few problems showed up during the deployment, which were the reasons why it took so long. These problems are tracked in the issue tracker under the XMPP Migration milestone.

Apart from these rather minor issues, the biggest time killer was me trying to convert the users from the old database to the new one. Originally I wanted to use the PIEFXIS module from ejabberd, but porting it to MongooseIM would have taken longer than the available time frame and wouldn't export all the data (for example no MUC configurations).

So I ended up writing a quick & dirty conversion tool for the database dump, which in the end did the job.

Apologies for the long downtime to everyone using the service.

Feel free to write comments and suggestions to the milestone and if possible report other problems that might arise in the next few days and weeks.

We're going to move to a new deployment with a different XMPP server software and also switch to NixOS instead of Debian GNU/Linux, which is also being deployed using NixOps instead of manually editing configuration files.

Right now the old network is running on a single server with 4 virtual hosts (headcounter.org, aszlig.net, noicq.org and no-icq.org). It is still running on Debian Squeeze LTS and the configuration is a big mess and hard to maintain.

Switching to NixOps has the advantage that the deployment is reproducible and allows for better participation and transparency. Everyone can contribute and review the code using this very repository.

Post-switch Problems

With the current configuration the following problems are still around (but should be resolved eventually).

  • Pubsub service is currently down because of a critical bug (which reappeared in 2.10.0) that affects user's privacy as it may reveal other user's JIDs in error messages.

  • Registrations are currently deactivated because of a spam incident targeting Google on February 2013. All of the spam accounts since then have been deleted.

  • SSL certificates for aszlig.net, noicq.org and no-icq.org are still using 1024bit CA root certificates and need to be updated in par with the already updated headcounter.org certificate.

  • ICQ transport move to Spectrum.

Pre-switch Problems

With the previous configuration, a whole bunch of problems eventually arose which made the switch to the new deployment an urgent matter:

  • IPv6 problems, we were using a SixXS tunnel (disabled on 2015-08-25) and no native IPv6. Having only one IPv6 address has lead to wrong certificates being used for aszlig.net, noicq.org and no-icq.org.
  • Cryptographic ciphers are too weak and there is no support for elliptic curve cryptography. There are already XMPP servers which enforce elliptic curves, one example being jabber.ccc.de, which hosts quite a lot of accounts out there.
  • Pubsub service is currently down because of a critical bug (which reappeared in 2.10.0) that affects user's privacy as it may reveal other user's JIDs in error messages.
  • Registrations are currently deactivated because of a spam incident targeting Google on February 2013. All of the spam accounts since then have been deleted.
  • MSN transport is still running even though the service has been discontinued. As far as I could tell it's still working but doesn't allow for new users.
  • ICQ transport is very old and uses an older OSCAR protocol version that doesn't have transport encryption.
  • SSL certificates for aszlig.net, noicq.org and no-icq.org are still using 1024bit CA root certificates and need to be updated in par with the already updated headcounter.org certificate.
  • There is no support for XEP-0280 or XEP-0198, thus using mobile clients can lead to a very frustrating chat experience.
  • Updating the current ejabberd server is very cumbersome because it contains Headcounter-specific patches that do not apply to the latest upstream version of ejabberd.

What we're migrating to

We'll be switching to MongooseIM, which is an XMPP server implementation forked from ejabberd but is more integrated into the Erlang/OTP design principles (which means better customization and less trouble with updates) and has a comprehensive automated test suite.

Apart from that, it also addresses almost all of the mentioned problems, except that it's lacking Pubsub support, however, our current situation is that we don't have Pubsub running anyway.

Also, we're going to migrate the transports to Spectrum 2, which should give us better support for ICQ and even more transports to offer, for example, Facebook, Skype, Twitter, and more.

Migration process

At first, we're going to migrate the XMPP server itself and we're (re-)adding more features on the go. We already switched DNS to be served via our new infrastructure (in 8d5b5cf) and now the only blocking issue is #4.

We'll use the XMPP Migration milestone to track further development after the XMPP server switch commenced. Feel free to participate in discussion and to report new issues or ideas.

So, for current users of the service this essentially means that directly after the XMPP server switch the following services will not be available (yet):

Of course, the goal is to bring them back as fast and wherever possible.

Add a custom footer
Add a custom sidebar

Clone this wiki locally