HTTP interface into Idavoll

This interface is meant for services that want to exchange updates on its resources through XMPP publish-subscribe. We assume that each resource is represented by one publish-subscribe node with exactly one item. That item holds an ​Atom representation of that resource, in the form of an Atom Entry document.

Web services are traditionally built around HTTP only. HTTP is a stateless request-response protocol. Such services, typically written in PHP, have a CGI model of processing requests, a script is run on each request. XMPP on the other hand, has persistent connections from a client to its server, has addressable end points and allows two way traffic.

To allow PHP based services to use XMPP, in particular its publish-subscribe extension, we thought up the concept of a gateway between the HTTP and XMPP world. Unlike generic approaches like ​XMPP over BOSH, we opted for an application specific solution that focuses on being able to publish to local publish-subscribe nodes, and allow a service to subscribe to remote nodes.

Publishing to nodes

To publish to a node, the service sends an HTTP POST request to the gateway. The message body contains a self-contained Atom Entry document. There are two different situations: the node does not yet exist or it does exist.

Publishing to a new node

The service makes a connection to the gateway (in this case at localhost, port 8081) and sends a POST request:

POST /publish HTTP/1.1
Host: localhost:8081
Content-Type: application/atom+xml;type=entry;charset=utf-8
Content-Length: 301

<?xml version="1.0"?>
<entry xmlns="http://www.w3.org/2005/Atom">
  <title>Atom-Powered Robots Run Amok</title>
  <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
  <updated>2003-12-13T18:30:42Z</updated>
  <author><name>John Doe</name></author>
  <content>Some text.</content>
</entry>

The gateway, that by nature is an XMPP publish-subscribe service, creates a new node with a generated NodeID, and publishes the passed Atom Entry document to the node as a new item, giving it the ItemID current. It responds to the service with the ​XMPP URI node that is needed for publishing updates to the node.

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 81

{"uri": "xmpp:example.org?;node=generic\/60c531da-5894-4328-b69c-c8af83ba7b70"}

The returned URI can also be used to be published on a web page to allow other services to subscribe to it. When subscribing directly through XMPP, the URI needs to be dereferenced. In this example, the JID of the publish-subscribe service is example.org, and the NodeID is generic/60c531da-5894-4328-b69c-c8af83ba7b70.

Publishing to an existing node

For publishing subsequent updates of a resource to a node, we use the XMPP URI of the node that was returned on creating the node, as shown above. The URI is passed in the query component of the URI we use to POST the updated Atom Entry document to:

POST /publish?uri=xmpp:example.org?%3Bnode=generic/60c531da-5894-4328-b69c-c8af83ba7b70 HTTP/1.1
Host: localhost:8081
Content-Type: application/atom+xml;type=entry;charset=utf-8
Content-Length: 309

<?xml version="1.0"?>
<entry xmlns="http://www.w3.org/2005/Atom">
  <title>Atom-Powered Robots Run Amok</title>
  <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
  <updated>2003-12-13T18:35:22Z</updated>
  <author><name>John Doe</name></author>
  <content>Some updated text.</content>
</entry>

On success, the publish-subscribe service will asynchronously send XMPP publish-subscribe notifications to all registered subscribers:

<message from='example.org' to='example.com' id='n181'>
  <event xmlns='http://jabber.org/protocol/pubsub#event'>
    <items node='generic/60c531da-5894-4328-b69c-c8af83ba7b70'>
      <item id='current'>
        <entry xmlns="http://www.w3.org/2005/Atom">
          <title>Atom-Powered Robots Run Amok</title>
          <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
          <updated>2003-12-13T18:35:22Z</updated>
          <author><name>John Doe</name></author>
          <content>Some updated text.</content>
        </entry>
      </item>
    </items>
  </event>
</message>

Subscribing to remote nodes and receiving notifications

To subscribe from the local service to changes to remote resources though HTTP, the gateway has a publish-subscribe client and exposes it through HTTP. The HTTP client provides the XMPP URI of the (remote) node, and a callback URI. The latter is used to forward received notifications for the node by performing a POST request on the URI.

Notifications can be about new or updated items for that node, or the deletion of the node. In case of item notifications, an Atom Entry document or Atom Feed document is the embedded entity in the request. For node deletion, the POST request does not carry an entity, but has the Event header set to DELETED.

Registering a callback for subscribing to a (remote) node

The service POSTs a JSON document to the gateway, that holds the XMPP URI of the node to subscribe to in (key uri) and the callback URI (key callback):

POST /subscribe HTTP/1.1
Host: localhost:8081
Content-Type: application/json
Content-Length: 127

{"callback": "http:\/\/localhost\/callback",
 "uri": "xmpp:example.com;?node=generic\/609bbd8f-730d-4c89-be0a-02a4690176ae"}

On receiving the request, the gateway dereferences the given XMPP URI into the JID of the (remote) publish-subscribe service and NodeID to subscribe to. It then requests the publish-subscribe service to subscribe to the node, sending notifications to itself:

<iq from='example.org' to='example.com' type='set' id='s521'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe node='generic/609bbd8f-730d-4c89-be0a-02a4690176ae'
               jid='example.org'/>
  </pubsub>
</iq>

If the subscription request is accepted, the publish-subscribe service responds with the status of the subscription.

<iq from='example.com' to='example.org' type='result' id='s521'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscription node='generic/609bbd8f-730d-4c89-be0a-02a4690176ae'
                  jid='example.org'
                  subscription='subscribed'/>
  </pubsub>
</iq>

The gateway then responds to the HTTP client. As there is no additional information to relay back to the HTTP client, the status code is 204.

HTTP/1.1 204 No Content

Receiving notifications

For nodes that are subscribed to, notifications are received asynchronously by the gateway.

<message from='example.com' to='example.org' id='n291'>
  <event xmlns='http://jabber.org/protocol/pubsub#event'>
    <items node='generic/609bbd8f-730d-4c89-be0a-02a4690176ae'>
      <item id='current'>
        <entry xmlns="http://www.w3.org/2005/Atom">
          <title>Havoc by Atom-Powered Robots</title>
          <id>urn:uuid:1aed3592-28f9-4381-abe2-550232e89638</id>
          <updated>2003-12-14T01:13:32Z</updated>
          <author><name>Jane Doe</name></author>
          <content>Some commentary.</content>
        </entry>
      </item>
    </items>
  </event>
</message>

Upon receiving a notification, the gateway matches the origin publish-subscribe service and NodeID to its list of registered callback URIs, and sends an HTTP POST request to the callback URI. The XMPP URI of the origin node is passed in the Referer HTTP header.

POST /callback HTTP/1.1
Host: localhost
Content-Type: application/atom+xml;type=entry;charset=utf-8
Content-Length: 284
Referer: xmpp:example.com;?node=generic/609bbd8f-730d-4c89-be0a-02a4690176ae

<entry xmlns="http://www.w3.org/2005/Atom">
  <title>Havoc by Atom-Powered Robots</title>
  <id>urn:uuid:1aed3592-28f9-4381-abe2-550232e89638</id>
  <updated>2003-12-14T01:13:32Z</updated>
  <author><name>Jane Doe</name></author>
  <content>Some commentary.</content>
</entry>

For now, the response to the HTTP request is ignored, but at some point it might be used to retry the notification on temporary failure or unsubscription on permanent failure.

POST /other_callback HTTP/1.1
Host: localhost
Content-Type: application/atom+xml;type=feed;charset=utf-8
Content-Length: 509
Referer: xmpp:example.com;?node=news_feed

<feed xmlns="http://www.w3.org/2005/Atom">
  <entry>
    <title>Item1</title>
    <id>urn:uuid:4c22de69-ad82-49d7-820d-1d1ccee87b4a</id>
    <updated>2008-03-02T13:49:01Z</updated>
    <author><name>Jane Doe</name></author>
    <content>Some text.</content>
  </entry>
  <entry>
    <title>Item2</title>
    <id>urn:uuid:0deddff7-8ff3-4e6a-a0eb-52a08cc588f1</id>
    <updated>2008-03-02T13:49:03Z</updated>
    <author><name>Jane Doe</name></author>
    <content>Some other text.</content>
  </entry>
</feed>