Social Computing or Let the bots talk!

This is a long article to follow up my talk at SSWC, so I will start with a summary for the lazy reader :)


With connected devices, JavaScript enabled web sites and the extensibility of the XMPP protocol, we are at the beginning of a new kind of applications that are operating in the social context of their owners. Google Wave is just the first well known application doing other-than-chatting things on top of XMPP. LinkedProcess is trying to lay a foundation for a much more general computing and distribution infrastructure for context-aware, social-spreadable and trusted application scenarios that potentially enable users to regain control over their own graph of information and control access to it using social connections.

What is XMPP?

XMPP is an XML instant messaging protocol that was developed by the Jabber foundation, as a reaction to the closed protocols of ICQ, Microsoft Messenger etc etc. It is based on the concepts of

  • server side contact lists (Roster in XMPP terminology)
  • open XML streams between the XMPP server and any connected client once the connection is established
  • every connected XMPP client gets a unique ID (e.g. “”) under the common XMPP “bare” ID ( and is uniquely addressable by XMPP packets. This can be used in situations where there is no direct IP number available or it is changing (e.g. mobile devices), making any Internet-connected device addressable as a potential server, with a constant address over time and space – an URI.
  • extensible payloads of XML packets as defined by the XMPP Specification, the most important reflecting the 3 paradigms of asynchronous communication, and supported by all major XMPP servers
    • broadcast (“<presence>“)
    • one-to-one or one-to-many (“<message>“)
    • request/response with acknowledgement (“<iq>“), similar to the HTTP protocol
  • handling of status, presence of resources and routing of messages to the right receivers by the XMPP servers
  • from the beginning built to scale for the Internet with distributed infrastructure, as explained in this blog post
  • large existing XMPP networks to tap into (GoogleTalk, iChat, Facebook Connect, holding part of your existing social graph in the contact lists) and open XMPP servers to set up your own XMPP network
  • SSL support built into the base specification, distributed server-to-server communication even between XMPP networks like GoogleTalk and iChat.


XMPP principle setup

Why should I care about XMPP?

  • XMPP is the currently most popular infrastructure for instant communication, used by systems like iChat, Google Talk and Google Wave. These are just a few of the possible applications we can do on top of XMPP, Twitter and TiVo are rumored to follow
  • via the use of a server side contact list of peers in any XMPP communication, suddenly there is a social or trust-like aspect to any XMPP conversation between two or more participating systems, users or devices
  • any modern mobile phone, javascript-enabled website, J2ME, Java or Flash-capable home automation device (e.g. BluRay players or media servers) or server application can talk XMPP and thus become a true peer to the other clients, without any new server side infrastructure or cloud service development more than existing XMPP servers for routing via e.g., HTML5 web sockets or various XMPP libraries for JavaScript, ActionScript, for an XMPP server and Smack for Java.

What can we do with this?

Now, with all these characteristics in place, we can look at the different emerging platforms for XMPP communication:

Chat – simple text exchange

This is the simple tracking of presence for XMPP clients and the exchange of only human-readable text messages between them. However, a certain coolness is added by being able to distinguish between different clients logged in under the same XMPP account (e.g. different browser windows logging into GTalk and still sending and receiving in the right window). This has been in use for the past 15 years and is nothing new, even if the clients have gotten lighter and fancier over the years (see figure, talking from the browser to on iChat).


Normal XMPP chat setup

Google Wave – concurrent document editing on steroids

Google Wave is introducing a concept where a central XML document (the Wave), reflecting the total of a conversation, is simultaneously edited by several participants (via XMPP clients) of the conversation at the same time. While the incremental changes are sent to the XMPP Google Wave server for merging into the central Wave, they are at the same time even sent to the other online clients in the Roster of the current conversation just as chat messages, creating the real-time response impression of the Google Wave experience. However, these packages do not contain simple chat but actual machine-data interpreted by special Google Wave clients in e.g. JavaScript (see the Wave Protocol).
Google Wave even supports extensions to its functionality, see this blog post which means that a lot of the infrastructure can be used to e.g. support new functionality and data packets.


Google Wave setup

LinkedProcess – turn any connected device into a server with a social context

LinkedProcess (LoP), driven by Marko Rodriguez and Joshua Shinavier is taking the approach of providing a set of XMPP tags for sending scripts and code snippets from XMPP clients (“LoP Villeins”) to other XMPP peers acting as servers (“LoP Farms”), get them executed there and the result sent back. The thought is to be able to speed up the processing aspect of the emerging Semantic Web/Web of Data by moving the processes to the data instead of the usual move of the data to the processor.
One huge application is the processing of the emerging Semantic Web or Linked Data (Tim Berners Lee talks about that at TED 2009) by sending processing instructions to the big servers holding large chunks of this emerging graph structure instead of having to pull all data onto your own device prior to processing it (see the LinkedProcess introduction).


LinkedProcess distributed processing of LinkedData setup

This has a number of interesting aspects:

  • any XMPP capable client (mobile phones, websites etc) can be turned into a server being in direct contact with its peers
  • data can stay at one logical place (e.g. the mobile phone) while others can send scripts to operate to the data in order to get ad-hoc queries executed
  • code can be spread within the social context, so scripts can be shared and spread through the social network of resources and people rapidly, without special installation hassle, giving birth to truly viral “web- and mobile only” applications and social networks of both humans, content and hardware
  • the access to the computing resources and data is controlled via the contact list of the XMPP client executing the script. Thus, it is always known who is trying to access the LinkedProcess server
  • via the use of existing or new social chat networks, this extends the social aspect of your contact list to any XMPP speaking resource under your account, persons, devices and software alike.

So, what is social about LinkedProcess?

Well, with all this said, we have suddenly a couple of very interesting concrete scenarios:

1. Personal and Social Could Computing

This is the concept of spreading code to execute on other servers and devices that are available within the social context of the sender – either a number of servers being set up under the same bare account or friends and colleagues providing their resources to others in their social vicinity. E.g. if I want to transcode a HD movie from TED to fit my mobile phone, I could send a script to my home server instructing the server to download the appropriate encoder, download the original high-resolution link and put the result onto a URL and return that to me. Totally Ad Hoc, without any new programs installed in my phone or the server afterwards.


The Social Cloud

2. Totally distributed and viral social apps – my personal Facebook

A user writes, downloads or is recommended a script – “SOClet” (written in e.g. LUA or Python for Android phones, see Android Sctripting Environment) that asks all its friends phones to evaluate if there is any content related to the asking phones owner. If yes, return the URIs of matching content on the friends phones. That results in an aggregated feed on “Pictures of me by my friends, friends statuses etc” – a Facebook- like setup, totally up to every individual to customize and no central storage. Of course if desired there can be a central server as an equal XMPP client in the middle in order to help holding state, much like the Google Wave setup.


Viral apps spreading in the social device context across mobile devices and e.g. web pages

3. Installing servers on phones and other resources

The technical use setup is as the previous one, but here, long running server code is executed on your friends devices and thus, we create new services on top of the LinkedProcess infrastructure, and turn the devices into mini-servers. This serves potential use cases like Opera Unite, although not with a central namespace like Unite but totally distributed. Peter could for instalnce install a script on Maries phone serving an RSS feed back to Peter when he asks for new content every 5 minutes, including the current location. This can easily be displayed as a GeoRSS over a map on Peters phone or used to mash up the feed with other information using e.g. Yahoo Pipes. This could even be done from a web page (e.g. in Facebook), resulting in a Facebook application that talks to the phone directly and can mash up live phone data with the rest of the Facebook data.

4. Privacy and ownership of everybody’s personal information

By turning the physical holder of your own data – the phone or your home desktop – into a server that accepts or denies requests for operations on that data, users regain control over the personal data. Imagining different protection zones (private, business, open), access can then be granted via the social contact list to individuals, groups of contacts (e.g. friends, co-workers in a project) or anyone.
If unknown or public clients like e.g. Google bots want to access that information, even commercial transactions can be attached and making the commercial client sending the script or access request pay for access to e.g. the GPS location, thus reversing the current notion of users spreading their data and its connections into multiple commercial cloud services, paying with their copyright, usage patterns and privacy only to be forced to aggregate the total personal information again via services like FriendFeed, Plaxo etc.
In the longer run, the total personal information, turned into a consistent personal graph of information with data models like Neo4j, a Graph Database would of course be held physically at more convenient locations than a non-backed up mobile phone and linked to the other personal sources of new information like desktop computers and embedded devices like cameras and GPS.


Socially controlled access to the personal information

5. Social home automation and media sharing, the Internet Of Things

With media and home automation devices having Internet access (TV, set top boxes, media servers, stereo), installing XMPP aware software on them or providing a DLNA/XMPP gateway on my home computer under my personal account will expose these as different accessible resources on the Internet, without the intervention of firewalls, wireless routers etc. So, now my friends LinkedProcess enabled phone can discover my living room TV, since I am in Maries Roster, and the TV is AVAILABLE in XMPP. It discovers the capabilities of the TV much like DLNA and UPnP and then either sends over the latest photo stream to my TV directly, or negotiates means of doing so, e.g. via FTP to a third server. Voila, the home automation is freed onto the Internet and visible and controlled by me and the friends I trust. The same goes for lamps, fridges, home security systems etc with the appropriate bridging in place on a convenient machine, e.g. your home media server. For an actual implementation of light-control via XMPP, see this bridging lab


Home autmomation devices, directly or via an UPnP/XMPP bridge exposed to LinkedProcess


These are just some use cases that one can think of when it comes to merging the key ingredients:

  • status aware, realtime, lightweight messaging infrastructure with contact list support
  • widely used infrastructure with existing contact networks (GTalk, iChat etc)
  • Open Platforms like LinkedProcess providing the protocol for communication and exchange of machine-readable data packets

With the advent of Google Wave, I think the time has come to realize the great potential in Social Computing – introducing social aspects into core programming, security and distribution paradigms.

/peter neubauer

Co-founder, NeoTechnology

Consultant, Jayway

This Post Has 7 Comments

  1. Kristopher

    I think this is one of the most vital information for
    me. And i’m glad reading your article. But want to remark on few general things, The site style is ideal, the articles is really nice : D. Good job, cheers

  2. online programs

    Hmm is anyone else having problems with
    the images on this blog loading? I’m trying to find out if its a problem on my end or if it’s the
    blog. Any responses would be greatly appreciated.

  3. see this page

    After I originally left a comment I seem to
    have clicked the -Notify me when new comments are added- checkbox
    and from now on every time a comment is added I get four emails
    with the exact same comment. Perhaps there is an easy
    method you are able to remove me from that service? Thanks!


    Great blog here! Also your site loads up very fast!
    What host are you using? Can I get your affiliate link to
    your host? I wish my site loaded up as fast as yours lol

  5. Bhagavan


    Very informative.

    You said this is a fallow up article to your talk at “SSWC”,
    Is there any link available for the talk?


  6. Rupert

    I’m trying to get an installation of FreeSwitch working. In one of the “man” pages, it says that XMPP is needed to allow communication with an external SIP server. Is this the same thing as I’m completely lost?

Leave a Reply