Getting some tinkerers together was no problem, so we got three families with kids, 4 pumpkins, one LEGO Mindstorms set and 2 Arduinos to start with. I recently discovered the awesome HiTechnic Prototype board, turning the LEGO Mindstorms set into a full IO-enabled prototyping experience with the added convenience of the thought-through LEGO sensors and the comfort of the LeJOS Java OS for the Brick. You get 6 analog inputs and 6 digital input or output pins - not bad.

So, I got one of these and started with wrapping the Proto Sensor into a LeJOS java class using the I2CSensor base. The communication protocol is fairly straightforward and not tight to the HiTechnic plugin software for the LEGO IDE they provide:

package org.peter;

import lejos.nxt.I2CPort;
import lejos.nxt.I2CSensor;

public class ProtoSensor extends I2CSensor {

 public static final int A0 = 0x42;
 public static final int A1 = 0x44;
 public static final int A2 = 0x46;
 public static final int A3 = 0x48;
 public static final int A4 = 0x4A;
 public static final int DOUBLE = 2;
 public static final int SINGLE = 1;
 public static final int DIGITAL_READ = 0x4C;
 public static final int DIGITAL_WRITE = 0x4D;
 public static final int IO_MODE = 0x4E;
 public static final int PIN_OUTPUT = 1;
 public static final int PIN_INPUT = 0;
 public static final int B0 = 1;
 private byte[] buf = new byte[2];

 public ProtoSensor(I2CPort port) {
 super(port);
 }

 public ProtoSensor(I2CPort port, int mode) {
 super(port, mode);
 }

 /**
 * Reads A0 Analog input
 *
 * @return byte[2], first byte is high, second byte low part
 */
 public byte[] readA0() {

   return getAnalog(A0);
 }
 private byte[] getAnalog(int address) {
   int res = getData(address, buf, DOUBLE);
   if (res != 0) {
     System.out.println("Error: " + res);
   }
   return buf;
 }

 /**
 *
 * @return the one byte read on all digital PINs on bits 0-5. bist 6 and 7
 *         will be ignored
 */
 public byte[] readDigital() {
   byte[] result = new byte[8];
   int res = getData(DIGITAL_READ, result, SINGLE);
   if (res != 0) {
     System.out.println("Error: " + res);
   }
   return result;
 }

 /**
 * set the input (0) or output (1) mode for the digital pins
 *
 * @param d0
 * @param d1
 * @param d2
 * @param d3
 * @param d4
 * @param d5
 * @return 0 if success, fail otherwise
 */
 public int setDigitalModes(int d0, int d1, int d2, int d3, int d4, int d5) {
   int mode = d5 * 32 + d4 * 16 + d3 * 8 + d2 * 4 + d1 * 2 + d0 * 1;
   return sendData(IO_MODE, (byte) mode);
 }

 public int setDigitalOutput(int d0, int d1, int d2, int d3, int d4, int d5) {
   int mode = d5 * 32 + d4 * 16 + d3 * 8 + d2 * 4 + d1 * 2 + d0 * 1;
   return sendData(DIGITAL_WRITE, (byte) mode);
 }

 public boolean isHigh(int pin) {
   return (readDigital()[0] & pin)==pin;
 }
}

Now, you can communicate with the Prototype sensor with only minimal hassle. The first test was to just have a light resistor connected to one analog input, and light a LED upon shadowing it (see here) . Oskar and me settled on listening for the touch sensor to be pressed (sticking through the nose of the pumpkin). On press, we simply light the 2 LED-eyes (cool blue color) and play a short "Happy Halloween" file tw times that Oskar has been recording on an old iPaq, transforming it with Audacity into 8bit ECM WAV that the LEGO brick can play:

import java.io.File;

import lejos.nxt.Button;
import lejos.nxt.SensorPort;
import lejos.nxt.Sound;
import lejos.nxt.TouchSensor;

import org.peter.ProtoSensor;

public class Helloween {
 private static ProtoSensor proto;

 private static TouchSensor touch;

 public static void main(String[] args) throws Exception {
 proto = new ProtoSensor(SensorPort.S1);
 touch = new TouchSensor(SensorPort.S2);
 // set all digital pins to OUT
 proto.setDigitalModes(1, 1, 1, 1, 1, 1);

 while (!Button.ESCAPE.isPressed()) {
   if (touch.isPressed()) {
     proto.setDigitalOutput(1, 1, 1, 1, 1, 1);
     sayHello();
   } else {
     proto.setDigitalOutput(0, 0, 0, 0, 0, 0);
   }
 }

}

 private static void sayHello() {
   File hello = new File("helloshort.wav");
   try {
     Sound.playSample(hello, 100);
     Thread.sleep(2000);
     Sound.playSample(hello, 100);
     Thread.sleep(1500);
   } catch (InterruptedException e) {
   }
 }
}

And voilá, the super-scary pumpkin is done and we even got time for some Kanelbullar and hot chocolate!
The competing comic-art-eyes super-cool-pumpkin is made by David Cuartilles of Arduino and 1Scale1 fame. It won the competition without problems - thanks all Pumpkin Pimpers for a great 3 hours with the kids!

Happy Halloween!
/peter

Summary

With connected devices, JavaScript enabled web sites and the extensibility of the XMPP protocol, we are at the beginning of a new kind of applications that are operating in the social context of their owners. Google Wave is just the first well known application doing other-than-chatting things on top of XMPP. LinkedProcess is trying to lay a foundation for a much more general computing and distribution infrastructure for context-aware, social-spreadable and trusted application scenarios that potentially enable users to regain control over their own graph of information and control access to it using social connections.

What is XMPP?

XMPP is an XML instant messaging protocol that was developed by the Jabber foundation, as a reaction to the closed protocols of ICQ, Microsoft Messenger etc etc. It is based on the concepts of

• server side contact lists (Roster in XMPP terminology)
• open XML streams between the XMPP server and any connected client once the connection is established
• every connected XMPP client gets a unique ID (e.g. "neubauer.peter@gmail.com/phone1/xyz") under the common XMPP "bare" ID (neubauer.peter@gmail.com) and is uniquely addressable by XMPP packets. This can be used in situations where there is no direct IP number available or it is changing (e.g. mobile devices), making any Internet-connected device addressable as a potential server, with a constant address over time and space - an URI.
• extensible payloads of XML packets as defined by the XMPP Specification, the most important reflecting the 3 paradigms of asynchronous communication, and supported by all major XMPP servers
  • broadcast ("<presence>")
  • one-to-one or one-to-many ("<message>")
  • request/response with acknowledgement ("<iq>"), similar to the HTTP protocol
• handling of status, presence of resources and routing of messages to the right receivers by the XMPP servers
• from the beginning built to scale for the Internet with distributed infrastructure, as explained in this blog post
• large existing XMPP networks to tap into (GoogleTalk, iChat, Facebook Connect, holding part of your existing social graph in the contact lists) and open XMPP servers to set up your own XMPP network
• SSL support built into the base specification, distributed server-to-server communication even between XMPP networks like GoogleTalk and iChat.

XMPP principle setup

Why should I care about XMPP?

• XMPP is the currently most popular infrastructure for instant communication, used by systems like iChat, Google Talk and Google Wave. These are just a few of the possible applications we can do on top of XMPP, Twitter and TiVo are rumored to follow
• via the use of a server side contact list of peers in any XMPP communication, suddenly there is a social or trust-like aspect to any XMPP conversation between two or more participating systems, users or devices
• any modern mobile phone, javascript-enabled website, J2ME, Java or Flash-capable home automation device (e.g. BluRay players or media servers) or server application can talk XMPP and thus become a true peer to the other clients, without any new server side infrastructure or cloud service development more than existing XMPP servers for routing via e.g. http://www.kaazing.com, HTML5 web sockets or various XMPP libraries for JavaScript, ActionScript, for an XMPP server and Smack for Java.

What can we do with this?

Now, with all these characteristics in place, we can look at the different emerging platforms for XMPP communication:

Chat - simple text exchange

This is the simple tracking of presence for XMPP clients and the exchange of only human-readable text messages between them. However, a certain coolness is added by being able to distinguish between different clients logged in under the same XMPP account (e.g. different browser windows logging into GTalk and still sending and receiving in the right window). This has been in use for the past 15 years and is nothing new, even if the clients have gotten lighter and fancier over the years (see figure, marie@neubauer.se talking from the browser to neubauer.peter@gmail.com on iChat).

Normal XMPP chat setup

Google Wave - concurrent document editing on steroids

Google Wave is introducing a concept where a central XML document (the Wave), reflecting the total of a conversation, is simultaneously edited by several participants (via XMPP clients) of the conversation at the same time. While the incremental changes are sent to the XMPP Google Wave server for merging into the central Wave, they are at the same time even sent to the other online clients in the Roster of the current conversation just as chat messages, creating the real-time response impression of the Google Wave experience. However, these packages do not contain simple chat but actual machine-data interpreted by special Google Wave clients in e.g. JavaScript (see the Wave Protocol).
Google Wave even supports extensions to its functionality, see this blog post which means that a lot of the infrastructure can be used to e.g. support new functionality and data packets.

Google Wave setup

LinkedProcess - turn any connected device into a server with a social context

LinkedProcess (LoP), driven by Marko Rodriguez and Joshua Shinavier is taking the approach of providing a set of XMPP tags for sending scripts and code snippets from XMPP clients ("LoP Villeins") to other XMPP peers acting as servers ("LoP Farms"), get them executed there and the result sent back. The thought is to be able to speed up the processing aspect of the emerging Semantic Web/Web of Data by moving the processes to the data instead of the usual move of the data to the processor.
One huge application is the processing of the emerging Semantic Web or Linked Data (Tim Berners Lee talks about that at TED 2009) by sending processing instructions to the big servers holding large chunks of this emerging graph structure instead of having to pull all data onto your own device prior to processing it (see the LinkedProcess introduction).

LinkedProcess distributed processing of LinkedData setup

This has a number of interesting aspects:

• any XMPP capable client (mobile phones, websites etc) can be turned into a server being in direct contact with its peers
• data can stay at one logical place (e.g. the mobile phone) while others can send scripts to operate to the data in order to get ad-hoc queries executed
• code can be spread within the social context, so scripts can be shared and spread through the social network of resources and people rapidly, without special installation hassle, giving birth to truly viral "web- and mobile only" applications and social networks of both humans, content and hardware
• the access to the computing resources and data is controlled via the contact list of the XMPP client executing the script. Thus, it is always known who is trying to access the LinkedProcess server
• via the use of existing or new social chat networks, this extends the social aspect of your contact list to any XMPP speaking resource under your account, persons, devices and software alike.

So, what is social about LinkedProcess?

Well, with all this said, we have suddenly a couple of very interesting concrete scenarios:

1. Personal and Social Could Computing

This is the concept of spreading code to execute on other servers and devices that are available within the social context of the sender - either a number of servers being set up under the same bare account or friends and colleagues providing their resources to others in their social vicinity. E.g. if I want to transcode a HD movie from TED to fit my mobile phone, I could send a script to my home server instructing the server to download the appropriate encoder, download the original high-resolution link and put the result onto a URL and return that to me. Totally Ad Hoc, without any new programs installed in my phone or the server afterwards.

The Social Cloud

2. Totally distributed and viral social apps - my personal Facebook

A user writes, downloads or is recommended a script - "SOClet" (written in e.g. LUA or Python for Android phones, see Android Sctripting Environment) that asks all its friends phones to evaluate if there is any content related to the asking phones owner. If yes, return the URIs of matching content on the friends phones. That results in an aggregated feed on "Pictures of me by my friends, friends statuses etc" - a Facebook- like setup, totally up to every individual to customize and no central storage. Of course if desired there can be a central server as an equal XMPP client in the middle in order to help holding state, much like the Google Wave setup.

Viral apps spreading in the social device context across mobile devices and e.g. web pages

3. Installing servers on phones and other resources

The technical use setup is as the previous one, but here, long running server code is executed on your friends devices and thus, we create new services on top of the LinkedProcess infrastructure, and turn the devices into mini-servers. This serves potential use cases like Opera Unite, although not with a central namespace like Unite but totally distributed. Peter could for instalnce install a script on Maries phone serving an RSS feed back to Peter when he asks for new content every 5 minutes, including the current location. This can easily be displayed as a GeoRSS over a map on Peters phone or used to mash up the feed with other information using e.g. Yahoo Pipes. This could even be done from a web page (e.g. in Facebook), resulting in a Facebook application that talks to the phone directly and can mash up live phone data with the rest of the Facebook data.

4. Privacy and ownership of everybody's personal information

By turning the physical holder of your own data - the phone or your home desktop - into a server that accepts or denies requests for operations on that data, users regain control over the personal data. Imagining different protection zones (private, business, open), access can then be granted via the social contact list to individuals, groups of contacts (e.g. friends, co-workers in a project) or anyone.
If unknown or public clients like e.g. Google bots want to access that information, even commercial transactions can be attached and making the commercial client sending the script or access request pay for access to e.g. the GPS location, thus reversing the current notion of users spreading their data and its connections into multiple commercial cloud services, paying with their copyright, usage patterns and privacy only to be forced to aggregate the total personal information again via services like FriendFeed, Plaxo etc.
In the longer run, the total personal information, turned into a consistent personal graph of information with data models like Neo4j, a Graph Database would of course be held physically at more convenient locations than a non-backed up mobile phone and linked to the other personal sources of new information like desktop computers and embedded devices like cameras and GPS.

Socially controlled access to the personal information

5. Social home automation and media sharing, the Internet Of Things

With media and home automation devices having Internet access (TV, set top boxes, media servers, stereo), installing XMPP aware software on them or providing a DLNA/XMPP gateway on my home computer under my personal account will expose these as different accessible resources on the Internet, without the intervention of firewalls, wireless routers etc. So, now my friends LinkedProcess enabled phone marie@neubuaer.se/phone1 can discover my living room TV, peter@neubaeur.se/tv1 since I am in Maries Roster, and the TV is AVAILABLE in XMPP. It discovers the capabilities of the TV much like DLNA and UPnP and then either sends over the latest photo stream to my TV directly, or negotiates means of doing so, e.g. via FTP to a third server. Voila, the home automation is freed onto the Internet and visible and controlled by me and the friends I trust. The same goes for lamps, fridges, home security systems etc with the appropriate bridging in place on a convenient machine, e.g. your home media server. For an actual implementation of light-control via XMPP, see this bridging lab

Home autmomation devices, directly or via an UPnP/XMPP bridge exposed to LinkedProcess

Conclusion

These are just some use cases that one can think of when it comes to merging the key ingredients:

• status aware, realtime, lightweight messaging infrastructure with contact list support
• widely used infrastructure with existing contact networks (GTalk, iChat etc)
• Open Platforms like LinkedProcess providing the protocol for communication and exchange of machine-readable data packets

With the advent of Google Wave, I think the time has come to realize the great potential in Social Computing - introducing social aspects into core programming, security and distribution paradigms.

/peter neubauer

Co-founder, NeoTechnology

Consultant, Jayway

It was an amazing experience to see the kids play 20-finger-games, soldier, cut, bend and slice open stuff and see inside old stuff to discover a lot of reusable parts.

Everyone got a laser-cut MDF keyring with Goofy on it, and we got lots of magnets, loudspeakers, key pads, mirrors and switches for later use in a "build from scratch" session

For a fast result, we built a very easy Walkie-Talkie by just wiring together two loudspeakers - it actually worked!

Thanks for making this possible Jayway, 1Scale1 and Österportskolan - the kids loved it and we think this could certainly be a stepping stone to the goal of getting more young engineers into higher education!

Thanks to all involved - David, David, Jannike, Elisabeth and my wife Marie.

We will submit this even to OnceUponASchool.org, thanks TED and Dave Eggers for the inspiration and stay tuned!

Happy tinkering,

/peter

we decided to join Earth Hour and try to get all people at Jayway, Testway, Realway, Dotway and Leadway to save as much energy as they can on Saturday, 20.30 - 21.30. If you read this, please join in and register yourself or your organization! It won't save the planet but it will set a sign of our willingness to find energy saving lifestyles and solutions that in total can help to slow down the transformation of our habitat to an unknown state.

Cheers

/peter

Øredev was great success, especially all the people and the organization - a big Thank You to Michael and Emily for making this possible, you rock!

Now, it was really awesome to exchange Android games with Mike Jennings fro the Android team, and get him excited over the recent Android port to the Neo FreeRunner! Thanks Anders Hedberg for coming over to fix the FR, and to all the OpenMoko enthusiasts doing the hard work!

/peter neubauer

However, yesterday @Øredev, I met Mike Jennings from the Android team and together we looked into this problem over a beer in the Oracle booth. Looking into the current SVN source at http://svn.igniterealtime.org/svn/repos/smack/trunk/source, it seemed that a fast hack to get things working (especiallly the keystore problem on the device) would be to simply disable SSL negotiation.This does of course not work for SSL stuff, but it gets you going and it is enough for talk.google.com:5222. It is enough to just disable SSL negotiation by hardcoding it away in the ConnectionConfiguration.java, and comment out everything related to java.beans and other packages that are not present on the Android platform to get things working.

After doing that, voila things started to work and I am now happily logging in programatically via SVN Smack again!

Thanks Mike for the hints, help and a great BouncingBall application!

For anyone intersted, I am attaching the patch.

/peter neubauer

smackdiff

The Consumer JRE

The ads

In its endeavours to again push Java onto the desktop, Sun announced that the JRE
will be slimmed down to approx 2 Mb, containing only the essential classes needed
to run “Hello world” and other basic applications. That will make it comparable to
e.g. the Flex runtime and significantly improve the load time perception. Other
modules will be downloaded and installed on demand. There was even a new good
looking Windows installer, default Java codecs for media and a deployment toolkit
that lets you install Java for your app in the browser with a single line of JavaScript.

The reality

Some of the things mentioned above are actually already being deployed, such as
the improved installer. The latest Java6 versions have a much friendlier installation
experience. SUN seems to be so committed to Java that even the stock ticker sym-
bol has been changed to JAVA.
The latest take was that the Leaner Meaner JRE should come at the latest in Q1
2008 but it has been very quiet on release dates. I still hope for it, since it would
make the Java experience so much more pleasant.

OpenJDK

The ads

Java has finally been released under an “open” license, loosely based on the GPLv2,
some parts with the classpath exception. This enables Suns JDK and JRE to be
shipped via Ubuntu, Debian and other Linux distributions. To guide the transition
to open code, a Governance Board, consisting of a number of high profile OSS
people even outside Sun, has been formed. The release comprises both the JDK 1.5
(followed by Java6 and 7) and interestingly, the CLDC and CDC JavaME parts.

The reality

It seems things are progressing nicely, Java7 milestones are appearing, and a com-
munity is being established. However, now the question is what Apaches Harmony,
the Classpath project and OpenJDK bring to the table and how we all can avoid a
fracturing of Java. Interesting times. Anyway the opening up of the JDK is a great
move and OpenJDK will probably continue to be the reference and major imple-
mentation of the JDK in the future as well.

Java (FX) is everywhere

The ads

Java technology nowadays has spread almost everywhere except in the original Ap-
plet domain. We find Java in SmartCards, routers, phones, servers, desktops, web
cameras, Lego Mindstorms Robots, to name but a few. To leverage this installation
power for the new strategy of going visual with even Java, Sun is launching a new
initiative called JavaFX for bringing interactive content to Java based systems. After
that Applets and Swing failed to live up to the expectations, JavaFX tries to stand-
ardize the different Java platforms regarding interactive content delivery, and adds a
scripting language, JavaFX Script, based on the F3 (Form Follows Function) project,
to the mix.
How relevant the system is for the target group, the content creators, design-
ers and artists, remains to be seen. Right now the only tool for JavaFX Script is a
command shell, so there is no Flash or Silverlight-like tooling around. Even from a
platform point of view, it has not been clarified in what incarnations (e.g. a unified
animation API) JavaFX is going to emerge on existing platforms such as J2SE or
J2ME.

The reality

On the development tools side, JavaFX plugins for NetBeans 6.0, M10 is out, so
take a look at it. For the designers, being the most important target group for FX
Script, Sun seems honestly to be working on something closer to FlexDesigner than
Netbeans, but no details are to be leaked right now.

Glassfish for president

The ads

Sun pushed Glassfish very hard as the best application server around. Glassfish ver-
sion 2 is about to be released. Among the notable new developments are Ericsson
contributing a SIP servlet engine to Glassfish, enabling open development of public
telecommunication services like VoIP, Multimedia and Instant Messaging.

The reality

Glassfish v2 has just been released. It is the reference implementation for JavaEE,
and will be the basis of SUN’s commercial Application Server offerings. Even Net-
beans 6.0 will be packaged with a Glassfish server. Seems SUN really wants to take
market share from JBoss, IBM and BEA.

Scripting support in the JVM and IDE

The ads

Sun has hired 3 of the 5 core JRuby developers. Even the Groovy folks are work-
ing closely with Sun to make these languages a real alternative to classic class based
development. The next version of Suns JVM will go one step further and explicitly add support for dynamic methods and other goodies, if JSR292 (Supporting Dy-
namically Typed Languages on the JavaTM Platform) can be pushed through until
then.

The reality

Both Netbeans 6.0 and Eclipse via the DLTK (Dynamic Languages Tool Kit) and
RDT (Ruby Development Tools) plugins provide good support for JRuby, JavaS-
cript and others. Refactoring is not a strong side of dynamic languages but that is
partly made up for by shortened testing cycles without the building and packaging
phase.
For the JVM part, the JSR292 is underway to ease the execution of dynamic
code, but there is some debate on whether that will really have the desired effects
and what the timeframe is. Meanwhile, JRuby and others are getting to a state where
commercial products are built on them, Thoughtworks Mingle platform being one
of the most public ones. It seems dynamic languages on the JVM are here to stay,
partly because they have their uses in a lot of areas, and partly because most of them
need a VM, and SUN’s JVM has proven itself to be a rock solid foundation to build
on.

Netbeans - the one and only

The ads

Netbeans has been pushed very hard at JavaOne. I could not spot any single oc-
currence of Eclipse at any seminar. Speakers seem to have been instructed to show
any code or examples exclusively in Netbeans. Netbeans has very good support for
J2ME development, JSP integration and visual tools, whereas these are optional
plugins in Eclipse.

The reality

Netbeans is probably just as flexible and powerful as Eclipse, but its approach is a
bit different. Whereas Netbeans and IDEA try only to be an IDE, thus being able to
package that product more tightly, Eclipse with its OSGi and RCP core is more of
a generic platform, the Java IDE being just a subset of plugins, bundled in the JDT,
WTK and related projects.
Furthermore, Netbeans and IDEA are driven by a single company whereas the
Eclipse foundation consists of very big players in the Java market. Still, Netbeans de-
livers a highly integrated and consistent Java IDE, making it an obvious alternative to
today’s ruling Eclipse trend. The support for J2ME development is particularly nice,
with visual development of Midlet screen flows, easy emulation and good support
for signing and deploying Midlets.

Summary

After 10 years in existence, it seems the pace of revolutionary improvements in Java
Standard Edition is starting to slow down. This might be a good thing, as it means
even more compatibility and stability for long running systems. Nevertheless, with
the existing platform there is plenty of space for progress and development outside
the core Java platform, like the Java IDEs, scripting languages being ported to the
JVM and new efforts like Spring and QI4J (www.qi4j.org). I am very confident in
Java - and the surrounding technologies - having the power to give us new ways to
build better systems in the future too and being able to reinvent itself in the face of
mobile computing, scripting and new programming paradigms bubbling up.

Originally published in JayView.