Drawing in Flash
using iPhone

Specialmoves Labs

Written by Dine Albrecht,

For this R&D experiment, we wanted to find a way for one/multiple iPhones to communicate with a Flash Website in real time. So we decided to build a prototype of how this could work by creating a Drawing Pad application where the iPhone functions as a controlling device and the output is displayed on a Flash website in real time.

Implementation

Challenges

Flash can’t receive any data outside of its sandbox (due to security issues) – in our case, outside of the domain to which the application is deployed.
Finding a communication channel that allows data to be sent between Flash and the iPhone.
This communication needs to be in realtime.

Facing the challenges

To get the communication working in realtime we had to decide on a fast way to send data across the web. There are two protocols that can be used for this kind of communication – TCP and UDP. TCP is more reliable as it allows for error checking and the validation of packets. The downside is that it’s slower due to the obvious overhead that validation and error checking incur and the fact that the receiving client will always have to wait until all previous packets have arrived before being able to use the latest packet. UDP is faster, but there is no guarantee that all packets will arrive. After assessing both of these options, we decided that UDP packets were the way to go – speed was far more important than the quality of the data for this particular project (as the communication needs to be pretty instantaneous).

The most obvious way to get the iPhone to communicate with Flash was to send the UDP messages directly to Flash. Unfortunately one limitation of Flash is that it doesn’t support sockets (which are needed for both TCP and UDP communication). However, Adobe Air does support sockets, but as we don’t want users to have to install an Air app (adding one more step in the user journey), we had to come up with a more elegant solution.

Thinking about possible communication methods we hit on a solution. Instead of requiring the iPhone user to install the Air app on their computer, we decided to install the Air app on our server, forwarding the received data to Flash via Flash Media Server (which can run on the same server as the Air app).

1) We start by sending UDP data from the iPhone:


        udpSocket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
 
setsockopt(udpSocket, SOL_SOCKET, SO_BROADCAST, 1, sizeof(int));
 
memset((char *) &amp;targetAddress, 0, sizeof(targetAddress));
 
targetAddress.sin_family = AF_INET;
 
targetAddress.sin_addr.s_addr = htonl(0xFFFFFFFF);
 
targetAddress.sin_port = htons(kAccelerometerSimulationPort);
 
targetAddress.sin_len = sizeof(targetAddress);
 
const char *msg = [[NSString stringWithFormat:@"%1.3f,%1.3f\n", loc.x, loc.y] UTF8String];
 
int error = sendto(udpSocket, msg, strlen(msg), 0, (struct sockaddr*)&amp;targetAddress, sizeof(targetAddress));

2) We receive this data in the Air app and update a shared object:


        //Create the socket
 
datagramSocket = new DatagramSocket();
 
datagramSocket.addEventListener( DatagramSocketDataEvent.DATA, dataReceived );
 
//Bind the socket to the local network interface and port
datagramSocket.bind( localPort, localIP );
 
//Listen for incoming datagrams
datagramSocket.receive();
 
// whenever data is received:
private function dataReceived( event:DatagramSocketDataEvent ) : void
{
    //Read the data from the datagram
    srcAddress  = event.srcAddress;
    srcPort     = String(event.srcPort);
    sentMessage = event.data.readUTFBytes( event.data.bytesAvailable );
    updateSharedObject(sentMessage);
}
 
// send data via FMS:
private function updateSharedObject ( s:String) : void
{
    so.setProperty("message", s);
}

3) We receive the shared object from Flash:


        // whenever Shared Object changes:
private function onSoSync(e:Event) : void
{
    if ( so.data.message)
    {
        coordinates = so.data. message.split(",");
        // do stuff width the coordinates
    }
}

This works well and data can be sent from several iPhones at the same time – everyone drawing onto the same Flash Bitmap.

Potential

This prototype allows several iPhone users to send data to the same Flash website. This thought could be developed further so different user groups could each have their own canvas and draw on it using their phones.

The iPhone/Flash communication model could also be used pretty successfully for playing a Flash game on a big screen – the controllers being iPhone. In fact after writing this post, this project went on to inform our Consequences project for Creative Review and Tate Britain.

Have a look here:
http://vimeo.com/26764840

Draw in Flash using iPhone - Specialmoves

Drawing in Flash using iPhone