Divvi: collaborative
photo sharing

Specialmoves Labs

Written by Sunil Jolly,

Adobe have been promoting the use of AIR to develop cross platform applications, especially since Apple relaxed their rules around 3rd party tools for App development, thereby making AIR a valid tool for iOS development. Recently a number of devices have come to market that have made this more appealing too, notably the Blackberry Playbook and Android tablets like the Xoom.

So we thought, why don’t we try and develop something across all these devices and see what happens?

This post focuses on Android development with AIR.

We had an R&D idea a while ago that lent itself nicely to the idea of a cross platform app. A common situation is that you have photos on your device that you want to share with other people. Not just on Facebook or via email, but in a way that the other person ‘owns’ the photo themselves.

Our idea is a virtual table that you can chuck your images onto. A place where other people can pick them up and add them to their own device. The table is shared so that any image you placed on the table is visible to other users. And moving the image around the table is reflected on the other users’ devices.

Adobe AIR and mobile devices

We’re building Divvi in Flash Builder 4.5 which has several mobile focussed improvements over previous versions. These improvements include new workflows for mobile development (including debugging devices using the Flex debugger) and components optimised for touch events and different screen densities (previously called Flex Hero).The bulk of the work however was done using a pre-release version of Flash Builder (Burrito) so there may be differences between our techniques and the current recommended techniques.

Mobile development vs web and desktop development

When discussing mobile development, technical limitations are the most obvious difference to web/desktop development. Mobile devices are far less powerful and have much less memory, so you have to be wary of that. As a test of AIR performance on devices, we tried to run an Augmented Reality app with marker detection in AIR on an Android device and the whole thing ground to a halt.Developing for a single device like the iPhone4 is great because you have a set screen size. The difficulty with Android is that the resolutions between tablets and devices varied between 800 x 600 and 1280 x 800, so that’s quite a difference. That doesn’t take into account pixel density either, which makes things more confusing again. (There are tools in AIR to deal with this, but more on that later)

Hello World! Security, permissions and deploying to devices

We started at the most obvious places. Getting “Hello World!” to run on all these devices.This was one of the most time consuming parts of the project. It isn’t particularly tricky, but requires ploughing through documentation, going step by step and hoping you don’t hit an obscure error.In this post we’ll deal with Android deployment, the next post will discuss iOS and Playbook deployment. We also created an Ant script which packages and deploys across multiple devices, more on that in the next post too.

Android Deployment

Compared to iOS and the Playbook, Android deployment is simple. You can deploy and install your app straight onto your device via USB from Flash Builder. No certificates, no identification… If only all devices were this easy. (Note – If you want to get your app into the Android Market then you will have to jump through some security hoops). You’ll need to install the drivers for your phone first. The Flex SDK includes some Android drivers but we had to download ours from the HTC and Motorola sites.

Developing Flex Mobile applications

We’re not going to get into the specifics of Flex Mobile development here, since there are several great articles on the web already (see below).As an overview, Flash Builder 4.5 adds new views and navigation elements suited to mobile apps. It’s very quick to get up and running and we’re using Robotlegs as our framework with no issues. Follow these tutorials and you’ll have something running in no time.

Testing locally

When you test locally you can run the app in a variety of emulators with pre-configured sizes to match different devices. To test on a physical Android device you can connect via USB and set the run configuration to “deploy on device”. The debug mode is particularly great to see running, with the debugger working as expected as you test the app.We had some problems with the Galaxy crashing or sometimes not allowing debugging, but a restart seems to fix that. Not sure why that happened. The Motorola Xoom had particular trouble with the debugger.

A few technical notes

There are a few interesting parts to Divvi and Mobile development that we think are worth sharing.

Retrieving an image from your device
Resizing the image
Sending the image to all connected devices
Creating a draggable, rotatable and clickable visual element
Synchronising the image manipulation across devices
Saving an image to your device
Multitasking in Android
Getting rid of system menus
Creating a splash screen

1. Retrieving an image from your device

The CameraRoll class takes care of all this. “browseForImage” opens the native gallery browser and returns the image when you select it. The image is returned as an FilePromise (see the example for how to use this), which contains a reference to the loaded file which you can upload using File.upload. We’re uploading the image to a PHP server since we weren’t able to get Flash Media Server to successfully save the file. It should be possible but we didn’t want to spend too long on that part and knew it was simple in PHP (and we could use ImageMagick to resize and alter it in the future if we wanted to).

2. Resizing the image

This was the first point where mobile performance became an issue. To try and save a little bit of time, we thought we’d try resizing the image on the client to create the thumbnail. The lack of processing power was very obvious here with large images taking up to 10 seconds to resize. Definitely better here to have the server resize the image using Imagemagick.

3. Sending the image to all connected devices

We’re using Flash Media Server to handle communication between multiple devices. Port issues were the biggest problem getting FMS up and running. FMS tries connecting on various ports to get around security issues so it shouldn’t have been an issue (see: http://stream-recorder.com/forum/rtmp-rtmpt-and-rtmps-t4201.html). Took a little bit of port experimentation to get it working.We used a server shared object to send the image. Shared Objects are objects that all connected clients can listen to. So calling a method on the shared object will call that method on all clients connected to that shared object. It’s a very quick and easy way to get communication between devices. On the server side the only thing you need to do is create an empty application to connect to. To do this, just create a folder with the name of your app in the applications directory of Flash Media Server – that’s it!There are limitations to this. Firstly it can get confusing since calling a method on the shared object will call that method on all connected clients, including the one that called the method in the first place. So you need to filter out calls depending on whether your client sent it in the first place, not a major deal but a bit messy.Also, the server isn’t retaining any information about the state of the application. So if you had a few devices connected and they were all sharing a few photos on the table, then you wanted to add another device and have that update it’s state with all the photos on the table you’d have do something like having a “master” client that could send the data out. So that isn’t ideal and pushes responsibility and load onto one of the clients – having the state on the server would make it easy and gives you more options for future development.The obvious benefits (and the reason we used it) are that you don’t need to learn anything about Flash Media Server or any other backend technology.

4. Creating a draggable, rotatable and clickable visual element

There are two different modes for handling touch events in Flex. Gesture mode or Touch Point mode.Touch Point mode is the “basic” mode where a user’s touch is relayed directly, i.e. taps and drags.Gesture mode is where AIR interprets touch events into a gesture. So you get events like TransformGestureEvent and PressAndTapGestureEvent. In this mode, “Touch Point” events like touch are changed into Mouse Events (i.e. click).In touchPoint mode there is a touchDrag event (http://help.adobe.com/en_US/as3/dev/WS1ca064e08d7aa93023c59dfc1257b16a3d6-7ffc.html) which is a quick way to get touch and drag working. Because we wanted to capture the rotate event, we used the usual startDrag method as if you were using a mouse. A note on the limitations of Flex Mobile here. If you place a single finger on the screen and then place a second finger to rotate, it won’t work. Only the original finger will register which is a shame. Also there’s no indication of the centre of rotation. You only get the rotation amount. So if you wanted to do very sensitive rotations then you might have to try and work around that.Getting the click to work was more troublesome. The click event fired constantly, even when trying just to drag. So we had to use a workaround which was to time the gap between pressing and releasing. If the time was short enough we considered it a click/press.Here’s our warts-n-all class complete with TODO comments where we’ve cut some corners:


        package com.specialmoves.touch
{
	import flash.display.Bitmap;
	import flash.display.DisplayObject;
	import flash.display.MovieClip;
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.events.MouseEvent;
	import flash.events.TransformGestureEvent;
	import flash.filters.DropShadowFilter;
	import flash.geom.Matrix;
	import flash.geom.Point;
	import flash.utils.Timer;
 
	import mx.core.IVisualElementContainer;
	import mx.core.UIComponent;
 
	import spark.core.SpriteVisualElement;
 
	public class DraggableRotatableSprite extends SpriteVisualElement
	{
		public var graphic:Bitmap;
		//TODO: these are quick fixes, a dedicated event would be better
		public static const ADJUSTED:String = "adjusted";
		public static const PRESSED:String = "pressed";
		public static const CLICK_DRAG_THRESHOLD:Number = 150;
 
		private var clickStartTime:Number;
 
		public function DraggableRotatableSprite(graphic:Bitmap)
		{
			filters = [new DropShadowFilter(4,45,0,0.5)];
			this.graphic = graphic;
			addChild(graphic);
			addEventListeners();
		}
 
		public function set disabled(disabled:Boolean):void
		{
			if (disabled)
			{
				removeEventListeners();
				graphic.alpha = 0.25;
			}
			else
			{
				addEventListeners();
				graphic.alpha = 1;
			}
		}
 
		private function addEventListeners():void
		{
			addEventListener(TransformGestureEvent.GESTURE_ROTATE , onRotate);
			addEventListener(MouseEvent.MOUSE_DOWN, startDragging);
			addEventListener(MouseEvent.MOUSE_UP, stopDragging);
		}
 
		private function removeEventListeners():void
		{
			removeEventListener(TransformGestureEvent.GESTURE_ROTATE , onRotate);
			removeEventListener(MouseEvent.MOUSE_DOWN, startDragging);
			removeEventListener(MouseEvent.MOUSE_UP, stopDragging);
		}
 
		protected function onRotate (e:TransformGestureEvent):void
		{
			stopDrag();
			var rotateMatrix:Matrix = transform.matrix;
			var rotatePoint:Point = rotateMatrix.transformPoint(
				new Point((graphic.width/2), (graphic.height/2)));
			rotateMatrix.translate(-rotatePoint.x, -rotatePoint.y);
			rotateMatrix.rotate(e.rotation*(Math.PI/180));
			rotateMatrix.translate(rotatePoint.x, rotatePoint.y);
			transform.matrix = rotateMatrix ;
		} 
 
		protected function startDragging(event:MouseEvent):void
		{
			clickStartTime = new Date().time;
			moveToFront();
			startDrag();
		}
 
		protected function stopDragging(event:MouseEvent):void
		{
			var timeSinceClickStart:Number = new Date().time - clickStartTime;
			if (timeSinceClickStart &lt; CLICK_DRAG_THRESHOLD)
				dispatchEvent(new Event(PRESSED));
			stopDrag();
		}		
 
		//TODO: breaks OOP, another class should handle element index
		private function moveToFront():void
		{
			IVisualElementContainer(parent).setElementIndex(this, parent.numChildren - 1);
		}	
 
	}
}

For more information see:

5. Synchronising the image manipulation across devices

The synchronising was done via Flash Media Server. We didn’t know what to expect with the latency but it worked out fine. The other worry was that having lots of events being dispatched from the DragEvent might cause issues, but we haven’t noticed anything yet. Occasionally the drag sync gets left behind on a device and then catches up in a flurry of movement a bit later.

6. Saving an image to your device

The CameraRoll class comes to the rescue here again. addBitmapData will save your image to the device’s camera roll. There are no options to which folder you want to save it in which is a shame, but it works well none the less.

7. Multitasking in Android

Android doesn’t exit the application when you close it – it is still running in the background. Fortunately Flex has DEACTIVATE and ACTIVATE events which tell you when a user enters/exits an app. You can call NativeApplication.nativeApplication.exit(); to exit the application and unload it from memory.Be careful when using this. When you use the CameraRoll to select an image you’re actually exiting the app temporarily and then returning to the app. So when we initially tried this, the user would select an image from the CameraRoll and then return to the device homepage because Divvi had exited.The two solutions to this would be to have a timeout value or to implement a exit button in the app.

8. Getting rid of system menus on Android

In the app config file you can set “fullScreen” to true. That will remove the Android message bar at the top. Note that on the Xoom (and probably for all Honeycomb devices) you can’t completely get rid of Android bars since there are no physical back/home buttons. So a bar with Home/Back is always visible on Honeycomb no matter what you do.One thing to note is that the app appears to start with the bar and then resizes. So the initial stageWidth/Height property isn’t correct (i.e. it still has the bar in it). You’ll have to wait for a resize event to get proper dimensions.

9. Creating a splash screen

All flex mobile apps are contained in a MobileApplication spark container which has a “splashScreenImage” property. There are various settings for the splash screen (http://blog.everythingflex.com/2010/11/09/air-on-android-splash-screen/) but it doesn’t seem like you can force the orienation of it.

Conclusions

Flex mobile development for Android was a smooth and straightforward experience and I’d say that Adobe have done a good job. The documentation is pretty good, even this early on. Touch events are functional but not brilliant and performance seems good as you long as you take into account the reduced processing power of mobile and tablet devices.

Divvi: photo sharing - Specialmoves

Divvi: collaborative photo sharing