Thoughts for mobile in 2013 and beyond

Today Canonical announced the mobile version of Ubuntu. For me as a Linux fan this is very exciting news, however, the developer in me screams: “Not another phone I need to target when writing mobile apps”. I’m surely not the only one thinking that.

Currently the major smartphone OSes on the market are Apple iOS, Google Android (and Microsoft Windows Phone). Upcoming OSes that I know of and follow are: Mozilla’s Firefox OS, Jolla’s Sailfish, Blackberry’s BB10, Samsung’s Tizen and Canonical’s Ubuntu. Some of those will be released this year, other in the following years. Some might not be released at all and new ones will also try to enter the market.

The current generation of devices mostly have a OS specific user experience (with an exception of Android where manufacturers can somewhat customise the experience). The clear trend is that device manufacturers and operators want to differentiate themselves with different user experiences and we (as consumers) also want to have a unique device that separates us from our friends, which fits with our personalities – meaning there is demand and room for different operating systems.

If a new device is to succeed it has to have quite a large repository of apps that the user can use. That can be a taunting task for the newcomers as getting developers to make an app for their device is next to impossible due to the huge costs associated with it. In most cases that process involves rewriting the application from scratch in a different language.

Here is a list of the official languages and native (UI) frameworks for each of the platforms:

  • Android – Java w/ Android framework
  • iOS – objC w/ Cocoa
  • Windows Phone – C# w/ Silverlight
  • Sailfish – C++ w/ QML
  • Firefox OS – JavaScript w/ HTML5
  • BB10 – C++ w/ QML
  • Tizen – C w/ Enlightenment Foundation Libraries
  • Ubuntu – C++ w/ QML

As anyone who has been involved in the app developing process knows, it’s a major headache developing two mostly identical apps, but it’s more or less manageable. Developing the app for even more platforms would be infeasible (especially financially).

We need tools, which would enable us to stop writing the same code over and over again for multiple platforms. There are various technologies available which do just that and can be used to create 3rd party apps, the most popular of which is the over-hyped HTML5. Others include Flash (for iOS, Android), .NET with Xamarin’s Mono products (for Android, iOS), various JavaScript based abstraction layers (PhoneGap, Titanium, etc) and the list goes on.

Many see HTML5 as the silver bullet here, but I tend to disagree for a couple of reasons. First the bar for performance was set really high by Apple when they released iOS and currently HTML5 cannot get anywhere near it. It is also quite difficult to get a single sane way of integrating with the device user experience due to the fragmented nature of HTML and browser technologies.

Flash based apps do run fine on iOS and Android, but they lack integration with the look and feel of the underlying device. Meaning the technology is only useful for games, where the user doesn’t expect them to integrate well with the system. There is also a problem of being locked into the platform more than with other technologies – when Adobe decides to discontinue their Flash products (or export to iOS/Android feature) that’s that.

The Xamarin team has brought the powerful C# language and .NET framework to both iOS and Android platforms. They have created bindings for almost 100% of the platform provided APIs, so the applications written using their technologies are as good as the ones written with the official platform frameworks. This means that when targeting multiple platforms (Andorid, iOS or Windows Phone) most of the code, which doesn’t deal with the UI directly, can be reused – resulting in fewer bugs, faster development time and decreased costs while still providing the user with a native performance and look-n-feel they’ve grown accustomed to. And because it’s all based on the open-source Mono project it is all true and tested, it’s fast and stable – even Unity, the powerful gaming engine, uses Mono as it’s internal game logic engine.

The last technology, which promises extensive code reuse, native performance and device specific UX is Digia’s Qt Quick (QML). Most of the new upcoming mobile OS’es use this technology and Digia is committed to enabling the use of this technology on iOS, Android and even Windows 8 devices. The power of Qt Quick is that it separates app code and UI very well, enabling designers to quickly create and experiment with UIs for different form factors and devices. The moment it is available on more than one smartphone platform, this technology will become a serious contender when developing apps for more than one platform, which is one of the reasons why most of the upcoming OSes are using it for their native apps.

And now for some predictions for 2013.

Instead of having to write and maintain several versions for the app in objC, Java and .NET companies and developers will be looking for technologies like .NET/Mono and QML to write performance critical apps and JS/HTML5 to write not-so performance critical apps. Reusing code enables more innovation on the UI side for multiple platforms.

This means that when currently objC/iOS and Java/Android developers are very valuable to companies creating mobile apps, their value will decrease towards the end of the year and .NET/iOS+Android+WP and C++/QML developers will become a sought-after employee in 2014.

There will be many more smartphones which aren’t iOS or Android devices and which will look and feel awesome, they will have access to most Android apps in addition to apps optimised for those devices. The hardware manufacturers, most of whom currently have almost no way to differentiate their offerings, can regain market share and release cool products that the consumers will want to buy. (Except Nokia. Sorry, but you dance to Microsoft’s slow and steady beat now. Maybe next year. Probably 2015 though.)

This also means that the user experience will range vastly from device to device. Giving room for experimentation and much more choice. Apps will be available everywhere and they will fit in and be of better quality. The overall application store concept will transcend platforms and a competition of who can provide the best analytical and developer tooling will start.

And now, lets wait and see what the next 12 months brings us; whatever it is, it won’t be boring!

Getting ProGuard to work in Eclipse

Up until yesterday, I thought that enabling ProGuard for an Android project would be just adding one or two lines somewhere. I was kind of right — it was adding a single line to enable it, and a day of Googling and experimenting to get it actually working.

I wasn’t using the latest SDK tools when I started, but while fiddling everything I upgraded to Android SDK Tools rev 20 and Android SDK Platform-tools rev 12, so the following applies to those versions, but might work for earlier or future versions as well.

During the trial and error process I encountered several different errors, some of them being more common than others.

The first problem was easy to find a solution on the web; ProGuard was throwing out warnings about the support-v4 library. Those just had to be disabled by adding an extra line to proguard.cfg:

-dontwarn android.support.**

After which there was my first encounter with the dreaded  ”Conversion to Dalvik format failed with error 1″ failed. This kept appearing and disappearing a lot. And as there is absolutely no logs as to why it happened my only guess is that ProGuard failed to produce Dalvik compatible bytecode — which in turn means something is wrong somewhere.

There are many-many alleged solutions to this problem on Stackoverflow, but most of them had little or no impact on solving the problem. However, two of them combined worked for me.

First, you need to double-check your build paths (Project Properties → Java Build Path) and under the Source tab remove all <library_project>_src folders if you have any; and under Libraries tab you want to remove everything except the Android x.x. After having done that you need to re-add the library dependencies by invoking Android Tools → Fix Project Properties.

Second, there appears to be an annoying bug in the actual ADT themselves that mess with ProGuard somehow. What you need to do is disable automatic building in Eclipse (Project → Build Automatically), then clean your entire project and then rebuild it (Project → Build All). Credits for this step go to Regex Rookie on Stack Overflow.

There was a third problem as well, some IllegalArgumentException in ProGuard, saying that some method in the compatibility library was using something from SDK v14 and it couldn’t find it.

Unexpected error while evaluating instruction:
  Class       = [android/support/v4/view/AccessibilityDelegateCompat$AccessibilityDelegateJellyBeanImpl]
  Method      = [newAccessiblityDelegateBridge(Landroid/support/v4/view/AccessibilityDelegateCompat;)Ljava/lang/Object;]
  Instruction = [18] areturn
  Exception   = 1 (Can't find any super classes of [android/support/v4/view/AccessibilityDelegateCompatIcs$1] (not even immediate super class [android/view/View$AccessibilityDelegate]))
Unexpected error while performing partial evaluation:
  Class       = [android/support/v4/view/AccessibilityDelegateCompat$AccessibilityDelegateJellyBeanImpl]
  Method      = [newAccessiblityDelegateBridge(Landroid/support/v4/view/AccessibilityDelegateCompat;)Ljava/lang/Object;]
  Exception   = 1 (Can't find any super classes of [android/support/v4/view/AccessibilityDelegateCompatIcs$1] (not even immediate super class [android/view/View$AccessibilityDelegate]))
java.lang.IllegalArgumentException: Can't find any super classes of [android/support/v4/view/AccessibilityDelegateCompatIcs$1] (not even immediate super class [android/view/View$AccessibilityDelegate])
	at proguard.evaluation.value.ReferenceValue.generalize(ReferenceValue.java:287)
	...

Solution for that was simple: bump up the target SDK version of the project to the latest one. Only thing to watch out with this one is that backwards compatibility stuff won’t be used when you run your app on newer platforms now – so you need to know what has changed in the platform and take that into consideration (ie with v14 the default AsyncTask behaviour changed).

And to recap, the check list for getting ProGuard to work:

  1. If using support library, bump up your target SDK to latest and add -dontwarn into proguard.cfg
  2. Check your build paths for rogue entries
  3. Disable automatic building, clean everything and rebuild

Hope this saves someone a couple of hours trying to figure this out.

Usage pattern for dynamically registered BroadcastReceivers

When registering BroadcastReceivers dynamically in your application, you need to keep track of them and unregister them. And when unregistering you need to make sure that you haven’t done it before, else you’ll get an IllegalArgumentException. Another annoyance is when you need to check if you actually registered the receiver already or not.

The following code sample illustrates a nice way to keep track of registered receivers and only allow deleting them once.

Getting started with Android C2DM

UPDATE: As of June 26, 2012 C2DM is deprecated and the new official way to send messages to your Android device is the GCM (Google Cloud Messaging). With GCM both ClientLogin and OAuth2 don’t work any more, so this article is effectively useless.

Having realized you need push notifications in your Andorid application you head over to the official C2DM page to read more about it. After finishing it you still don’t have a clear understanding on how to actually implement the 3rd party application server so that it would satisfy all of the criteria mentioned there. Here we will go through the very first steps to get you started: signing up for C2DM and getting a better understanding on how to authenticate with Google so you could access the API.

Signing Up for C2DM

As Android C2DM is still in early test phases you need to explicitly sign up for it by filling out a form at the sign up page. All of the fields are very straight forward except one, the “Role (sender) account email” - what is this “sender email”? On the C2DM page it is described as: This ID is typically role-based rather than being a personal account – for example, my-app@gmail.com.

This basically means that you need to create a new Google account, however, it does not specify if the Gmail inbox is required or can it be a plain Google account, maybe even a Google Apps account? Just to be on the safe-side it is wise to create a Google account with a Gmail inbox. You can do this from the Gmail page when logged out of your other accounts (or browsing incognito).

When signing up you will be automatically signed up for Google+, but don’t worry about it as the G+ profile can be easily deleted once you’ve created the account. You can do so on the Account overview page by clicking the “Delete profile and remove associated Google+ features” link at the very bottom of it.

Now that your Android app has its very own my.app@gmail.com Google account you can return to the C2DM sign up page and finish completing the form there. After you’ve submitted the form you will shortly receive an email to the contact email you specified and you will have access to the C2DM APIs.

Authenticating Yourself for API Access

The next step is getting yourself authenticated for actually communicating with the C2DM API. The C2DM documentation page mentions that you should use ClientLogin for Installed Applications, but the link itself is (as of writing this) dead. Quick search yields the correct location for ClientLogin information. And at the top of the page, there is a big notice in red, that this has been deprecated and future code should use OAuth 2.0. Sadly, there is not much information out there about using C2DM with OAuth 2.0.

After reading the documentation about using OAuth 2.0 with Google APIs we realize that we’ve got 3 different ways to approach this:

  1. Use the deprecated ClientLogin. Out of the three, this is the only one that actually has sample code. But it comes with some caveats – you never know when Google removes ClientLogin forcing you to update your code and ClientLogin cannot be completely automated, as occasionally it requires captcha completion to successfully get the authentication token.
  2. Use OAuth 2.0 and the Web Server authentication mechanism to obtain the access token. Even though it doesn’t seem to be a very good option at first as it requires the user to authorize the application in a web browser, it was to work by Alex.
  3. Use OAuth 2.0 and the Service Account authentication mechanism to obtain the access token. From all of the OAuth options this actually seems the best one to use for an unattended server application.

The ClientLogin’s quirk to force captcha checkis is a big no-no for most unattended server applications, so option 1 is out. Even though option 3 seems to be the best suited, it seems that the C2DM does not yet support that OAuth machanism. That leaves us with option 2, using the Web Server authentication with OAuth 2.0.

To get started with OAuth API access you first need to head on over to the Google APIs Console (while logged in with your application Google account you created previously). Because the C2DM is still in trial phases it does not show up under the Services tab there, but worry not, it still works.

Now you need to go to the API Access tab and create your web application the credentials it can use for OAuth 2.0 protocols. Hit the “Create another client ID…” button and a dialog should pop up. In there you need to select the “Web application” under Application type and hit (more options) link to expose 2 text areas. In the “Authorized Redirect URIs” text area replace the content with “https://code.google.com/oauthplayground/” (you need this in the next steps), and clear the “Authorized JavaScript Origins” text area. Hitting the “Create client ID” button at the bottom of the dialog window will create a new client ID for you to use.

Create Client ID dialog

Google APIs Console - Create Client ID dialog

The newly created Client ID will appear with it’s associated email address and client secret. Now we will use Google’s OAuth 2.0 Playground to do the initial authorization and obtain the refresh token, that can be used in your web application to obtain the access token that gives access to the C2DM APIs.

On the OAuth 2.0 Playground page open up the settings menu and fill it in like in the image below, into the OAuth Client ID and OAuth Client secret fields you need to copy the strings from the API Access tab in the API console that appeared after you created the new Client ID.

Playground C2DM Settings

OAuth 2.0 Playground - C2DM Settings

After filling in the configuration in the settings pane, you need to enter “https://android.apis.google.com/c2dm” into the “Input your own scopes” text box on the left hand side and press Authorize APIs. You will be redirected to another page that asks your permission to use your account – double check that this is the account you created, not your personal and click “Allow access” button.

C2DM Scope

OAuth 2.0 Playground - C2DM Scope

You will be redirected back to the Playgroung page with “Step 2″ being expanded on the left hand column. There you need to click the “Exchange authorization code for tokens” button to acquire the refresh token (if this does not appear double check that your “Access type” is Offline in the settings) and access token. Now you need to write down the refresh token as this is the one you will be storing in your web application configuration files for accessing the C2DM APIs.

To quickly test it all worked and you have access to C2DM API, you can use the access token that was returned and curl on the command line like this (don’t forget to replace the <access_token> with your access_token from the previous step):

$ curl -k -H 'Authorization: Bearer <access_token>' \
  https://android.apis.google.com/c2dm/send \
  -d 'registration_id=0bcfed8655ad4a0abddb051ac65da432' \
  -d 'collapse_key=0' -d 'data.test=test'

The C2DM should return Error=InvalidRegistration, unless you replaced the registration_id with a value that is actually associated with your sender account.

Now you are ready to write the rest of the server that confirms to the criteria cited in the C2DM documentation. Good luck!

QEMU as a separate machine on LAN

With QEMU you basically got 2 choices for networking:

  • Usermode networking, where QEMU does some magic and you can access internet from the Guest OS and redirect ports to the Guest OS,
  • TUN/TAP interface way which gives the Guest OS a separate IP on your LAN.

Usermode is not very good for running servers on them, so TUN/TAP it is. The problem with TUN/TAP is that on the internet there are many posts explaining it, but too many of them make it annoyingly hard or even worse, go about it the wrong way.

So, the easy and nice way: first you need to set-up your network connection as a bridged one. To do that globally we need to edit the /etc/network/interfaces file and make it look like this:

auto lo
iface lo inet loopback

iface eth0 inet manual

auto br0
iface br0 inet dhcp
	pre-up ifconfig eth0 down
	pre-up brctl addbr br0
	pre-up brctl addif br0 eth0
	pre-up ifconfig eth0 up
	post-down ifconfig eth0 down
	post-down brctl delif br0 eth0
	post-down brctl delbr br0

You probably want to reboot your computer for the changes to take effect.

Now the default /etc/qemu-ifup on Ubuntu looks like this:

#!/bin/sh

switch=$(/sbin/ip route list | awk '/^default / { print $5 }')
/sbin/ifconfig $1 0.0.0.0 up
/usr/sbin/brctl addif ${switch} $1

… and /etc/qemu-ifdown looks like:

#!/bin/sh

# NOTE: This script is intended to run in conjunction with qemu-ifup
#       which uses the same logic to find your bridge/switch

switch=$(/sbin/ip route list | awk '/^default / { print $5 }')

/usr/sbin/brctl delif $switch $1
/sbin/ifconfig $1 0.0.0.0 down

QEMU executes those scripts automatically, however, they expect to be run as root, which I don’t like, so we will have to execute them manually with sudo if we want to run qemu as a normal user. A simple QEMU wrapper that I use looks like:

#!/bin/sh

IFACE=$(sudo tunctl -b -u $(whoami))
sudo /etc/qemu-ifup $IFACE

qemu -net nic -net tap,ifname=$IFACE,script=no $@

sudo /etc/qemu-ifdown $IFACE
sudo tunctl -d $IFACE

This creates a new TAP interface, runs the qemu-ifup script to add it to the default bridge (br0) you should be using, runs QEMU, and afterwards cleans up after itself. And for me this is much more elegant solution compared to most out there.