insanelab.com insanelab.com

June 26, 2018 - Mobile Development

MultiDex in Xamarin

Nowadays modern mobile users are very exigent – they want outstanding, unique material design, in-app maps, fast and precise geolocation, social (let’s use the synonym – Facebook) sign-in, and more.

The problem is that these elegant features require significant space. Every line of code you write is transformed into some weird thing called virtual machine bytecode – which, eventually, is going to be executed by a virtual machine.

Read also: Get Familiar with Xamarin Workbooks

Android Runtime uses two virtual machines – Dalvik and ART (available since Android 4.4, standard since Android 5.0). Both of those virtual machines execute Dalvik instructions stored in .dex files. The problem with Dalvik instruction set is: they are capable of referencing at most 65,536 methods (every method gets its unique id from to 0 to 65,536 – id has 16 bytes size)

You might wonder: so what? 65,536 is a pretty big number, I will never reach that!

Well in theory – yes. In practice – no.

Every library you add, every .dll you reference adds methods/classes/and more. Also, it has to be transformed to a dex file.

For instance: including Google Play Services adds about 30,000 methods to your small, tiny Android project. Almost half of the available limit!

Let’s see what happens when you reach said limit.

65,536 Limit in Practice

We will start with a small sample.

  1. Go to your Tools/Options/Build and Run options (VS)
  2. Check MSBuild project build output verbosity to “Detailed” – we want to get a detailed error message in case of build failure.
  3. Create a clean Android project and add packages.xml file (see below).
  4. Type in Nuget Packages Manager Console Update-Package -Reinstall.
# csharp
<?xml version=”1.0″ encoding=”utf-8″?>
<packages>
<package id=”Bolts” version=”1.4.0″ targetFramework=”monoandroid60″ />
<package id=”HockeySDK.Xamarin” version=”4.1.0″ targetFramework=”monoandroid60″ />
<package id=”MvvmCross.Binding” version=”4.3.0″ targetFramework=”monoandroid60″ />
<package id=”MvvmCross.Core” version=”4.3.0″ targetFramework=”monoandroid60″ />
<package id=”MvvmCross.Droid.Shared” version=”4.3.0″ targetFramework=”monoandroid60″ />
<package id=”MvvmCross.Droid.Support.V4″ version=”4.3.0″ targetFramework=”monoandroid60″ />
<package id=”MvvmCross.Platform” version=”4.3.0″ targetFramework=”monoandroid60″ />
<package id=”NineOldAndroids” version=”2.4.0″ targetFramework=”monoandroid60″ />
<package id=”Xam.Plugins.Android.SlidingUpPanel” version=”3.3.0″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.Animated.Vector.Drawable” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.CustomTabs” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.Design” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.v4″ version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.v7.AppCompat” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.v7.CardView” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.v7.Palette” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.v7.Preference” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.v7.RecyclerView” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Android.Support.Vector.Drawable” version=”23.4.0.1″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Azure.NotificationHubs.Android” version=”0.4.0″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Bindings.AdvancedRecyclerView” version=”0.86.0″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Bindings.CircleImageView” version=”2.1.0″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Bindings.MaterialDateTimePicker” version=”2.5.0″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Bindings.UniversalImageLoader” version=”1.0.4″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.Facebook.Android” version=”4.13.2″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.GooglePlayServices.Ads” version=”29.0.0.2″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.GooglePlayServices.Base” version=”29.0.0.2″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.GooglePlayServices.Basement” version=”29.0.0.2″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.GooglePlayServices.Location” version=”29.0.0.2″ targetFramework=”monoandroid60″ />
<package id=”Xamarin.GooglePlayServices.Maps” version=”29.0.0.2″ targetFramework=”monoandroid60″ />
</packages>

Make sure you have Java Max Heap Size set to 1G in your Android project options. Without that, Java process used during app compilation will crash due to OutOfMemoryException.

If you try to build this project you will get:

# csharp
 trouble writing output: Too many field references: 65723; max is 65536.

C:Program Files (x86)MSBuildXamarinAndroidXamarin.Android.Common

… and we even haven’t written a single line of code.

We have Material Design libraries – either provided by Android Support Libraries or custom one, we have HockeyApp crash-log support, Geolocation, Maps, Ads support and… and we failed.

MultiDex to the Rescue

As I have previously described – the problem lies in .dex / Dalvik Instruction set limitation. MultiDex is a technique of splitting your application code into multiple .dex files – each containing at most 65 KB of bytecode.

To enable MultiDexing in Xamarin.Android just go to your project settings and check “Enable Multi-Dex” (pic rel.)

MultiDex in Xamarin

Now try to build the project. Build error should go away, the application should work.

Make sure you use Android SDK Build Tools with version 23.0.3 – so you will have same reproduction sample as the one built on my computer.

To force Xamarin to use custom build tools version edit your .csproj file – add this line:

# csharp
<AndroidSdkBuildToolsVersion>23.0.3</ AndroidSdkBuildToolsVersio

Inside your Configuration PropertyGroup.

Too Easy to Be True?

Sample app with packages config presented above works fine on all API levels. However, change packages.xml to this:

The next step is to add custom Application class:

# csharp
[Application]
[Register(“com.app.CustomApp”)]
public class CustomApp :  Application
{
protected CustomApp(IntPtr javaReference, JniHandleOwnership transfer) : base(javaReference, transfer)
{
}

public CustomApp()
{
}

public override void OnCreate()
{
base.OnCreate();

new CustomClass();
}

class CustomClass
{

}
}

Let’s run our new sample application on device/emulator with Android 5.0 or higher.

Works as expected – the app does nothing.

Try that on a bit older device with Android less than 5.0 – I run it on Jellybean 4.2.

It’s crashing. By examining logcat logs we can see it crashed with NoClassDefFoundException.

MultiDex in Xamarin

Why Does the Application Crash on API Less Than 21?

As I have previously mentioned, the MultiDex technique relies on splitting your application code into multiple .dex files. To explain why it has crashed on an API lower than 21, first you need to understand how Android handles .dex files on different API levels – from Android documentation:

Android 5.0 (API level 21) and higher uses a runtime called ART which natively supports loading multiple dex files from application APK files. ART performs pre-compilation at application install time which scans for classes(..N).dex files and compiles them into a single .oat file for execution by the Android device. For more information on the Android 5.0 runtime, see Introducing ART.

Versions of the platform prior to Android 5.0 (API level 21) use the Dalvik runtime for executing app code. By default, Dalvik limits apps to a single classes.dex bytecode file per APK. In order to get around this limitation, you can use the multidex support librarywhich becomes part of the primary DEX file of your app and then manages access to the additional DEX files and the code they contain.

The problem is: some important code related to MutliDex/Application initialization can accidentally land in a secondary .dex file – therefore we will see an exception similar to the one shown on the picture from previous paragraph (cause code from secondary dex can’t be loaded on app Startup!).

The reason why we have added loads of libraries and custom application class is to “manually” provoke moving Application initialization classes to secondary dex file (this step usually is done automatically – by your in-app source code).

Sometimes, your application might even work fine on API less than 21, but it can crash when it’s loaded by some kind of service (like application launched by push notification handler service) – code needed to launch an app by push notification can be moved to secondary dex file (real-world example I had in past)!

Can We Fix That, Please?

In an ideal world, it should work automatically. However, Android SDK Build tools is not bug-free. During the build, Android SDK tools automatically detect what is important and what should be kept in the first .dex file. It does not always work perfectly though.

Android has some kind of “MultiDex.keepthat classes in first dex” file concept. You add class names you want to preserve in a text file and they won’t be moved to secondary dex file.

Xamarin has not always supported this feature – but after my feature request (https://bugzilla.xamarin.com/show_bug.cgi?id=35491) they have added Build action called “MultiDexMainDexList”

We just need to find out which classes we need to preserve.

The fastest and most reliable way to do that is by examining and decompiling .dex bytecode.

Download two tools:

  1. https://github.com/pxb1988/dex2jar – convert .dex bytecode to jar
  2. https://github.com/java-decompiler/jd-gui/releases – decompile and browse jar

Steps to examine the dex file:

1. Go to your %Project Root Folder%/obj/Debug/android/bin, copy classes2.dex file to your dex2jar tool folder.

2. Type in command line: dex2jar classes2.dex -o classes2Jar.jar

MultiDex in Xamarin

3. Launch jd-gui tool and open generated classes2jar.jar

MultiDex in Xamarin

The missing class is mono/MonoPackageManager.class, lets use this information.

Add a new text file to project (ex. multidex.keep), set its build action to “MutliDexMainDexList”:

# csharp
mono/android/app/ApplicationRegistration.class
mono/android/app/NotifyTimeZoneChanges.class
mono/MonoRuntimeProvider.class
mono/MonoPackageManager.class
mono/MonoPackageManager_Resources.class

I have added classes from /android/app/ as well – as it looks that they might be used during app initialization too.

Rebuild the app. It should work now.

Summary

By now you should know how to bypass the 65kB limit and how MultiDex process works – if you use it, make sure you completely test your app on API less than 21. We have to be careful because MultiDex has its pitfalls.

Before going with MultiDexing – try to decrease application size using Proguard or Xamarin Linker. MultiDex can increase Startup Time up to 15% on Dalvik-based Android devices (due to secondary dex file loading on startup).

Want more news and tips on mobile development?

Sure you do.

What is your challenge?

Tell us with any means provided. We'd love to hear from you!