1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Integrate a C++ library with your Android build system

Discussion in 'Papers' started by Stéphane Lenclud, Nov 7, 2017.

Tags:
  1. Stéphane Lenclud

    Stéphane Lenclud Founder Staff Member

    Purpose & Scope

    This article is intended for a software engineer audience developing Android applications.
    We will explain how to integrate a CMake based C++ library with your Android build system.
    That includes compiling and debugging your C++ library from Android Studio along side your application.
    This solution was implemented in October 2017 on Android Studio 2.3 and proved to work just fine with Android Studio 3.0 too.

    Disclaimer

    I started coding aged 12 and been working as a software engineer since 1997 however do not take my words for granted as I still have very limited experience on the Android platform.
    Considering the flexibility of Gradle there is most certainly various ways to accomplish our goals nevertheless the solution presented in this document worked well for us.

    Background

    Our team of three software engineers inherited quite a large project with almost 400K source code lines in Java and nearly 3M in C++.
    Parts of that code base has been under development for over a decade.
    The Continuous Integration setup from the former team was using two build machines running separate instances of Jenkins:
    • Linux build machine doing the C++ library builds using a CMake custom tool chain.
    • Windows build machine doing the Android builds after downloading the native library binaries.
    We use Android Studio and Visual Studio on our Windows developer workstations.
    To build that C++ library on Windows developer's machines the former team gave us a Linux Virtual Machine image that was put together years ago.
    That setup worked well enough but had the following inconvenience:
    • Two build machines to administrate.
    • Linux build machine setup arcane. None of us had the knowledge to redeploy such a build machine if it failed.
    • Convoluted C++ build process, slowing down developers.
    • No debugger support for C++ code, slowing down maintenance and new feature development.
    Therefore we decided to build that C++ library with Gradle together with other Java modules and our application.

    Proof of concept

    Native build from application

    Luckily for us, nowadays the Android Gradle Plugin does support inclusion of CMake projects through that ExternalNativeBuild.cmake property.
    Therefore, in theory, it should be as simple as referencing our top level CMakeLists.txt from our application build.gradle.
    It appears this is the approach taken by every code sample I came across such as this hello-gl2 sample.
    That worked well enough for our proof of concept and within a couple of days we could build our C++ library.

    Not variant friendly

    However this results in building your library once per-variant of your application which was not the intended behaviour and caused inflated build times. In fact, the documentation linked above mentions that the cmake block "Encapsulates per-variant configurations for your external ndk-build project".
    As a consequence this approach did not scale well with our application's 17 variants. In fact it would take Android Studio about 45 minutes just to synchronise after touching our build.gradle.
    For the purpose of our proof of concept we therefore commented out most variants.

    Specify your ABIs

    To further speed up your build you will also need to tell Gradle which ABI you want to build as by default it seems to be generating all supported ones.
    This is done using that abiFilters property, see samples below.

    Debugging

    Our C++ build was now working, albeit without variants, but still debugging was not.
    It turns out that, by default, Gradle for Android strips out debug information from your binaries before packaging them in your APK.
    To tell Gradle to keep debug information you will need something like that:
    Code (Groovy):
    1. android {
    2.     // ...
    3.     packagingOptions {
    4.         // By default .so libraries are striped from debug information when creating APK
    5.         // To prevent this make sure your .so files are matching that doNotStrip pattern
    6.         // Our patterns matches only debug variant of our libraries as named by our CMake configuration
    7.         doNotStrip "**D.so"
    8.     }
    9.     // ...
    10. }

    Final solution

    Embolden by our proof of concept success we were then looking for a solution we could actually use in our production environment.
    That meant fixing our variant scaling issue.
    In the following samples we will be assuming our C++ library is called Engine.

    Native build from library

    The way forward was to move our native build from our application to an Android library which would be included by our application as a dependency.
    Where an android application build produces an APK the packaged output of an Android library will be an AAR zip archive. If like myself you are new to the world of Android or even Java libraries I strongly recommend you read through this excellent Android Library documentation.
    We would then now have a directory structure similar to that one:
    Code (Text):
    1.  
    2. project
    3. ├ build.gradle
    4. ├ settings.gradle
    5. ├ app
    6. │ └ build.gradle
    7. └ lib
    8.   └ build
    9.     ├ AndroidManifest.xml
    10.     ├ build.gradle
    11.     └ CMakeLists.txt
    12.  
    Below you can find samples of some of the key files in that directory structure.

    project/settings.gradle

    Code (Groovy):
    1.  
    2. // Here we define our Engine module making sure it is named properly and not just using the parent folder of the Gradle file.
    3. include 'Engine'
    4. project(':Engine').projectDir = file('lib/build')
    5.  

    project/app/build.gradle

    Code (Groovy):
    1. apply plugin: 'com.android.application'
    2.  
    3. android {
    4.  
    5.     packagingOptions {
    6.         // By default .so libraries are striped from debug information when creating APK
    7.         // To prevent this make sure your .so files are matching that doNotStrip pattern
    8.         // See Gradle pattern documentation for details:
    9.         // https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/util/PatternFilterable.html
    10.         // Needed in both build.gradle file, library and application
    11.         doNotStrip "**D.so"
    12.     }
    13.  
    14.     buildTypes {
    15.         debug {
    16.             debuggable true
    17.             jniDebuggable true
    18.             renderscriptDebuggable true
    19.             minifyEnabled false
    20.         }
    21.  
    22.         release {
    23.             debuggable false
    24.             jniDebuggable false
    25.             renderscriptDebuggable false
    26.             minifyEnabled true
    27.         }
    28.     }
    29. }
    30.  
    31. // Here we define our dependency on our Engine library
    32. dependencies {
    33.     // Select Engine Release or Debug build.
    34.     // See: https://developer.android.com/studio/projects/android-library.html#publish_multiple_variants
    35.     // To debug Engine uncomment the following line and comment out the next one.
    36.     //debugCompile project(path: ':Engine', configuration: 'debug')
    37.     debugCompile project(path: ':Engine', configuration: 'release')
    38.     releaseCompile project(path: ':Engine', configuration: 'release')
    39. }
    40.  
    Note that by default we still use the release variant of our library even for application's debug builds as Engine debug builds do run significantly slower than the release ones.
    Developers must then locally enable debug build to be able to use C++ breakpoints.

    project/lib/build/build.gradle

    Code (Groovy):
    1. apply plugin: 'com.android.library'
    2.  
    3. android {
    4.     compileSdkVersion 24
    5.     buildToolsVersion '25.0.0'
    6.     // Make sure we publish debug variant too
    7.     // See: https://developer.android.com/studio/projects/android-library.html#publish_multiple_variants
    8.     publishNonDefault true
    9.  
    10.     sourceSets {
    11.         main {
    12.             manifest.srcFile 'AndroidManifest.xml'
    13.             // If your native library comes with some JNI you'll need to compile them too
    14.             // We want to compile the following Java files
    15.             java.srcDirs '../java'
    16.         }
    17.     }
    18.  
    19.     packagingOptions {
    20.         // By default .so libraries are striped from debug information when creating APK
    21.         // To prevent this make sure your .so files are matching that doNotStrip pattern
    22.         // See Gradle pattern documentation for details:
    23.         // https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/util/PatternFilterable.html
    24.         // Needed in both build.gradle file, library and application
    25.         doNotStrip "**D.so"
    26.     }
    27.  
    28.     defaultConfig {
    29.         minSdkVersion 21
    30.         targetSdkVersion 25
    31.  
    32.         multiDexEnabled true
    33.         dimension "default"
    34.  
    35.         externalNativeBuild {
    36.             cmake {  
    37.                 // Build only the following targets
    38.                 abiFilters 'armeabi'  
    39.                 // Android options
    40.                 arguments '-DANDROID_PLATFORM=android-14',
    41.                         '-DANDROID_TOOLCHAIN=gcc',
    42.                         '-DANDROID_STL=gnustl_static',
    43.                         // CMAKE options
    44.                         "-DCMAKE_INSTALL_PREFIX=.externalNativeBuild/engine_install",
    45.                         // Engine options
    46.                         "-DENGINE_DUMMY_BUILD_OPTION:STRING=ON",
    47.             }
    48.         }
    49.     }
    50.  
    51.     externalNativeBuild {
    52.         cmake {
    53.             path 'CMakeLists.txt'
    54.         }
    55.     }
    56. }
    57.  

    Conclusions

    We've seen how native libraries should be integrated in their own Android library module rather than directly from an Android application module.
    To further optimise your native libraries' build time you should specify whatever ABI you really need.
    To be able to debug native code you want to make sure your binaries are not striped from their debug information before packaging.

    References

     
    Last edited: Nov 15, 2017

Share This Page