Idea: First-class Android NDK integration

tools and infrastructure
1

This is mainly a brain dump of my thoughts during my recent experiments. Time is still limited on my side due to job hunting, but I think it’s never too late to get the conversation going.

Motivation

Many of us probably would like to see Rust being used to develop native libraries for Android. There is already the jni-sys crate and a wrapper around it, so I went ahead and recorded the roadblockers I’ve hit. These are:

  • No way to compile for multiple NDK arches apart from multiple Cargo invocations
  • NDK home is not automatically discovered (ANDROID_NDK_HOME is the canonical environment variable to look at AFAIK)
  • The toolchain sysroot can’t be automatically derived without using manually-exported standalone toolchains, in which case a gcc wrapper is needed
  • RegisterNatives is not wrapped in the jni crate, so one must invoke the unsafe interface directly to register native methods if symbol names like Java_Adder_add__II are undesirable

Vision for first-class Android NDK integration

Overall from an Android developer’s perspective, one would expect the Rust build tools to integrate with the NDK, like when using CMake or ndk-build. This integration could go in a Cargo subcommand and various support crates, instead of bloating rustc or cargo, but if things could be better streamlined it’s also OK.

There are several areas that need work for such integration:

  • JNI wrapper crates:
    • better ergonomics exporting functions
    • native method announcement with RegisterNatives
  • Proposed Cargo subcommand (cargo android?) or Cargo itself:
    • Discovery of NDK home
    • Configuration of NDK arches, toolchain types (gcc or clang) and sysroot API level
    • Passing of said information to build scripts
  • gcc-rs:
    • Deriving sysroot for linking with the information provided, in addition to the Android support present today

What does such an integrated workflow look like?

Ideally one just specifies the arches and API level they want, perhaps in a [target] section in Cargo.toml, or some other file like .cargo-android.toml:

[ndk]
# host defaults to the native platform of the build tool, or you can override it if at all necessary:
# host = "linux-x86_64"
# platform API level
api-level = 16
# toolchain to use
toolchain = "gcc-4.9"

[target]
# NDK arches to compile for
# MIPS Android targets are TODO
arch = ["armeabi", "armeabi-v7a", "x86", "x86_64"]

Then write JNI wrappers and export them:

#![feature(proc_macro)]

extern crate jni;
extern crate jni_macros;

use jni::JNIEnv;
use jni::objects::JClass;
use jni::sys::jint;
use jni_macros::jni_export;

#[jni_export("com.example.Test.testMethod", "(II)I"]
fn test_method(_: JNIEnv, _: JClass, x: jint, y: jint) -> jint {
    x + y
}

After that one simply cargo android ndk-build to have all arches built, ready for use. Ideally the whole process could be driven from Gradle side, but let’s wait for Rust’s IDE support to mature before trying that.

I have implemented a simple jni_export macro in my jnisupport crate (not published yet; a name change is of course needed :joy:). But the rest is still very much TODO and have plenty of room for improvement. Hence the thread!

5 Likes
2

You can look at my crate rust_swig.

It also contains android-example to demonstrace rust + java cooperation on adnroid.

That interface

#[jni_export("com.example.Test.testMethod", "(II)I"]
fn test_method(_: JNIEnv, _: JClass, x: jint, y: jint) -> jint {
    x + y
}

is not near to enough, you need way to generate "Java classes" or do a lot of work, that possible to automate.

I doubt that such crates are usefull, you can get them via one call of bindgen,

plus if you use jni-sys, you can have problems with android/bitmap.h because of it also uses jni types, thus if you use jni-sys and call bindgen only for android/bitmap.h you get two variants of jni types inside different modules, write by hands such binding too boring, so I prefer just generate rust modules from c header on the fly with bindgen and not use trivial -sys modules.

3

I propose the name jni_in_a_bottle.

3 Likes
4

Sorry I'm not being super clear about the use case. What I have in mind is a primarily Java codebase seeking to gradually replace bits of itself with native code, for portability, code reuse with other platforms and such. Hence the interface is dictated by the Java side and the Java class is already written, no need to generate anything Java.

Auto-generating interface classes may be good for obfuscation since the symbol names could be mangled simultaneously, or if the component is meant to be reasonably isolated from the rest. But I believe for the use case I described above automating the native library side only is enough.

I know of the crate, but as I have said my experiment is initially very simple, and I didn't want to introduce additional complexity by having to write a build.rs for bindgen. Larger or legacy projects likely want to manage all the native deps themselves, of course.

5

I don't see much difference. Yes, primary goal of rust_swig to be glue between core part of project in Rust langauge and other part of project in another language.

So if you have several native functions spread around you several Java classes, it looks like it is not suitable.

But I think such smear of your native code around of Java code is bad for architecutre for many reasons (dependicy managment, not compile time checking, clearness).

Thus if you have several static native functions why not auto generate class like RustNativeMethods with all of that functions?

6

Oh no, of course software is not written like that. I mean first you have a class completely written in Java, then you gradually rewrite it in Rust, finally leaving only native methods and wrappers providing the same interface. Hence the class is not to be completely generated from the Rust side.

However I clearly see that putting native methods in auto-generated classes and referencing them later costs almost the same. In fact I'm planning to eventually do so due to obfuscation needs and the concerns you listed, but I'm only describing things step by step so the implementation is only preliminary at this stage. I'd like the whole process be driven by Gradle so that generated classes are correctly marked as such in Android Studio plus you only have to interact with only one build system, so I'm not touching this part yet. To put it in other words, I'm probably not going to add such functionality without first fleshing out Gradle support. Cargo-driven generation is already done by you, and I appreciate your effort, so perhaps we're just focusing on different priorities (Rust- versus Java-centric development).

7

@xen0n

I gave a link to android example (see my first post), it contains gradle + cargo integration, in fact 20 lines or so of groovy code.

And in my example you can build and run your java + rust project from android studio, in the same way like you do with ordinal java only project. Also generation ofr java code by cargo automaticaly seen by android studio, thanks to IntelliJ guys for file watching functionality.

You edit foreigner_class! in Rust langauge , press Run button in android studio, and all just works. And thanks to rust plugin for IntelliJ you can edit java and rust code in the same IDE - android studio.

Also gradle automaticaly build for arm + x86, so you can debug in emulator and on real device at the same time.

The real thing that bother me is need to create separate toolchain, just for linking, what you mention in the top post.

8

To solve the problem of using the Android NDK at work, I created a cargo plugin aptly named cargo-ndk.

Usage is quite straightforward, for example:

cargo ndk --target aarch64-linux-android --android-platform 25 -- build --release

The code is quite straightforward. Patches welcomed. :slight_smile:

3 Likes
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.