Android NDK binaries with rfo-Basic!  (Sep 2015 - outdated)

---

The only other time I remember was the very first output of my very first compiled language when a teenager.
Subsequent events fade into boredom and indifference. And that wonderment gets forgotten, especially as languages and systems become monotonously similar as the years drag on.

Surprisingly, the same feeling came back when executing a native c-binary from a basic! app on my Android phone. 
'The phone is running 'c' made code !'  For a few brief minutes, I was again a teenager.

I don't know wether it was because of my usual bias against Java running android phones, or that I had forgotten that a phone was really just a small sized PC, or that I had mentally placed NDK as just another mass of geeky dependent-hell files designed not to work for 'normal' people. But that the binary actually ran, freaked me out and gave me quite a buzz for the rest of the day.

The Android Native Development Kit is primarily a system of building cross compiled binaries of other languages to interface with Android's OS (which runs on Java). It relies heavily on using JNI as an interface to make library calls in-between the running binary and Java. But as you might have noticed, this website (about rfo-basic) avoids Java/XML due to it's complexities. So why use NDK at all?

Why Use It ?

It so happens that you can still compile NDK binaries and have them execute without JNI as long as you don't call the Java libraries. Such needs are typically tasks which are cpu intensive and don't need graphics or any other user interaction. e.g Background or service utilties e.g servers, batch file operations, or self contained utitlities which have already been proven and written in other languages.

Simple Interface

All we need to do is to find a way to execute and perhaps devise a simple way for the binary to talk to the basic! program.

Launching is not a problem, Basic! already has a built-in shell in the form of the system. command, of which we can use to launch the binary.

The next requirement is somewhere to put the binary, where we can give it permissions to execute.
All android apps are entitled a private area of storage called Primary Internal Private Storage. This is usually at location /data/data/<package_name>. It is created when the app is installed. The app has full rights here, so putting the binary here means we can give it the right to execute. Another bonus, is that the directory is wiped when the app is uninstalled.

For communication between the binary and basic!, you can either use a protocol of read/write to temporary files, or alternatively the stdin/stdout of the shell/binary.

But first things first, let's see how we can build an actual binary with NDK.

The development environment is more simple than you think. There is no special IDE, no drivers to install and you can install to anywhere you want. You don't even have to set any environment variables if you don't want. Configuration It is mostly setting values inside provided scripts.

1. Download the latest NDK distribution.

The NDK distro is a gigantic collection of compilers and toolchains (utilities for compilation) and examples that cater for windows, linux and mac.
Pick the distro that matches your development PC.
Download and run the self-extracter which will extract a tree of files to a directory of your choice.

2. Check for build script ndk-build.

In the main directory there is script file called 'ndk-build'

This is the main script to call for compilation. Note it's location, you will need it for later.

(For windows, you can use 'ndk-build.cmd' otherwise you need to install cygwin to be able to use the linux script)

3. Navigate to the standalone example file.

Open a terminal and cd into tests/standalone/basic-c-compile/jni/

(although we won't use jni, this is the standard sub-directory structure.)

This is where the source files and makefile go, and where we will be operating from.

You will find a file called main.c which is the 'hello world' c example.

Now we check if a makefile is here..

4. Create Android.mk

Below is the makefile. If one is not provided, create this one (infact, you should probably use this one);

Android.mk

This makefile takes one source file main.c, it will compile and link it with android's bionic library (a cut down version of the standard c library for android), and generate a binary called main.

In the middle of the makefile are 3 commented lines. The 3 lines determine wether PIE is turned on or off.

#APP_PLATFORM := android-16
#LOCAL_CFLAGS += -fPIE
#LOCAL_LDFLAGS += -fPIE -pie

#commented = PIE disabled ( off )
un-commented = PIE enabled ( on )

If your Android phone version is JellyBean or later (4.1+) i.e api16+, then un-comment the lines (PIE on).
If your Android phone version is older, i.e is less than 4.1,  then #comment the lines out (PIE Off)

(see later for explanation of PIE)

5. Create a compile script.

In this directory, create a script called go.sh (or go.cmd) and type in the path to the main build script ndk-build. e.g

go.sh

This means you can run this script instead of setting up environment variables with a path.
You will run 'go.sh' whenever you need to compile in this directory.

6. Compile

Run the compile script ./go.sh
This should generate any executables inside ../libs/armeabi/
i.e ../libs/armeabi/main
(It will also generate some files in ../obj/local/armeabi/ but we aren't interested in those.)

main is the binary executable which you will copy to your phone (see later).

For editor mode, the package_name is "com.rfo.basic".
For Apk, the package name is whatever you named your package(with your apk builder),
For both cases, the project directory will be /mnt/sdcard/<package_name>/

1. Copy the binary you compiled to your <>/data directory. e.g /mnt/sdcard/rfo-basic/data/main

2. In your program, initialise the basic shell and go to private storage /data/data/<package_name>

3. Copy over the binary to private storage

note: the binary should first be in <base>/data. If making a package either the apk-builder should ensure this or your basic program should pre-copy this from assets to <base> /data.

4. Make the binary executable

5. Run It.

6. Get the Output

7. Cleanup and close the shell

When you run this basic program, it should copy over and execute the binary in a shell, then spool any output to the screen. Any shell errors will normally show by the spooling. If all goes well, you will see "hello world" .

Basic! can execute binaries because of it's system command. This is a shell by which you can pass commands to. You can only open one shell at a time but even so, it is a powerful tool.

Process Properties

The shell is a truly spawned parallel process.
The executed binary is also a truly spawned parallel process.
Basic, the shell and the binary will all have differing PIDs.

Both the shell and the binary might be seen under 'Basic' in a task manager but the shell process is named "sh" and the binary process is named with it's called command e.g "./main"

Conditions for exit

If Basic exits or ends, it will also close the shell (if still open).
If the shell is closed, any spawned binaries will continue to run.
If Basic crashes, then both the shell and the binaries may continue executing.

Communication
Because the binary is forked from the shell, they will share the same environment. This means stdin/stdout will be shared.

Data from binary/shell to Basic.

Data from Basic to binary/shell.
Any system.write will be sent to stdin.
If the shell or binary is not reading stdin, the data will be buffered and Basic will continue to the next instruction (will not be blocked).

If the binary is reading from stdin and there is no data sent (from Basic), then the binary will be blocked.

In Jun 2012, Google began to enable PIE (Position Independent Executables) in Android as a security measure.
They sneaked this into the OS starting with JellyBean (4.1) and ending with KitKat (4.4). This was the 'grace period' whereby any phone within this range is able to execute both PIE and non-PIE compiled code. Any phone after this range (starting with Lollipop (5.0)) will only execute PIE binaries.

This means older phones (< v4.1) don't work with PIE while newer phones (>4.4) don't work without it.

The only choice left for binary makers then, was either to force the customer to upgrade their phones or make sure the package ships with both types of binaries. For the latter case, the app will need to choose which binary to execute at runtime.

Fortunately in Basic, you can get the OS version like this

a=0
device a
bundle.get a, "OS", s$

With a bit of manipulation, you can convert this to an integer value to decide which binary to use.
(For apk making, you will need permission READ_PHONE_STATE for the 'device' command)

  Copy over the binary to private storage matching the OS pie or no-pie version. (modify the program above like this;)

    Note that the final binary copied to private storage need not have the pie/nopie suffix.

Ofcourse this means you have to compile two versions of your binary, main-pie and main-nopie.
Android.mk can build both versions at the same time.

Android.mk

note: since NDK 10d:
PIE is supposed to be anabled by default if APP_PLATFORM := android-16
but experience suggests this is not enough. To be sure that PIE is enabled, also set :
LOCAL_CFLAGS += -fPIE
LOCAL_LDFLAGS += -fPIE -pie