android - Loading shared libs that depend on other shared libs

Question

Problem:

I am building Android app in Eclipse which uses shared lib libgstreamer-0.10.so (GStreamer-android NDK Bundle libs compiled for android-8 platform). I made new folder libs/armeabi in project root folder and put it there. Also, I have put all other libs that came with it (158 of them) in the same folder. If I put this in my main activity code:

static{
    System.loadLibrary("gstreamer-0.10");
}

And build/install/run my app on Android-8 emulator, it throws this error:

06-15 21:54:00.835: E/AndroidRuntime(402): Caused by: java.lang.UnsatisfiedLinkError: Cannot load library: link_image[1962]:    33 could not load needed library 'libglib-2.0.so' for 'libgstreamer-0.10.so' (load_library[1104]: Library 'libglib-2.0.so' not found)

Now, libglib-2.0.so is in the same folder as libgstreamer-0.10.so, and why is it not loaded? I get that linker tries to load it from /system/lib and libglib-2.0.so just is not there, but why is it not loading it from the location where libgstreamer-0.10.so is?

So I went to discover which libs libgstreamer-0.10.so depends on with this command:

arm-linux-androideabi-readelf -d libgstreamer-0.10.so

Results:

Dynamic section at offset 0x118b64 contains 29 entries:
  Tag        Type                         Name/Value
 0x00000001 (NEEDED)                     Shared library: [libglib-2.0.so]
 0x00000001 (NEEDED)                     Shared library: [libgobject-2.0.so]
 0x00000001 (NEEDED)                     Shared library: [libgthread-2.0.so]
 0x00000001 (NEEDED)                     Shared library: [libgmodule-2.0.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x0000000e (SONAME)                     Library soname: [libgstreamer-0.10.so]
 0x00000010 (SYMBOLIC)                   0x0

First four libglib-2.0.so, libgobject-2.0.so, libgthread-2.0.so, libgmodule-2.0.so are all located in the same folder libgstreamer-0.10.so is located in (/data/data/com.marko.gstreamer_test/lib) on the device.

Logical solution:

So, I tried to load these four libs before I load libgstreamer-0.10.so and, it worked:

static{
    System.loadLibrary("glib-2.0");
    System.loadLibrary("gthread-2.0");
    System.loadLibrary("gobject-2.0");
    System.loadLibrary("gmodule-2.0");
    System.loadLibrary("gstreamer-0.10");
}

My questions are:

1. Can I somehow tell the linker to load libs also from the app location? Like add path to some environment variable or something... similar to PATH on Linux.

2. Does my solution have some bad side-effects? I mean, linker would do this also before it loads the libgstreamer-0.10.so. But will this make any problems?

3. Can I install my libs to /system/lib folder on unrooted device?

Answer 1

According to https://groups.google.com/forum/?fromgroups#!msg/android-ndk/J3lzK4X--bM/4YaijymZy_AJ

Yes, and this is the documented behaviour: you must load libraries in reverse dependency order explicitely. [...] It is a limitation of the system.

In a nutshell, the dynamic linker doesn't know anything about your application (e.g. where its libraries live), it only knows about the LD_LIBRARY_PATH value that was set when the process was created. When you start an Android application, you really fork the Zygote process, you don't create a new one, so the library search path is the initial one and doesn't include your app's /data/data//lib/ directory, where your native libraries live. This means that dlopen("libfoo.so") will not work, because only /system/lib/libfoo.so will be searched.

When you call System.loadLibrary("foo") from Java, the VM framework knows the application's directory, so it can translate "foo" into "/data/data//lib/libfoo.so", then call dlopen() with this full path, which will work.

It libfoo.so references "libbar.so", then the dynamic linker will not be able to find the latter.

Add to this that even if you update LD_LIBRARY_PATH from native code, the dynamic linker will not see the new value. For various low-level reasons, the dynamic linker contains its own copy of the program's environment as it was when the process was created (not forked). And there is simply no way to update it from native code. This is by design, and changing this would have drastic security constraints. For the record, this is also how the Linux dynamic linker works, this forces any program that needs a custom library search path to use a wrapper script to launch its executable (e.g. Firefox, Chrome and many others).

I've emailed the author asking where this is documented.

Tor Lillqvist goes on to provide a workaround: https://groups.google.com/d/msg/android-ndk/J3lzK4X--bM/n2zUancIFUEJ

To be more verbose, what that lo_dlopen() function does is:

• Searches where the shared object in question is. It searches a set of directories passed to it by the Java code. The Java code looks at LD_LIBRARY_PATH and adds the app's lib directory to that.
• Opens the found shared object file and reads the ELF structures in it . Not all, but just enough to find out what shared objects it needs (the DT_NEEDED ones as displayed by arm-linux-androideabi-readelf -d). It calls itself recursively on the needed shared objects.
• Only after that, i.e. after making sure that all needed other shared objects have been loaded, it calls the real dlopen() on the found full pathname to the shared object.

You can find his code at http://cgit.freedesktop.org/libreoffice/core/tree/sal/android/lo-bootstrap.c?id=5510127e89d6971a219ce3664e4631d6c6dda2b1

UPDATE: According to http://code.google.com/p/android/issues/detail?id=34416 this code was integrated into Android as of December 2012. Yay! Dependencies are loaded automatically for devices with API level 18 and up. If you are supporting older API levels than that you still need to list the dependencies.