"local symbol '__bss_start' in global part of symbol table" only in Android NDK aarch64 build

Issue

I'm creating an Android library with NDK (r19b) for arm, aarch64, x86, and x86_64. All is OK, except that when I build an app for aarch64 architecture, I get the following error message.

ld.lld: error: found local symbol '_edata' in global part of symbol table in file libmystuff.so
ld.lld: error: found local symbol '_end' in global part of symbol table in file libmystuff.so
ld.lld: error: found local symbol '__bss_start' in global part of symbol table in file libmystuff.so

When I checked each build variant with readelf -s libmystuff.so, I noticed that only aarch64 is different.

[arm]
    4021: 007a30f0     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    4022: 007c6b10     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    4023: 007a30f0     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[x86]
    3848: 00c82c88     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    3849: 00ca4b28     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    3850: 00c82c88     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[x86_64]
    3874: 0000000000c9b890     0 NOTYPE  GLOBAL DEFAULT  ABS _edata
    3875: 0000000000ce5f68     0 NOTYPE  GLOBAL DEFAULT  ABS _end
    3876: 0000000000c9b890     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start

[aarch64]
       3: 0000000000b4f168     0 NOTYPE  LOCAL  DEFAULT  ABS _edata
       4: 0000000000b990e8     0 NOTYPE  LOCAL  DEFAULT  ABS _end
       5: 0000000000b4f168     0 NOTYPE  LOCAL  DEFAULT  ABS __bss_start

     865: 0000000000b9e3e8     0 NOTYPE  GLOBAL DEFAULT  ABS __end__
    2468: 0000000000b54168     0 NOTYPE  GLOBAL DEFAULT  ABS __bss_start__

I can surely see _edata, _end, and __bss_start are in LOCAL instead of GLOBAL, but I don't (or at least I think I don't) do anything special for aarch64; they all use the same build configuration.

jni/Application.mk
    NDK_TOOLCHAIN_VERSION := clang
    APP_STL := c++_static
    APP_CFLAGS := -fstack-protector-all -fvisibility=hidden -ffunction-sections -fdata-sections
    APP_CPPFLAGS := -fstack-protector-all -std=c++11 -fvisibility=hidden -ffunction-sections -fdata-sections -frtti
    APP_LDFLAGS := -Wl,--gc-sections,-fvisibility=hidden,--strip-debug

So, why is aarch64 different? Better yet, how do I move those to GLOBAL?

[UPDATE] Thanks to the gentle folks at https://github.com/android-ndk/ndk/issues/927, I found the solution works best; notice "-fuse-ld=lld" at the end.

APP_LDFLAGS := -Wl,--gc-sections,--strip-debug -fvisibility=hidden -fuse-ld=lld

This way, I still get to keep --gc-sections, and --no-fatal-warnings is not necessary either.

[MORE UPDATE] The example shown in this question is using ndk-build, and if you are using Android Studio with Gradle, you'll most likely have CMake. In that case, add your compiler flags as the following.

[app/build.gradle]
android {
    ...
    defaultConfig {
        externalNativeBuild {
            cmake {
                cppFlags '-fuse-ld=lld'
            }

Solution

APP_LDFLAGS := -Wl,-fvisibility=hidden

Does this do anything? This option doesn't appear in the help page for bfd, but those symbols are emitted by the linker so I'm wondering if that's what's causing this.

Failing that, try adding -fuse-ld=gold (or -fuse-ld=lld if you're feeling adventurous, but there are a number of Windows KIs with that if that's something you need to support) to your APP_LDFLAGS. The biggest difference between arm64 and the other architectures in the NDK is that we still use bfd for arm64.

Since these are symbols that are emitted by the linker, this seems like the most likely culprit.

Answered By - Dan Albert