"ld: error: undefined symbol" when using "repeated" keyword in protobuf

Issue

After hours of debugging, I have the following minimalist .proto file:

syntax = "proto3";

message PbCaptureResult {
    bool checkedValid = 1;
}

message PbCaptureResultSequence {
    PbCaptureResult captureResults = 1;
}

It compiles and links successfully. But, if I add a "repeated" like:

syntax = "proto3";

message PbCaptureResult {
    bool checkedValid = 1;
}

message PbCaptureResultSequence {
    repeated PbCaptureResult captureResults = 1;
}

then I have a link error and get:

cmd.exe /C "cd . && C:\Android\Sdk\ndk\22.0.6917172\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=aarch64-none-linux-android29 --gcc-toolchain=C:/Android/Sdk/ndk/22.0.6917172/toolchains/llvm/prebuilt/windows-x86_64 --sysroot=C:/Android/Sdk/ndk/22.0.6917172/toolchains/llvm/prebuilt/windows-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security  -Wno-deprecated-declarations -O0 -fno-limit-debug-info  -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libatomic.a -Wl,--build-id=sha1 -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -shared -Wl,-soname,libnative-lib.so -o libnative-lib.so @CMakeFiles/native-lib.rsp  && cd ."
ld: error: undefined symbol: google::protobuf::internal::RepeatedPtrFieldBase::AddOutOfLineHelper(void*)
>>> referenced by repeated_field.h:1767 (../../../../imported-lib/include\google/protobuf\repeated_field.h:1767)
>>>               CMakeFiles/native-lib.dir/src/main/cpp/authenticationLib/CaptureResultSequence.pb.cc.o:(google::protobuf::RepeatedPtrField<PbCaptureResult>::TypeHandler::Type* google::protobuf::internal::RepeatedPtrFieldBase::Add<google::protobuf::RepeatedPtrField<PbCaptureResult>::TypeHandler>(google::protobuf::RepeatedPtrField<PbCaptureResult>::TypeHandler::Type*))
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

If the "repeated" come before a "standard type" such as string then it compiles:

syntax = "proto3";

message PbCaptureResult {
    bool checkedValid = 1;
}

message PbCaptureResultSequence {
    repeated string captureResults = 1;
}

it's only if I try to repeat a custom message that I have an issue.

But in the protobuf website I found the following example:

message SearchResponse {
  repeated Result results = 1;
}

message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
} 

I have put the example as is in my .proto file and it failed to compile with the same linking error. I concluded that it's not a .proto syntax problem.

It's protobuf 3.15.5.

The generating command is: ./bin/protoc.exe --cpp_out=.. CaptureResultSequence.proto

I am static linking against libprotobuf.a (not libprotobuf-lite.a)

I have cross compiled protobuf myself

I initially thought that the issue was related to linking, but for me it doesn't explain why I can link without "repeated" but can't link with it.

I have spent 2 days on this, and I am sure it's obvious...

EDIT:

Indeeed "AddOutOfLineHelper" is defined in "repeated_field.cc"

using ar x libprotobuf.a, I can confirm that repeated_field.cc.o is included in the library.

What is strange it that without "repeated" messages, I have no link issues.

I assume that libprotobuf.a is also needed and linked if I don't use any "repeated" message. Maybe not ? how can I check ?

As requested. Here is my build command for protobuf:

#!/bin/bash
NDK_LOCATION=/home/xxx/Android/android-ndk-r21
INCLUDE_LOCATION=/home/xxx/Android/3rdparty/include
LIB_LOCATION=/home/xxx/Android/3rdparty/lib
ABI_LIST="arm64-v8a"
#ABI_LIST="arm64-v8a armeabi-v7a x86 x86_64"
SRC_LOCATION=/home/xxx/Android/protobuf-3.15.5/cmake
BUILD_LOCATION=${SRC_LOCATION}/build

for ABI in ${ABI_LIST}
do
    [ -d ${BUILD_LOCATION} ] && echo "the build location exists: deletting" && rm -rf ${BUILD_LOCATION}

    mkdir -p ${BUILD_LOCATION}
    cd ${BUILD_LOCATION}
    cmake ${SRC_LOCATION} -DCMAKE_TOOLCHAIN_FILE=${NDK_LOCATION}/build/cmake/android.toolchain.cmake \
    -Dprotobuf_BUILD_EXAMPLES=OFF \
    -Dprotobuf_BUILD_TESTS=OFF \
    -Dprotobuf_BUILD_SHARED_LIBS=FALSE \
    -Dprotobuf_BUILD_LIBPROTOC=FALSE \
    -Dprotobuf_BUILD_PROTOC_BINARIES=FALSE \
    -Dprotobuf_DISABLE_RTTI=ON \
    -Dprotobuf_WITH_ZLIB=OFF \
    -DANDROID_ABI="${ABI}" \
    -DANDROID_STL=c++_shared \
    -DCMAKE_BUILD_TYPE=Release \
    -DANDROID_NATIVE_API_LEVEL=android-28 &&
    make -j4
done

and on the Android side ndk section of Gradle file:

ndk {
    // Specifies the ABI configurations of your native
    // libraries Gradle should build and package with your APK.
    abiFilters 'arm64-v8a' //'armeabi-v7a', 'arm64-v8a'
}

externalNativeBuild {
    cmake {

        // Passes optional arguments to CMake.
        arguments  "-DANDROID_TOOLCHAIN=clang", "-DANDROID_STL=c++_shared", "NDK_DEBUG=1"

        // Sets a flag to enable format macro constants for the C compiler.
        // cFlags "-D__STDC_FORMAT_MACROS"

        // Sets optional flags for the C++ compiler.
        cppFlags "-Wno-deprecated-declarations" //, "-fexceptions", "-frtti"
    }
}

CMakeList.txt :

cmake_minimum_required(VERSION 3.10.0)


add_library( libprotobuf STATIC IMPORTED )
set_target_properties( libprotobuf PROPERTIES IMPORTED_LOCATION
        ${PROJECT_SOURCE_DIR}/imported-lib/${ANDROID_ABI}/libprotobuf.a )


find_library( zlib z )
find_library( log-lib log )
find_library( camera-lib camera2ndk )
find_library( media-lib mediandk )
find_library( android-lib android )
find_library( gl-lib GLESv2 )

file( GLOB_RECURSE app_src_files
        "${PROJECT_SOURCE_DIR}/src/main/cpp/*.c*" )


add_library( native-lib
             # Sets the library as a shared library.
             SHARED
             # Provides a relative path to your source file(s).
             ${app_src_files} )

include_directories(${PROJECT_SOURCE_DIR}/src/main/cpp)

# because we have cyclic dependencies, we need to link several times the same libary.
target_link_libraries( native-lib
        libprotobuf
        ${log-lib}
        ${zlib}
        ${camera-lib}
        ${media-lib}
        ${android-lib}
        ${gl-lib}
        
        libprotobuf
        ${log-lib}
        ${zlib}
        ${camera-lib}
        ${media-lib}
        ${android-lib}
        ${gl-lib}
        
        libprotobuf
        ${log-lib}
        ${zlib}
        ${camera-lib}
        ${media-lib}
        ${android-lib}
        ${gl-lib}

        )

Solution

After days on the issue, the problem was related to the include files.

Because I am doing a cross compilation for Android, I haven't done the "make install" step. That makes no sens to install the package on my dev machine. Thus, I just grab the .a files from the compilation folder and the include files from the sources.

Here was my mistake !

The include files that I have to put with the compiled libraries are only a subset of all the files found on the src/include folder... sounds obvious afterwards...

Thus I had to specify a temporary CMAKE_INSTALL_PREFIX and run make install. Then grab the include folder from that location.

Then everything works as expected.

Answered By - Blup1980