Tarantool child process crashes on crypto module loading on MacOS

added bug label

changed the description

As far as I can remember, we've had problems with mac os build related to some symbols being optimized out if they're not invoked directly, i.e. via dlopen/dlsym. Not sure if this is related.

Have you tried git bisecting? At some point in the past picodata definitely was working at least on some mac os machines

I'll use git bisect when I get totally desperate (now I still have some other ways for debugging - in the comment below).

The main suspect is the crypto module:

We produce SIGABRT when load crypto.lua file, than uses crypto module exported functions.
The src/lib/crypto module in the core of tarantool has a name conflict on the build stage with openssl crypto module (this problem was solved with openssl-prefix directory in the build.rs)

I've inspected my current tarantool (not picodata) builds on MacOS and they all dynamically link openssl@3, while picodata statically links vendor version of openssl@1.1g (light weight version without docs and tests). So, I have tried several building options for picodata:

Dynamically link system openssl@1.1 - same SIGABRT
Dynamically link system openssl@3 - same SIGABRT

This approach failed and didn't give me any information. Next steps:

Try to remove lj_trace_abort() from jit_secure_call() for debug purpose (I need a non-flushed stack).
Manually comment code blocks inside crypto.lua until I'll detect the problem.

Ok, finally commenting parts of the crypto.lua lead me on the trail: the crypto module exported functions don't have crypto_ prefix (we export only a pre-defined list of symbols in tarantool). For example, we fail with SIGABRT on calling ffi.C.crypto_EVP_get_digestbyname(class) function, while the picodata binary contains only _EVP_get_digestbyname symbol:

darthunix@macbook-arena picodata % nm -g target/debug/picodata | grep _EVP_get_digestbyname                      
0000000100dc9b40 T _EVP_get_digestbyname

While tarantool build contains the one:

darthunix@macbook-arena picodata % nm -g ~/git/builds/tarantool_picodata/bin/tarantool| grep _EVP_get_digestbyname 
                 U _EVP_get_digestbyname
000000010055d5f0 T _crypto_EVP_get_digestbyname

The symbol _EVP_get_digestbyname comes from libcrypto-ssl.a:

darthunix@macbook-arena picodata % nm --quiet -g target/debug/build/tarantool-sys/openssl-prefix/lib/libcrypto-ssl.a | grep _EVP_get_digestbyname   
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
0000000000000200 T _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname
                 U _EVP_get_digestbyname

I've manually played with export symbols on MacOS. test.c:

#include <stdlib.h>


int f() {
        return 0;
}

int export_f() {
        return f();
}

exports:

_export_f

A command to build a shared library:

cc -fpic -c test.c -o test.o
cc test.o -shared -dynamiclib -undefined suppress -flat_namespace -exported_symbols_list exports -o test.dylib

Exported symbols:

darthunix@macbook-arena exports % nm -g test.dylib                                                                                                 
0000000000003fb0 T _export_f

The only question is - why compiler inlines crypto_EVP_get_digestbyname into EVP_get_digestbyname for picodata build?

const EVP_MD *
crypto_EVP_get_digestbyname(const char *name)
{
	return EVP_get_digestbyname(name);
}

Interesting... _crypto_EVP_get_digestbyname symbol exists in the picodata's core libcrypto.a:

darthunix@macbook-arena picodata % nm --quiet -g \
./target/debug/build/tarantool-sys/tarantool-prefix/src/tarantool-build/src/lib/crypto/libcrypto.a | \
grep EVP_get_digestbyname
                 U _EVP_get_digestbyname
0000000000001710 T _crypto_EVP_get_digestbyname

But it is absent in the resulting picodata binary:

darthunix@macbook-arena picodata % nm --quiet -g ./target/debug/picodata | \
grep EVP_get_digestbyname 
0000000100dc9b40 T _EVP_get_digestbyname

If I manually link libtarantool.a and libcrypto.a together, all symbols are exported correctly.

darthunix@macbook-arena linking % libtool -static -o big_tarantool.a libtarantool.a libcrypto.a 
darthunix@macbook-arena linking % nm -g big_tarantool.a| grep EVP_get_digestbyname
                 U _EVP_get_digestbyname
0000000000001710 T _crypto_EVP_get_digestbyname

So, what is wrong with the picodata build logic?

Ok, it is the problem of the cargo compiler.

We compile the libpicodata-XXX.rlib with -l 'static:+whole-archive,-bundle=crypto flag. According to documentation:

When building a rlib -bundle means that the native static library is registered as a dependency of that rlib "by name", and object files from it are included only during linking of the final binary, the file search by that name is also performed during final linking.

So, the resulting libpicodata-XXX.rlib doesn't contain the _crypto_EVP_get_digestbyname symbol.
Then we link all the rlibs into the picodata binary. And at this stage we use only the rlibs (and no other dependencies) and as a result there is no _crypto_EVP_get_digestbyname symbol form the libcrypto.a in picodata.

The problem is that the linking modifiers +bundle and +whole-archive are not compatible with each other when generating rlibs (fixed in f40b5641).

Ok, I was wrong - libpicodata-XXX.rlib is a separate target from the picodata binary (we build it for binaries as well). But the problem is the same for binary - we use only rlibs that don't contain _crypto_EVP_get_digestbyname.

cargo build log: log1

I've added --print link-args to the final rustc command that builds the picodata binary. Here is the file with the output: log3

As we can see, after libpicodata-757f23358dc59070.rlib there are lots of linking options like

"-Wl,-force_load" "-Wl,/Users/darthunix/git/picodata/target/debug/build/tarantool-sys/tarantool-prefix/src/tarantool-build/src/lib/core/libcore.a"

but no libcrypto.a... Seem like libpicodata-757f23358dc59070.rlib doesn't depend on libcrypto.a. for some reason.

Very strange. I've added libcrypto.a to the libpicodata-757f23358dc59070.rlib (hacked the linking options for module crypto in the build.rs to "cargo:rustc-link-lib=static:-whole-archive,+bundle=crypto"), but still the binary made from this rlib doesn't contain _crypto_EVP_get_digestbyname

darthunix@macbook-arena picodata % nm -g \
/Users/darthunix/git/picodata/target/debug/deps/picodata-9e0291764b338dc0 | \
grep EVP_get_digestbyname 
0000000100dcab40 T _EVP_get_digestbyname

darthunix@macbook-arena picodata % nm --quiet -g \
/Users/darthunix/git/picodata/target/debug/deps/libpicodata-757f23358dc59070.rlib | \
grep EVP_get_digestbyname
                 U _EVP_get_digestbyname
0000000000001710 T _crypto_EVP_get_digestbyname

Oh... It seems that the linker uses -dead_strip option. It strips the unused code on the linking phase (and crypto_EVP_get_digestbyname() fucntion is used only inside Lua ffi calls in crypto.lua - not in the C code). I have found it out when used --print link-args:

RUSTFLAGS="--print link-args" cargo build -vvv --locked &> /tmp/log4

the log file: log4

RUSTFLAGS="-C link-dead-code=on" saved the day! Now it contains _crypto_EVP_get_digestbyname symbol!

mentioned in merge request !476 (merged)

closed with merge request !476 (merged)

It didn't work. To be fixed again in either !480 (closed) or !482 (merged).

reopened

mentioned in merge request !480 (closed)

mentioned in merge request !482 (merged)

closed with merge request !482 (merged)

mentioned in issue #240 (closed)

This is still reproducible on macos arm in --release builds.

Loading crypto module fails due to missing crypto_EVP_get_digestbyname symbol, which was supposed to be fixed in !476 (merged). The solution was to generate code which uses all of the exported symbols by calling a dummy black_box function. But the macos optimizer seems to have outsmarted us, and this trick doesn't work in release mode. Replacing our own black_box with std::hint::black_box fixes the disappearing exported symbols, but another error happens:

dyld[33618]: symbol not found in flat namespace '_luaT_tolstring.4'

More investigation is needed

reopened

removed bug label

added icebox label

added triaged label

mentioned in merge request !1414 (merged)

closed with merge request !1414 (merged)

(erroneously closed by sbroad import)

reopened

Tarantool child process crashes on crypto module loading on MacOS

Child items ...

Activity