When using git describe fetch depth for shallow clones needs to be deep
enough to reach exiting tag. We used to only deepen our submodules for
tarantool build but now we reached the same problem for picodata. So
fetch tags for it as well.

allow_failure in test-helm-docker-image makes CI properly fail and
prevent merges when there are failures. Without it it didnt go past
"manual action required" icon and allowed to merge CI without fixing
other test jobs

Having dynamic build as a default was dictated by certification and now
when we merged certification patches back to master it started to cause
friction. So this patch makes static build the default and introduces
dynamic_build cargo feature instead of static_build one.

New feature `static_build` is added to Cargo.toml. By default dynamic
build is being used. build.rs is changed to respect it.

Use `cargo build --features static_build` to build static binary.

Previous attempt directly modified static-build cmake files which
is less convenient and doesnt allow us to support both build types
at the same time

Replaces https://git.picodata.io/picodata/tarantool/-/merge_requests/143 and https://git.picodata.io/picodata/picodata/picodata/-/merge_requests/817

Web UI feature is already enabled in CI `test` stage. This patch adds it
to the `pack` stage.

To do that, a path to a webui buldle can be passed explicitly, and it
does in CI which now builds it in a separate stage.

In default Rust features `webui` remains disabled by default to keep
minimal development environment truly minimal.

Dmitry Rodionov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

It has been observed that on Fedora -export-dynamic triggers some
reincarnation of this bug:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=47390

TLDR: it treated leading -e as entrypoint argument so resulting binary
was completely broken, i e there was no main in it and entrypoint was
set to unrelated deregister_tm_clones. Big thanks to @gmoshkin for
finding the solution and to @funbringer for assistance with debugging

Dmitry Ivanov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This fixes failing builds on Arch Linux, Fedora and other Linux
distros which don't provide static libgomp or build it without -fPIE.

See https://git.picodata.io/picodata/tarantool/-/merge_requests/107

Some code in libldap which we don't need requires libresolv,
so we should relax linking rules to let linker drop it.

Closes #240

Feodor Alexandrov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

This commit reverts "build: fix libcrypto renaming" (9485e9c1). The fix
proposed in that commit actually broke the whole build instead of fixing
re-build. Since there are two `libcrypto.a` (tarantool one and symlinked
openssl), the linker sometimes loads the same library twice (`-lcrypto` and
`-lcrypto-ssl`).

This patch provides another solution - tarantool lib is renamed so now
`-lcrypto` uniquely corresponds to openssl and
`-ltcrypto` to the tarantool library.

Renaming libcrypto breaks the rebuid. Repro:

```bash
cargo build -vv
rm -rf target/debug/picodata-*
cargo build -vv
```

It results in

```
[build.rs:204] running ["cmake" "--build" "target/debug/build/tarantool-sys/tarantool-prefix/src/tarantool-build"]
gmake[2]: *** No rule to make target 'target/debug/build/tarantool-sys/openssl-prefix/lib/libcrypto.a', needed by 'src/tarantool'.  Stop.
```

Instead of renaming this patch makes a symlink.

Current commit introduces distributed SQL in picodata (we use
'sbroad' library). Now it is possible to execute cluster-wide
SQL with 'pico.sql([[query]], {params})' lua method.

Current limitations:
1. DDL is still not implemented in picodata, so space sharding
   is mocked by '_pico_space' space (we'll remove it later).
2. We still have duplicating symbols of the Tarantool module in
   sbroad and in picodata rlibs.
3. We need to reuse piocdata's sharding functions and handle the
   case when different spaces in the query have different sharding
   functions.

Co-authored-by: Georgy Moshkin <gmoshkin@picodata.io>

Denis Smirnov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

We don't want to strip unused symbols from the static libraries on
the linking stage. This leads to problems with tarantool's crypto
module. Some symbols ("crypto_EVP_get_digestbyname" for example)
were used only by the ffi calls inside `crypto.lua` file. So the
linker stripped these symbols as it was not able to find their
usage in the rest of the C code. This produced SIGABRT on the
picodata startup when tarantool_lua_init() initialized the crypto
module.

Denis Smirnov authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

An attempt to start picodata on macos returns an error:
symbol not found in flat namespace '_res_9_getservers'

The reason is that macos executables require libresolv.dylib to
use resolv.h.

Yaroslav Dynnikov authored 2 years ago and Alexey Protsenko committed 2 years ago

The build process was refactored recently in b9aeab59 and it broke the
packaging in a tricky way.

Packpack wraps the whole build process in a Makefile which exports its
own `MAKEFLAGS` environment variable:

```
MAKEFLAGS="w -j -- BUILDDIR=/build"
```

The `cmake` crate used to override it with `CARGO_MAKEFLAGS` provided by
cargo. But after we've replaced it with `std::process::Command`, the
`BUILDDIR` setting has made its way to the `icu` build, which suddenly
clashed one of internal `make` variables and wrote there resource files.
As a result, the `icu` build started to fail with the error:

```
icupkg: unable to open input file
"/build/usr/src/debug/picodata-22.11.0.158/
   /target/debug/build/tarantool-sys/icu-prefix/src/icu-build/
   /data/out/build/icudt62l/af.res"
```

while all resources were copied directly to `/build/af.res`.

This patch repeats the behavior of the `cmake` crate.

It also makes logging more verbose and aligns the output. Replacing
`eprintln` with `println` is necessary since `make` and `cmake` both
write the logs to the `stdout`.

Yaroslav Dynnikov authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

Cargo used to complain:

```
error: the linking modifiers `+bundle` and `+whole-archive` are not
compatible with each other when generating rlibs
```

Also remove unnecessary subdir

1. Encapsulate printing rustc flags in their own mod.
2. Arrange paths formatting.

1. Remove build dependency on cmake crate. It doesn't fulfill all our
   needs, so we run cmake directly via `std::process::command`. It also
   makes `build.rs` easier to understand.
2. Arrange paths variable names.

Provide a supplementary `Run` trait for the
`std::process::Command`.

We haven't used rustc < 1.61 for a long time.