Dmitry Ivanov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

Instead of using ctest, let's simply run the CheckDependencies.cmake as
a post build command if Tarantool was built without dependencies. The
good thing about it is that the check will run even if the static build
is created directly, without the /static-build/CMakeLists.txt wrapper.

Part of #9242

NO_DOC=build
NO_TEST=build
NO_CHANGELOG=build

(cherry picked from commit fa4939bd)

Similarly to release_asan_clang but to test debug build. It is also run
only under `asan-ci` and `full-ci` labels.

Fiber stack size is 2 times bigger than in the release workflow for luajit
tests to pass. Note that this factor is a wild guess.

Part of #7327

NO_TEST=ci
NO_CHANGELOG=ci
NO_DOC=ci

(cherry picked from commit 980ad3f4)

Performance tests added to perf directory are not automated and
currently we run these tests manually from time to time. From other side
source code that used rarely could lead to software rot [1].

The patch adds CMake target "test-perf" and GitHub workflow, that runs
these tests in CI. Workflow is based on workflow release.yml, it builds
performance tests and runs them.

1. https://en.wikipedia.org/wiki/Software_rot

NO_CHANGELOG=testing
NO_DOC=testing
NO_TEST=testing

(cherry picked from commit 5edcb712)

Remove hard-coded compiler version for the `test-release-asan` target
in the .test.mk file.

NO_DOC=make
NO_TEST=make
NO_CHANGELOG=make

(cherry picked from commit ed35713e)

All LuaJIT related LSan warnings were suppressed in the scope of the
commit 985548e4 ("asan: suppress all
LSAN warnings related to LuaJIT"), since all compiler flags tweaks were
enclosed in LuaJIT CMake machinery. As a result of the commit in LuaJIT
submodule tarantool/luajit@a86e376 ("build: introduce LUAJIT_USE_ASAN
option") ASan and LSan support has been finally added to LuaJIT runtime,
so it was decided to remove LSan suppressions for LuaJIT functions.
Unfortunately, it was not so easy as it looked like.

At first, Lua global state is not closed properly at Tarantool instance
exit (see <tarantool_free> in src/main.cc and <tarantool_lua_free> in
src/lua/init.c for more info), so LSan false-positive leaks are detected
(for more info, see #3071). Hence, the original LSan suppression for
lj_BC_FUNCC is returned back (temporarily) until the aforementioned
issue is not resolved.

Furthermore, the internal LuaJIT memory allocator is not instrumented
yet, so to find any memory faults it's worth building LuaJIT with system
provided memory allocator (i.e. enable LUAJIT_USE_SYSMALLOC option).
However, again, since Tarantool doesn't finalize Lua universe the right
way, so running Tarantool testing routine with LUAJIT_USE_SYSMALLOC
enabled generates false-positive LSan leaks. Return back here to enable
LUAJIT_USE_SYSMALLOC, when #3071 is resolved.

Last but not least, the default value of fiber stack size is 512Kb, but
several tests in test/PUC-Rio-Lua-5.1-test suite in LuaJIT submodule
(e.g. some cases with deep recursion in errors.lua or pm.lua) have
already been tweaked according to the limitations mentioned in #5782,
but the crashes still occur while running LuaJIT tests with ASan support
enabled. Experiments once again confirm the notorious quote that "640 Kb
ought to be enough for anybody".

Anyway, LuaJIT tests are added to <test-release-asan> target in .test.mk
and LUAJIT_TEST_ENV is extended with required ASan and LSan options.

Follows up #5878

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit bacf4e56)

Since JIT is disabled in the previous commit, regular testing on
macOS/M1 can be returned back to Tarantool CI routine. Furthermore, all
specialized targets in .test.mk and auxiliary environment tweaks in
macOS-related workflows are also removed in scope of this patch.

Follows up #8252

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

(cherry picked from commit d807022b)

- Drop testing for macOS 11 since macOS 13 is now available

- Add missing testing for macOS 12:
  - debug build (x86_64)
  - debug, release, and static-cmake builds (aarch64)

- Add testing for macOS 13:
  - debug, release, release-lto, and static-cmake builds (x86_64)
  - debug, release, release-lto, and static-cmake builds (aarch64)

Closes #6739
Closes tarantool/tarantool-qa#301

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

As a result of the commit 1eb0a696 ("ci:
change runner dispatch for LuaJIT testing") <inputs.host> parameter has
become obsolete. The testing workflow has been updated in scope of the
commit tarantool/luajit@fcaecf8fb42ff8a35582fbd8d034eb6f3b9b5b68 ("ci:
use strategy matrix for integration workflow"). Hence, the only changes
required to finish the transition from <inputs.host> to <inputs.arch> +
<inputs.os> are the following:
* Drop <inputs.host> parameter from the LuaJIT integration workflow
* Make both <inputs.arch> and <inputs.os> parameters obligatory

Besides, there is no need to obtain the kernel name and the machine
hardware name in scope of the separate workflow step, since all info
need to be passed to .test.mk is already passed via workflow inputs.
Anyway, .test.mk need to be adjusted to the values used for the new
workflow parameters.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

Reviewed-by: Yaroslav Lobankov <y.lobankov@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

As it breaks sane usage of region as a data stack:

	size_t region_svp = region_used(&fiber()->gc);
	/* some allocation on fiber gc and usage of allocated memory. */
	region_truncate(&fiber()->gc, region_svp);

If in the above snippet one calls a function that in turn calls
`fiber_gc` then the snippet code may have use-after-free and later UB
on truncation.

For this reason let's get read of fiber_gc. However we need to make sure
we won't introduce leaks this way. So before actually removing
fiber_gc we make it perform leak check instead and only after fixing
all the leaks the fiber_gc was removed.

In order to find the leak easily the backtrace of the first fiber gc
allocation that is not truncated is saved and then reported.

In order to catch leaks that are not triggered by the current test suit
and to prevent introducing leaks in future patches the leak check is
added on fiber exit/recycle and for long living system fibers on every loop
iteration.

Leak check in release build is on but without leak backtrace info by
default for performance reasons. Backtrace can be provided by using
`fiber.leak_backtrace_enable()` knob before starting leaking fiber.

Regularly leaks are only reported in log but it will not help to
catch errors when running test suits so build option ABORT_ON_LEAK
is added. When it is on we abort on leak. This option is turned off
for all builds that used in CI.

Closes #5665

NO_CHANGELOG=internal
NO_DOC=internal

LuaJIT tests are disabled in the scope of the issue [1].

[1] https://github.com/tarantool/tarantool/issues/4819

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

In order to support parallel cmake build in the `test-jepsen` target
we need to run Jepsen testing in a `debian-buster` container instead
of `debian-stretch` due to the old `cmake` version in it. The `cmake`
utility started to support `--parallel` option since 3.12 version [1].

In Debian Stretch `cmake` has version 3.7.
In Debian Buster `cmake` has version 3.13.

[1] https://cmake.org/cmake/help/latest/release/3.12.html#command-line

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci

This patch introduces the new test makefile `.test.mk` instead of the
ancient `.travis.mk`. The `.test.mk` file is a fully reworked version of
`.travis.mk` with the huge refactoring and contains only those targets
that are indeed used.

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci