- Mar 10, 2021
-
-
Igor Munkin authored
LuaJIT submodule is bumped to introduce the following changes: * test: adjust LuaJIT test suite for Tarantool * test: change LuaJIT test suite to match b4e6bf0 * test: change LuaJIT test suite to match 5a61e1a * test: change LuaJIT test suite to match c198167 * test: change LuaJIT test suite to match de5568e * test: add LuaJIT-test-cleanup test suite * test: fix Lua command in utils.selfrun * test: fix luacheck invocation for non-real paths Within this changeset LuaJIT-test-cleanup suite[1] is added to Tarantool testing. Considering Tarantool specific changes in runtime the suite itself is adjusted in LuaJIT submodule. However, there is <strict> module enabled by default in Debug build, so the testing environment need to be tweaked via test/luajit-test-init.lua script to be run prior to every LuaJIT suite test is started. [1]: https://github.com/LuaJIT/LuaJIT-test-cleanup/tree/014708b/test Closes #4064 Part of #4473 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Sergey Ostanevich <sergos@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Mar 08, 2021
-
-
pkubaj authored
On my ARM system __CC_ARM is not defined. Checking whether we don't run on non-x86 may also make supporting other architectures (like POWER or RISC-V) easier.
-
Alexander Turenko authored
This commit updates test-run with an update of tarantool-python, which eliminates the test-run submodule from tarantool-python. In other words, we have no test-run submodule inside test-run anymore. The goal is to don't confuse luacheck with old revisions of test-run / tarantool-python code and to make recursive clone faster. See [1] and [2] for details. [1]: https://github.com/tarantool/test-run/pull/268 [2]: https://github.com/tarantool/tarantool-python/commit/dccf18fd07ac684195fdd386d2a5285b9582d380
-
- Mar 05, 2021
-
-
Alexander Turenko authored
Now `make luacheck` gracefully handles different cases[^1]: in-source and out-of-source build (within the source tree or outside), current working directory as a real path or with symlink components. As result of looking into those problems I filed the issue [1] against luacheck. It seems, there are problems around absolute paths with symlinks components. [^1]: We have the similar problems with LuaJIT's luacheck rule. They will be fixed in a separate patch. [1]: https://github.com/mpeterv/luacheck/issues/208 Reviewed-by:
Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by:
Igor Munkin <imun@tarantool.org>
-
HustonMmmavr authored
The static build isn't as portable as it should be. In fact, the OPENSSLDIR is configured at build time, but in runtime, it may (and do) differ: ``` $ strace tarantool -e 'require("http.client").get("https://...")' ... openat(AT_FDCWD, "/__w/sdk/sdk/build/tarantool/static-build/" "openssl-prefix/ssl/cert.pem", O_RDONLY) = -1 ENOENT (No such file or directory) ... stat("/__w/sdk/sdk/build/tarantool/static-build/" "openssl-prefix/ssl/certs/...", ...) = -1 ENOENT (No such file or directory) ... ``` As a result, statically build tarantool can't open some https links with an error: ``` tarantool> http_client = require('http.client').new({max_connections = 5}) tarantool> http_client:get('https://..../', {verbose = true}) * Trying X.Y.Z.W:443... * TCP_NODELAY set * Connected to .... (X.Y.Z.W) port 443 (#0) * ALPN, offering http/1.1 * SSL certificate problem: self signed certificate in certificate chain * Closing connection 0 --- - status: 495 reason: SSL peer certificate or SSH remote key was not OK ... ``` Also this error may occur at dynamically build tarantool in case of linkage tarantool binary with incorrectly configured openssl. In this patch, we adopt the approach from the golang ecosystem [1-3]. At startup, tarantool scans several popular locations and uses it for `SSL_CERT_DIR` and/or `SSL_CERT_FILE` setting. See also [4,5] - it's fun. [1] https://serverfault.com/a/722646 [2] https://golang.org/src/crypto/x509/root_unix.go [3] https://golang.org/src/crypto/x509/root_linux.go [4] https://github.com/edenhill/librdkafka/blob/cb69d2a8486344252e0fcaa1f959c4ab2d8afff3/src/rdkafka_ssl.c#L845 [5] https://www.happyassassin.net/posts/2015/01/12/a-note-about-ssltls-trusted-certificate-stores-and-platforms/ Close #5615 Co-authored-by:
Yaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
-
- Mar 04, 2021
-
-
Igor Munkin authored
* core: fix cdata decrementing Closes #5820 Follows up #5187
-
- Mar 03, 2021
-
-
Vladislav Shpilevoy authored
Fiber_cond to pop fibers from the queue uses rlist_shift() + fiber_wakeup() calls. The shift wasn't needed, because fibers are linked with 'state' member, which is moved by fiber_wakeup() anyway. Closes #5855
-
Sergey Bronnikov authored
Closes #5463 Reviewed-by:
Vladislav Shpilevoy <v.shpilevoy@tarantool.org> Reviewed-by:
Igor Munkin <imun@tarantool.org>
-
Alexander V. Tikhonov authored
Added cleanup routine for performance workflows before sources checkout to avoid of fails on previously created files from inside docker with any UID:GID.
-
Alexander V. Tikhonov authored
Decided to use performance image with pre-installed benchmarks from docker.io repository instead of gitlab-ci.
-
- Mar 02, 2021
-
-
Serge Petrenko authored
Follow-up #5762
-
Sergey Bronnikov authored
W611 (A line consists of nothing but whitespace) Note: changes were made automatically in vim with regexp "%s/^\s*$//g". Closes #5739
-
Sergey Bronnikov authored
Closes #5464
-
Sergey Bronnikov authored
W621 (Inconsistent indentation (SPACE followed by TAB)) Part of #5464
-
Sergey Bronnikov authored
W612 (A line contains trailing whitespace) W613 (Trailing whitespace in a string) W614 (Trailing whitespace in a comment) Note: changes were made automatically in vim with regexp "%s/\s\+$//e". Part of #5464
-
Sergey Bronnikov authored
W542 (An empty if branch) Part of #5464
-
Sergey Bronnikov authored
W512 (Loop can be executed at most once) Part of #5464
-
Sergey Bronnikov authored
W511 (Unreachable code) Part of #5464
-
Sergey Bronnikov authored
W311 (Value assigned to a local variable is unused) Part of #5464
-
Sergey Bronnikov authored
W231 (Local variable is set but never accessed) Part of #5464
-
Sergey Bronnikov authored
W213 (Unused loop variable) Part of #5464
-
Sergey Bronnikov authored
W212 (Unused argument) Part of #5464
-
Sergey Bronnikov authored
W211 (Unused local variable) Part of #5464
-
Sergey Bronnikov authored
W113 (Accessing an undefined global variable) Part of #5464
-
Sergey Bronnikov authored
Part of #5464 Closes #5741
-
- Feb 28, 2021
-
-
Igor Munkin authored
This patch adds LuaJIT tests to every CI job type except the one for static build testing routine on OSX: there is no way to run LuaJIT tests for out of source build on OSX due to SIP[1]. [1]: https://developer.apple.com/library/archive/documentation/Security/Conceptual/System_Integrity_Protection_Guide/FileSystemProtections/FileSystemProtections.html Follows up #4862 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Timur Safin <tsafin@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
Igor Munkin authored
LuaJIT submodule is bumped to introduce the following changes: * test: run luacheck static analysis via CMake * test: fix warnings found with luacheck in misclib* * test: run LuaJIT tests via CMake * build: replace GNU Make with CMake * build: preserve the original build system Since LuaJIT build system is ported to CMake in scope of the changeset mentioned above, the module building the LuaJIT bundled in Tarantool is completely reworked. There is no option to build Tarantool against another prebuilt LuaJIT due to a91962c0 ('Until Bug#962848 is fixed, don't try to compile with external LuaJIT'), so all redundant options defining the libluajit to be used in Tarantool are dropped with the related auxiliary files. To run LuaJIT related tests or static analysis for Lua files within LuaJIT repository, <LuaJIT-test> and <LuaJIT-luacheck> targets are used respectively as a dependency of the corresponding Tarantool targets. As an additional dependency to run LuaJIT tests, prove[1] utility is required, so the necessary binary packages are added to the lists with build requirements. [1]: https://metacpan.org/pod/TAP::Harness#prove Closes #4862 Closes #5470 Closes #5631 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Timur Safin <tsafin@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
Igor Munkin authored
If Lua source path given to <lua_source> function is relative, the output file is generated in the binary directory. At the same time if the given path to be compiled to *.lua.c is absolute, the output file is generated in source directory instead of the binary one. This patch fixes the latter case, providing the valid behaviour for out of source build type. Needed for #4862 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Timur Safin <tsafin@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Feb 26, 2021
-
-
Serge Petrenko authored
row_count % WAL_ROWS_PER_YIELD == 0 is tested by both local recovery and relay on each recovered row, so it makes sense to have this constant a power of two, so that the compiler could replace '%' with bitwise and. WAL_ROWS_PER_YIELD used to be 32000, so change it to the closest power of two: 32768 == 2 ^ 15. Follow-up #5762
-
Serge Petrenko authored
While sending a WAL, relay only yields in `coio_write_xrow`, once it sees the socket isn't ready for writes. It may happen that the socket is always ready for a long period of time, and relay doesn't yield at all while recovering a whole .xlog file. This may take well more than a minute. During this period of time, relay doesn't read replica's ACKs due to relay reader fiber not being scheduled, and once the reader is finally live it times out immediately, causing the replica to reconnect. The problem mostly occurs when replica is syncing with master, so there are always some WALs to read, and is amplified by the fact that replica waits for replication_timeout to pass prior to reconnecting, which lets master pile up even more ready WALs, and effectively making it impossible for the replica to sync. To fix the problem let's yield explicitly in relay_send_row every WAL_ROWS_PER_YIELD rows. The same is already done in local recovery, and serves the same purpose: to not block the event loop for too long. Closes #5762
-
- Feb 24, 2021
-
-
Sergey Bronnikov authored
Tarantool has been integrated to OSS-Fuzz infrastructure, and implemented fuzzers are passed now [1]. Patch adds a new workflow that will execute fuzzers on each push to a master branch using OSS-Fuzz infrastructure [2]. To reduce a load on testing machines job triggers on changes in directory with Tarantool's source code, fuzzers source code, corpus files and GH Actions workflow file for fuzzing. OSS-Fuzz provides web interface with fuzzing statistics, found errors and other useful information. Access to OSS-Fuzz web interface is available for persons whose email addresses specified in project.yml, committed to oss-fuzz repository [4]. Bugs found in OSS-Fuzz reported to bugtracker [5]. Using fuzzing testing documented in Tarantool's wiki [6]. 1. https://github.com/google/oss-fuzz/pull/4723 2. https://google.github.io/oss-fuzz/getting-started/continuous-integration/#integrating-into-your-repository 3. https://google.github.io/oss-fuzz/further-reading/clusterfuzz/ 4. https://github.com/google/oss-fuzz 5. https://bugs.chromium.org/p/oss-fuzz/issues/list 6. https://github.com/tarantool/tarantool/wiki/Fuzzing Follows up #1809
-
Cyrill Gorcunov authored
Instead of shifting flags inside txn_x_flag() helpers lets define precompiled value. This allows us to operate with several flags at onces and shrink code a bit. Closes #5128 Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
mechanik20051988 authored
There was a bug in the netbox module related to access to previously released memory. To understand the essence of error, you need to understand how GC works in Lua: - GC checks the reachability of objects in Lua in one cycle and cleans out those that were unreachable. - Lua GC object is an entity whose memory is managed by the GC, for example: table, function, userdata, cdata. In our case it's cdata object, with struct error payload. - ffi.gc allows us to clean up Lua GC object payload at the time of deleting the GC object. - Finalizer in ffi.gc is hung on the Lua GC object. So after ffi.cast in our case first err object becomes unreachable. It will be cleaned after some time and if finalizer hangs on it, payload will also be cleaned. So payload in new err object (struct error in our case) becomes invalid.
-
- Feb 19, 2021
-
-
Alexander V. Tikhonov authored
Removed last OSX 14 job from testing, gitlab-ci configuration removed.
-
Alexander V. Tikhonov authored
Found that in commits: c53c650e ("github-ci: port perf jobs from gitlab-ci") 5c68884a ("github-ci: add jepsen* test jobs") was mistakenly missed "or" check "if:" condition, fixed. Follows up #5663 Follows up tarantool/tarantool-qa#79
-
- Feb 18, 2021
-
-
Sergey Bronnikov authored
http_parser() function resolves symbol to lower case using array, where an char code maps to an appropriate symbol. Some symbols may have negative ASCII code, but array index cannot be negative, it is an undefined behaviour. I read parts relevant to message headers in "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing" [1] and "Hypertext Transfer Protocol -- HTTP/1.1" [2] and found that only ASCII symbols only allowed in message headers. It means that symbols with negative ASCII code are not allowed. Patch adds a check for each processed symbol to have positive ASCII code, otherwise function returns HTTP_PARSE_INVALID. 1. RFC 7230 - 3.2 Header Fields, https://tools.ietf.org/html/rfc7230#section-3.2 2. RFC 2616 - Message Headers, https://www.ietf.org/rfc/rfc2616.txt Bug found using LibFuzzer Closes https://github.com/tarantool/security/issues/6 Needed for: https://github.com/google/oss-fuzz/pull/4723
-
Alexander V. Tikhonov authored
Found more flaky tests running testings with real parallel even for tests in fragile lists.
-
Alexander V. Tikhonov authored
Added Github local Action 'perf' to be used for perf docker images preparations and benchmarks runs. Ported jobs from gitlab-ci: perf_cbench perf_linkbench_ssd perf_nosqlbench_hash perf_nosqlbench_tree perf_sysbench perf_tpcc perf_tpch perf_ycsb_hash perf_ycsb_tree Closes #5663
-
Alexander V. Tikhonov authored
Pip and jq installation may fail if the host is real, in this way it should be manually pre-installed.
-
Alexander V. Tikhonov authored
Moved from gitlab-ci to github-ci 'jepsen*' jobs for testing Tarantool with long Jepsen tests: jepsen-single-* jobs run by cron each 3 hours jepsen-cluster-* jobs run manualy or with API backend request. Closes tarantool/tarantool-qa#79
-