Commits · 16d60dfb0c72059799f1b1c6607568881c6bebba · core / tarantool

Dec 11, 2024

Revert "fix(qpromote): drop flaky test" · 16d60dfb
Dmitry Ivanov authored 3 months ago
```
This reverts commit 13577990.
```
16d60dfb
ci: make tarantool test artifacts available as gitlab artifacts · 530ce01f
Виталий Шунков authored 3 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
530ce01f

fix(picodata): don't close stderr & stdout in log_destroy · 236b2b5a

Dmitry Ivanov authored 3 months ago

Previously, log_destroy would close log->fd even if it's one of the
standard streams. This behavior almost never makes sense, unless
one's trying to write a unix daemon which is obviously not the case.

Closing stderr has the following side effects:
- it breaks a reinit of the default logger;
- it inhibits ASan's final leak report;
- it causes a EPOLLHUP during a restart via `--entrypoint-fd` (Picodata).

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

236b2b5a

feat(build): dump all include directories for box · 4a8134c0

Dmitry Ivanov authored 5 months ago

This stores all of the include directories of box into a file
called box-include-args. We use this to generate certain FFI
bindings using rust-bindgen.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

4a8134c0

ci: set pull_policy always · 36195597
Виталий Шунков authored 5 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
36195597
picodata(ci): add ASan build · 1273af73
Dmitry Ivanov authored 5 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
1273af73

picodata(ci): build CI's image from docker/Dockerfile.ci · d89772f2

Виталий Шунков authored 5 months ago and

Dmitry Ivanov committed 3 months ago


This image is suitable for release, debug and debug-asan builds.
The image is somewhat heavy (1.08 GiB) due to llvm-18, but don't run
tarantool's CI that often.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Co-authored-by: Dmitry Ivanov <d.ivanov@picodata.io>
Co-authored-by: Vitaly Shunkov <v.shunkov@picodata.io>

d89772f2

misc: fix clang-specific warnings in luajit's asserts · 6a6e9222

Dmitry Ivanov authored 5 months ago

Apparently, clang 18 is not particularly happy about luajit's static
assert implementation:

```
In file included from tarantool/src/box/box.cc:38:
In file included from tarantool/src/lua/utils.h:47:
In file included from tarantool/third_party/luajit/src/lj_state.h:9:
tarantool/third_party/luajit/src/lj_obj.h:488:1: error: variable length arrays in C++ are a Clang extension; did you mean to use 'static_assert'? [-Werror,-Wvla-extension-static-assert]
  488 | LJ_STATIC_ASSERT(offsetof(Node, val) == 0);
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
tarantool/third_party/luajit/src/lj_def.h:370:71: note: expanded from macro 'LJ_STATIC_ASSERT'
  370 |   extern void LJ_ASSERT_NAME(__COUNTER__)(int STATIC_ASSERTION_FAILED[(cond)?1:-1])
      |                                                                       ^~~~~~~~~~~
tarantool/third_party/luajit/src/lj_obj.h:488:18: note: cast that performs the conversions of a reinterpret_cast is not allowed in a constant expression
  488 | LJ_STATIC_ASSERT(offsetof(Node, val) == 0);
      | ~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~
tarantool/src/trivia/util.h:273:33: note: expanded from macro 'offsetof'
  273 | #define offsetof(type, member) ((size_t) &((type *)0)->member)
      |                                 ^
tarantool/third_party/luajit/src/lj_def.h:370:72: note: expanded from macro 'LJ_STATIC_ASSERT'
  370 |   extern void LJ_ASSERT_NAME(__COUNTER__)(int STATIC_ASSERTION_FAILED[(cond)?1:-1])
      |
```

Luckily, we can just mute this.

NO_DOC=<nothing interesting here>
NO_TEST=<tested during build time>
NO_CHANGELOG=<nothing interesting here>

6a6e9222

fix: check sql statement is not busy on unprepare · 2794e2d8
Denis Smirnov authored 6 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
2794e2d8

feat: unprepare statement references after sql execution · 4ea0e3f7

Denis Smirnov authored 6 months ago and

Dmitry Ivanov committed 3 months ago

Picodata SQL now manages the Tarantool statement cache using a
dedicated SQL fiber that handles preparation and unpreparation of
statements based on an LRU eviction policy. Prepared statements
can be executed across different sessions by SQL clients.

Previously, when a client executed a prepared statement, it
increased the reference count in the statement cache and linked
the statement to the client's session. While this approach seemed
fine, it caused issues during eviction, as references to these
statements remained in client sessions, preventing proper eviction.

This commit addresses the issue by ensuring that if a statement is
added to the current session during execution, it is removed and
the session state is restored once execution is complete.

NO_DOC=internal
NO_CHANGELOG=internal

4ea0e3f7

feat: avoid schema version bump for temporary spaces · 297e1fd4

Denis Smirnov authored 7 months ago and

Dmitry Ivanov committed 3 months ago

Temporary spaces, used for cluster-wide SQL data materialization,
were causing unnecessary netbox schema version bumps, leading to
schema downloading via netbox, excessive Lua garbage and GC blocks.
Since these tables are for internal SQL use, we don't need to inform
netbox clients about schema changes. We now maintain separate schema
versions: one for netbox clients and one for the internal prepared
statement cache.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

297e1fd4

fix(qpromote): drop flaky test · 8063ad6c
Dmitry Ivanov authored 6 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
8063ad6c
test(qsync): fix tests and drop the ones incompatible with new RAFT_PROMOTE · b1d7f001
Dmitry Ivanov authored 7 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
b1d7f001

test(qsync): add multiple tests for RAFT_PROMOTE queue · f356bb3b

Dmitry Rodionov authored 1 year ago and

Dmitry Ivanov committed 3 months ago


* smoke test that verifies that cluster can be successfully bootstrapped
* test diverging limbo owner (litmus test)
  - doesn't work on fork master, works with promote chains
* test aba leader
  - two cases where elected leader fails to deliver promote to others
* attempt to model 2+2 quorum lowering case
* two cases with two outstanding promotes
* use new injections for particular xrow types

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Co-authored-by: Dmitry Ivanov <ivadmi5@gmail.com>

f356bb3b

fix(qsync): account for pending RAFT_PROMOTE in box.ctl.wait_rw() · 81ec3896
Dmitry Ivanov authored 7 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
81ec3896
fix(qsync): improve handling of non-ack RAFT_PROMOTE · 14015dba
Dmitry Ivanov authored 7 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
14015dba
fix(qsync): on_parameters_change may apply pending RAFT_PROMOTE · d5fb312e
Dmitry Ivanov authored 8 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
d5fb312e

feat(qsync): RAFT_PROMOTE now requires quorum and RAFT_CONFIRM · 67f2ec8a

Dmitry Ivanov authored 9 months ago

Further reading:
* https://git.picodata.io/picodata/tarantool/-/merge_requests/175
* https://github.com/tarantool/tarantool/pull/10334

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

67f2ec8a

Dec 09, 2024

feat: tarantoolctl cat now prints more known keys · a580cef7
Dmitry Rodionov authored 1 year ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Co-authored-by: Dmitry Ivanov <ivadmi5@gmail.com>
```
a580cef7

fix: wrong sql cache byte counter · e815853f

Arseniy Volynets authored 7 months ago and

Dmitry Ivanov committed 3 months ago

- If you prepare and execute statement with params
in the projection and then unprepare the
statement, byte counter may show the wrong value
or even overflow.
- The problem is that when we compile sql
statement, we set parameter type to 'any'.
But when we execute the statement we set parameter
type to actual type. Then we use this type in
calculation of estimated of sql cache entry size.
This leads to different estimated sizes of cache
entry during prepare and during unprepare after
statement was executed
- Fix this by resetting type to 'any' after
executing the statement

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

e815853f

refactor!: add session id to prepare funcs · 7b364dbc

Arseniy Volynets authored 7 months ago and

Dmitry Ivanov committed 3 months ago

BREAKING CHANGE!:
1. add session id argument to sql_prepare_ext
2. introduce sql_unprepare_ext function.
This function removes prepared stmt using
given session id.

In picodata SQL, we may prepare stmt in one
session and unprepare it in some
other session, which does not know in
what session the statement was prepared. Now
sql_prepare_ext returns not only statement id,
but also a session id. This way statement can
be unprepared from other session using
sql_unprepare_ext.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

7b364dbc

ci: repair rocks install in rockylinux docker image · 4a298aab
Дмитрий Кибирев authored 7 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
4a298aab
auth: fix buffer overflow in auth_md5's bytes_to_hex · b12cf6a2
Dmitry Ivanov authored 7 months ago
```
NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=internal
```
b12cf6a2
fix(picodata): skip luaL_loadfile is picodata cb is set · 59bb30ec
Dmitry Ivanov authored 7 months ago
```
NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=internal
```
59bb30ec
fix(test): use waiters instead of timeout for replication_test.lua and wal_ext_test.lua · c5d16101
Дмитрий Кольцов authored 7 months ago and Dmitry Ivanov committed 3 months ago
```
Replace fiber.sleep with luatest.helpers.retrying
to make tests less flaky

NO_DOC=internal
NO_CHANGELOG=internal
```
c5d16101

feat(wal_ext): add support for WAL extensions. · 6050497b

godzie44 authored 1 year ago and

Dmitry Ivanov committed 3 months ago

WAL extensions allows to add auxiliary information to each
write-ahead log record. WAL extensions configured by
`box.cfg.wal_ext` option. Currently, there is only one builtin
extension: `new_old`. `new_old` extension add information about
new and old tuples for ddl operations.

NO_DOC=internal
NO_CHANGELOG=internal

6050497b

ci: try raise timeouts for debug job · 304fd1eb
Dmitry Rodionov authored 7 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=internal
```
304fd1eb
ci: make test jobs interruptible so they're cancelled · 7d6371fd
Dmitry Rodionov authored 7 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=internal
```
7d6371fd
ci: split linux tests with gitlab matrix · 3f294578
Виталий Шунков authored 7 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
3f294578
ci: fix build on tag · a74ef36f
Виталий Шунков authored 8 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
a74ef36f

ci: Centos 7 EOL · c796a78b

Дмитрий Кибирев authored 8 months ago and

Dmitry Ivanov committed 3 months ago

CentOS 7 reached its end-of-life on 30/06/2024, making its
repositories unavailable. Therefore, continuing to build CI
images for it is no longer sensible. We have now switched to
Rocky Linux 8.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

c796a78b

refactor!: replace sql_prepare with sql_prepare_ext symbol · cf0172a6

Denis Smirnov authored 8 months ago and

Dmitry Ivanov committed 3 months ago

BREAKING CHANGE!:
1. remove sql_prepare from the export list;
2. introduce sql_prepare_ext.

The sql_prepare symbol previously included the tarantool port as
an output parameter. However, this structure was inconvenient for
libraries using the C API, as they primarily required just the
statement ID. To address this issue, the sql_prepare symbol was
replaced with the sql_prepare_ext symbol.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

cf0172a6

feat: share prepared statements among sessions · 6ecad604

Denis Smirnov authored 8 months ago and

Dmitry Ivanov committed 3 months ago

Previously, users with multiple connections to tarantool instance
couldn't share prepared statements across sessions. They had to
manually call prepare in each session before execution.

This commit automates this process for the exported version of
SQL prepared statement execution (sql_execute_prepared_ext symbol).
Original Lua execution keeps the old behavior for backward
compatibility.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

6ecad604

refactor: reorder arguments in SQL execution functions · 1b1b3fa5

Denis Smirnov authored 8 months ago and

Dmitry Ivanov committed 3 months ago

Previously, sql_prepare_and_execute and sql_execute_prepared
functions didn't follow a convention to keep output parameters
(the port to be exact) at the end of the argument list.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

1b1b3fa5

refactor!: change exported C API for SQL · d86ccc23

Denis Smirnov authored 8 months ago and

Dmitry Ivanov committed 3 months ago

BREAKING CHANGE!:
1. sql_bind_list_decode - removed
2. sql_execute_prepared_ext - new arguments
3. sql_prepare_and_execute_ext - exported

There were several reasons to refactor the API.
1. sql_bind_list_decode (decodes message pack parameters into
   internal C bind structure) is very difficult to use without
   memory leaks (as it allocates results on fiber()->gc).
2. sql_execute_prepared_ext missed vdbe step limit in parameters
   and used the default value.
3. Sometimes SQL queries don't fit into prepared statement cache
   and the user still wants to execute them via a slow pass with
   full compilations from the query text. That was the reason to
   export sql_prepare_and_execute_ext symbol.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

d86ccc23

ci: switch kaniko registry · b2d4c2b2
Виталий Шунков authored 9 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
b2d4c2b2

fix(sql): remove dead code from sql_trigger_begin · b7c1f66e

Dmitry Ivanov authored 9 months ago

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

The following commit introduced a tautological if expression:

```gitcommit
sql: introduce structs assembling DDL arguments during parsing (ba56b145fafaa3)
```

Due to the changes in commit

```gitcommit
sql: refactor memory allocation system (cb777136dd7a0c)
```

the allocations in sql expressions became infallible, which means
that we may safely fix static analysis warnings by dropping the
tautological comparison altogether.

Original patch by Feodor Alexandrov.

b7c1f66e

ci: retry checkouts since they sporadically fail · 40d57836
Dmitry Rodionov authored 9 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
40d57836
doc: add copyrights for iproto-ssl sources · 59e78b29
godzie44 authored 10 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
59e78b29
test/fuzz: set base-builder docker image hash · 724ad5ed
Егор Ивков authored 10 months ago and Dmitry Ivanov committed 3 months ago
```
NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal
```
724ad5ed