Антон Фетисов authored 2 months ago and Антон Фетисов committed 2 months ago

CONFIG_SITE is an environment variable which may point to a global autoconf initialization script,
which is run at the start of every autoconf invocation and can modify its parameters.
Most systems don't have that script, and don't set that variable. On OpenSUSE, the variable is set.
The default script overwrites paths to local builds of libraries from `lib` to `lib64`, breaking our
builds. This override makes no sense for our project, since all libraries are linked statically, and
only ever built in 64-bit versions (the script allows side by side 32- and 64-bit builds).

It is unlikely that this script is used in practice, since the possibility of breaking random builds is
too high. It's also relevant only for building C code which relies on autoconf. Most new projects and
projects in modern languages don't do that.

For this reason I unset CONFIG_SITE in our default builds of Tarantool.

Added a `wait_index` parameter to `proc_replication_demote` to improve
synchronization during demotion operations.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Enhanced `test_long_term_transaction_causing_rpc_timeouts` to validate
that the `raft_term` remains unchanged during timeouts.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Updated `test_successful_wakeup_after_ddl` to validate that `raft_term`
remains unchanged after a successful wakeup operation.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Key updates include:
Adding the term field to relevant RPC requests (ConfigureReplicationRequest, ReplicationSyncRequest, and DemoteRequest).
Utilizing node.status().check_term(req.term) in critical sections of the replication logic.

Part of #998

Dmitry Ivanov authored 3 months ago and Dmitry Rodionov committed 2 months ago

This patch migrates tarantool-sys from a patched 2.11.2 to a patched 2.11.5.
Here's some stats:

```
$ git log --oneline 2.11.2..2.11.5 | wc -l
308
$ git log --oneline 2.11.2..2.11.2-picodata | wc -l
173
$ git log --oneline 2.11.5..2.11.5-picodata | wc -l
155
```

The difference in the number of patches (155 vs 173) is due to:
* Squashes and cleanups (e.g. the notorious `Part of #37`).
* Already applied upstream patches.

It's not a vanilla 2.11.5, though. The branch also includes a handful
of upstream vinyl fixes which haven't been released yet:

```
$ git log --oneline 2.11.5..release/2.11 | rg -i vinyl
abbfac3181 vinyl: disable tautological DELETE optimization for deferred DELETEs
980bcdd1d3 vinyl: move cache invalidation from vy_tx_write to vy_lsm_set
bfab99a10f vinyl: do not invalidate cache on commit after prepare
03b982e977 vinyl: fix cache invalidation on rollback of DELETE statement
878f3c04ac vinyl: fix handling of duplicate multikey entries in transaction write set
6f63cd1e0c vinyl: skip invisible read sources
edf02d65d1 vinyl: fix handling of overwritten statements in transaction write set
```

Branch: https://git.picodata.io/core/tarantool/-/commits/2.11.5-picodata

The patch adds support of ldap auth method to pgproto. For ldap we
request user password in clear text using corresponding protocol
message. Afterwards password is passed to tarantool `authenticate`
method which handles ldap based auth.

Important: since password is transferred as clear text it is advised to
setup ssl.

glauth was installed in CI but we unconditionally skipped the test.
Run the test depending on the glauth availability.

Maksim Kaitmazian authored 2 months ago and Dmitry Rodionov committed 2 months ago

Previously, using a default port of another protocol would result in an
error like the following
```
error: Invalid value "localhost:8080" for '--pg-listen <[HOST][:PORT]>': PGPROTO cannot listen on port 8080 because it is default port for HTTP
```

This commit allows default ports to be used by other protocols,
as long as the port is not already occupied.

Close #1254

The bug was introduced when I changed the behaviour of conf_change in
regard to instances with target_state Expelled. As a result we would
sometimes arbitrarily demote healthy voters in presence of degenerate
ones.

For example we could have this situation:

instance i1: raft_id=1, target_state=Online, raft_configuration=voter
instance i2: raft_id=2, target_state=Expelled, raft_configuration=voter (!)
instance i3: raft_id=3, target_state=Offline, raft_configuration=voter (!)
instance i4: raft_id=4, target_state=Online, raft_configuration=learner (!)
instance i5: raft_id=5, target_state=Online, raft_configuration=learner (!)

Dmitry Rodionov authored 2 months ago and Егор Ивков committed 2 months ago

The check arrived late when elections were already completed.
Circumstances are a bit weird. For some reason the check hang for two
seconds. Only after that it received conn refused, and during these two
seconds elections happened and leader was chosen so check can never
succeed after that.

Artur Sabirov authored 2 months ago and Maksim Kaitmazian committed 2 months ago

Follow-up for !1425

BREAKING CHANGE: as soon as PostgreSQL protocol
is enabled by default, we must be careful with
`--pg-listen` cli flag when creating more than
a single instance, because we will suddenly get
a port confict error (busy port)

Logs help to debug flaky tests. Because sometimes just by instance logs
it is not that easy to guess what the test was doing at that time.
Especially when there is a repetitive pattern in the test.

For now instance logs and test output are printed in separate sections.
It still allows to match events by time easier compared to looking only
at instance log (it is way bigger compared to test log).

Additionally this patch sets up a `framework` package intended to host
reusable test code without polluting conftest.py

I found this by compiling tarantool with CMAKE_BUILD_TYPE=Debug

We used to automatically truncate the index in compact_log if the caller
requested to compact too many entries. This made it so that the
requirement of not compacting any un-applied entries was implicit in our
code base, which is not good as it allows for some bugs to creep in
(like the one we fix a couple commits ago).

Now this is changed and instead of silently adjusting the index of last
compacted entry, we just assert that it's no greater than the applied
index. As a consequence there's a minor improvement in do_raft_log_auto_compaction
function.