Closes #996

This was the intended behavior all along, but I erroneously remove it
thinking that it's a bad thing that we're going to be having references
from _pico_plugin_migration to plugins which don't exist. If you drop
the plugin leaving it's data, the only way to drop that data would be
to re-create the same plugin and then drop it using `DROP PLUGIN WITH
DATA` syntax.

Also note that there's currently no way to drop the plugin's data
without dropping the plugin itself (other then from lua via
pico.migration_down but no analogue in SQL).

test_plugin_remove is split into 2: test_drop_plugin_basics and
test_drop_plugin_with_or_without_data. The use of lua plugin api is
replaced with SQL plugin api (except for down_migration, because there's
no alternative in SQL API).

There was also a bug in storage::Plugins::get_all_versions.

Close #1283

Kurdakov Alexander authored 4 months ago and Егор Ивков committed 2 months ago

Prints all current members of the cluster and their status.

The same result can be achieved by executing SQL
query to system tables.

Антон Фетисов authored 2 months ago and Антон Фетисов committed 2 months ago

We actually need only a single field: target_dir. It makes no sense to make network calls
and get back a dump of json.

Plugin tests should generate the plugin metadata files automatically in
the body of the test, so it's simpler to read and understand where the
data comes from.

Maksim Kaitmazian authored 2 months ago and Maksim Kaitmazian committed 2 months ago

Here is how DROP USER IF EXISTS query can result in such error:

- [client 1]: check user exists
- [client 1]: send CAS request 1
- [client 2]: check user exists
- [client 2]: send CAS request 2
- [leader]: recv CAS request 1 -> access_check: ok -> check predicate: ok -> apply(async)
- [leader]: recv CAS request 2 -> access_check: no such user
                               /|\
                                |
                                |
                             applied

To address such errors, access_check was made to not return errors when
trying to drop non-existent users or roles.

Now the above example is handled as follows:
When a user is dropped during handling the request, it causes a schema change
that leads to the rejection of the operation on the predicate check.
Upon retry, the initiator will detect that the user has been dropped
and handle it accordingly.

Replicaset name: {tier_name}_{replicaset_number_in_this_tier}
Instance name: {tier_name}_{replicaset_number}_{instance_number_in_replicaset}

Антон Фетисов authored 2 months ago and Антон Фетисов committed 2 months ago

CONFIG_SITE is an environment variable which may point to a global autoconf initialization script,
which is run at the start of every autoconf invocation and can modify its parameters.
Most systems don't have that script, and don't set that variable. On OpenSUSE, the variable is set.
The default script overwrites paths to local builds of libraries from `lib` to `lib64`, breaking our
builds. This override makes no sense for our project, since all libraries are linked statically, and
only ever built in 64-bit versions (the script allows side by side 32- and 64-bit builds).

It is unlikely that this script is used in practice, since the possibility of breaking random builds is
too high. It's also relevant only for building C code which relies on autoconf. Most new projects and
projects in modern languages don't do that.

For this reason I unset CONFIG_SITE in our default builds of Tarantool.

Added a `wait_index` parameter to `proc_replication_demote` to improve
synchronization during demotion operations.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Enhanced `test_long_term_transaction_causing_rpc_timeouts` to validate
that the `raft_term` remains unchanged during timeouts.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Updated `test_successful_wakeup_after_ddl` to validate that `raft_term`
remains unchanged after a successful wakeup operation.

Diana Tikhonova authored 3 months ago and Diana Tikhonova committed 2 months ago

Key updates include:
Adding the term field to relevant RPC requests (ConfigureReplicationRequest, ReplicationSyncRequest, and DemoteRequest).
Utilizing node.status().check_term(req.term) in critical sections of the replication logic.

Part of #998

Dmitry Ivanov authored 3 months ago and Dmitry Rodionov committed 2 months ago

This patch migrates tarantool-sys from a patched 2.11.2 to a patched 2.11.5.
Here's some stats:

```
$ git log --oneline 2.11.2..2.11.5 | wc -l
308
$ git log --oneline 2.11.2..2.11.2-picodata | wc -l
173
$ git log --oneline 2.11.5..2.11.5-picodata | wc -l
155
```

The difference in the number of patches (155 vs 173) is due to:
* Squashes and cleanups (e.g. the notorious `Part of #37`).
* Already applied upstream patches.

It's not a vanilla 2.11.5, though. The branch also includes a handful
of upstream vinyl fixes which haven't been released yet:

```
$ git log --oneline 2.11.5..release/2.11 | rg -i vinyl
abbfac3181 vinyl: disable tautological DELETE optimization for deferred DELETEs
980bcdd1d3 vinyl: move cache invalidation from vy_tx_write to vy_lsm_set
bfab99a10f vinyl: do not invalidate cache on commit after prepare
03b982e977 vinyl: fix cache invalidation on rollback of DELETE statement
878f3c04ac vinyl: fix handling of duplicate multikey entries in transaction write set
6f63cd1e0c vinyl: skip invisible read sources
edf02d65d1 vinyl: fix handling of overwritten statements in transaction write set
```

Branch: https://git.picodata.io/core/tarantool/-/commits/2.11.5-picodata

The patch adds support of ldap auth method to pgproto. For ldap we
request user password in clear text using corresponding protocol
message. Afterwards password is passed to tarantool `authenticate`
method which handles ldap based auth.

Important: since password is transferred as clear text it is advised to
setup ssl.

glauth was installed in CI but we unconditionally skipped the test.
Run the test depending on the glauth availability.

Maksim Kaitmazian authored 2 months ago and Dmitry Rodionov committed 2 months ago

Previously, using a default port of another protocol would result in an
error like the following
```
error: Invalid value "localhost:8080" for '--pg-listen <[HOST][:PORT]>': PGPROTO cannot listen on port 8080 because it is default port for HTTP
```

This commit allows default ports to be used by other protocols,
as long as the port is not already occupied.

Close #1254

The bug was introduced when I changed the behaviour of conf_change in
regard to instances with target_state Expelled. As a result we would
sometimes arbitrarily demote healthy voters in presence of degenerate
ones.

For example we could have this situation:

instance i1: raft_id=1, target_state=Online, raft_configuration=voter
instance i2: raft_id=2, target_state=Expelled, raft_configuration=voter (!)
instance i3: raft_id=3, target_state=Offline, raft_configuration=voter (!)
instance i4: raft_id=4, target_state=Online, raft_configuration=learner (!)
instance i5: raft_id=5, target_state=Online, raft_configuration=learner (!)