- May 22, 2023
-
-
Georgiy Lebedev authored
Change original `space_by_name` to `space_by_name0` and `space_index_by_name` to `space_index_by_name0`, since they accept NULL-terminated names, and add `space_by_name` and `space_index_by_name` for arbitrary strings. Needed for #8146 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
- May 19, 2023
-
-
Vladislav Shpilevoy authored
Previously it wasn't allowed to change instance UUID in _cluster. When needed, it had to be done manually by deleting the instance from _cluster and inserting it back with a new UUID. Or not to be done at all. Re-UUID (like re-name) was reported to be used when people didn't want to register new replica IDs. They wanted to rejoin lost replicas from scratch but keep the numeric ID. With UUID they could deal by either setting it explicitly to the old value on a new instance, or by doing the manual re-UUID like described above. This commit is supposed to make things simpler. If a replica has a name, then its re-join with another UUID is not an error. Its record in _cluster is automatically updated to store the new UUID. That is only possible if the old-UUID-instance is not connected anymore and is not listed in replication cfg. Closes #5029 @TarantoolBot document Title: Instance rebootstrap with new UUID but same ID and name If an instance has a non-empty instance name (`box.cfg.instance_name`), then at rebootstrap it can keep the name and its old numeric ID (space `_cluster['id']` field). This might be needed if one doesn't want to pollute `_cluster` with new rows, and somewhy doesn't want to or can't just drop the rows belonging to the dead replicas. In order for this to work 1) the rebootstrapping replica must keep its old non-empty instance name, 2) the other instances should not have any alive connections to the old dead replica. Ideally, the old replica should be just deleted from `box.cfg.replication` everywhere. When that works, the old row in `_cluster` is automatically updated with the new instance UUID.
-
Vladislav Shpilevoy authored
The instance name is carried with instance UUID everywhere in the replication protocols. It is visible in all other instances via _cluster and is displayed in monitoring. Part of #5029 @TarantoolBot document Title: `box.cfg.instance_name` and `box.info.name` The new option `box.cfg.instance_name` allows to assign the instance name to a human-readable text value to be displayed in the new info key - `box.info.name`. Instances can see names of their peers in `box.info.replication[id].name`. The name is broadcasted in "box.id" built-in event as "instance_name" key. It is string when set and nil when not set. When set, it has to be unique in the instance's replicaset. If a name wasn't set on cluster bootstrap (was forgotten or the cluster is upgraded from a version < 3.0), then it can be set on an already running instance via `box.cfg.instance_name`. To change or drop an already installed name one has to use `box.cfg.force_recovery == true` in all instances of the cluster. After the name is updated and all the instances synced, the `force_recovery` can be set back to `false`. The name can be <= 63 symbols long, can consist only of chars ['0'-'9'], '-' and 'a'-'z'. It must start with a letter. When upper-case letters are used in `box.cfg`, they are automatically converted to lower-case. The names are host- and DNS-friendly.
-
Vladislav Shpilevoy authored
The replicaset name is carried with replicaset UUID wherever any sanity validations are needed like whether 2 instances belong to the same replicaset. Part of #5029 @TarantoolBot document Title: `box.cfg.replicaset_name` and `box.info.replicaset.name` The new option `box.cfg.replicaset_name` allows to assign the replicaset name to a human-readable text value to be displayed in the new info key - `box.info.replicaset.name` - and to be validated when the instances in the replicaset connect to each other. The name is broadcasted in "box.id" built-in event as "replicaset_name" key. It is string when set and nil when not set. When set, it has to match in all instances of the entire replicaset. If a name wasn't set on cluster bootstrap (was forgotten or the cluster is upgraded from a version < 3.0), then it can be set on an already running instance via `box.cfg.replicaset_name`. To change or drop an already installed name one has to use `box.cfg.force_recovery == true` in all instances of the cluster. After the name is updated and all the instances synced, the `force_recovery` can be set back to `false`. The name can be <= 63 symbols long, can consist only of chars ['0'-'9'], '-' and 'a'-'z'. It must start with a letter. When upper-case letters are used in `box.cfg`, they are automatically converted to lower-case. The names are host- and DNS-friendly.
-
Vladislav Shpilevoy authored
The patch adds 2 new entities to replication: the concept of a cluster which has multiple replicasets and a name for this cluster. The name so far doesn't participate in any replication protocols. It is just stored in _schema and is validated against the config. The old mentions of 'cluster' (in logs, in some protocol keys like in the feedback daemon) everywhere are now considered obsolete and probably will be eventually replaced with 'replicaset'. Part of #5029 @TarantoolBot document Title: `box.cfg.cluster_name` and `box.info.cluster.name` The new option `box.cfg.cluster_name` allows to assign the cluster name to a human-readable text value to be displayed in the new info key - `box.info.cluster.name` - and to be validated when the instances in the cluster connect to each other. The name is broadcasted in "box.id" built-in event as "cluster_name" key. It is string when set and nil when not set. When set, it has to match in all instances of the entire cluster in all its replicasets. If a name wasn't set on cluster bootstrap (was forgotten or the cluster is upgraded from a version < 3.0), then it can be set on an already running instance via `box.cfg.cluster_name`. To change or drop an already installed name one has to use `box.cfg.force_recovery == true` in all instances of the cluster. After the name is updated and all the instances synced, the `force_recovery` can be set back to `false`. The name can be <= 63 symbols long, can consist only of chars '0'-'9', '-' and 'a'-'z'. It must start with a letter. When upper-case letters are used in `box.cfg`, they are automatically converted to lower-case. The names are host- and DNS-friendly.
-
Vladislav Shpilevoy authored
The new function check_global_ids_integrity() checks that the replicaset UUID specified in the config and found in the data match. Instance UUID is created at bootstrap and validated at the beginning of recovery, not in the end. Hence not checked here. For now this function is not very useful, but soon there will be more global IDs stored in WAL which will need validation. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
Node name stores a DNS- and host- friendly string name. It will be used in the next patches for some new global names: cluster, replicaset, and instance. Part of #5029 NO_DOC=internal NO_CHANGELOG=internal
-
Vladislav Shpilevoy authored
It was named 'cluster', but really was just about the replicaset. This is going to be even more confusing soon, because there will be introduced an actual concept of cluster as multiple replicasets. The patch renames it to 'replicaset'. `box.info.cluster` now means the whole cluster and is empty so far. Next patches will add here the cluster name. Part of #5029 @TarantoolBot document Title: `box.info.cluster` is renamed to `box.info.replicaset` Done since 3.0.0. The old behaviour can be reverted back via the `compat` option `box_info_cluster_meaning`. `box.info.cluster` key is still here, but now means a totally different thing - the entire cluster with all its replicasets. <h2>Compat documentation</h2> `box.info.cluster` default meaning is the whole cluster with all its replicasets. To get info about only the current replicaset `box.info.replicaset` should be used. In old versions (< 3.0.0) `box.info.cluster` meant the current replicaset and `box.info.replicaset` didn't exist. <h3>Old and new behaviour</h3> New behaviour: ``` tarantool> box.info.cluster --- - <some cluster keys> ... tarantool> box.info.replicaset --- - uuid: <replicaset uuid> - <... other attributes of the replicaset> ... ``` Old behaviour: ``` tarantool> box.info.cluster --- - uuid: <replicaset uuid> - <... other attributes of the replicaset> ... tarantool> box.info.replicaset (= nil on < 3.0.0) --- - uuid: <replicaset uuid> - <... other attributes of the replicaset> ... ``` <h3>Known compatibility issues</h3> VShard versions < 0.1.24 do not support the new behaviour. <h3>Detecting issues in you codebase</h3> Look for all usages of `box.info.cluster`, `info.cluster`, and even just `.cluster`, `['cluster']`, `["cluster"]`. For the new behaviour to work all of them have to use 'replicaset' key.
-
Vladislav Shpilevoy authored
Replicaset UUID was stored in _schema['cluster'] tuple. This is going to be confusing soon, because there will be introduced an actual concept of cluster as multiple replicasets. The patch renames it to 'replicaset_uuid'. Part of #5029 @TarantoolBot document Title: Update '_schema' with new 'replicaset_uuid' key Currently _schema system space is documented to have 'cluster' key with replicaset UUID value. Now this key is deleted (since 3.0) and the UUID is stored in 'replicaset_uuid' key.
-
Vladislav Shpilevoy authored
box_on_join() was called not only on IPROTO_JOIN but also on IPROTO_REGISTER. The name was a bit misleading. It is now called box_register_replica(). The old box_register_replica() is renamed to box_insert_replica_record(). It says "insert record", because this is what it does - inserts a new tuple into _cluster space. It also skips the check whether the instance is read-only. It allows to use the function on the bootstrap master so as it could register itself. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
_cluster on_replace trigger in alter.cc was a huge multi-screen function with many indentation levels. It was not too bad in its old state. But soon it is going to get more complicated as _cluster will get a new field - 'name'. Its update will require own on commit and rollback triggers, own checks, errors. Trying to fit name processing into the old monstrous function didn't look too tempting, so the trigger now is split into multiple functions serving update, insert, and delete separately. At least it helps to reduce the indentation. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
To tell whether the instance is anon there used to be just one flag in C code: replication_anon. Having one flag both for cfg and for the actual state is bad because if cfg is updated, then there is a moment when that flag can't be safely used to check the actual state. For example, when replication_anon had been true and was set to false, it took time to register the instance. In the meantime the C flag replication_anon was already false, although the instance is still anon (not present in _cluster). In the existing code it could lead to insignificant errors like when an anon instance was being registered, it could already accept IPROTO_REGISTER requests. It would fail on ER_READONLY instead of ER_UNSUPPORTED. It wasn't a critical problem, but still it wasn't correct to use cfg flag for checking the actual state. Now there is a separate cfg flag and a function for checking the real state. This patch is done because soon there will be a new option which also takes time to change: instance name. This commit sets a pattern how to deal with such options. In scope of #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
If attempt to set `box.cfg{replication_anon = false}` failed, the instance's ballot event had is_anon = false nonetheless. This was because on reconfig failure the option's scope guard did revert the option itself in C++ code, but didn't update the ballot. NO_DOC=bugfix
-
Vladislav Shpilevoy authored
There were a few places where instance and replicaset UUIDs from box.cfg where passed as arguments in box.cc functions. It was fine although sometimes could cause struggling like "where along the callstack replicaset UUID is created when it was nil in cfg". But soon the situation will get more complicated. There will be up to 3 new arguments - cluster, replicaset, and instance names. Passing all these identifiers as parameters would be cumbersome. The patch makes the UUIDs fetched from the config by the functions which need them. The same will be done with the names where they are relevant. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
box.cfg.force_recovery used to be needed only during box.cfg() in a few places, but its usage is going to extend. In future commits about cluster/replicaset/instance names it will be needed to allow rename. It won't be entirely legal (hence can't be done without any flags), but won't be fully illegal either. The "valid" rename will be after upgrading, when an old cluster updated to a new version and wants to start using the names. Then it will have to set force_recovery, set the names, sync the instances, drop force_recovery. One-time action to allow old installations use the new feature - the names. Part of #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
_schema on_replace trigger used to treat replace as commit. No support for rollback at all and all changes are immediately visible. That is fine most of the time but still incorrect. The patch makes the space properly respect transactions. This is done as a preparation for adding several new _schema keys which will be transactional from the start. Would be strange to leave certain keys ignoring transactions. Hence this fix is done. In scope of #5029 NO_DOC=bugfix NO_CHANGELOG=couldn't happen with legal usage of public APIs
-
Vladislav Shpilevoy authored
Deletion of the own entry from _cluster space is allowed during the join stage, because the remote master could have already had the joining instance UUID in _cluster space but then deleted it. Then for the joining instance it looks like deletion of self from _cluster. But that is fine - in the end of join the master will register the replica again. The case is handled, but not covered with a test. The patch adds one. NO_DOC=test NO_CHANGELOG=test
-
Vladislav Shpilevoy authored
The function replica_check_id() is called on any change in _cluster: insert, delete, update. It was supposed to check if the replica ID is valid - not nil, not out of range (VCLOCK_MAX). But it was also raising an error when the ID matched this instance's ID unless the instance was joining. That happened even if a _cluster tuple was updated without changing the ID at all. For example, if one would just do _cluster:replace(_cluster:get(box.info.id)). That was a surprising side effect of the ID checker which blocked next patches. The next commits are going to introduce a new field in _cluster (replica name) which will be mutable. Such behaviour of replica_check_id() wouldn't allow to update even that new field. Better do the check in the only place where the mutation can happen - on deletion. Since replica ID is a primary key in _cluster, it can't be updated there. Only inserted or deleted. Needed for #5029 NO_DOC=bugfix and refactoring NO_CHANGELOG=couldn't happen unless user touched _cluster in a weird way NO_TEST=covered by next commits, too insignificant for an own test
-
Vladislav Shpilevoy authored
It wasn't allowed to drop it, but was allowed to update. The patch bans it. Firstly, it was not supposed to work. Secondly, a future patch will introduce a new tuple in _schema, which would store replicaset UUID too. It won't allow UUID update. Would be strange, if the update would be let through one _schema tuple and wouldn't work via another. Needed for #5029 NO_DOC=bugfix
-
Yaroslav Lobankov authored
The next major release of Tarantool (3.0.0) will not support Cartridge. So tet's be ready in advance. This patch removes the cartridge and crud integration test runs to make the integration tests pass for #8289. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
- May 18, 2023
-
-
Vladimir Davydov authored
This directory belongs to the EE repository. Also, delete test/enterprise-tap because we don't have any TAP tests in the EE repository. Follow-up commit d8faca69 ("lua: implement stubs for etcd client"). NO_DOC=gitignore NO_TEST=gitignore NO_CHANGELOG=gitignore
-
Sergey Bronnikov authored
Patch made following changes: - add a CMake function lua_multi_source that unpacks symbol names and Lua source paths in passed list and compiles Lua source into *.lua.c sources using lua_source() function. Function is needed for embedding Lua modules consisting from a number of files to Tarantool EE. - add using lua_multi_source instead of lua_source for etcd client module. - remove function prototypes and declarations related to extra features. All this code is moved to Tarantool EE source code. Part of #8620 Follows up https://github.com/tarantool/tarantool-ee/issues/443 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Aleksandr Lyapunov authored
Fefactor lua name resolution, simplify and comment. Add an ability to specify path with brackets, for example in 'box.space[512]:get' or 'box.space["test"]:get'. Only literals (strings and numbers) are supported. Closes #8604 @TarantoolBot document Title: square brackets in procedure resolution for Lua calls Square brackets are now supported in Lua call procedure resolution. This is applicable to `net.box` connection objects `call` method as well as `box.schema.func.call`. Examples of function calls with square brackets can be found in the test to this patch.
-
- May 16, 2023
-
-
Maksim Kokryashkin authored
Default token for actions provided by GitHub lacks read and write permissions for private projects. This commit changes the token used for the stale PRs workflow to the dedicated token with needed permissions. NO_DOC=Workflow fix NO_TEST=Workflow fix NO_CHANGELOG=Workflow fix
-
Oleg Babin authored
Sometimes we need negative timestamps to work with dates before 1970. But seems such cases were even covered in tests. So there wasn't any handling of negative timestamps with fraction part. Such datetime objects had incorrect string representation (e.g. "1963-11-22T12:30:02.-999"). This patch fixes it. Closes #8570 NO_DOC=bugfix
-
Oleg Babin authored
Seems that problem code part was ported from Lua as is. But there is some difference between modulo operator in C and in Lua. Lua always returns positive value but in C result could be negavive. This difference led to the case when after subtraction nsec part of datetime object become negative that yielded weird result on attempt to get string representation (e.g."2008-02-03T03:36:43.-100Z"). This patch fixes it. Part of #8570 NO_DOC=bugfix NO_CHANGELOG=see next commit
-
- May 15, 2023
-
-
Andrey Saranchin authored
Currently, to export IPROTO constants to Lua, we define a translation table in C manually. As a result, some constants are forgotten and some are exported with mistakes. Fortunately, we have a mechanism to generate enums and its stringified names in compile-time. Let's rewrite iproto constants using such mechanism and generate translation table automatically. Closes #8443 Closes #8574 Closes #8577 NO_DOC=bugfix
-
Andrey Saranchin authored
We are going to generate iproto_key enum automatically to generate iproto constants for Lua as well. This mechanism generates *_MAX constant for enum using its name, which is in lower case. So let's convert IPROTO_KEY_MAX to form which is appropriate for enum generator. Part of #8443 NO_TEST=rename constant NO_CHANGELOG=internal NO_DOC=internal
-
Serge Petrenko authored
This commit adds another possible bootstrap_strategy to accompany "auto" and "config": "supervised". Such a strategy may be useful to pin the desired bootstrap leader on an active cluster (so that the user may join the replicas from the desired node without changing their box.cfg) or to manually set the bootstrap leader among the nodes that managed to start without issues. More details are in the docbot request. Closes #8509 @TarantoolBot document Title: new bootstrap strategy - "supervised" The `bootstrap_strategy` configuration option may now be set to "supervised". This strategy works as follows: When bootstrapping a new replicaset, the nodes do not choose a bootstrap leader automatically and instead wait for it to be appointed by the user. The configuration will fail if no bootstrap leader is appointed during a `replication_connect_timeout`. In order to apppoint a bootstrap leader, the user has to issue `box.ctl.make_bootstrap_leader()` call on the desired node. This must be done during the initial `box.cfg()` call. Possible ways to achieve this are: 1. In interactive mode: ```lua fiber.create(box.cfg, desired_config) box.ctl.make_bootstrap_leader() ``` 2. Via an init script: ```lua -- Init script: console.listen(admin_port) box.cfg(desired_config) -- User console: tarantoolctl enter admin port > box.ctl.make_bootstrap_leader() ``` When joining a new replica with `bootstrap_strategy` = "supervised" to an existing replica set, the replica will not choose the bootstrap leader automatically, but will instead join to the node on which `box.ctl.make_bootstrap_leader()` was issued last. In case such a node isn't found after a `replication_connect_timeout`, the configuration fails.
-
Serge Petrenko authored
Both triggers are used to wait for specific applier state (applier is connected, remote node ballot is known, etc.) and are often used one after another. Keeping these two trigger lists only adds complexity of creating 2 triggers instead of 1 in some places. Let's unify the triggers. Now the triggers might be fired multiple times, so make all the existing trigger functions reenterable. In-scope-of #8509 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Serge Petrenko authored
The idea is to simplify applier_on_connect_f as much as possible, so that it only wakes the waiter up, but doesn't participate in any decisions regarding the replicaset connection state. In-scope-of #8509 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Serge Petrenko authored
Now that the event isn't used to pass the success status, we may pass applier there for consistency with on_state triggers. Besides, this allows to simplify applier_ballot_data struct, which doesn't need to store applier anymore. In-scope-of #8509 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Serge Petrenko authored
Ballot watcher used to pass a boolean as an event to ballot_update triggers, with "false" meaning there was some error while waiting for ballot and one has to check diag for the error. This was intended as a guard against stale diag contents (diag might contain some leftovers from previous errors). Let's make ballot watcher fiber clear the diagnostics when it's not planning on using them. Now ballot update triggers may rely on `diag_is_empty()` test instead of the "success" parameter. In-scope-of #8509 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Vladislav Shpilevoy authored
It was used for getting _schema tuple[1] field. Worked because BOX_CLUSTER_FIELD_UUID is also 1. This commit introduces BOX_SCHEMA_FIELD_VALUE. It is going to be used in the next commits more extensively for new _schema tuples. Part of #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Oleg Babin authored
This patch fixes a case when timestamp is passed to datetime.set function at the same time with nsec, usec or msec. It works fine for datetime.new but some logic was missed for set function. Here we fix that and introduce a test. Closes #8583 NO_DOC=bugfix
-
Vladimir Davydov authored
With MVCC off (box.cfg.memtx_use_mvcc_engine = false), a memtx space read view may include a dirty (not committed to WAL) record. To prevent such records from being written to a snapshot, we sync WAL after creating a read view for a snapshot. The problem is that it doesn't work for long (yielding) DDL operations, such as building a new index, because such operations yield before waiting on WAL. As a result, a dirty DDL record may make it to a snapshot even though it may fail eventually. To fix that, let's keep track of all yielding DDL statements and exclude them from a read view using the memtx snapshot cleaner. Closes #8530 NO_DOC=bug fix
-
Vladimir Davydov authored
We don't use this functionality in net.box (sync number is always 0 for all watch/event packets), but other clients may actually use it. Closes #8393 @TarantoolBot document Title: Document that `IPROTO_EVENT` has sync number Initially the sync number sent by a client in an `IPROTO_WATCH` request was ignored and `IPROTO_EVENT` packet didn't have a sync number. There were complaints about it from users so we consider this to be a bug. Now the server sends the same sync number in an `IPROTO_EVENT` packet as the one sent by the client in the last corresponding `IPROTO_WATCH` request.
-
- May 12, 2023
-
-
Georgiy Lebedev authored
Closes #8636 NO_TEST=<dropped functionality> @TarantoolBot document Title: Remove mentions about `IPROTO_CALL16` from `net.box` Support for `IPROTO_CALL16` was dropped: remove mentions about `call_16` option for `net.box` connections.
-
- May 11, 2023
-
-
Sergey Ostanevich authored
Add necessary wait for replication to appear on the replica, enforce correct txn isolation to avoid memtx/vinyl discrepancy. Remove the test from the fragile list. Closes tarantool/tarantool-qa#292 NO_DOC=test fix NO_CHANGELOG=test fix
-
Maksim Kokryashkin authored
The setup-python action defaults to searching for a dependency file (requirements.txt or pyproject.toml for pip, Pipfile.lock for pipenv or poetry.lock for poetry) in the repository, and uses its hash as a part of the cache key. We don't have one, so the caching option is disabled. NO_DOC=Workflow fix NO_TEST=Workflow fix NO_CHANGELOG=Workflow fix
-