- May 25, 2023
-
-
Nikolay Shirokovskiy authored
Currently on tuple encoding failure we raise Lua error. In many placess the error is not handled in Lua C code and we get misc leaks. Let's instead pass error as return value. Note that generally speaking encoding code can raise an error on OOM. Which will lead to leak again. Hopefully application will be killed by OOM killer instead. Other then that we expect no more errors in the code. If code calls a user defined callback then pcall is used (see lua_field_inspect_ucdata for example). So the turn from raising errors to returning error code seems the right direction. Closes #7939 NO_DOC=bugfix
-
Nikolay Shirokovskiy authored
This will bring new ibuf_truncate method. Part of #7939 NO_TEST=internal NO_CHANGELOG=internal NO_DOC=internal
-
- May 24, 2023
-
-
Igor Munkin authored
* Fix IR_RENAME snapshot number. Follow-up fix for a32aeadc. * OSX: Disable unreliable assertion for external frame unwinding. * Disable unreliable assertion for external frame unwinding. * Handle on-trace OOM errors from helper functions. * LJ_GC64: Make ASMREF_L references 64 bit. * lldb: introduce luajit-lldb * x64/LJ_GC64: Fix emit_rma(). * Limit path length passed to C library loader. Closes #7745 Part of #4808 Part of #8069 Part of #8516 NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump
-
Ilya Verbin authored
Currently is_nullable property of a functional index part disables the unique property of the index. The bug is in func_index_compare(), which compares functional keys first, and if they are equal it compares the primary keys. This behaviour is correct only when some part of the key is NULL (and for non-unique indexes), but for now the primary keys are compared unconditionally. Fix this by checking for NULL key parts. Closes #8587 NO_DOC=bugfix
-
- May 23, 2023
-
-
Mergen Imeev authored
This patch adds a check that sqlXPrintf() does not fail in the built-in SQL function printf(). There are two possible problems: the result might get too large, or there might be an integer overflow because internally int values are converted to size_t. Closes #tarantool/security#122 NO_DOC=bugfix
-
Mergen Imeev authored
This patch fixes problems with INSERT INTO ... SELECT FROM optimization. These problems appeared after 6b8acd8f, where the check became redundant, but was not updated. Two problems arose: 1) an assertion or segmentation fault when optimization was used and the source space does not have an index; 2) optimization can be used even if the indexes are incompatible. The second problem does not result in changes that are user-visible, so there is no test. Closes #8661 NO_DOC=bugfix
-
- May 22, 2023
-
-
Gleb Kashkin authored
Hostname is a useful piece of information in state reports. So it was decided to add it to box.info. Hostname is obtained on requested and is not cached. Closes #8605 @TarantoolBot document Title: Add hostname to box.info This patch adds hostname to box.info, it can be useful e.g. to supplement various instance state reports. It is not cached and is requested on each call.
-
Eli Kobrin authored
The error of invalid negation occurred because of invalid check, which did not cover the case when value is equal to INT64_MIN, so the negation of INT64_MIN equal to itself. This must be fixed, because negation of INT64_MIN is undefined behavior. It is fixed by the explicit check for the value variable. NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
Add support for accepting IPROTO requests with space or index name instead of identifier (name is preferred over identifier to disambiguate missing identifiers from zero identifiers): mark space identifier request key as present upon encountering space name, and delay resolution of identifier until request gets to transaction thread. Add support for sending DML requests from net.box connection objects with disabled schema fetching by manually specifying space or index name or identifier: when schema fetching is disabled, the space and index tables of connections return wrapper tables that store necessary context (space or index name or identifier, determined by type, connection object and space for indexes) for performing requests. The space and index tables cache the wrapper table they return. Closes #8146 @TarantoolBot document Title: Space and index name in IPROTO requests Refer to design document for details: https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#f4d4b3fa2b3646f1949319866428b6c0
-
Georgiy Lebedev authored
Change original `space_by_name` to `space_by_name0` and `space_index_by_name` to `space_index_by_name0`, since they accept NULL-terminated names, and add `space_by_name` and `space_index_by_name` for arbitrary strings. Needed for #8146 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
- May 19, 2023
-
-
Vladislav Shpilevoy authored
Previously it wasn't allowed to change instance UUID in _cluster. When needed, it had to be done manually by deleting the instance from _cluster and inserting it back with a new UUID. Or not to be done at all. Re-UUID (like re-name) was reported to be used when people didn't want to register new replica IDs. They wanted to rejoin lost replicas from scratch but keep the numeric ID. With UUID they could deal by either setting it explicitly to the old value on a new instance, or by doing the manual re-UUID like described above. This commit is supposed to make things simpler. If a replica has a name, then its re-join with another UUID is not an error. Its record in _cluster is automatically updated to store the new UUID. That is only possible if the old-UUID-instance is not connected anymore and is not listed in replication cfg. Closes #5029 @TarantoolBot document Title: Instance rebootstrap with new UUID but same ID and name If an instance has a non-empty instance name (`box.cfg.instance_name`), then at rebootstrap it can keep the name and its old numeric ID (space `_cluster['id']` field). This might be needed if one doesn't want to pollute `_cluster` with new rows, and somewhy doesn't want to or can't just drop the rows belonging to the dead replicas. In order for this to work 1) the rebootstrapping replica must keep its old non-empty instance name, 2) the other instances should not have any alive connections to the old dead replica. Ideally, the old replica should be just deleted from `box.cfg.replication` everywhere. When that works, the old row in `_cluster` is automatically updated with the new instance UUID.
-
Vladislav Shpilevoy authored
The instance name is carried with instance UUID everywhere in the replication protocols. It is visible in all other instances via _cluster and is displayed in monitoring. Part of #5029 @TarantoolBot document Title: `box.cfg.instance_name` and `box.info.name` The new option `box.cfg.instance_name` allows to assign the instance name to a human-readable text value to be displayed in the new info key - `box.info.name`. Instances can see names of their peers in `box.info.replication[id].name`. The name is broadcasted in "box.id" built-in event as "instance_name" key. It is string when set and nil when not set. When set, it has to be unique in the instance's replicaset. If a name wasn't set on cluster bootstrap (was forgotten or the cluster is upgraded from a version < 3.0), then it can be set on an already running instance via `box.cfg.instance_name`. To change or drop an already installed name one has to use `box.cfg.force_recovery == true` in all instances of the cluster. After the name is updated and all the instances synced, the `force_recovery` can be set back to `false`. The name can be <= 63 symbols long, can consist only of chars ['0'-'9'], '-' and 'a'-'z'. It must start with a letter. When upper-case letters are used in `box.cfg`, they are automatically converted to lower-case. The names are host- and DNS-friendly.
-
Vladislav Shpilevoy authored
The replicaset name is carried with replicaset UUID wherever any sanity validations are needed like whether 2 instances belong to the same replicaset. Part of #5029 @TarantoolBot document Title: `box.cfg.replicaset_name` and `box.info.replicaset.name` The new option `box.cfg.replicaset_name` allows to assign the replicaset name to a human-readable text value to be displayed in the new info key - `box.info.replicaset.name` - and to be validated when the instances in the replicaset connect to each other. The name is broadcasted in "box.id" built-in event as "replicaset_name" key. It is string when set and nil when not set. When set, it has to match in all instances of the entire replicaset. If a name wasn't set on cluster bootstrap (was forgotten or the cluster is upgraded from a version < 3.0), then it can be set on an already running instance via `box.cfg.replicaset_name`. To change or drop an already installed name one has to use `box.cfg.force_recovery == true` in all instances of the cluster. After the name is updated and all the instances synced, the `force_recovery` can be set back to `false`. The name can be <= 63 symbols long, can consist only of chars ['0'-'9'], '-' and 'a'-'z'. It must start with a letter. When upper-case letters are used in `box.cfg`, they are automatically converted to lower-case. The names are host- and DNS-friendly.
-
Vladislav Shpilevoy authored
The patch adds 2 new entities to replication: the concept of a cluster which has multiple replicasets and a name for this cluster. The name so far doesn't participate in any replication protocols. It is just stored in _schema and is validated against the config. The old mentions of 'cluster' (in logs, in some protocol keys like in the feedback daemon) everywhere are now considered obsolete and probably will be eventually replaced with 'replicaset'. Part of #5029 @TarantoolBot document Title: `box.cfg.cluster_name` and `box.info.cluster.name` The new option `box.cfg.cluster_name` allows to assign the cluster name to a human-readable text value to be displayed in the new info key - `box.info.cluster.name` - and to be validated when the instances in the cluster connect to each other. The name is broadcasted in "box.id" built-in event as "cluster_name" key. It is string when set and nil when not set. When set, it has to match in all instances of the entire cluster in all its replicasets. If a name wasn't set on cluster bootstrap (was forgotten or the cluster is upgraded from a version < 3.0), then it can be set on an already running instance via `box.cfg.cluster_name`. To change or drop an already installed name one has to use `box.cfg.force_recovery == true` in all instances of the cluster. After the name is updated and all the instances synced, the `force_recovery` can be set back to `false`. The name can be <= 63 symbols long, can consist only of chars '0'-'9', '-' and 'a'-'z'. It must start with a letter. When upper-case letters are used in `box.cfg`, they are automatically converted to lower-case. The names are host- and DNS-friendly.
-
Vladislav Shpilevoy authored
The new function check_global_ids_integrity() checks that the replicaset UUID specified in the config and found in the data match. Instance UUID is created at bootstrap and validated at the beginning of recovery, not in the end. Hence not checked here. For now this function is not very useful, but soon there will be more global IDs stored in WAL which will need validation. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
Node name stores a DNS- and host- friendly string name. It will be used in the next patches for some new global names: cluster, replicaset, and instance. Part of #5029 NO_DOC=internal NO_CHANGELOG=internal
-
Vladislav Shpilevoy authored
It was named 'cluster', but really was just about the replicaset. This is going to be even more confusing soon, because there will be introduced an actual concept of cluster as multiple replicasets. The patch renames it to 'replicaset'. `box.info.cluster` now means the whole cluster and is empty so far. Next patches will add here the cluster name. Part of #5029 @TarantoolBot document Title: `box.info.cluster` is renamed to `box.info.replicaset` Done since 3.0.0. The old behaviour can be reverted back via the `compat` option `box_info_cluster_meaning`. `box.info.cluster` key is still here, but now means a totally different thing - the entire cluster with all its replicasets. <h2>Compat documentation</h2> `box.info.cluster` default meaning is the whole cluster with all its replicasets. To get info about only the current replicaset `box.info.replicaset` should be used. In old versions (< 3.0.0) `box.info.cluster` meant the current replicaset and `box.info.replicaset` didn't exist. <h3>Old and new behaviour</h3> New behaviour: ``` tarantool> box.info.cluster --- - <some cluster keys> ... tarantool> box.info.replicaset --- - uuid: <replicaset uuid> - <... other attributes of the replicaset> ... ``` Old behaviour: ``` tarantool> box.info.cluster --- - uuid: <replicaset uuid> - <... other attributes of the replicaset> ... tarantool> box.info.replicaset (= nil on < 3.0.0) --- - uuid: <replicaset uuid> - <... other attributes of the replicaset> ... ``` <h3>Known compatibility issues</h3> VShard versions < 0.1.24 do not support the new behaviour. <h3>Detecting issues in you codebase</h3> Look for all usages of `box.info.cluster`, `info.cluster`, and even just `.cluster`, `['cluster']`, `["cluster"]`. For the new behaviour to work all of them have to use 'replicaset' key.
-
Vladislav Shpilevoy authored
Replicaset UUID was stored in _schema['cluster'] tuple. This is going to be confusing soon, because there will be introduced an actual concept of cluster as multiple replicasets. The patch renames it to 'replicaset_uuid'. Part of #5029 @TarantoolBot document Title: Update '_schema' with new 'replicaset_uuid' key Currently _schema system space is documented to have 'cluster' key with replicaset UUID value. Now this key is deleted (since 3.0) and the UUID is stored in 'replicaset_uuid' key.
-
Vladislav Shpilevoy authored
box_on_join() was called not only on IPROTO_JOIN but also on IPROTO_REGISTER. The name was a bit misleading. It is now called box_register_replica(). The old box_register_replica() is renamed to box_insert_replica_record(). It says "insert record", because this is what it does - inserts a new tuple into _cluster space. It also skips the check whether the instance is read-only. It allows to use the function on the bootstrap master so as it could register itself. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
_cluster on_replace trigger in alter.cc was a huge multi-screen function with many indentation levels. It was not too bad in its old state. But soon it is going to get more complicated as _cluster will get a new field - 'name'. Its update will require own on commit and rollback triggers, own checks, errors. Trying to fit name processing into the old monstrous function didn't look too tempting, so the trigger now is split into multiple functions serving update, insert, and delete separately. At least it helps to reduce the indentation. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
To tell whether the instance is anon there used to be just one flag in C code: replication_anon. Having one flag both for cfg and for the actual state is bad because if cfg is updated, then there is a moment when that flag can't be safely used to check the actual state. For example, when replication_anon had been true and was set to false, it took time to register the instance. In the meantime the C flag replication_anon was already false, although the instance is still anon (not present in _cluster). In the existing code it could lead to insignificant errors like when an anon instance was being registered, it could already accept IPROTO_REGISTER requests. It would fail on ER_READONLY instead of ER_UNSUPPORTED. It wasn't a critical problem, but still it wasn't correct to use cfg flag for checking the actual state. Now there is a separate cfg flag and a function for checking the real state. This patch is done because soon there will be a new option which also takes time to change: instance name. This commit sets a pattern how to deal with such options. In scope of #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
If attempt to set `box.cfg{replication_anon = false}` failed, the instance's ballot event had is_anon = false nonetheless. This was because on reconfig failure the option's scope guard did revert the option itself in C++ code, but didn't update the ballot. NO_DOC=bugfix
-
Vladislav Shpilevoy authored
There were a few places where instance and replicaset UUIDs from box.cfg where passed as arguments in box.cc functions. It was fine although sometimes could cause struggling like "where along the callstack replicaset UUID is created when it was nil in cfg". But soon the situation will get more complicated. There will be up to 3 new arguments - cluster, replicaset, and instance names. Passing all these identifiers as parameters would be cumbersome. The patch makes the UUIDs fetched from the config by the functions which need them. The same will be done with the names where they are relevant. Needed for #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
box.cfg.force_recovery used to be needed only during box.cfg() in a few places, but its usage is going to extend. In future commits about cluster/replicaset/instance names it will be needed to allow rename. It won't be entirely legal (hence can't be done without any flags), but won't be fully illegal either. The "valid" rename will be after upgrading, when an old cluster updated to a new version and wants to start using the names. Then it will have to set force_recovery, set the names, sync the instances, drop force_recovery. One-time action to allow old installations use the new feature - the names. Part of #5029 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=already covered
-
Vladislav Shpilevoy authored
_schema on_replace trigger used to treat replace as commit. No support for rollback at all and all changes are immediately visible. That is fine most of the time but still incorrect. The patch makes the space properly respect transactions. This is done as a preparation for adding several new _schema keys which will be transactional from the start. Would be strange to leave certain keys ignoring transactions. Hence this fix is done. In scope of #5029 NO_DOC=bugfix NO_CHANGELOG=couldn't happen with legal usage of public APIs
-
Vladislav Shpilevoy authored
Deletion of the own entry from _cluster space is allowed during the join stage, because the remote master could have already had the joining instance UUID in _cluster space but then deleted it. Then for the joining instance it looks like deletion of self from _cluster. But that is fine - in the end of join the master will register the replica again. The case is handled, but not covered with a test. The patch adds one. NO_DOC=test NO_CHANGELOG=test
-
Vladislav Shpilevoy authored
The function replica_check_id() is called on any change in _cluster: insert, delete, update. It was supposed to check if the replica ID is valid - not nil, not out of range (VCLOCK_MAX). But it was also raising an error when the ID matched this instance's ID unless the instance was joining. That happened even if a _cluster tuple was updated without changing the ID at all. For example, if one would just do _cluster:replace(_cluster:get(box.info.id)). That was a surprising side effect of the ID checker which blocked next patches. The next commits are going to introduce a new field in _cluster (replica name) which will be mutable. Such behaviour of replica_check_id() wouldn't allow to update even that new field. Better do the check in the only place where the mutation can happen - on deletion. Since replica ID is a primary key in _cluster, it can't be updated there. Only inserted or deleted. Needed for #5029 NO_DOC=bugfix and refactoring NO_CHANGELOG=couldn't happen unless user touched _cluster in a weird way NO_TEST=covered by next commits, too insignificant for an own test
-
Vladislav Shpilevoy authored
It wasn't allowed to drop it, but was allowed to update. The patch bans it. Firstly, it was not supposed to work. Secondly, a future patch will introduce a new tuple in _schema, which would store replicaset UUID too. It won't allow UUID update. Would be strange, if the update would be let through one _schema tuple and wouldn't work via another. Needed for #5029 NO_DOC=bugfix
-
Yaroslav Lobankov authored
The next major release of Tarantool (3.0.0) will not support Cartridge. So tet's be ready in advance. This patch removes the cartridge and crud integration test runs to make the integration tests pass for #8289. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
- May 18, 2023
-
-
Vladimir Davydov authored
This directory belongs to the EE repository. Also, delete test/enterprise-tap because we don't have any TAP tests in the EE repository. Follow-up commit d8faca69 ("lua: implement stubs for etcd client"). NO_DOC=gitignore NO_TEST=gitignore NO_CHANGELOG=gitignore
-
Sergey Bronnikov authored
Patch made following changes: - add a CMake function lua_multi_source that unpacks symbol names and Lua source paths in passed list and compiles Lua source into *.lua.c sources using lua_source() function. Function is needed for embedding Lua modules consisting from a number of files to Tarantool EE. - add using lua_multi_source instead of lua_source for etcd client module. - remove function prototypes and declarations related to extra features. All this code is moved to Tarantool EE source code. Part of #8620 Follows up https://github.com/tarantool/tarantool-ee/issues/443 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Aleksandr Lyapunov authored
Fefactor lua name resolution, simplify and comment. Add an ability to specify path with brackets, for example in 'box.space[512]:get' or 'box.space["test"]:get'. Only literals (strings and numbers) are supported. Closes #8604 @TarantoolBot document Title: square brackets in procedure resolution for Lua calls Square brackets are now supported in Lua call procedure resolution. This is applicable to `net.box` connection objects `call` method as well as `box.schema.func.call`. Examples of function calls with square brackets can be found in the test to this patch.
-
- May 16, 2023
-
-
Maksim Kokryashkin authored
Default token for actions provided by GitHub lacks read and write permissions for private projects. This commit changes the token used for the stale PRs workflow to the dedicated token with needed permissions. NO_DOC=Workflow fix NO_TEST=Workflow fix NO_CHANGELOG=Workflow fix
-
Oleg Babin authored
Sometimes we need negative timestamps to work with dates before 1970. But seems such cases were even covered in tests. So there wasn't any handling of negative timestamps with fraction part. Such datetime objects had incorrect string representation (e.g. "1963-11-22T12:30:02.-999"). This patch fixes it. Closes #8570 NO_DOC=bugfix
-
Oleg Babin authored
Seems that problem code part was ported from Lua as is. But there is some difference between modulo operator in C and in Lua. Lua always returns positive value but in C result could be negavive. This difference led to the case when after subtraction nsec part of datetime object become negative that yielded weird result on attempt to get string representation (e.g."2008-02-03T03:36:43.-100Z"). This patch fixes it. Part of #8570 NO_DOC=bugfix NO_CHANGELOG=see next commit
-
- May 15, 2023
-
-
Andrey Saranchin authored
Currently, to export IPROTO constants to Lua, we define a translation table in C manually. As a result, some constants are forgotten and some are exported with mistakes. Fortunately, we have a mechanism to generate enums and its stringified names in compile-time. Let's rewrite iproto constants using such mechanism and generate translation table automatically. Closes #8443 Closes #8574 Closes #8577 NO_DOC=bugfix
-
Andrey Saranchin authored
We are going to generate iproto_key enum automatically to generate iproto constants for Lua as well. This mechanism generates *_MAX constant for enum using its name, which is in lower case. So let's convert IPROTO_KEY_MAX to form which is appropriate for enum generator. Part of #8443 NO_TEST=rename constant NO_CHANGELOG=internal NO_DOC=internal
-
Serge Petrenko authored
This commit adds another possible bootstrap_strategy to accompany "auto" and "config": "supervised". Such a strategy may be useful to pin the desired bootstrap leader on an active cluster (so that the user may join the replicas from the desired node without changing their box.cfg) or to manually set the bootstrap leader among the nodes that managed to start without issues. More details are in the docbot request. Closes #8509 @TarantoolBot document Title: new bootstrap strategy - "supervised" The `bootstrap_strategy` configuration option may now be set to "supervised". This strategy works as follows: When bootstrapping a new replicaset, the nodes do not choose a bootstrap leader automatically and instead wait for it to be appointed by the user. The configuration will fail if no bootstrap leader is appointed during a `replication_connect_timeout`. In order to apppoint a bootstrap leader, the user has to issue `box.ctl.make_bootstrap_leader()` call on the desired node. This must be done during the initial `box.cfg()` call. Possible ways to achieve this are: 1. In interactive mode: ```lua fiber.create(box.cfg, desired_config) box.ctl.make_bootstrap_leader() ``` 2. Via an init script: ```lua -- Init script: console.listen(admin_port) box.cfg(desired_config) -- User console: tarantoolctl enter admin port > box.ctl.make_bootstrap_leader() ``` When joining a new replica with `bootstrap_strategy` = "supervised" to an existing replica set, the replica will not choose the bootstrap leader automatically, but will instead join to the node on which `box.ctl.make_bootstrap_leader()` was issued last. In case such a node isn't found after a `replication_connect_timeout`, the configuration fails.
-
Serge Petrenko authored
Both triggers are used to wait for specific applier state (applier is connected, remote node ballot is known, etc.) and are often used one after another. Keeping these two trigger lists only adds complexity of creating 2 triggers instead of 1 in some places. Let's unify the triggers. Now the triggers might be fired multiple times, so make all the existing trigger functions reenterable. In-scope-of #8509 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Serge Petrenko authored
The idea is to simplify applier_on_connect_f as much as possible, so that it only wakes the waiter up, but doesn't participate in any decisions regarding the replicaset connection state. In-scope-of #8509 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-