- Oct 27, 2023
-
-
Georgiy Lebedev authored
Add new MsgPack extension for tuples MP_TUPLE with the following structure: +------+-----------------+----------+ |MP_EXT| MP_UINT | MP_ARRAY | +------+-----------------+----------+ ^ ^ format identifier tuple data Needed for #8147 NO_CHANGELOG=<internal change> NO_DOC=<internal change>
-
Georgiy Lebedev authored
When data is coming from IPROTO and is passed to Lua, it is convenient to pass raw MsgPack data (see `box.schema.func::takes_raw_args` option). Currently, the port's MsgPack is retrieved directly via `port_get_msgpack` and then a MsgPack object is constructed via `luamp_push`. In the future, ports can have a MsgPack decoding context that will also be needed to be passed to the MsgPack object constructor, so we encapsulate the MsgPack object construction completely into a port method by introducing a new operation mode to `port_dump_lua`. This new operation mode is essentially a no-op for a MsgPack port (which is the case for data coming from IPROTO), and is completely inefficient for other types of ports (since an intermediate conversion to MsgPack occurs), but we do not expect users to be calling Lua stored procedures with the `takes_raw_args` locally. To handle the latter case we introduce a `port_dump_lua_mp_object_mode_slow` helper function which facilitates retrieving MsgPack data from ports via region allocation, creating a Msgpack object and freeing the MsgPack data. Needed for #8147 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
Currently, `port_dump_lua` has 2 modes of operation: 'flat' and 'tabular', which are controlled via a boolean flag. We plan to extend it with another mode of operation, 'Msgpack object', so we need to replace the flag with a enumeration. Plus, this refactoring should improve readability. Needed for #8147 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
In scope of #8633 call and eval encoding will require an additional argument (whether to encode tuples as extensions and send formats), and method encoders already have 3 arguments (MsgPack stream, IPROTO sync and IPROTO stream identifier): to simplify further extension of method encoders move arguments to a netbox_method_encode_ctx struct. Needed for #8633 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
The MsgPack extension encoder has a strange interface: it returns the type it has encoded the value to which at the same time indicates whether it has succeeded in encoding the value — instead, let's return the type in an output parameter and return the encoding status as a boolean value. Needed for #8147 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
The MsgPack context structure is the essential data structure for collecting and retrieving various meta information during MsgPack encoding/decoding. This structure will be a base for a box-specific MsgPack context (and, possibly, others), so we make it extendable by adding a padding (to avoid dynamic allocations). Refactor existing MsgPack encoding interfaces to use the MsgPack context structure: 1. extend the port dumping interface, so that we can collect meta information about dumped tuples' formats; 2. extend the port MsgPack retrieval interface, so that we can also retrieve meta information about tuple formats for decoding tuples coming from IPROTO (needed when decoding is postponed to a MsgPack object), the context ownership is acquired by the caller to avoid copying; 3. extend the MsgPack encoding/decoding interfaces to accept a MsgPack context to collect meta information during encoding or to use this meta information for decoding — the context is passed down up to the extension encoder/decoder, which is the point where future box tuple extension will be processed. Needed for #8147 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
The tuple format map is the essential data structure for collecting tuple formats during box tuple encoding and for looking up tuple formats during box tuple decoding. Needed for #8147 NO_CHANGELOG=<internal data structure> NO_DOC=<internal data structure> Co-authored-by:
Aleksandr Lyapunov <alyapunov@tarantool.org>
-
Nikita Zheleztsov authored
At the last itaration of review it was decided to throw an alerts, when no UUID was passed to config and name is not set in _cluster. This leads to alerts, thrown during replicaset bootstrap. However, _cluster:on_replace trigger wasn't updated for that change, it asssumed that when insert of a new replica is done, no alerts was thrown. Let's fix the behavior of on_replace trigger so that it deletes alerts as soon as replica joined with name. Follow-up #8978 NO_DOC=bugfix NO_TEST=<already tested> NO_CHANGELOG=bugfix
-
Nikolay Shirokovskiy authored
New commits: - test: fix test build for Debian 9 and alike - test: fix unused variable warning in matras test NO_TEST=submodule bump NO_CHANGELOG=submodule bump NO_DOC=submodule bump
-
Nikita Zheleztsov authored
It's impossible to set names on Tarantool below 3.0.0, as all DDL is forbidden before schema upgrade. Let's make names NoOp on schema below Tarantool 3.0.0 and set names automatically only when schema upgrade is done. Follow-up #8978 NO_DOC=tarantool/doc#3661
-
Nikita Zheleztsov authored
This commit moves all code, related to working with versions and which was used in box/lua/upgrade.lua, into a separate module and exports it to Lua API as 'internal.version' This is needed, as in the following commit we set names automatically only when schema version is more than 3.0.0. This module is used their in order to avoid code duplication. Follow-up #8978 NO_DOC=internal NO_TEST=<already tested> NO_CHANGELOG=internal
-
Nikita Zheleztsov authored
For now it's impossible to use config module in order to recover from snaps, which don't have names set in them. Calling box.cfg with names on recovery fails with MISMATCH error, which is caused by difficult implementation of setting names on first box.cfg, as names can be set only on rw instance. This commit doesn't call box.cfg with names, if these names are missing from the snapshot file. Instead it creates a fiber, which will set names, when it's possible to do so. Only master sets the names for the whole cluster. Closes #8978 NO_DOC=tarantool/doc#3661 NO_CHANGELOG=following commits
-
Nikita Zheleztsov authored
When the name is manually set on master by replace in _cluster space, calling box.cfg on replica with the same name causes its hang. The problem is the fact, that resubscribe is initiated and waiting for APPLIER_REGISTERED status is started. As applier knows, that no registration should be done, this never happens. Let's don't initiate registration, when instance name is already set. Needed for #8978 NO_DOC=bugfix NO_CHANGELOG=not released yet
-
Nikita Zheleztsov authored
For now it's impossible to drop created alert in any way except manual searching for _alerts table. However, we need to drop alerts on missing names, when the names are set. Let's introduce simple key-value alerts in order to easily drop them by key. Needed for #8978 NO_DOC=internal NO_TEST=following commit NO_CHANGELOG=internal
-
Nikita Zheleztsov authored
Currently only instance_uuid is validated before recovery process. All names and replicaset_uuid are checked only when recovery is done, which can take a long time. It can be frustrating to users, which have been waiting for several hours only to get name mismatch error. Let's read the small part of snapshot file before calling box.cfg in order to figure out, whether the names and uuids, passed to configuration match the ones, saved inside the snapshot. During config reload there's no sense in reading snapshot file, as data is already saved inside spaces, let's read them. We still check that names in config and names in spaces don't contradict during config reload. This commit also introduces methods, for getting names, which are not set in snap (or memory), this'll be used in consequent commits to set names automatically. Needed for #8978 NO_DOC=tarantool/doc#3661
-
Nikita Zheleztsov authored
box.schema has a number of constants, e.g. IDs of system spaces, which may be useful for the user. Let's allow to access box.schema before box.cfg is called. It is used in checking names, as we need to know, which snapshot spaces to scan. Needed for #8978 NO_DOC=minor change
-
Nikita Zheleztsov authored
This commit introduces the new method for xlog module: xlog.meta(). It opens an xlog file, reads and returns the meta block of the file, which includes its filetype, instance_uuid and vclocks. It's needed in order to introduce checking of names inside the config module in the following commit. Needed for #8978 @TarantoolBot document Title: xlog.meta(file-name) method Description: Open an xlog file, and return its meta block. Possible errors: Failed to open a file or it does not contain properly formatted meta block. Example: ```lua tarantool> xlog = require('xlog') --- ... tarantool> xlog.meta('00000000000000000000.snap') --- - filetype: SNAP prev_vclock: {} instance_uuid: 87b2e60f-275c-4efa-9b0e-e9562e309692 vclock: {} ... ```
-
Nikita Zheleztsov authored
Config's box.cfg applier scans snapshot_dir in order to find out, whether recovery is going to be done. It's needed in order to determine, whether the instance should be started into ro mode firstly. Let's move info about snapshot into separate file in utils. The commit also introduces snapshot_path, which will be used in the following commits in order to validate names. Needed for #8978 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
- Oct 26, 2023
-
-
Georgy Moshkin authored
Closes #9237 Add exports for fiber_set_name_n, fiber_name, fiber_id, fiber_csw & fiber_find. Also make fiber_set_joinable, fiber_set_ctx & fiber_get_ctx interpret NULL as the current fiber. @TarantoolBot document Title: add basic fiber api to ffi exports. 5 basic functions can now be used via ffi api, which were previously only accessible via lua api: fiber_set_name_n, fiber_name, fiber_id, fiber_csw & fiber_find. fiber_set_joinable now interprets NULL as current fiber.
-
Ilya Verbin authored
Old: "Failed to allocate 2147483648 bytes in hash_table for key" New: "Failed to allocate 16384 bytes in hash_table for key" ERRINJ_INDEX_ALLOC cannot be used to test this error, because it fails earlier, so ERRINJ_HASH_INDEX_REPLACE is introduced. Follow-up #3594 NO_DOC=minor NO_CHANGELOG=minor
-
Ilya Verbin authored
The issue is fixed in the `small' submodule by the following commit: * matras: fix matras_view::block_count overflow Closes #3594 NO_TEST=The test requires more than 64 GB of RAM. @TarantoolBot document Title: Document maximum number of tuples in hash index Product: Tarantool Root document: https://www.tarantool.io/en/doc/latest/book/box/limitations/ Number of tuples in hash index: 2147483648
-
Vladimir Davydov authored
The bug was fixed in the small library: - slab: fix NULL ptr deref in assertion in slab_get https://github.com/tarantool/small/commit/ef77efacd452cb90caea2caf22d266f791c95ec3 - slab: fix uint32_t overflow in slab_capacity https://github.com/tarantool/small/commit/77203600a7c645d97bce56f901eec25de0b29d6e The small library submodule was updated in commit ebafd684 ("small: bump version"). Closes #9218 NO_DOC=changelog NO_TEST=changelog
-
Alexander Turenko authored
Recently added persistent instance/replicaset/cluster names have certain validation rules (see #5029 and #9148). An instance name and a replicaset name that are provided in a declarative configuration are stored in the database, so they should follow the same rules. This patch implements the validation for instance/replicaset/group names, for `--name` CLI option and `TT_INSTANCE_NAME` environment variable. Part of #8862 Related to #9148 NO_DOC=The user visible change is about a better error message if a wrong name is passed. The naming rules are documented in https://github.com/tarantool/doc/issues/3466, https://github.com/tarantool/doc/issues/3467, https://github.com/tarantool/doc/issues/3468.
-
Alexander Turenko authored
Sometimes shell quoting is needed in tests to trigger a validation error. For example, if the argument is empty or contains whitespaces. Left the default unchanged to don't affect existing tests. Part of #8862 NO_DOC=testing helper change NO_CHANGELOG=see NO_DOC NO_TEST=see NO_DOC
-
Nikolay Shirokovskiy authored
We hit #3807 in release/2.11 for release ASAN build with ASAN-friendly small allocators. Follow-up #7327 NO_CHANGELOG=internal NO_DOC=internal
-
Vladimir Davydov authored
Closes #9277 @TarantoolBot document Title: Document `box_space_execute_priv` compatibility option Historically, it was possible to grant the `execte` privilege on a space although this action had no effect. Since Tarantool 3.0 it isn't allowed anymore. The new `compat` module option `box_space_execute_priv` was added to revert to the old behavior. Please create a documentation page for the new compatibility option at https://tarantool.io/compat/box_space_execute_priv Example: ``` tarantool> box.cfg{log_level = 'error'} --- ... tarantool> box.schema.user.create('alice') --- ... tarantool> box.schema.user.grant('alice', 'execute', 'space') --- - error: Unsupported space privilege 'execute' ... tarantool> require('compat').box_space_execute_priv = 'old' --- ... tarantool> box.schema.user.grant('alice', 'execute', 'space') --- ... ```
-
- Oct 24, 2023
-
-
Vladimir Davydov authored
Configuring log modules work differently with log.cfg and box.cfg: box.cfg{log_modules=...} overwrites the current config completely while log.cfg{modules=...} overwrites the currently config only for the specified modules. Let's fix this inconsistency by making log.cfg behave exactly as box.cfg. Closes #7962 NO_DOC=bug fix
-
- Oct 23, 2023
-
-
Sergey Bronnikov authored
This reverts commit 0ea1ba87. Bump is reverted due to regression in Curl 8.4.0. Needed for #9283 NO_DOC=libcurl submodule bump NO_TEST=libcurl submodule bump NO_CHANGELOG=libcurl submodule bump
-
Sergey Ostanevich authored
We decided to allow underscore in names to provide easier support for many existent installations. Closes #9148 NO_CHANGELOG=no updates to the feature description NO_DOC=update existent doc tickets 3466, 3467, 3468
-
Nikolay Shirokovskiy authored
Message body resides in one of rotating input buffers for the connection. When we don't need message body anymore we advance the reading end of the input buffer by the size of the message. But message processing order can differ from the order of messages in the wire. Thus this advancing a bit hacky. Let's instead mark the data in the input buffer as read when we process all the messages in the input buffer. We cannot reuse the buffer any earlier anyway. Follow-up #7327 NO_TEST=refactoring NO_CHANGELOG=refactoring NO_DOC=refactoring
-
Nikolay Shirokovskiy authored
The API functions additionally poison related data in ASAN build. Follow-up #7327 NO_TEST=refactoring NO_CHANGELOG=refactoring NO_DOC=refactoring
-
Nikolay Shirokovskiy authored
New commits: - ibuf: change ibuf_consume_before argument type to const void ptr - ibuf: introduce ibuf_consume/ibuf_consume_before API - matras: fix matras_view::block_count overflow - slab: fix uint32_t overflow in slab_capacity - debug: add assertions of ASAN version to non ASAN - ibuf: add ibuf_discard - slab: fix NULL ptr deref in assertion in slab_get NO_TEST=submodule bump NO_CHANGELOG=submodule bump NO_DOC=submodule bump
-
Nikolay Shirokovskiy authored
The test is quite a flacky in debug ASAN CI workflow. The issue is test check upper boundary of it's execution time. I run many instances of this test on in parallel and got average time of 40s for memtex and 70s for vinyl. The time quota is already changed by the commit 84cb1e04 ("sql: increase time quota for selectG test on vinyl") for laptops with HDD. I did not check execution time for HDD though. I guess the bottleneck for debug ASAN is CPU. Follow-up #7327 NO_CHANGELOG=internal NO_DOC=internal
-
- Oct 20, 2023
-
-
Sergey Ostanevich authored
The error message was not quite clear in case instance has any name defined and config provides a different one. Closes #8692 NO_CHANGELOG=no previous release with this error message NO_DOC=not needed since no changes to the public API
-
Vladimir Davydov authored
We install a signal handler that prints the stack trace on SIGSEGV, SIGBUS, SIGILL, SIGFPE. The signal handler uses the current stack. This works fine for most issues, but not for stack overflow, because the latter makes the current stack unusable, leading to a crash in the signal handler. Let's install an alternative signal stack in each thread so that we can print the stack trace on stack overflow. Note that we skip this for ASAN because it installs its own signal stack. (Installing a custom stack would result in a crash.) Closes #9222 NO_DOC=bug fix
-
Sergey Vorontsov authored
Modify Linux workflows for running jobs inside Docker containers. It helps isolate the building and testing environment from the runner's system and other workflows. Closes tarantool/tarantool-qa#327 NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
Magomed Kostoev authored
The macro expands to inline keyword and always_inline attribute if it's supported. This attribute forces the compiler to inline the function if it's possible and raise a diagnostic if it's not. Needed for tarantool/tarantool-ee#580 NO_DOC=internal NO_TEST=internal NO_CHANGELOG=internal
-
- Oct 19, 2023
-
-
Gleb Kashkin authored
With this patch the following log/alert structure is introduced: * all informational messages (e.g. some privs granted) use `log.verbose()` * all warnings about postponed apply use `config:_alert{type = 'warn'}` * all apply errors are reported with `config:_alert{type = 'error'}` Debug logs are removed. Part of #8967 NO_DOC=logging and alerts NO_CHANGELOG=see NO_DOC NO_TEST=see NO_DOC
-
Gleb Kashkin authored
Before this patch luatest.server used to fail if the instance status was different from 'ready'. Now it allows 'check_warnings' state too. It is required to test legal corner cases inside config applier. Part of #8967 NO_DOC=test helper update NO_TEST=see NO_DOC NO_CHANGELOG=see NO_DOC
-
Vladimir Davydov authored
There are four problems we have to address to make this possible: 1. memtx_engine_shutdown may delete the tuple referenced by box_tuple_last so that tuple_free, which is called later by box_free, will crash trying to free it. Fix this by clearing box_tuple_last in memtx_engine_shutdown. 2. tuple_format_destroy and tuple_field_delete, called by it, expect all constraints to be detached. Let's destroy the constraints if this isn't the case. This effectively reverts commit 7a87b9a5 ("box: do not call constraint[i].destroy() in tuple_field_delete()"). 3. tuple_field_delete, called by tuple_format_destroy, expects the default value function to be unpinned. Let's unpin it if this isn't the case. To avoid linking dependencies between the tuple and box libraries, we have to introduce a virtual destructor for field_default_func. 4. The tuple_format unit test calls tuple_free after box_free. If box_free calls tuple_free by itself, this leads to a crash. Fix this by removing tuple_free and tuple_init calls from the test. Closes #9174 NO_DOC=code health NO_CHANGELOG=code health NO_TEST=checked by existing tests
-