- Jul 05, 2023
-
-
Magomed Kostoev authored
1. Make double-formatted fields accept integer and float values. 2. Make indexes compare the values as double if the field key type is FIELD_TYPE_DOUBLE. 3. Make hashers cast double key field to double before hashing, so we are able to insert and select any int, uint, float or double if their value casted to double is equal (for double keys). Notes about tuple_compare.cc: Since now `mp_compare_double` casts any value placed in field to double it was renamed to `mp_compare_as_double` to not semantically conflict with existing `mp_compare_double_*` functions. Notes about tuple_hash.cc: The hashee cast result is encoded in MP_DOUBLE and hashed for backward compatibility reasons. Since now the field hashing function (tuple_hash_field) requires field type to hash the field correctly, a new parameter has been introduced. By the way added assertions to the generic `field_hash` to prevent invalid hashing for new precompiled hashers and made `key_hash_slowpath` static cause it's only used in this file. Closes #7483 Closes #5933 Unblocks tarantool/crud#298 @TarantoolBot document Title: It's not required to ffi-cast integral floating point to double anymore. The page describing tarantool data model states that: > In Lua, fields of the double type can only contain non-integer > numeric values... If the patch is merged this isn't the case anymore, so this statement and the code snippet below it should be updated. Link to the document: [Data storage](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-type-details). Affected segments: > double. The double field type exists mainly to be equivalent > to Tarantool/SQL’s DOUBLE data type. In msgpuck.h (Tarantool’s > interface to MsgPack), the storage type is MP_DOUBLE and the > size of the encoded value is always 9 bytes. In Lua, fields of > the double type can only contain non-integer numeric values and > cdata values with double floating-point numbers. Examples: 1.234, > -44, 1.447e+44. > > To avoid using the wrong kind of values inadvertently, use > ffi.cast() when searching or changing double fields. For example, > instead of space_object:insert{value} use ffi = require('ffi') > ... space_object:insert({ffi.cast('double',value)}). Example: > > ``` > s = box.schema.space.create('s', {format = {{'d', 'double'}}}) > s:create_index('ii') > s:insert({1.1}) > ffi = require('ffi') > s:insert({ffi.cast('double', 1)}) > s:insert({ffi.cast('double', tonumber('123'))}) > s:select(1.1) > s:select({ffi.cast('double', 1)}) > ```
-
Magomed Kostoev authored
This update pulls the following commits: * Add mp_read_double_lossy without direct convertibility checks * Fix mp_read_double_lossy tests freebsd build These commits introduce a function required to compare and hash msgpack values of double key fields as double. Need for #7483, #5933 NO_DOC=see the next commit NO_CHANGELOG=see the next commit
-
Kirill Yukhin authored
NO_DOC=no code changes NO_TEST=no code changes NO_CHANGELOG=no code changes
-
Alexander Turenko authored
Follows up #8724. NO_DOC=just add a changelog entry NO_TEST=see NO_DOC
-
Yaroslav Lobankov authored
Tarantool 3.0.0 is not going to be distributed as a set of packages for specific distros. Instead, it is planned to have a few `deb` and `rpm` packages with a statically compiled Tarantool binary inside, which will work on almost any distro. Also, we drop building packages with GC32. Now only GC64 packages will be shipped. Static build packaging was added at #8771. NO_DOC=ci NO_TEST=ci
-
- Jul 04, 2023
-
-
Igor Munkin authored
* test: fix flaky <unit-jit-parse.test.lua> * Fix use-def analysis for vararg functions. * Fix use-def analysis for BC_VARG. * Fix TNEW load forwarding with instable types. * Fix memory probing allocator to check for valid end address, too. * Another fix for lua_yield() from C hook. * Fix lua_yield() from C hook. * Fix saved bytecode encapsulated in ELF objects. * x64: Fix 64 bit shift code generation. * Fix canonicalization of +-0.0 keys for IR_NEWREF. * test: add utility for parsing `jit.dump` * test: split utils.lua into several modules * test: rewrite lj-49-bad-lightuserdata test in C * test: rewrite misclib-sysprof-capi test in C * test: rewrite misclib-getmetrics-capi test in C * test: introduce utils.h helper for C tests * test: introduce module for C tests * test: fix setting of {DY}LD_LIBRARY_PATH variables * build: fix build with LUAJIT_USE_GDBJIT enabled Closes #8718 Part of #7900 Part of #8516 NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump
-
Alexander Turenko authored
The following syntax variants are fobidden now: * user@ * user:pass@ * user@host:port * user:pass@host:port Only host:port is allowed (inet URI or unix socket URI). The idea is that we shouldn't distribute login/password information using the configuration, so it is better to explicitly forbid such usage of the option. Closes #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
Before this commit all the peers of the given replicaset were obligated to have suitable URI to connect -- either iproto.advertise.peer or one of iproto.listen URIs. Now replication.peers is successfully constructed if at least one suitable URI is constructed (except URI of the given instance itself). However, if an unknown user is used in the iproto.advertise.peer option of any peer, it is considered as an error (startup failure), just like before. The testing code for unsuccessful cases of replication.peers autobuilding is rewritten to reflect that just one bad peer doesn't cause a startup failure anymore. Now it also needs stripping of log messages, because the peers autobuilding tells about unsuitable URIs. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
This failover mode enables automatic leader election on a replicaset. Assigning a leader manually (`leader` option) and assigning RO/RW mode (`database.mode` option) are forbidden in this failover mode. Configuration example: ```yaml replication: failover: election # !! groups: group-001: replicasets: replicaset-001: instances: instance-001: {} instance-002: {} instance-003: {} ``` All the replicaset instances are so called candidates by default: they can vote for a leader, they can be elected as a leader. However, it is possible to set `replicaset.election_mode` to 'voter', 'off' or 'manual'. See more detailed description in the box_cfg applier comments in the code and in the box.cfg() options documentation [1]. Note: Unlike box.cfg()'s option 'election_mode', the replication.election_mode = "off" forces the instance to be read-only (if failover = "election" is enabled). box.cfg() call has no cluster configuration and it doesn't know, whether the given replicaset is managed by the built-in election algorithm or an external coordinator. The election failover may be used to improve cluster's availability. [1]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-replication-election-mode Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
This failover mode allows to set a leader for a replicaset by its instance name instead of per-instance database.mode option. For example: ```yaml replication: failover: manual # !! groups: group-001: replicasets: replicaset-001: leader: instance-001 # !! instances: instance-001: {} instance-002: {} instance-003: {} ``` The "manual" failover mode doesn't allow several leaders in a replicaset. If it is desired, use the "off" failover mode. The "manual" mode doesn't perform a proper leader switching at the moment. An administrator should resign the old leader (by unsetting the leader option or setting it to `null`), wait till all the data arrives to the upcoming leader and then configure it as the new leader. The proper leader switching is subject of a future work. Also, the next commit will add replicaset.failover = "election", which handles such problems. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
It has the only one value for now: 'off'. Two other ones are planned: 'manual' and 'election'. The 'off' value means exactly what is implemented now: the database.mode option controls whether a particular instance should serve write requests. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
If the given instance has no existing snapshot, it means that it is not registered in the replicaset and will attempt to join as a new replica. It has no chance to succeed if all the instances in the replicaset are in the read-only mode: neither of them can register the new instance. Report an error and fail the startup in the case to make the situation clear. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
If the startup may take a long time (reads of an existing snapshot or fetches it from a remote master), then * If there are other instances in the replicaset, force the read-only mode on the database startup[^1]. * After the database startup, re-read the configuration and apply the fresh one (now without any RO/RW mangling). If the startup shouldn't take a long time[^2], the configuration is read and applied once. [^1]: The read-only mode is not forced if there is no existing snapshot. It may lead to missing bootstrap leader in the replicaset. See details in the comment in the box_cfg applier code. [^2]: There is the only case that we consider as such: startup of an instance without an existing snapshot without other instances in the same replicaset. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC NO_TEST=it is not trivial to verify automatically
-
Alexander Turenko authored
Enabling read-write by default is unsafe for an instance in a replicaset with more than one instance. The only meaningful default here is read-only. On the other hand, a signleton instance (the only one in its replicaset) almost always started in the read-write mode in practice. Let's use these values as defaults for these situations. The name of the option is changed from `rw` (boolean) to `mode` (enum with allowed values `'ro'` and `'rw'`). We agreed on the enum after a long controversy whether it should be `ro` or `rw`. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
In brief: * client -- for external clients * peer -- for connections within the cluster, in particular for replicas * sharding -- for routers and a rebalancer See the instance_config.lua file for the details. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
The following new syntax variants are introduced for `iproto.advertise`. * `user@` -- use the given user, a password from the `credentials` section and `host:port` from `iproto.listen` * `user:pass@` -- use the given user and password, use `host:port` from `iproto.listen` * `user@host:port` -- use the given user, host and port, use a password from the `credentials` section It allows to don't repeat the same information in different places of the config. The `test_no_advertise_*` test cases are generalized: now the same boilerplate code is used for these and new cases. The `test.luatest_helpers.server` util gains ability to parse all the new `iproto.advertise` variants. The replicaset and etcd configuration examples are updated to use the `user@` syntax. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
There are a few nuances in using the `iproto.listen` option as a peer URI at replicaset building (constructing a list of peer URIs). * `iproto.listen` is a comma separated list of URIs: the single URI is just a special case. * `iproto.listen` may contain `0.0.0.0` or `::` host and/or zero port. It is perfectly valid for listening, but doesn't give enough information to connect. Let's handle `iproto.listen` appropriately: split it the URI list, walk over the URIs and check whether the URI is suitable to connect. The first suitable URI is used as the peer URI. If no suitable URI is found, raise an error (it means a startup failure if occurs at a startup). Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
Aside of a fast feedback for a user that feeds an incorrect configuration, this validation simplifies further box_cfg applier code: we can assume that the URI list is valid. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
The following values shouldn't be accepted as an advertise URI: * Not an URI. * Comma separated list of URIs. * An URI with IPv4/IPv6 INADDR_ANY host (`0.0.0.0` or `::`). * An URI with zero TCP port. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
We can't just pass it over and start the instance: 1. If it is an initial bootstrap (there is no data for the given instance yet), the instance will form its own replicaset instead of joining into the existing one. 2. If it is a startup of an existing instance, the local data may be outdated and serving requests is undesirable. Part of #8810 NO_DOC=the old behavior was not released, the documentation request will be registered manually NO_CHANGELOG=see NO_DOC
-
Georgiy Lebedev authored
In scope of #4693, we now create runtime tuple formats from format data, so we need to adapt space read views accordingly: they use a field definition array for creating tuple formats and for accessing field names and types — for the latter case we will also allocate a field definition array separately. Follows-up #4693 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Sergey Bronnikov authored
Previous attempt to fix flakiness in commit 6a2c73f8 ("test: fix flakiness in gh_6128_background_mode_test") used a constant buffer size in check_err_msg function. Tarantool 2.10 has a bit bigger log before a desired message that other versions of Tarantool and it leads to a this resulted in a truncated message ("entering the even" instead of "entering the event loop"). Patch replaces check_err_msg() implementation to grep_log used in luatest, it reads the whole log. Also patch renames check_err_msg to check_msg, because "entering the event loop" is not an error message. Follows up #6128 NO_CHANGELOG=fix test NO_DOC=fix test
-
- Jul 03, 2023
-
-
Mergen Imeev authored
This patch reworks the way the meta is populated. This is done to incrementally populate the metadata, instead of set the metadata at the end of reading data from the source. This allows to get the correct meta in cases where getting data from the source failed. Follow-up #8789 NO_DOC=feature not yet released NO_TEST=tested in EE NO_CHANGELOG=feature not yet released
-
Vladimir Davydov authored
The bug was fixed in the libyaml repository. This commit just updates the submodule and adds a test. Closes #8782 NO_DOC=bug fix
-
Magomed Kostoev authored
Function execution permissions should only be checked on constraint creation. So when the function is used to check a tuple access rights don't have to be checked on each call for the current user. Closes #7873 NO_DOC=bugfix
-
Magomed Kostoev authored
Function execution permissions should only be checked on functional index creation and on functional index function set. So when the function is used by key_list_iterator its rights don't have to be checked on each call for the current user. Part of #7873 NO_DOC=bugfix NO_CHANGELOG=see the next commit
-
Georgiy Lebedev authored
When a space is altered, we abort all in-progress transactions and delete all stories related to that space: the problem is we don't delete the stories' read gaps, which are also linked to the stories' transactions, which get cleaned up on transaction destruction — this, in turn, results in heap-use-after-free. To fix this, clean up stories' read gap in `memtx_on_space_delete` — we don't do this in `memtx_tx_story_delete` since it expects the story to not have any read gaps (see `memtx_tx_story_gc_step`). Tested this patch manually against Nick Shirokovskiy's experimental small-ASAN integration branch. Closes #8781 NO_DOC=bugfix NO_TEST=<already covered by existing tests, but was not detectable by ASAN>
-
Sergey Vorontsov authored
The created packages can be found in the job artifacts. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci Co-authored-by:
Yaroslav Lobankov <y.lobankov@tarantool.org>
-
Yaroslav Lobankov authored
This target is a wrapper on the command to build DEB and RPM packages with a statically compiled Tarantool binary inside. It just runs the `./static-build/make_packages.sh` script with the properly defined VERSION env variable. Example of usage: $ make -f .pack.mk package-static NO_DOC=make NO_TEST=make NO_CHANGELOG=make
-
Sergey Vorontsov authored
This patch adds facility to make DEB and RPM packages with a statically compiled Tarantool binary inside. The build is performed in a Docker container, using PackPack docker image (centos-7) and CPack. The packpack/packpack:centos-7 image has all the necessary dependencies for building Tarantool and quite old glibc 2.17 which theoretically provides compatibility of the created packages with any distro where glibc >= 2.17. The build can be run with the command below: $ VERSION=3.0.0 ./static-build/make_packages.sh In the `static_build` directory, there will be the following packages: tarantool_3.0.0-1_amd64.deb tarantool-dev_3.0.0-1_amd64.deb tarantool-3.0.0-1.x86_64.rpm tarantool-devel-3.0.0-1.x86_64.rpm `tarantool_3.0.0-1_amd64.deb`, `tarantool-3.0.0-1.x86_64.rpm` are packages with the Tarantool server binary inside. `tarantool-dev_3.0.0-1_amd64.deb`, `tarantool-devel-3.0.0-1.x86_64.rpm` are packages with the Tarantool server development files inside. NO_DOC=build NO_TEST=build Co-authored-by:
Yaroslav Lobankov <y.lobankov@tarantool.org>
-
Georgiy Lebedev authored
Introduce `box.tuple.format` object, a Lua wrapper around tuple format: these objects own a tuple format, which is almost equivalent to `space:format`, except for check constraints and foreign key constraints being disabled (though they appear to be present for compatibility with `space:format`). Add an option table argument to `box.tuple.new` with 'format' option, allowing to create formatted spaceless tuples. Closes #4693 @TarantoolBot document Title: Formats for standalone tuples and `box_tuple_new_vararg` compat opt A new box.tuple.format library was added, with a tuple format constructor (`new`) and a tuple format validator (`is`). New tuple format objects (userdata) were added, which can be used with the same format clause as for the `space:format` method (except that check constraints and foreign keys are disabled for them): NO_WRAP ```lua f = box.tuple.format.new(box.space._space:format()) f = box.tuple.format.new{{name = 'field1', type = 'string', is_nullable = true, nullable_action = 'none', collation = 'unicode_uk_s2', default = 'UPPER("string")', constraint = {ck = 'box.schema.user.info'}, foreign_key = {fk = {space = '_space', field = 'name'}}}, {name = 'field2', nullable_action = 'ignore', foreign_key = {fk = {space = '_space', field = 1}}}} ``` NO_WRAP Format objects have several introspection methods: `:pairs`, `:ipairs`, `totable`, and also have a `__serialize` metamethod — these methods return the original (i.e., user-provided) format clause. `:pairs` is an alias to `ipairs` (since the format clause is an array by nature), and the `totable` method is an alias to the `__serialize` metamethod, which returns an array of field definitions. Format objects also have a `:tostring` method, which simply returns a "box.tuple.format" literal. The standalone tuple constructor, `box.tuple.new` was extended with an options parameter which currently has one available option, `format` (default value is `nil`, i.e., no format). The format option is either a tuple format object previously created using `box.tuple.format.new` or a format clause. Examples of standalone tuple creation with formats: NO_WRAP ```lua box.tuple.new({1}, {format = {{name = 'field', type = 'number'}}}) box.tuple.new({1}, {format = {{'field', type = 'number'}}}) box.tuple.new({1}, {format = {{'field', 'number'}}}) f = box.tuple.format.new({{name = 'field', type = 'number'}}) box.tuple.new({}, {format = f}) box.tuple.new({1}, {format = f}) box.tuple.new({'str'}, {format = f}) -- error: Tuple field 1 (field) type does not match one required by operation: expected number, got string box.tuple.new({'str'}, {format = f}) ``` NO_WRAP See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#a33e2d7418d249679969e5f21ef2832c A new `box_tuple_new_vararg` compatibility option was introduced: a new page needs to be created for it (https://tarantool.io/compat/box_tuple_new_vararg) This option controls whether `box.tuple.new` should interpret an argument list as an array of tuple fields (i.e., vararg, old behaviour), or as a value plus a tuple format (new default behaviour). The value can be either a scalar, an array or a box tuple. The old behaviour does not allow creating formatted standalone tuples. Old behaviour examples: ```lua box.tuple.new(1) box.tuple.new{1} box.tuple.new(1, 2, 3) box.tuple.new{1, 2, 3} -- This won't create a formatted tuple: the format option will become the -- second tuple field. box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}}) ``` New behaviour examples: ```lua box.tuple.new(1) box.tuple.new(1, {format = box.tuple.format.new{{'field'}}}) box.tuple.new{1} box.tuple.new({1}, {format = box.tuple.format.new{{'field'}}}) box.tuple.new(1, 2, 3) -- error box.tuple.new(1, 2, 3, {format = box.tuple.format.new{{'field'}}}) -- error box.tuple.new{1, 2, 3} box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}}) ``` See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#6f74f0c70005463b8438830edd1a0117.
-
Georgiy Lebedev authored
Previously, reusable tuple formats were only used for ephemeral spaces and by `net.box` (which only used 'name' definitions), so non-space tuple format definitions were ignored in tuple format comparison and hash functions, but in scope of #4693 reusable tuple formats will be interchangeable with space formats, so now we need to account for these too. Needed for #4693 NO_CHANGELOG=internal NO_DOC=internal
-
Georgiy Lebedev authored
In scope of #4693 we need to save the MsgPack encoding of the original (i.e, user-provided) format clause in the tuple format structure for serialization to Lua and IPROTO (#8147, #8633): since tuple formats for spaces are created from space definitions, we need to also save the MsgPack encoding in the latter. Since we need to pass the format clause MsgPack encoding to the runtime tuple format constructor, refactor it to perform field definition decoding in-place. We cannot use default field definition array decoding for external formats (67578d1f), so introduce a new field name decoder for this case. Needed for #4693 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
userdata is the preferred type for Lua objects wrapping C structures, but tuple formats were made cdata for no good reason: change them to userdata. Needed for #4693 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
Since we are going to make public Lua tuple format objects, introduce a new `tuple_format` submodule and move the code related to these objects there. Needed for #4693 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=refactoring
-
- Jun 30, 2023
-
-
Sergey Bronnikov authored
Test runs an external process with tarantool that writes to a log file. Then test reads that log file and searches a string with required message in it (see function check_err_msg). Test was flaky on macOS and I suspect it was happening due to a high log level - timeout was not enough to wait message in the log file. Patch decreases a log level to a default value and replaces io functions with the similar alternatives in a fio module. Using fio functions allows to not block fibers. NO_CHANGELOG=test fix NO_DOC=test fix
-
Vladimir Davydov authored
The xlog reader Lua module uses the xlog_cursor_next_row, which decodes the row header with xrow_header_decode. The latter silently ignores any unknown fields, which complicates catching bugs when garbage is written to a row header by mistake, for example, see #8783. Let's parse a row header without using xrow_header_decode in the xlog reader module, like we parse a row body, and output all unknown/invalid keys as is. To do that, we have to extend the xlog cursor API with the new method xlog_cursor_next_row_raw that returns a pointer to the position in the tx buffer where the next xrow is stored without advancing it. To avoid a memory leak in case the caller fails to parse an xrow returned by this function, we also have to move the call to xlog_tx_cursor_destroy from xlog_tx_cursor_next_row to xlog_cursor_next_tx. While we are at it, - Don't raise an error if a key type encountered in a row body is invalid (not an integer). Instead, silently ignore such keys. - Remove the useless body MsgPack validness check because we already check it after decoding the header. - Add error injection based tests to check all the corner cases. NO_DOC=bug fix
-
Vladimir Davydov authored
To avoid garbage written to xlog. Closes #8783 NO_DOC=bug fix NO_TEST=next commit
-
- Jun 29, 2023
-
-
Nikolay Shirokovskiy authored
These are misc tests that can be related to the issue. Not sure all of them do not work before the patch set for the issue. It is nice to have them and be sure everything works fine. Closes #8658 NO_DOC=minor
-
Nikolay Shirokovskiy authored
We can replace the last `xrow_update_err_double` occurrence with `xrow_update_err_no_such_field`. The last is correct one for this place as `xrow_update_op_do_field_##op_type` is called only for non terminal paths. Thus if XUPDATE_SCALAR is encountered then the field referenced does not exist. Part of #8658 NO_DOC=minor NO_CHANGELOG=minor
-