- Oct 26, 2023
-
-
Georgy Moshkin authored
Closes #9237 Add exports for fiber_set_name_n, fiber_name, fiber_id, fiber_csw & fiber_find. Also make fiber_set_joinable, fiber_set_ctx & fiber_get_ctx interpret NULL as the current fiber. @TarantoolBot document Title: add basic fiber api to ffi exports. 5 basic functions can now be used via ffi api, which were previously only accessible via lua api: fiber_set_name_n, fiber_name, fiber_id, fiber_csw & fiber_find. fiber_set_joinable now interprets NULL as current fiber.
-
- Oct 10, 2023
-
-
Pavel Balaev authored
This patch fixes issue: $ tarantoolctl rocks --version 1>/dev/null Warning: failed to load command module luarocks.cmd.help NO_DOC=bugfix NO_CHANGELOG=not released yet
-
- Sep 28, 2023
-
-
Pavel Balaev authored
luarocks version updated to version 3.9.2 Closes #6597 NO_DOC=The engine has been updated, the functionality has not changed NO_TEST=see NO_DOC
-
- Jul 12, 2023
-
-
Igor Munkin authored
There was a little mess in exports file regarding LuaJIT symbols to be visible from Tarantool binary, so some functions (e.g. <lua_tonumberx> and <lua_tointegerx>) were hidden unintentionally. As a result of the patch almost all public LuaJIT symbols are exported from Tarantool binary; there are still several functions reasonably hidden (you can find the rationale around the corresponding cases in the test file). Closes #3680 @TarantoolBot document Title: clean up LuaJIT exported functions The list of the LuaJIT-related functions being exported (i.e. public) from Tarantool should be updated. The actual list of the exported symbols can be found within changeset (either extra/exports file or related Lua test chunk).
-
- Jun 28, 2023
-
-
Vladimir Davydov authored
There's base64_bufsize for calculating the buffer size needed for base64_encode but there's no such function for base64_decode - one's supposed to pass a buffer > 3/4 of the input size. This is confusing. Let's rename base64_bufsize to base64_encode_bufsize and introduce base64_decode_bufsize. While we're at it, move base64_encode_bufsize body from base64.h to base64.c because otherwise the linker fails if this function, which is currently declared as extern inline, is used in lyaml.cc. Needed for #1629 NO_DOC=refactoring NO_CHANGELOG=refactoring
-
- Jun 23, 2023
-
-
Georgiy Lebedev authored
`cord_slab_cache` was not designed to offer a backward compatible API, and leaving it exposed inevitably leads to errors like those highlighted in #7124 and tarantool/memcached#96: hence, remove it from the public API export. Closes #7124 @TarantoolBot document Title: `cord_slab_cache` was removed from public API export The `cord_slab_cache` needs to be removed from the C API reference of the fiber module.
-
- May 26, 2023
-
-
Vladimir Davydov authored
** Implementation details ** We disable DDL by patching the existing on_replace_dd_system_space trigger callback installed for each system space so that now it raises an error in case the current schema version is less than the most recent one known to this build. Since to perform a schema upgrade we need to execute DDL, we suppress the error for the fiber that is currently running a schema upgrade. To achieve that, the upgrade script calls box_schema_upgrade_begin and box_schema_upgrade_end before starting and after completing a schema upgrade. The functions keep track of the fiber that is currently running a schema upgrade so that we can allow all DDL operations for it. We also allow DDL during recovery so that we can replay DDL statements written to the WAL. Since there may be a bug in the `box.schema.upgrade` implementation, we export `box.internal.run_schema_upgrade`, which runs the given function as a schema upgrade script (allowing DDL). The user may use this function to recover after a schema upgrade failure. ** Note about the tests ** A test server instance started by luatest grants permissions to the guest user so that luatest can execute commands on it. It means that if a test uses a generated snap file committed to the repository for a test server instance, it will fail because granting permissions is a DDL operation. To prevent this, we have to regenerate snap files so that they contain all required permissions. This works because a test server instance grants permissions with the `if_not_exists` flag. The problem is that it isn't easy to regenerate the snap files for the following tests because there's no generator script: - `test/box-luatest/gh_6794_recover_nonmatching_xlogs_test.lua` - `test/box-luatest/gh_7974_force_recovery_bugs_test.lua` So we temporarily disable these tests and file tickets to fix them. Other notes: - We drop `test/box-luatest/upgrade/2.9.1` and make the test using it use `test/box-luatest/upgrade/2.10.0` instead. We do this because 2.9.1 was never released and the earliest Tarantool version using the 2.9.1 schema version is 2.10.0. This shouldn't affect the test anyhow. - We drop the part of the `user_auth_history_last_modified_upgrade` test that checks that creating users/roles with an old schema works fine because this is forbidden now. - We wrap the code that creates a space with an old schema in the downgrade test in `box.internal.run_schema_upgrade`. Even though it's unsupported now, we still need to check that space creation works after a downgrade. Closes #7149 @TarantoolBot document Title: Document that DDL is disabled with an old system schema Executing DDL operations with an old (not upgraded) system schema is dangerous and might result in unexpected breakages. So we decided to explicitly forbid all DDL operations with an old system schema until `box.schema.upgrade()` is called. Note, one can still call `box.schema` functions with an old schema provided they do nothing, for example, if an object is created with the `if_not_exists` flag and the object with same id already exists: ```lua box.schema.create_space('test', {if_not_exists = true}) ``` Otherwise an attempt to create a space with an old schema will raise an error like shown below: ```yaml tarantool> box.schema.space.create('test') --- - error: Your schema version is 1.6.8 while Tarantool 3.0.0-entrypoint-262-g3eaba1cef686 requires a more recent schema version. Please, consider using box.schema.upgrade(). ... ```
-
- Apr 12, 2023
-
-
Mergen Imeev authored
This patch introduces the SHOW CREATE TABLE statement. This statement can be used to obtain a description of a space in the form of a corresponding CREATE TABLE and CREATE INDEX statements. Closes #8098 @TarantoolBot document Title: SHOW CREATE TABLE statement Statement can be used to obtain a description of a space in the form of a corresponding `CREATE TABLE` and `CREATE INDEX` statements. Result will be in form of set of statements and set of found errors. If errors were not detected, set of the statements should be enough to completely serialize space definition. There is two types of `SHOW CREATE TABLE` statement: 1) Get a description of a single space: ``` SHOW CREATE TABLE table_name; ``` This statement can be used to obtain a description of a space in the form of the corresponding `CREATE TABLE` and `CREATE INDEX` statements. The result will be in the form of a set of statements and a set of found errors. If no errors are found, the set of statements should be sufficient to fully serialize the space definition. Otherwise, it will certainly not be a complete space definition, and a `CREATE TABLE` statement is generally not guaranteed to be syntactically correct. 2) Get descriptions of all available non-system spaces: ``` SHOW CREATE TABLE; ``` This statement returns descriptions for each available non-system space in the form described above.
-
- Mar 30, 2023
-
-
Nikolay Shirokovskiy authored
Remove parts that used before flightrec reader API is added. - fix test_max_record_size case of prbuf unit test to use reader from file - drop prbuf reader from buffer in memory - drop prbuf Lua FFI interface Follow up: https://github.com/tarantool/tarantool-ee/issues/319 NO_DOC=internal NO_CHANGELOG=internal
-
- Mar 07, 2023
-
-
Georgiy Lebedev authored
In some cases unsafe extension decoding was done without bound and type checks: add necessary checks. Closes tarantool/security#73 NO_DOC=bugfix
-
- Feb 02, 2023
-
-
Georgiy Lebedev authored
The `tarantool_version` symbol identifies the binary as Tarantool (see also https://github.com/tarantool/tarantool/blob/f991f7c0be73558f0710f0af871d07e8bd506efe/tools/tarabrt.sh#L179-L180 ). It is not exported and thus can be optimized away by LTO — add it to the exports list. Closes #8129 Acked-by:
Aleksandr Lyapunov <alyapunov@tarantool.org> NO_CHANGELOG=<internal change> NO_DOC=<internal change> NO_TEST=<no convenient way to test devtools>
-
- Jan 30, 2023
-
-
Andrey Saranchin authored
Recently, ffi select was broken on M1 in commit ec1a71ff ("box: introduce pagination to memtx_tree and tuple position methods"). It turned out that ffi on M1 poorly supports a big quantity of arguments. Fortunately, there is a workaround - we can pass only 64-bit integer arguments beyond the 8th argument. Let's do it. Closes #7946 NO_TEST=reflected in existing tests NO_CHANGELOG=bugfix for unreleased feature NO_DOC=bugfix
-
- Jan 13, 2023
-
-
Aleksandr Lyapunov authored
If the first argument of box.atomic is a non-callable table then consider it as options table for box.begin{}. For test and debug purposes introduce internal getter of current transaction isolation level as box.internal.txn_isolation(). Closes #7202 @TarantoolBot document Title: Options in box.atomic Now it's allowed to pass transaction options in the first argument of box.atomic(..) call. The options must be a table, exactly as in box.begin(..). If options are passed as the first arguments, the second and the rest arguments are expected to be a functions and its arguments, like in usual box.atomic.
-
- Jan 12, 2023
-
-
Alexander Turenko authored
Fixed pthread-related CMake checks. The checks code is built with `-pedantic-errors` and it leads to errors of the following kind on clang 15: ``` <...>/CMakeFiles/CMakeScratch/TryCompile-78KaOK/src.c:4:17: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes] int main() { pthread_setname_np(pthread_self(), ""); } ^ void ``` Fixed a warning in the SQL code (it's an error in Debug build): ``` <...>/src/box/sql/vdbeaux.c:170:13: error: variable 'n' set but not used [-Werror,-Wunused-but-set-variable] static int n = 0; ``` Fixed several warnings from lemon.c of the following kind: ``` <...>/extra/lemon.c:173:6: warning: a function declaration without a prototype is deprecated in all versions of C and is treated as a zero-parameter prototype in C2x, conflicting with a subsequent definition [-Wdeprecated-non-prototype] void FindRulePrecedences(); ^ <...>/extra/lemon.c:766:6: note: conflicting prototype is here void FindRulePrecedences(struct lemon *xp) ``` See also https://github.com/tarantool/small/issues/57 Fixes #8110 NO_DOC=build fix NO_TEST=build fix
-
- Jan 11, 2023
-
-
Georgiy Lebedev authored
Add `IPROTO_UNKNOWN` command code for overriding the unknown request handler. Change request type variable types related to IPROTO to `uint32_t`. Add request handler hash table to transaction thread and request handler set to IPROTO threads for storing overridden request handlers: TX thread notifies IPROTO threads about overridden request handlers using IPROTO configuration message. If a given request handler is overridden, the IPROTO thread does not preprocess it and sends the package immediately over a dedicated route. If later it is necessary to fallback to the system handler, the message decoding and dispatching is done in the TX thread. Add new `box.iproto.override` method to Lua and `box_iproto_override` to C API, which allow setting IPROTO request handler callbacks. Closes #7901 @TarantoolBot document Title: Document overriding IPROTO request handlers feature For the API description and usage examples, see: * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#0f84694523214c0e9bf2f3d75cccace4); * tarantool/tarantool#7901.
-
- Dec 27, 2022
-
-
Mergen Imeev authored
This patch introduces new keyword SEQSCAN and new restrictions on SELECTs. These restrictions are disabled by default. Closes #7747 @TarantoolBot document Title: SEQSCAN Now scanning SELECT will not run and will throw an error if the new SEQSCAN keyword is not used for scanned spaces. This change only affects SELECT and does not affect UPDATE and DELETE. A SELECT is recognized as a scanning SELECT if `EXPLAIN QUERY PLAN SELECT ...` indicates that the SELECT `scans` rather than `searches`. For example, if we have spaces created with these queries: ``` CREATE TABLE t(i INT PRIMARY KEY, a INT); CREATE TABLE s(i INT PRIMARY KEY, a INT); ``` Then these queries will throw an error: ``` SELECT * FROM t; SELECT * FROM t WHERE a > 1; SELECT * FROM t WHERE i + 1 = 5; SELECT * FROM t, s; SELECT * FROM t JOIN s; ``` And these will not: ``` SELECT * FROM t WHERE i > 1; SELECT * FROM SEQSCAN t; SELECT * FROM SEQSCAN t WHERE i + 1 = 5; SELECT * FROM SEQSCAN t, SEQSCAN s; SELECT * FROM SEQSCAN t JOIN SEQSCAN s; ``` Scanning can be allowed or disallowed by default. To do this, a new session setting is introduced: `sql_seq_scan`. The default value for setting is `true`, i.e. scanning is allowed. When set to `false`, the scanning SELECTs will throw a `scanning is not allowed` error.
-
Sergey Bronnikov authored
Patch replaces encoding and decoding functions written in Lua with functions implemented in C. Performance of Lua implementation (before the patch): ``` uri.escape 152.37 runs/sec uri.unescape 263.44 runs/sec ``` Performance of C implementation (after the patch): ``` uri.escape 4983.03 runs/sec uri.unescape 4197.19 runs/sec ``` Follows up #3682 NO_CHANGELOG=see previous commit NO_DOC=see previous commit Co-authored-by:
Alexander Turenko <alexander.turenko@tarantool.org>
-
Georgiy Lebedev authored
Add translation table for `box.iproto.key` constants encoding to simplify packet assembly. Add new `box.iproto.send` method to Lua and `box_iproto_send` function to C API, which allow sending arbitrary IPROTO packets, using active IPROTO sessions. Packets are sent asynchronously using Kharon. Add `xregion_join` to the `xalloc` API. Change gh-7894 test: instead of simply comparing `box.iproto` table to the reference table, iterate over `box.iproto` and check that corresponding non-{function, thread, userdata} type values exist in the reference table. Closes #7897 @TarantoolBot document Title: Document sending arbitrary IPROTO packets feature For the API description and usage examples, see: * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#a2cc04da89d34fad8f8564c150cd9977); * tarantool/tarantool#7897.
-
- Dec 20, 2022
-
-
Mergen Imeev authored
This patch refactors the SQL memory allocation system. There are three main changes: 1) now, when allocating memory, no additional 8 bytes are allocated to remember the size of the allocated memory, so instead of sql_malloc()/sqlRealloc()/sql_free(), the malloc()/realloc()/free() functions are used; 2) the malloc()/realloc() functions were used through the xmalloc()/xrealloc() macros, so checks for memory allocation errors were removed; 3) there is no need for an explicit "sql *db" argument for most of the functions, so it has been omitted. Part of #1544 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Mergen Imeev authored
This patch fixes possible memleak. NO_DOC=Possible bug NO_TEST=No proper way to test NO_CHANGELOG=Possible bug
-
- Dec 14, 2022
-
-
Ilya Verbin authored
Before commit 24323448 ("log: add log.new() function that creates a new logger"), log_write_flightrec() was called from log_vsay() regardless of the log level. After, the log level is checked in Lua, so log_vsay() may not be called. This patch restores the original behaviour by moving log_write_flightrec() calls to say_default() and say(). Part of https://github.com/tarantool/tarantool-ee/issues/320 NO_DOC=bugfix NO_CHANGELOG=unreleased NO_TEST=will be enabled in EE, because there are no flightrec in CE
-
- Dec 12, 2022
-
-
Vladimir Davydov authored
Since commit f6ea7180 ("Try to load several variants of libssl.") the digest module uses an internal version of SHA1. Back then, we didn't link the OpenSSL library. Instead, we tried to load it dynamically. Since on some distributions the library could be missing, it was decided to implement an internal version of SHA1, see #405. However, since commit 59a55740 ("Link against libssl and libcrypto. Issue #1382") we link the OpenSSL library unconditionally so there's no need in having an internal implementation of SHA1. Let's drop it and switch the digest module to the version of SHA1 implemented by the crypto module using OpenSSL. Part of #7987 NO_DOC=code cleanup NO_TEST=code cleanup NO_CHANGELOG=code cleanup
-
- Dec 09, 2022
-
-
Ilya Verbin authored
It allows to create a new instance of a log module, with a custom name: local my_log = require('log').new('my_module') The name is added to the log message after fiber name: YYYY-MM-DD hh:mm:ss.ms [PID]: CORD/FID/FIBERNAME/MODULENAME LEVEL> MSG Part of #3211 NO_DOC=See next commit NO_CHANGELOG=See next commit Co-authored-by:
AnastasMIPT <beliaev.ab@tarantool.org>
-
Vladimir Davydov authored
This commit introduces an abstraction for the authentication code so that one can easily add new methods. To add a new method, one just needs to define a set of authentication callbacks in a struct auth_method and register it with auth_method_register. The IPROTO_AUTH and _user.auth formats were initially designed with extensibility in mind: both take the authentication method name (currently, only 'chap-sha1' is supported) so no changes to the schema are required. Note that although 'chap-sha1' is now implemented in its own file src/box/auth_chap_sha1.c, we don't merge src/scramble.c into it. This will be done later, in the scope of #7987. Since we call authentication plug-ins "methods" (not "mechanisms"), let's rename BOX_USER_FIELD_AUTH_MECH_LIST to BOX_USER_FIELD_AUTH while we are at it. Anyway, the corresponding field of the _user system space is called 'auth' (not 'auth_mech_list'). Closes #7986 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
- Dec 07, 2022
-
-
Georgiy Lebedev authored
In order to send IPROTO packets using `box_iproto_send` (#7897) we need to have some session identifier source in the C API: for now, export an analog of `box.session.id`. Closes #7895 @TarantoolBot document Title: Document export of current session's identifier to C API For the API description and usage examples, see: * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#0900ff9cb6b148378ce0b185d3f628b9); * tarantool/tarantool#7895.
-
- Nov 29, 2022
-
-
Mergen Imeev authored
This patch removes three rules in the parser. NO_CHANGELOG=will be added later @TarantoolBot document Title: Changes in parsing rules. Three rules related to foreign keys have been removed. The first is the "reference trigger action" rule, which determines the behavior when a referenced tuple is deleted. This rule has been dropped because the new foreign keys only support RESTRICT. The second is "constraint check time" rule, which determines when a foreign key constraint should be validated. This rule has been dropped because the new foreign keys only support INITIALLY IMMEDIATELY. The third is "match type" rule, which determines how inserted values are validated. This rule has been dropped because the new foreign keys only support FULL.
-
- Nov 28, 2022
-
-
Andrey Saranchin authored
The patch introduces pagination to pairs: one can create iterator which will start after specific tuple or position, described by option after. Tuple (both cdata and array) and position (obtained by index:tuple_pos or fetch_pos option of select) can be passed as option after. NO_CHANGELOG=next commit NO_DOC=next commit
-
- Nov 18, 2022
-
-
Ilya Grishnov authored
Added a description for the --format flag of the tarantoolctl cat command. Fixes #7099 NO_DOC=bugfix NO_TEST=help message fix
-
- Nov 01, 2022
-
-
Nikolay Shirokovskiy authored
See [1] for some details on why the code for log configuration needs some care. In short log config validity checks are spread thru many places, on some code paths we use one checks and on other code paths other checks. And we make repetitive validity checks many times in runtime on single configure call. We can also reuse code for setting default values, checking options type and resetting values to default. - As a side effect of refactoring one can now reset values to default thru `log.cfg()` so now `log.cfg()` is on par with `box.cfg` in this respect. - This patch also drops conversion `log_level` from string to number. Before (shorten): tarantool> box.cfg{log_level='warn'} tarantool> box.cfg.log_level - info tarantool> log.cfg.level - 5 Also: tarantool> log.cfg{level='info'} tarantool> log.cfg.level - 5 tarantool> box.cfg{} tarantool> box.cfg.log_level - 5 After patch if `log_level`/`level` is given as string than it is saved and returned as string too. I guess it should not affect users but looks more handy. - Also fixed issue with inconsistent setting `log_nonblock` thru `box.cfg()` and `log.cfg()`. In former case `nil` means setting default depending on logger type. In the latter case `nil` meant setting `nonblock` to `false`. - Also patch fixes #7447. Closes #7447. [1] PR for this refactoring https://github.com/tarantool/tarantool/pull/7454 NO_DOC=refactoring/tiny API improvemnent
-
- Oct 18, 2022
-
-
Ilya Verbin authored
By default a user might not have privileges to access the _schema space, that will cause an error during schema_needs_upgrade(), which calls get_version(). Fix this by using C variable dd_version_id, which is updated in the _schema.version replace trigger. There's a special case for upgrade() during bootstrap() - triggers are disabled during bootstrap, that's why dd_version_id is not being updated. Handle this by passing _initial_version=1.7.5 to the upgrade function. Part of #7149 NO_DOC=internal NO_CHANGELOG=internal
-
- Sep 29, 2022
-
-
Andrey Saranchin authored
The patch introduces C and Lua methods for extracting position of tuple in index. Multikey and functional indexes are not supported. Also this patch extends method index_create_iterator_after and adds implementation of iterator_position for memtx tree. Options after and fetch_pos are added to Lua select as well (both FFI and LuaC). All the types of memtx tree indexes are supported (multikey, functional). NO_CHANGELOG=see later commits NO_DOC=see later commits Closes #7633 Closes #7636
-
- Sep 13, 2022
-
-
Georgy Moshkin authored
Before this change there was no way to create a fiber that accepts parameters without yielding from the current fiber using the c api. You could pass the function arguments when calling fiber_start, but that forces you to yield, which is not acceptable in some scenarios (e.g. within a transaction). This commit introduces 2 new functions to the api: fiber_set_ctx for setting an pointer to a context of the given fiber and fiber_get_ctx for accessing that context. Closes https://github.com/tarantool/tarantool/issues/7669 @TarantoolBot document Title: fiber: add fiber_set_ctx & fiber_get_ctx functions Add 2 api functions: `fiber_set_ctx` & `fiber_get_ctx` which can be used for passing data to a fiber. Previously this could be done via the `fiber_start` function, except that this would force the current fiber to yield, which is not acceptable in some scenarios (e.g. during a transaction). Now you can create a fiber with `fiber_new`, set it's contents with `fiber_set_ctx`, make it ready for execution with `fiber_wakeup` and keep executing the current fiber.
-
- Sep 12, 2022
-
-
Vladimir Davydov authored
strerror() is MT-Unsafe, because it uses a static buffer under the hood. We should use strerror_r() instead, which takes a user-provided buffer. The problem is there are two implementations of strerror_r(): XSI and GNU. The first one returns an error code and always writes the message to the beginning of the buffer while the second one returns a pointer to a location within the buffer where the message starts. Let's introduce a macro HAVE_STRERROR_R_GNU set if the GNU version is available and define tt_strerror() which writes the message to the static buffer, like tt_cstr() or tt_sprintf(). Note, we have to export tt_strerror(), because it is used by Lua via FFI. We also need to make it available in the module API header, because the say_syserror() macro uses strerror() directly. In order to avoid adding tt_strerror() to the module API, we introduce an internal helper function _say_strerror(), which calls tt_strerror(). NO_DOC=bug fix NO_TEST=code is covered by existing tests
-
- Aug 26, 2022
-
-
Mergen Imeev authored
This commit introduces a new parse rule for compiling an unresolved single expression. This simplifies the current implementation of sql_expr_compile() and is useful for generating SQL expressions that can be used in the core check constraint. This rule is for internal use only. Part of #6986 NO_DOC=will be added later NO_TEST=refactoring NO_CHANGELOG=will be added later
-
- Aug 05, 2022
-
-
Alexander Turenko authored
The Rust module (see the issue) needs a getter and a setter for decimal values on the Lua stack. Let's make them part of the module API. Part of #7228 @TarantoolBot document Title: Lua/C functions for decimals in the module API The following functions are added into the module API: ```c /** * Allocate a new decimal on the Lua stack and return * a pointer to it. */ API_EXPORT box_decimal_t * luaT_newdecimal(struct lua_State *L); /** * Allocate a new decimal on the Lua stack with copy of given * decimal and return a pointer to it. */ API_EXPORT box_decimal_t * luaT_pushdecimal(struct lua_State *L, const box_decimal_t *dec); /** * Check whether a value on the Lua stack is a decimal. * * Returns a pointer to the decimal on a successful check, * NULL otherwise. */ API_EXPORT box_decimal_t * luaT_isdecimal(struct lua_State *L, int index); ```
-
- Aug 04, 2022
-
-
Alexander Turenko authored
The main decision made in this patch is how large the public `box_decimal_t` type should be. Let's look on some calculations. We're interested in the following values. * How much decimal digits is stored? * Size of an internal decimal type (`sizeof(decimal_t)`). * Size of a buffer to store a string representation of any valid `decimat_t` value. * Largest signed integer type fully represented in decimal_t (number of bits). * Largest unsigned integer type fully represented in decimal_t (number of bits). Now `decimal_t` is defined to store 38 decimal digits. It means the following values: | digits | sizeof | string | int???_t | uint???_t | | ------ | ------ | ------ | -------- | --------- | | 38 | 36 | 52 | 126 | 127 | In fact, decNumber (the library we currently use under the hood) allows to vary the 'decimal digits per unit' parameter, which is 3 by default, so we can choose density of the representation. For example, for given 38 digits the sizeof is 36 by default, but it may vary from 28 to 47 bytes: | digits | sizeof | string | int???_t | uint???_t | | ------ | ---------- | ------ | -------- | --------- | | 38 | 36 (28-47) | 52 | 126 | 127 | If we'll want to store `int128_t` and `uint128_t` ranges, we'll need 39 digits: | digits | sizeof | string | int???_t | uint???_t | | ------ | ---------- | ------ | -------- | --------- | | 39 | 36 (29-48) | 53 | 130 | 129 | If we'll want to store `int256_t` and `uint256_t` ranges: | digits | sizeof | string | int???_t | uint???_t | | ------ | ---------- | ------ | -------- | --------- | | 78 | 62 (48-87) | 92 | 260 | 259 | If we'll want to store `int512_t` and `uint512_t` ranges: | digits | sizeof | string | int???_t | uint???_t | | ------ | ------------ | ------ | -------- | --------- | | 155 | 114 (84-164) | 169 | 515 | 514 | The decision here is what we consdider as possible and what as unlikely. The patch freeze the maximum amount of bytes in `decimal_t` as 64. So we'll able to store 256 bit integers and will NOT able to store 512 bit integers in a future (without the ABI breakage at least). The script, which helps to calculate those tables, is at end of the commit message. Next, how else `box_decimal_*()` library is different from the internal `decimal_*()`? * Added a structure that may hold any decimal value from any current or future tarantool version. * Added `box_decimal_copy()`. * Left `strtodec()` out of scope -- we can add it later. * Left `decimal_str()` out of scope -- it looks dangerous without at least a good explanation when data in the static buffer are invalidated. There is `box_decimal_to_string()` that writes to an explicitly provided buffer. * Added `box_decimal_mp_*()` for encoding to/decoding from msgpack. Unlike `mp_decimal.h` functions, here we always have `box_decimal_t` as the first parameter. * Left `decimal_pack()` out of scope, because a user unlikely wants to serialize a decimal value piece-by-piece. * Exposed `decimal_unpack()` as `box_decimal_mp_decode_data()` to keep a consistent terminogoly around msgpack encoding/decoding. * More detailed API description, grouping by functionality. The script, which helps to calculate sizes around `decimal_t`: ```lua -- See notes in decNumber.h. -- DECOPUN: DECimal Digits Per UNit local function unit_size(DECOPUN) assert(DECOPUN > 0 and DECOPUN < 10) if DECOPUN <= 2 then return 1 elseif DECOPUN <= 4 then return 2 end return 4 end function sizeof_decimal_t(digits, DECOPUN) -- int32_t digits; -- int32_t exponent; -- uint8_t bits; -- <..padding..> -- <..units..> local us = unit_size(DECOPUN) local padding = us - 1 local unit_count = math.ceil(digits / DECOPUN) return 4 + 4 + 1 + padding + us * unit_count end function string_buffer(digits) -- -9.{9...}E+999999999# (# is '\0') -- ^ ^ ^^^^^^^^^^^^ return digits + 14 end function binary_signed(digits) local x = 1 while math.log10(2 ^ (x - 1)) < digits do x = x + 1 end return x - 1 end function binary_unsigned(digits) local x = 1 while math.log10(2 ^ x) < digits do x = x + 1 end return x - 1 end function digits_for_binary_signed(x) return math.ceil(math.log10(2 ^ (x - 1))) end function digits_for_binary_unsigned(x) return math.ceil(math.log10(2 ^ x)) end function summary(digits) print('digits', digits) local sizeof_min = math.huge local sizeof_max = 0 local DECOPUN_sizeof_min local DECOPUN_sizeof_max for DECOPUN = 1, 9 do local sizeof = sizeof_decimal_t(digits, DECOPUN) print('sizeof', sizeof, 'DECOPUN', DECOPUN) if sizeof < sizeof_min then sizeof_min = sizeof DECOPUN_sizeof_min = DECOPUN end if sizeof > sizeof_max then sizeof_max = sizeof DECOPUN_sizeof_max = DECOPUN end end print('sizeof min', sizeof_min, 'DECOPUN', DECOPUN_sizeof_min) print('sizeof max', sizeof_max, 'DECOPUN', DECOPUN_sizeof_max) print('string', string_buffer(digits)) print('int???_t', binary_signed(digits)) print('uint???_t', binary_unsigned(digits)) end ``` Part of #7228 @TarantoolBot document Title: Module API for decimals See the declarations in `src/box/decimal.h` in tarantool sources.
-
- Aug 01, 2022
-
-
Alexander Turenko authored
The Rust module [1] leans on several internal symbols. They were open in Tarantool 2.8 (see #2971 and #5932), but never were in the public API. Tarantool 2.10.0 hides the symbols and we need a way to get them back to use in the module. We have the following options: 1. Design and expose a module API for fiber channels. 2. Export the symbols with a prefix like `tnt_internal_` (to don't spoil the global namespace). 3. Provide a `dlsym()` alike function to get an address of an internal symbol for users who knows what they're doing. I think that the third way offers the best compromise between amount of effort, quality of the result and opportunities to extend. In this commit I hardcoded the list of functions to make the change as safe as possible. Later I'll return here to autogenerate the list. Exported the following function from the tarantool executable: ```c void * tnt_internal_symbol(const char *name); ``` I don't add it into the module API headers, because the function is to perform a dark magic and we don't suggest it for users. While I'm here, added `static` to a couple of fiber channel functions, which are only used within the compilation unit. [1]: https://github.com/picodata/tarantool-module Part of #7228 Related to #6372 NO_DOC=don't advertize the dangerous API NO_CHANGELOG=don't advertize the dangerous API
-
- Jul 26, 2022
-
-
Alexander Turenko authored
Added a function (see the API in the documentation request below), which reflects the `tuple[json_path]` Lua API (see #1285). Part of #7228 @TarantoolBot document Title: tuple: access a field using JSON path via module API The following function is added into the module API: ```c /** * Return a raw tuple field in the MsgPack format pointed by * a JSON path. * * The JSON path includes the outmost field. For example, "c" in * ["a", ["b", "c"], "d"] can be accessed using "[2][2]" path (if * index_base is 1, as in Lua). If index_base is set to 0, the * same field will be pointed by the "[1][1]" path. * * The first JSON path token may be a field name if the tuple * has associated format with named fields. A field of a nested * map can be accessed in the same way: "foo.bar" or ".foo.bar". * * The return value is valid until the tuple is destroyed, see * box_tuple_ref(). * * Return NULL if the field does not exist or if the JSON path is * malformed or invalid. Multikey JSON path token [*] is treated * as invalid in this context. * * \param tuple a tuple * \param path a JSON path * \param path_len a length of @a path * \param index_base 0 if array element indexes in @a path are * zero-based (like in C) or 1 if they're one-based (like * in Lua) * \retval a pointer to a field data if the field exists or NULL */ API_EXPORT const char * box_tuple_field_by_path(box_tuple_t *tuple, const char *path, uint32_t path_len, int index_base); ```
-
- Jul 22, 2022
-
-
Yaroslav Lobankov authored
The packpack-dont-decline-custom-version.patch file was introduced in 9707a7bd (github-ci: support the new version format) and tried to resolve the issue that actually did not exist. PackPack supports accepting a custom version (pay attention to `?=`) [1] and the patch is not needed. [1] https://github.com/packpack/packpack/blob/194f898750cbbee6dd35255fe8aa7ec878eb3836/pack/config.mk#L45 NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
- Jun 22, 2022
-
-
Mergen Imeev authored
This macro does nothing, so it is dropped. NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-