- Jan 11, 2023
-
-
Georgiy Lebedev authored
Add an optional dictionary field to MsgPack object which can be used during indexing for aliasing string keys. Needed for #7901 NO_CHANGELOG=<internal feature> NO_DOC=<internal feature>
-
Igor Munkin authored
* cmake: introduce CheckUnwindTables helper * x64/LJ_GC64: Fix type-check-only variant of SLOAD. * LJ_GC64: Fix ir_khash for non-string GCobj. * gdb: support full-range 64-bit lightuserdata Relates to #6481 Part of #7230 NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump
-
- Jan 10, 2023
-
-
Alexander Turenko authored
Use an array to convert a hex digit into a number instead of `isdigit()` and `tolower()`. Eliminate the `isxdigit()` check by reusing the same array lookup. Results on perf/lua/uri_escape_unescape.lua show 2.4x average boost (from 2.1x to 2.5x). Noise in the results: 4% in baseline, 11% in the new version (the maximal difference between min and max results). The average linear/standard deviation is within 3%. Measured on i7-10510U, acpi-cpufreq driver (no intel_pstate), no TurboBoost, userspace governor, 1.80GHz, HyperThreading. The workload is bound to a single logical core using `taskset`. NO_DOC=no behavior change NO_TEST=no behavior change NO_CHANGELOG=the function is new and not yet released
-
Anna Balaeva authored
Change runner label 'macos-11' to 'macos-11-self-hosted' and 'macos-12' to 'macos-12-self-hosted' to distinguish between self-hosted and GitHub-hosted runners. We want to use only self-hosted macOS runners because of test-run problems with python3.11 on GitHub-hosted macOS runners. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
Vladimir Davydov authored
Closes #8043 NO_DOC=bug fix
-
- Dec 29, 2022
-
-
Yaroslav Lobankov authored
The coverage workflow is a part of the default testing, so there is no sense to run this workflow when the 'full-ci' label is set. Moreover, it cancelled the run of the default testing and started the run of the full testing due to workflow `concurrency`. By default, a workflow only runs when a `pull_request` event's activity type is `opened`, `synchronize`, or `reopened`. That's why there is no sense in the following construction: pull_request: types: [ opened, reopened, synchronize ] So just removed the line related to event's activity type. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
viacheslav.kirichenko authored
Add workflow for integration testing of ddl and crud modules. Resolves tarantool/tarantool#6619 Resolves tarantool/tarantool#6620 NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci
-
- Dec 28, 2022
-
-
Mergen Imeev authored
This patch removes unused variables that were not caught by the compiler due to MAYBE_UNUSED or conversion to void. NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Mikhail Elhimov authored
Also pretty printer was introduced for struct tuple, so it can be printed with 'p' command as well. 'tt-tuple' command allows to specify some additional options (see 'help tt-tuple') Closes #7729 NO_DOC=gdb extension NO_CHANGELOG=gdb extension NO_TEST=gdb extension
-
Mikhail Elhimov authored
Supported extensions: - decimal - uuid - datetime - error - compression (only shows compression type and raw/compressed data size) - interval This functionality is similar to 'mp_str' function, but 'mp_str' can be invoked from gdb only when a process is alive, so to be able to print MsgPack during post-mortem analysis of core dump, 'mp_str' and all subsequent functions that decode and convert MsgPack into human-readable string were adapted to Python and put into the extension. In order to simplify implementation and maintenance of the gdb-extension implementation of the most of functions is kept as close to possible to their 'C'-counterparts (original 'C'-name can be found as a comment at the end of 'def' string) Part of #7729 NO_DOC=gdb extension NO_CHANGELOG=gdb extension NO_TEST=gdb extension
-
- Dec 27, 2022
-
-
Mergen Imeev authored
This patch introduces new keyword SEQSCAN and new restrictions on SELECTs. These restrictions are disabled by default. Closes #7747 @TarantoolBot document Title: SEQSCAN Now scanning SELECT will not run and will throw an error if the new SEQSCAN keyword is not used for scanned spaces. This change only affects SELECT and does not affect UPDATE and DELETE. A SELECT is recognized as a scanning SELECT if `EXPLAIN QUERY PLAN SELECT ...` indicates that the SELECT `scans` rather than `searches`. For example, if we have spaces created with these queries: ``` CREATE TABLE t(i INT PRIMARY KEY, a INT); CREATE TABLE s(i INT PRIMARY KEY, a INT); ``` Then these queries will throw an error: ``` SELECT * FROM t; SELECT * FROM t WHERE a > 1; SELECT * FROM t WHERE i + 1 = 5; SELECT * FROM t, s; SELECT * FROM t JOIN s; ``` And these will not: ``` SELECT * FROM t WHERE i > 1; SELECT * FROM SEQSCAN t; SELECT * FROM SEQSCAN t WHERE i + 1 = 5; SELECT * FROM SEQSCAN t, SEQSCAN s; SELECT * FROM SEQSCAN t JOIN SEQSCAN s; ``` Scanning can be allowed or disallowed by default. To do this, a new session setting is introduced: `sql_seq_scan`. The default value for setting is `true`, i.e. scanning is allowed. When set to `false`, the scanning SELECTs will throw a `scanning is not allowed` error.
-
Vladimir Davydov authored
This commit adds a few configuration options, function stubs, and error codes that will be used to perform extra security checks in EE: * box.cfg.auth_delay. Type: double. Default: 0. Unit: seconds. Description: If authentication of a user fails, the next authentication attempt for the same user will fail with ER_AUTH_DELAY error if called before box.cfg.auth_delay passes. Implementation: Error will be raised by security_check_auth_pre() called by authenticate() right before checking the challenged password. Authentication failures will be accounted per user in session_on_auth trigger. - box.cfg.disable_guest. Type: boolean. Default: false. Description: If set, an attempt to perform any request except 'auth', 'ping', 'id', or 'vote' over iproto without authentication or authenticated as guest will raise ER_AUTH_REQUIRED error. Implementation: Error will be raised by security_check_session() called by tx_check_msg(), which in turn is called before starting to process any message received over iproto. - box.cfg.password_lifetime_days. Type: number. Default: 0. Unit: days. Description: If > 0, an attempt to authenticate as a user that hasn't reset the password for more than box.cfg.password_lifetime_days will fail with ER_PASSWORD_EXPIRED error. Implementation: Error will be raised by security_check_auth_post() called by authenticate() right after successfully authenticating the user. Note, we can't raise the error in security_check_auth_pre() because that would enable user enumeration. All the new options are dynamic. The option values will be stored and used in C code so we'll have to define a configuration callback for them in EE: box.internal.cfg_set_security. Also note that since the new options configure authentication behavior, they should be set before the box port is opened: we'll do that in security_cfg() called by box_storage_init(). Needed for https://github.com/tarantool/tarantool-ee/issues/299 Needed for https://github.com/tarantool/tarantool-ee/issues/300 Needed for https://github.com/tarantool/tarantool-ee/issues/301 NO_DOC=ee NO_TEST=ee NO_CHANGELOG=ee
-
Vladimir Davydov authored
In contrast to all other request handlers, replication request handlers don't check the schema version. Strictly speaking, this is incorrect - if it's specified it must always be checked. It was fine, because we don't set the schema version in replication requests. However, in the future we're planning to add more checks in tx_check_msg, which will have to be performed for all requests, including replication. So let's call tx_check_msg before processing replication requests - it shouldn't hurt to check the schema version if available. Needed for https://github.com/tarantool/tarantool-ee/issues/301 NO_DOC=ee NO_TEST=ee NO_CHANGELOG=ee
-
Vladimir Davydov authored
After accepting an iproto message in tx, we call tx_check_schema to check if the schema version supplied by the user matches the actual one. If it doesn't, we bail out early with ER_WRONG_SCHEMA_VERSION. We need to add more checks that should be done before continuing with processing a message in tx. tx_check_schema looks like a perfect place for it. Let's rename it to tx_check_msg and pass an accepted message instead of the schema version to it. Needed for https://github.com/tarantool/tarantool-ee/issues/301 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Sergey Bronnikov authored
Closes #7931 @TarantoolBot document Title: Document a percent-encoding of params passed to http client The HTTP client has `params` option, where a user may provide a table of query parameters (added in #6832). Those parameters are encoded into a `?foo=bar&tweedledoo=tweedledee` string verbatim. If a name or a value of a query parameter contains `&`, `=` (or any another symbol with specific meaning in the URI query component), the query may be interpreted incorrectly by a server. Now key and values passed in a table as `params` option are percent-encoded and then encoded to a query string. This will be made automatically. Percent-encoding depends on used HTTP method: with `GET`, `HEAD` and `DELETE` parameters `uri.QUERY_PART` are used and with other HTTP method `uri.FORM_URLENCODED` is used.
-
Sergey Bronnikov authored
Commit "uri: encode table with http params to a string" (b31aec89) introduced two functions `params()` and `encode_kv()` that encodes a table with http key-value parameters to a http query string. However, keys and values could be interpreted wrong by http server if they contains reserved symbols that have special meaning for http server. Patch adds escaping for key-values before encoding params to query string. Needed for #7931 NO_CHANGELOG=internal NO_DOC=internal
-
Sergey Bronnikov authored
Patch replaces encoding and decoding functions written in Lua with functions implemented in C. Performance of Lua implementation (before the patch): ``` uri.escape 152.37 runs/sec uri.unescape 263.44 runs/sec ``` Performance of C implementation (after the patch): ``` uri.escape 4983.03 runs/sec uri.unescape 4197.19 runs/sec ``` Follows up #3682 NO_CHANGELOG=see previous commit NO_DOC=see previous commit Co-authored-by:
Alexander Turenko <alexander.turenko@tarantool.org>
-
Sergey Bronnikov authored
Added a simple benchmark for URI escape/unescape. Part of #3682 NO_DOC=documentation is not required for performance test NO_CHANGELOG=performance test NO_TEST=performance test
-
Sergey Bronnikov authored
Closes #3682 @TarantoolBot document Title: Document a new functions to encode and decode parts of URI New functions `uri.escape()` and `uri.unescape()` have been introduced. First one allows to escape symbols to a string and second one to unescape symbols to a string according to RFC 3986 [1]. Examples: ``` tarantool> uri.escape("тарантул") --- - '%D1%82%D0%B0%D1%80%D0%B0%D0%BD%D1%82%D1%83%D0%BB' ... tarantool> uri.unescape("%D1%82%D0%B0%D1%80%D0%B0%D0%BD%D1%82%D1%83%D0%BB") --- - тарантул ... ``` `uri.escape()` accepts a string that will be encoded and optionally a table with encoding options: string with unreserved symbols that will *not* be encoded and boolean option that enables/disables encoding of a space characters as '+'. By default `uri.escape()` uses a set of unreserved symbols defined in RFC 3986 ("2.3. Unreserved Characters") and encoding of space characters as '+' is disabled. Table with default encoding options is defined as `uri.RFC3986`. `uri.unescape()` accepts a string that will be decoded and optionally a table with decoding options: string with unreserved symbols (these symbols are actually unused by decoding function) and boolean option that enables/disables decoding of '+' as a space character. Table with default decoding options is defined as `uri.RFC3986`. See detailed description in RFC "http: add percent-encoding/decoding of query string in request" [2]. NO_WRAP 1. Uniform Resource Identifier (URI): Generic Syntax https://datatracker.ietf.org/doc/html/rfc3986 2. https://www.notion.so/tarantool/http-add-percent-encoding-decoding-of-query-string-in-request-76a2425a4c4744a1a72643527a4fe7f7 NO_WRAP
-
Georgiy Lebedev authored
Add translation table for `box.iproto.key` constants encoding to simplify packet assembly. Add new `box.iproto.send` method to Lua and `box_iproto_send` function to C API, which allow sending arbitrary IPROTO packets, using active IPROTO sessions. Packets are sent asynchronously using Kharon. Add `xregion_join` to the `xalloc` API. Change gh-7894 test: instead of simply comparing `box.iproto` table to the reference table, iterate over `box.iproto` and check that corresponding non-{function, thread, userdata} type values exist in the reference table. Closes #7897 @TarantoolBot document Title: Document sending arbitrary IPROTO packets feature For the API description and usage examples, see: * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#a2cc04da89d34fad8f8564c150cd9977); * tarantool/tarantool#7897.
-
Georgiy Lebedev authored
The translation table (and the future msgpack object dictionary) uses string keys coming from Lua and hence using `lua_hash{string}` as the hash function is beneficial. Needed for #7897 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring
-
Georgiy Lebedev authored
`test_key_def_dup` uses `box_key_def_dump_parts` which allocates memory on the fiber region, but the test does not clean it up: fix that by creating a region savepoint and truncating the fiber region in the end of the test. Needed for #7897 NO_CHANGELOG=<test fix> NO_DOC=<test>
-
Georgiy Lebedev authored
In some cases we need to convert a string to lower case (e.g., when normalizing an upper-case constant): add helper functions that do this in-place or by creating a copy of the original string. Needed for #7897 NO_CHANGELOG=internal NO_DOC=internal
-
- Dec 26, 2022
-
-
Nikita Zheleztsov authored
Currently replicaset state machine tracking the number of connected, loading and synced appliers may perform unnecessary decrementing of their count. On debug version this may lead to assertion failure. Here's the way it may happen: 1. Any kind of exception occurs in applier thread and leads to invoking its destructor (applier_thread_data_destroy), which is set with scoped guard; 2. Cbus call is made in order to remove the corresponding applier from the thread. According to the fact that cbus_call is synchronous, we yield, waiting for the result from the applier thread. 3. During yielding user calls reconfiguration, which invokes replicaset_update. Old appliers are pruned: for every replica trigger on changing state machine counter is deleted after which we stop fiber and wait its join. 4. If the first replica in replicaset_foreach is not the errored one and the errored fiber wakes up during yielding with fiber_join, then zero decrementing happens. Let's clear the above mentioned triggers for all replicas at the first place and only after that stop and join their applier fibers. Closes #7590 NO_DOC=bugfix
-
Gleb Kashkin authored
Before the change there was an unexpected behavior when using channel:close(), as it closed the channel entirely and discarded all unread events. This commit introduces graceful channel close option in tarantool.compat (gh-7000) that allows to select new or old behavior. With the new behavior `close()` marks channel as closed for writing. Only when all events are extracted, the channel is closed entirely. If there are no events in the channel, it is closed as usual. Document that describes new API can be found on notion (private): https://www.notion.so/fiber-channel-graceful-close-53b2788ed1f144598c4c0e1229c2eb69 Requires #7060 Requires #8007 Closes #7746 See also #7000 @TarantoolBot document Title: new compat option fiber_channel_close_mode New behavior is gracefully closing fiber channel by marking it read-only, instead of destroying. Full API description can be found on notion (private): https://www.notion.so/tarantool/Fiber-channel-graceful-close-53b2788ed1f144598c4c0e1229c2eb69
-
- Dec 25, 2022
-
-
Alexander Turenko authored
A jiggle in tarantool's initialization code can lead to trace numbers above 9. We can either: * accept trace numbers above 9 in the test * or drop traces from the initialization code in the test and assume that the new trace will have number 1. Igor Munkin suggested to stick with the second approach to avoid dependency on the initialization code. NO_DOC=test fixup, no user visible changes NO_CHANGELOG=see NO_DOC Co-authored-by:
Igor Munkin <imun@tarantool.org>
-
- Dec 23, 2022
-
-
Ilya Verbin authored
Currently all non-main threads have all the signals blocked, however according to `man pthread_sigmask': > If SIGBUS, SIGFPE, SIGILL, or SIGSEGV are generated while they > are blocked, the result is undefined, unless the signal was > generated by kill(2), sigqueue(3), or raise(3). On macOS they are actually blocked, causing the faulting instruction to loop indefinitely. While on Linux they are not blocked, however the signal handler registered by sigaction is not executed. Don't block them. Closes #8023 Closes #8083 NO_DOC=bugfix
-
Ilya Verbin authored
There are 4 fatal signals that cannot be blocked by sigmask if they are caused by the CPU exception (rather than kill, sigqueue or raise): SIGILL, SIGBUS, SIGFPE, SIGSEGV. Currently the crash module handles only SIGSEGV and SIGFPE. This patch adds handlers for SIGBUS and SIGILL. SIGBUS is usually raised by macOS on access to the unmapped memory, and SIGILL is possible, for example, while running AVX version of memcpy on a CPU without AVX support. Faulting address siginfo->si_addr is valid for all these signals, so print it unconditionally. Part of #8023 Part of #8083 NO_DOC=See next commit NO_CHANGELOG=See next commit
-
Andrey Saranchin authored
Currently, core constraints are dropped on commit. That is why it is impossible to drop constraint and drop objects it references to at the same transaction. Let's drop constraints in two steps - detach them when DDL occurs, then reattach on rollback or delete on commit. Closes #7339 NO_DOC=bugfix
-
- Dec 22, 2022
-
-
Andrey Saranchin authored
NO_TEST=no visible changes NO_CHANGELOG=stubs NO_DOC=stubs
-
Andrey Saranchin authored
The patch adds xalloc helpers for ibuf. NO_TEST=trivial NO_CHANGELOG=internal NO_DOC=internal
-
Mergen Imeev authored
This patch removes code that was used to implement the SQL check constraint as they are now replaced by BOX constraint. Also, the syntax for enabling/disabling check constraints has been removed as BOX constraints do not support this feature. Follow-up #6986 NO_DOC=Already introduced. NO_CHANGELOG=Already introduced.
-
Mergen Imeev authored
This patch removes code that was used to implement the SQL foreign key as they are now replaced by BOX foreign keys. Follow-up #6986 NO_DOC=Refactoring. NO_TEST=Refactoring NO_CHANGELOG=Refactoring.
-
Vladimir Davydov authored
- Add box.cfg.password_history_length configuration option. It will specify the max number of entries to keep in the auth_history field of the _user system space. - Add new error code ER_OLD_PASSWORD, which will be raised on an attempt to reuse an old password. - Set auth_history if box.internal.prepare_auth_history is defined. The function takes a user id. - Add auth_history argument to box.internal.check_password so that it can check if the new password matches an old one. - Add box_lua_security_init, because auth checks will be done in C. Needed for https://github.com/tarantool/tarantool-ee/issues/298 NO_DOC=ee NO_TEST=ee NO_CHANGELOG=ee
-
Vladimir Davydov authored
See the doc bot request for the description of the new fields. Note that we only store the value of the 'last_modified' field in struct user_def, because 'auth_history' will be used only in Lua code. Needed for https://github.com/tarantool/tarantool-ee/issues/298 Needed for https://github.com/tarantool/tarantool-ee/issues/299 NO_CHANGELOG=no user-visible effects in CE; will be added to EE @TarantoolBot document Title: Document auth_history and last_modified _user space fields Field name: auth_history. Field no: 6. Type: array. Description: The field stores an array of previous authentication data: when a user password is changed, the last value of the 'auth' field is appended to 'auth_history'. The length of the history is configured by the `box.cfg.password_history_length` option, which is available only in Tarantool EE, where it's used to prevent users from reusing old passwords. In Tarantool CE, the array is always empty. Field name: last_modified. Field no: 7. Type: unsigned. Description: The field stores the timestamp (seconds since Unix epoch) of the last user password update. It's never used in Tarantool CE. In Tarantool EE, it's used to disable users that haven't changed the password for more than `box.cfg.password_lifetime_days`. `box.schema.upgrade()` sets the new field values to an empty array and 0 for users that haven't updated them yet.
-
Vladimir Davydov authored
We disable system space triggers to generate a bootstrap snapshot, but this doesn't turn off space format checks. As a result, if a space format is updated during the upgrade sequence, it may turn out that data inserted earlier doesn't match the new format. We encountered this issue only once, when we added new fields to the _func system space, see commit 200a492a ("box: introduce Lua persistent functions"). Back then, we simply added the new fields to the earlier update function. However, modifying the update history looks ugly. We're planning to add new fields to the _user system space so we should find a way to avoid that. The fix is simple: we clear all system space formats before disabling system space triggers and reset them back after the bootstrap is done. To achieve that, we have to eliminate usage of any functions that access tuple fields by name. Needed for https://github.com/tarantool/tarantool-ee/issues/298 Needed for https://github.com/tarantool/tarantool-ee/issues/299 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
- Dec 21, 2022
-
-
Ilya Verbin authored
Use O_CLOEXEC flag instead. If Tarantool is forked before executing box.cfg{}, e.g. using io.popen(), the child process could start with stdin linked to /dev/urandom. This happens because wal_writer_singleton and vy_log_writer are not yet initialized, i.e. `fd' fields are 0, then atfork child handler wal_atfork() is called. It checks that xlog is opened (the check succeeded as 0 != -1) and closes its fd 0, in fact closing stdin (twice). Next, Tarantool opens the file /dev/urandom during initialization, and it receives the lowest unused file descriptor, which is 0. Then luaL_loadfile() loads stdin as a Lua chunk, in effect reading random numbers. This happens on glibc 2.28 and older, as newer versions do not invoke atfork handlers during io.popen(): https://sourceware.org/bugzilla/show_bug.cgi?id=17490 Closes #7886 NO_DOC=bugfix NO_TEST=Tested by test/app-luatest/gh_5747_crash_multiple_args_test.lua
-
Mergen Imeev authored
This patch fixes an issue where collation could change the computed type of a built-in function argument when choosing a function implementation. Closes #7992 NO_DOC=bugfix
-
Nick Volynkin authored
Workflow submodule_update now brings latest commits in stable branches further to tarantool/sdk, where integration tests can run with other enterprise products and modules. In tarantool/tarantool-ee a pull request is still opened, so the maintainers can merge it at any time. In tarantool/sdk the test branches are named in the following pattern: `TarantoolBot/test-ce-master` `TarantoolBot/test-ce-2.10` and so on. This pattern can be easily distinguished by workflows, which run integration tests and alert developers on failures. It is also consistent with branches with the same purpose of integration testing, that are created by the sdk_test workflow: `TarantoolBot/test-ce-{feature_branch_name}` NO_DOC=CI NO_TEST=CI NO_CHANGELOG=CI Resolves tarantool/sdk#355
-
Nick Volynkin authored
The submodule_update workflow only works with git repositories and doesn't require computational power. It's mostly limited by the network throughput. So it makes sense to run this workflow on lightweight runners, labeled with `flavor-1-2` (that is, 1 CPU and 2Gb RAM). New runners have granular labels: `self-hosted`, OS, arch and flavor. NO_DOC=CI NO_TEST=CI NO_CHANGELOG=CI
-