- Apr 25, 2022
-
-
Mons Anderson authored
Changed return value type in Lua for the following functions from `uint64_t` to `int64_t`: * clock.time64 * clock.realtime64 * clock.monotonic64 * clock.process64 * clock.thread64 * fiber.time64 * fiber.clock64 Changed return value type in Module API for the following functions from `uint64_t` to `int64_t`: * fiber_time64 * fiber_clock64 Also enhanced clock.test.lua tap test with more testcases Due to unsigned values returned from time64 family function time calculations may lead to undesired value overflows. Having two 64-bit timestamps I cannot just subtract them without prior comparison. It is used often, for example when calculating time to a deadline (in queues or for expiration). Let's measure time, that pass between two sequential clock.time calls: ```lua print(clock.time() - clock.time()) -- -9.5367431640625e-07 ``` It's ok, with double type time return, but what about 64-bit integers? ``` print(clock.time64() - clock.time64()) -- 18446744073709550616 ``` It returns weird integer overflow instead of expected `-1000` This is counterintuitive and gets conflicted with every time functions from system headers (which are always signed). If we consider some hypothetical code, that waits some time for a deadline, then instead of exiting loop, this code will go to "infinite" (actually 585 years) wait in the case when the deadline was passed. ```lua do local ts = clock.time64() + 1e9 -- some_timestamp_in_future, ex: now+1s -- ... local remaining = clock.time64() - ts -- ... if remaining > 0 then fiber.sleep( tonumber(remaining/1e9) ) end print "1 second passed" end ``` This is hard to notice (in real tests deadlines are always in the future), but in a real environment and under high load such cases occur. 1. Since Lua functions will return ffi.typeof `ctype<int64_t>` instead of old `ctype<uint64_t>` any code, that uses _strict_ ffi type check will fail. But I hardly believe such code ever exists. 2. Low-level Module API also changed, and code, which compiles with `-Werror` may also fail. This could be an issue and in such a case we may remove the update of module API change. Still, I consider this change reasonable, since all C-level API uses signed time values. @TarantoolBot document Title: Return type of time64 functions is changed These C functions in the public API used to return `uint64_t`, now they return `int64_t`: `clock_realtime64`, `clock_monotonic64`, `clock_process64`, `clock_thread64`, `fiber_time64()`, `fiber_clock64()`. These Lua functions used to return cdata of type `uint64_t`, now they return cdata of type `int64_t`: `fiber.time64()`, `fiber.clock64()`.
-
Valeria Khokha authored
Fix wording, punctuation, and formatting. Fix dashes, trailing whitespaces, indentation, and file names. Needed for: #6430 NO_DOC=changelog NO_TEST=changelog Co-authored-by:
Valeria Khokha <valeria.khokha@gmail.com> Co-authored-by:
Kseniia Antonova <xuniq.is.here@gmail.com>
-
Maxim Kokryashkin authored
LuaJIT submodule is bumped to introduce the following changes: * GC64: disable sysprof support * build: make -DLUAJIT_DISABLE_SYSPROF work * test: disable sysprof C API tests with backtrace * tools: introduce parsers for sysprof * sysprof: introduce Lua API * memprof: add profile common section * core: introduce lua and platform profiler * memprof: move symtab to a separate module * core: separate the profiling timer from lj_profile * vm: save topframe info into global_State Within this changeset a parser for binary data dumped via the sampling profiler to Tarantool binary. It is a set of the following Lua modules: * sysprof/parse.lua: decode the sampling profiler event stream * sysprof/collapse.lua: collapse stacks obtained while profiling * sysprof.lua: Lua script and module to display data Besides, all new public Lua C API introduced within this changeset is added to extra/exports. Closes #781 NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump
-
artembo authored
To make tarantool-gc64 available in repository we need to provide a product name in a package itself and in the repository. To build Tarantool with gc64 enabled, the flag -DLUAJIT_ENABLE_GC64 needs to be passed to cmake flags. For this purposes GC64 env var was added to all workflows for Debian, Ubuntu, Centos and Fedora. This variable is processed to rename PRODUCT_NAME. If GC64 is eanbled, PRODUCT_NAME is `tarantool-gc64`, otherwise it stays `tarantool`, which goes to repository name and the package itself. Thus, Tarantool with gc64 enabled can be installed by `<packaen_manager> install -y tarantool-gc64` in the one single repository. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci Closes: tarantool/tarantool-qa#161
-
Sergey Bronnikov authored
Changelog: https://curl.se/changes.html#7_80_0 NO_DOC=libcurl submodule bump NO_CHANGELOG=libcurl submodule bump NO_TEST=libcurl submodule bump Closes: #6029 Closes: https://github.com/tarantool/security/issues/10
-
Nikita Pettik authored
This patch introduces two stubs: one to log incoming requests and another one to log outgoing responses. Log itself takes place in TX thread, in function family `tx_process_*`. At the start of TX processing we are able to use raw msgpack from input buffer to log it as it was received. In the same way we can log raw msgpack for responses (before sending message to IProto thread). NO_DOC=<No user visible changes> NO_TEST=<No functional changes> NO_CHANGELOG=<No functional changes>
-
Nikita Pettik authored
It may turn out to be useful having output buffer savepoint pointing to the start of response. Using it one can restore the whole encoded IProto response. For instance, to dump it to auxiliary logs. NO_DOC=<No user visible changes> NO_TEST=<No functional changes> NO_CHANGELOG=<No functional changes>
-
Nikita Pettik authored
Let's add no-op interface functions and box cfg default parameters. NO_DOC=<No user visible changes> NO_TEST=<No functional changes> NO_CHANGELOG=<No functional changes>
-
Nikita Pettik authored
Let's add no-op interface functions and box cfg default parameters. NO_DOC=<No user visible changes> NO_TEST=<No functional changes> NO_CHANGELOG=<No functional changes>
-
Nikita Pettik authored
It can be used for instance to duplicate logs somewhere. In particular case it is going to be used for flight recorder. NO_DOC=<No user visible changes> NO_TEST=<No functional changes> NO_CHANGELOG=<No functional changes>
-
Nikita Pettik authored
prbuf entry contains pointer to raw memory (char *) and its length. Let's introduce :data() methods which would convert this memory chunk to Lua string. NO_DOC=<Feature for internal usage> NO_CHANGELOG=<Feature for internal usage>
-
Ilya Verbin authored
If a foreign key consists of multiple fields with `is_nullable=true` attribute, it's OK to insert a tuple with all-null key, however a combination of null and non-null fields is not allowed. Closes #7046 NO_DOC=bugfix
-
Boris Stepanenko authored
Losing quorum didn't modify RAFT leader behaviour, this led to easy split brain situation, when current leader gets disconnected from the rest of the replicaset. Old leader remained RW while new leader could be elected in the rest of the replicaset. Now when current leader looses connection to replica (connection being alive relay and applier for that replica) leader checks that he still has at least quorum of alive connections to replicas. If less than quorum of alive connection remain: leader resigns its role becoming follower without starting new term. This makes old leader RO. Old leader won't start new elections because of pre-vote, added in 421a0968. Fenced leader will "freeze" its limbo, and won't CONFIRM nor ROLLBACK any synchronous transactions until it regains leadership. If another leader is elected while old one is fenced, once old leader is reconnected to new one "frozen" transaction will be rolled back. When quorum of connections is regained and new leader is not elected old leader or another node will start elections and elect new leader. Closes #6661 @TarantoolBot document Title: RAFT leader fencing RAFT documentation must include that RAFT leader will resign its leadership if it has less than replication_synchro_quruom of alive connections to replicas (alive connection is connected relay and applier) if fencing is on. This applies to election_mode 'candidate' or 'manual'. Resigning leadership is becoming follower in current RAFT term, this leads to resigned leader becoming read-only. Introduce a new `box.cfg` option - `election_fencing_enabled`. If set to `true` fencing is on (default behaviour). If set to `false` fencing is off and leader doesn't resign it's leadership when looses quorum. If enabled on the current leader, when it doesn't have quorum of alive connections - leader will immediately resign its leadership.
-
Timur Safin authored
Extended datetime literal parser(s) with basic support for timezone abbreviations. Not all names could be directly supported at the moment, only non-ambiguous. NO_DOC=will be documented with Olson patchset NO_TEST=changelog only
-
Timur Safin authored
Use different return values for indication that recognized timezone suffix is ambiguous or not yet handled (before tzdata integration to come in next patchset). i.e. we could always parse '2020-02-02T01:03 MSK' as MSK is static +03:00, but it's ambiguous if there is something like '2020-02-02T01:03 AT' as there are multiple ATs (e.g. 'Alaska Time' and 'Atlantic Time'). NO_CHANGELOG=internal NO_DOC=internal
-
Timur Safin authored
Use tzindex field wherever we need to output datetime value to string (with or without format in `:format()` or `tostring()` correspondingly). NO_DOC=yet NO_CHANGELOG=yet
-
Timur Safin authored
Use our own list of generated timezone names for symbol table initialization, and then use this symbol table for datetime suffix parsing without format. Also, use newly integrated timezone names lookup functions in the `tnt_strptime` if `%Z` flag was ordered. Return corresponding `tm_gmtoff` (datetime->tzoffset) and `tm_tzindex` (datetime->tzindex) for later consumption. At the moment we could test only non-ambiguous abbreviations, which might be converted to tzoffset in a deterministic way. i.e. 'MSK' is always +0300, or 'MSD' is always +0400, but 'AT' may be either 'Alaska Time' (-0900) or 'Atlantic Time' (-0400/-0300) thus there is no (yet) any simple way to translate to their offset. NO_DOC=yet NO_CHANGELOG=yet
-
Timur Safin authored
We use gen-zone-abbrevs.pl script to generate file describing timezones and their fixed ids. Please see `perldoc gen-zone-abbrevs.pl` for more detailed usage description. Generator uses `zone-abbrevs.txt` as an input list of all known to the date timezone abbreviations. We use this page as original source https://www.timeanddate.com/time/zones/ Names encoding schema used by generator script supposed to be stable enough for input set of properly sorted zone names, including: - military zone names; - zone abbreviations; - human-readable IANA zone names. It generates _future compatible_, strictly ordered, sequential list. NO_DOC=yet NO_CHANGELOG=yet NO_TEST=yet
-
Timur Safin authored
NO_DOC=infra NO_CHANGELOG=infra NO_TEST=infra
-
Georgiy Lebedev authored
Empty or nil `select` calls on user spaces tend to be dangerous: we see Tarantool crashing with OOM, or Tarantool becoming unacceptably slow and its transaction thread utilizing 100% CPU usage. Also, debugging of such killer-requests is often too painful and exhausting. To mitigate this, create a critical log entry containing the current Lua stack traceback upon such function calls — a user can explicitly request a full scan though by passing `fullscan = true` to `select`'s `options` table argument in which case the log entry will not be created. These log entries are also rate limited. Closes #6539 @TarantoolBot document Title: new behaviour of empty or nil `select` calls on user spaces A critical log entry containing the current stack traceback is created upon empty or nil `select` calls on user spaces. A user can explicitly request a full scan though by passing `fullscan = true` to `select`'s `options` table argument in which a case a log entry will not be created.
-
Georgiy Lebedev authored
Introduce a `Ratelimit` class equivalent to `src/lib/core/ratelimit.h` to `log` module for internal usage: it acts as a wrapper around the log API. In scope of #6539 we need to make critical log entries: add S_CRIT alias for this purpose and a `log_crit` method to `Ratelimit`. Needed for #6539 NO_CHANGELOG=internal NO_DOC=internal NO_TEST=internal
-
Serge Petrenko authored
When a node has Raft turned on, issuing `box.ctl.promote()` on it triggers Raft elections. Such promotion bypasses the Pre-Vote check, because we want manual promotion to succeed even if it disrupts the current leader. However, in order to win elections, the node still has to collect votes from a quorum of peers. When there's no quorum, promote() might enter an infinite election loop, trying to promote the current node. Let's not start elections when it's known for sure that the node won't be able to win them, and make promote() throw an appropriate error. Now the node still triggers at least one election round on box.ctl.promote(), but it doesn't attempt re-running elections infinitely when there is no quorum. Follow-up #6654 NO_DOC=minor change
-
Serge Petrenko authored
Raft cluster members only treat the leader as alive as long as they preserve a direct connection to it. This means that as soon as one of the followers loses leader connection, it will start new elections bumping the term and forcing the existing leader to step off. Moreover, when a server is partitioned from the rest of the cluster, it will constantly rearrange elections by bumping the term and voting for itself. It will never get any votes, since it is partitioned, but it will increase its term to big numbers. Once such a server reconnects to the rest of the cluster, it will make everyone read-only, because the working majority will probably have a smaller term than this server. Such elections, performed while there is a working leader, like in the first example, or performed when there's no chance of winning, like in the second example, are considered disruptive. Diego Ongaro's thesis (https://github.com/ongardie/dissertation) proposes the following solution to the problem of disruptive servers. It's an additional election phase, called Pre-Vote. The basic idea is as follows: only bump the term and start elections once the quorum of peers reply OK to a Pre-Vote message. The peers would reply OK when: - the candidate's log is sufficiently up to date - the peer doesn't see the leader The downside of such approach is an additional request travelling to and from the followers. Tarantool replication architecture allows us to achieve the same goals without issuing Pre-Vote requests. Here's what's done in this patch to do so: 1) Track number of live peer connections, and only start elections when there is a quorum of connected peers. 2) Make every node broadcast whether it sees the leader of the current term or not. Every candidate collects info about who's seen as a leader in the current term. When at least one node sees a leader, the candidate doesn't start new elections. Closes #6654 @TarantoolBot document Title: elections: new `box.info` field and binary protocol changes * `box.info.election` got a new field: `leader_idle`. When elections are enabled, it shows time in seconds since the last interaction with the known leader. * `IPROTO_RAFT` request got 2 new fields: `IPROTO_RAFT_LEADER_ID = 0x04` - uint - the id of the current leader (as seen by the node which issued the request). `IPROTO_RAFT_IS_LEADER_SEEN = 0x05` - bool - whether the node has a direct connection to the leader.
-
Serge Petrenko authored
Support Pre-Vote in Raft: when a node knows, that some of the peers see the leader of the current term, it doesn't start elections. Even when it doesn't have a connection to the leader. Don't use the machinery yet: nodes never share info on whether they see the leader. Part-of #6654 NO_DOC=no user-visible changes NO_CHANGELOG=no user-visible changes
-
Serge Petrenko authored
Our checkpatch.pl script demands that every new function has a comment. One new function will be added to raft_ev.h in the following commits, and, according to the new rules, it has to have a comment. Having one function in a file commented looks stupid, so let's add comments to every such function in raft_ev.h NO_DOC=making lint happy NO_CHANGELOG=making lint happy NO_TEST=making lint happy
-
Serge Petrenko authored
raft_checkpoint_remote and raft_worker_handle_broadcast perform the same job of filling the raft message with a complete raft state. Let's fix that. Part-of #6654 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Serge Petrenko authored
Start counting replicas, whose connections are "healthy", e.g. there are both relay and applier in SUBSCRIBE state. Introduce triggers for the events of "healthy quorum" loss and gain. "healthy quorum" is the same as replication_synchro_quorum. The triggers will be used in raft pre-vote and synchro queue owner fencing. Both these features require tracking whether there is a quorum of nodes reachable from the local node. Pre-requisite #6654 Pre-requisite #6661 NO_DOC=no user-visible changes NO_CHANGELOG=no user-visible changes NO_TEST=preparatory patch
-
Aleksandr Lyapunov authored
MVCC must track a read set of transaction, including cases when the select returns an empty result. In this case MVCC has two options: to store a point key if the select has a full key or to store a gap otherwise. By a mistake the second variant was not implemented. Fix it now. Closes #7025 NO_DOC=bugfix
-
Vladimir Davydov authored
To implement online space upgrade, we need to add stub calls to the following code paths and data structures: - struct space: add a pointer to space upgrade state. - struct space_opts: add a pointer to space upgrade definition. - CheckSpaceFormat::prepare: skip space check if the format is changed in the scope of space upgrade. - alter_space_do: check space upgrade state and fail alter if upgrade is in progress. - alter_space_commit: run background worker for space upgrade. - space_on_final_recovery_complete: restart upgrade after recovery. - result_processor: apply space upgrade transformations to tuples returned to the user by box functions. We also need to: - Add a new error code ER_WRONG_SPACE_UPGRADE_OPTIONS, which we will use on error decoding upgrade options, stored in space options. - Load space upgrade Lua modules. The modules are supposed to define box.internal.space.upgrade method, which if available is used by box.schema.space.upgrade. - Add check_param, check_param_table and normalize_format helpers to box.internal, because we will use them from space.upgrade Lua code. Note, the space upgrade state will be reference counted, because background space upgrade may complete while some fiber is reading from the upgraded space (there may be yields in Vinyl). For this fiber to process the result correctly, it has to increment the reference counter of the space upgrade state before reading from the space. NO_DOC=ee NO_TEST=ee NO_CHANGELOG=ee
-
Vladimir Davydov authored
A convenient helper that is supposed to apply various transformations to a tuple fetched from a space before returning it to the user. Usage: struct result_processor res_proc; result_process_prepare(&res_proc, space); rc = index_get(index, key, part_count, result); result_process(&res_proc, &rc, result); We need to split the procedure into two parts, because index_get and similar methods may yield in case of Vinyl. The 'prepare' method is supposed to pin all data structures needed to process the result in the result_processor struct while the 'process' method is supposed to release them. Currently, the new methods do absolutely nothing. They will be used to convert tuples during space upgrade. NO_DOC=internal NO_TEST=internal NO_CHANGELOG=internal
-
Vladimir Davydov authored
The new helper function returns the space the iterator was created for. It looks up the space by id and caches it in iterator struct. We need it for space upgrade: we will use it in box_iterator_next() to update tuples returned by iterator_next(). NO_DOC=internal NO_TEST=internal NO_CHANGELOG=internal
-
Vladimir Davydov authored
NO_DOC=code cleanup NO_TEST=code cleanup NO_CHANGELOG=code cleanup
-
Vladimir Davydov authored
It's a more appropriate name, because the function doesn't update a space format - it just adjusts a format table to a normal form. NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Mergen Imeev authored
After this patch, it will be possible to perform permitted arithmetic operations on DATETIME and INTERVAL values. Closes #6773 @TarantoolBot document Title: Field type interval is available in SQL The INTERVAL field type is now available in SQL. INTERVAL values cannot be compared. INTERVAL values can be explicitly and implicitly cast to ANY. There are currently no literals for the INTERVAL field type.
-
Mergen Imeev authored
Prior to this patch, the right operand was checked before the left. This can lead to some complications when arithmetic operations are added for the DATETIME and INTERVAL values. To avoid this, let's check the left operand before the right one. This will cause the description of the error to change, but nothing else. Needed for #6773 NO_DOC=Not documented feature.
-
Mergen Imeev authored
Part of #6773 NO_DOC=Will be added later. NO_CHANGELOG=Will be added later.
-
Mergen Imeev authored
This patch introduces basic INTERVAL support in SQL. After this patch, it will be allowed to select INTERVAL values from spaces, insert them into spaces, and use them in functions. CAST() from INTERVAL to STRING and ANY is also supported. Part of #6773 NO_DOC=Will be added later. NO_CHANGELOG=Will be added later.
-
Mergen Imeev authored
This patch fixes the error description for comparison in case the left operand is comparable but the right operand is not. Part of #6668 Needed for #6668 NO_DOC=Will be added in another commit. NO_CHANGELOG=Will be added in another commit.
-
Mergen Imeev authored
After this patch, using MAP and ARRAY in GROUP BY will throw an error instead of an assertion. Part of #6668 Needed for #6773 NO_DOC=Will be added in another commit. NO_CHANGELOG=Will be added in another commit.
-
Mergen Imeev authored
After this patch, when DISTINCT is used with MAP or ARRAY, an error will be thrown instead of a segmentation fault. Part of #6668 Needed for #6773 NO_DOC=Will be added in another commit. NO_CHANGELOG=Will be added in another commit.
-