- Oct 10, 2024
-
-
Col-Waltz authored
Updates curl module to the version 8.10.1. The new version brings several new options, such as: CURL_USE_RUSTLS - Enables Rustls for SSL/TLS. Added in commit curl/curl@ed76a23fccc1 ("cmake: add rustls") CURL_USE_WOLFSSH - Option to use wolfSSH. Added in commit curl/curl@0d8fdd1c7421 ("cmake: add wolfSSH support") CURL_USE_GSASL - Option to use libgsasl. Added in commit curl/curl@66bf995d1cfc ("cmake: add CURL_USE_GSASL option with detection + CI test") CURL_DISABLE_SHA512_256 - Disables SHA-512/256 hash algorithm. Added in commit curl/curl@33629949488c ("build: add options to disable SHA-512/256 hash algo") CURL_USE_LIBUV - Use libuv for event-based tests. Added in commit curl/curl@f7d5f47059c3 ("cmake: add support for `CURL_USE_LIBUV` option") Corrected http_client test according to curl commit curl/curl@b7e769dc872d ("vtls: stop offering alpn http/1.1 for http2-prior-knowledge") Build file missed several options and paths not used in the current build. List of these options was added into curl-excluded-options.txt for the convenience of the following bumps. Closes #10576 https://curl.se/changes.html#8_10_1 https://github.com/curl/curl/releases/tag/curl-8_10_1 NO_DOC=libcurl submodule bump NO_TEST=libcurl submodule bump (cherry picked from commit 0919f390802f146852b462215327ef03e2730cfc) @Totktonada: the http_client test mentioned in the original commit message doesn't present in the `release/2.11` branch.
-
Col-Waltz authored
Some curl options appear to missed by several previous curl bumps. Here is a list of missed options with curl commits in which they first appeared: BUILD_EXAMPLES - Build libcurl examples. Added in curl 8.8.0 release in commit curl/curl@dfdd978f7c60 ("cmake: add `BUILD_EXAMPLES` option to build examples") USE_ECH - Enables ECH support. Added in curl 8.8.0 release in commit curl/curl@a362962b7289 ("TLS: add support for ECH (Encrypted Client Hello)") USE_HTTPSRR - Enables HTTPS RR support for ECH. Added in curl 8.8.0 release in commit curl/curl@a362962b7289 ("TLS: add support for ECH (Encrypted Client Hello)") BUILD_STATIC_CURL - Builds curl executable with static libcurl. Added in curl 8.3.0 release in commit curl/curl@1199308dbc90 ("cmake: support building static and shared libcurl in one go") CURL_DISABLE_NEGOTIATE_AUTH - Disables negotiate authentication. Added in curl 8.3.0 release in commit curl/curl@e92edfbef644 ("lib: add ability to disable auths individually") CURL_DISABLE_SRP - Disables TLS-SRP support. Added in curl 8.4.0 release in commit curl/curl@781242ffa44a ("cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS") NO_TEST=does not change tarantool behavior NO_DOC=does not change tarantool behavior (cherry picked from commit 97e3136ddc691f42cf0a5dff27881978b3c25d52)
-
- Oct 08, 2024
-
-
Sergey Bronnikov authored
The tests `array.test.lua` and `map.test.lua` require the remote Tarantool instance for running testcases. When tests are executed by test-run.py, it runs this Tarantool instance. The testcases that require the remote instance were ported to luatest to make tests able to run with ctest. The testcases `builtins-13.1` and `builtins-14.1` are companion testcases and were moved as well. Required by patches for CTest support, because allows to execute aforementioned tests without test-run.py. NO_CHANGELOG=codehealth NO_DOC=codehealth NO_TEST=codehealth (cherry picked from commit 47295fdebc1910b990c391c26fe92918a9c0c001)
-
Sergey Bronnikov authored
The patch set a path to `tarantool` executable binary explicitly in the aforementioned test. NO_CHANGELOG=codehealth NO_DOC=codehealth (cherry picked from commit 7844e6034bd67f3e5a57710443373da5200e441a)
-
Sergey Bronnikov authored
The test gh_5747_crash_multiple_args_test.lua uses a `tarantool` binary that is available in a current PATH environment variable. In some cases, PATH may point to a system `tarantool` executable and this could be unexpected for those who run the test. The patch set a path to `tarantool` executable derived from a command-line used to run the test. NO_CHANGELOG=codehealth NO_DOC=codehealth (cherry picked from commit 1e907ddbd8db2587418d50dde3e06e985c47bded)
-
Sergey Bronnikov authored
According to CMake policy CMP0110 whitespaces and other special characters are forbidden before CMake 3.19. This policy is triggered with the patch for CTest support and breaks test generation: NO_WRAP The following name given to add_test() is invalid if CMP0110 is not set or set to OLD: `test/box-luatest/gh_7217_repeatable_{in, up}sert_memtx_tx_conflict_test.lua´ NO_WRAP The patch renames file without using whitespaces and special characters. Required for CTest support. Follows up commit 654cf498 ("memtx: fix story delete statement list"). 1. https://cmake.org/cmake/help/latest/policy/CMP0110.html NO_CHANGELOG=codehealth NO_DOC=codehealth NO_TEST=codehealth (cherry picked from commit ec810b9d6b618c4ec5116dde2f02b5be01a13f6d)
-
Sergey Bronnikov authored
The aforementioned skipcond file is specific for Travis CI because it depends on the environment variable `TRAVIS_JOB_ID`. We have moved to GHA for a long time, it seems this skipcond is not useful anymore. NO_CHANGELOG=codehealth NO_DOC=codehealth NO_TEST=codehealth (cherry picked from commit 6d9620cd6d485f826acc2b54fb4ac7f971b56cde)
-
Sergey Bronnikov authored
The test has been renamed in commit 16d6e9d2 ("console: remove ERRINJ_STDIN_ISATTY injection") and it is not used the error injection anymore, so it was removed in suite.ini. NO_CHANGELOG=codehealth NO_DOC=codehealth NO_TEST=codehealth (cherry picked from commit b220f050261d4c4ba0ec679c8d77a5e817367a3f)
-
Vladimir Davydov authored
Currently, we just panic without providing any additional information if we failed to create the initial checkpoint on bootstrap. This complicates trouble shooting. Let's replace `panic()` with `say_error()` and raise the exception that caused the failure. The exception will be caught by `box_cfg()`, which will log it and then panic. NO_DOC=error logging NO_TEST=error logging NO_CHANGELOG=error logging (cherry picked from commit e1b5114d99ed2f224e9e9a17bf29882e50be3653)
-
- Oct 07, 2024
-
-
Nikita Zheleztsov authored
We decided to introduce new schema version, which does nothing in order to distinguish, which 2.11 schema we can safely use to allow persistent names on it. Follow up #10549 NO_DOC=internal NO_CHANGELOG=internal NO_TEST=nothing to test
-
Vladislav Shpilevoy authored
The function replica_check_id() is called on any change in _cluster: insert, delete, update. It was supposed to check if the replica ID is valid - not nil, not out of range (VCLOCK_MAX). But it was also raising an error when the ID matched this instance's ID unless the instance was joining. That happened even if a _cluster tuple was updated without changing the ID at all. For example, if one would just do _cluster:replace(_cluster:get(box.info.id)). Better do the check in the only place where the mutation can happen - on deletion. Since replica ID is a primary key in _cluster, it can't be updated there. Only inserted or deleted. This commit is backported to 2.11, since we want to allow using persistent names as early as we can in order to simplify the upgrade process. We also bump the schema version in the following commit in order to distinguish this version from overs 2.11.X, where persistent names doesn't work. Closes #10549 NO_DOC=bugfix and refactoring NO_CHANGELOG=cannot happen without touching system spaces NO_TEST=too insignificant for an own test (cherry picked from commit cb8f4715)
-
Sergey Bronnikov authored
There is no check for NULL for a value returned by `ibuf_alloc`, the NULL will be passed to `memcpy()` if the aforementioned function will return a NULL. The patch fixes that by replacing `ibuf_alloc` with macros `xibuf_alloc` that never return NULL. Found by Svace. NO_CHANGELOG=codehealth NO_DOC=codehealth NO_TEST=codehealth (cherry picked from commit b4ee146fde6e418aed590ac6054cff75c2a59626)
-
Astronomax authored
This patch optimizes the process of collecting ACKs from replicas for synchronous transactions. Before this patch, collecting confirmations was slow in some cases. There was a possible situation where it was necessary to go through the entire limbo again every time the next ACK was received from the replica. This was especially noticeable in the case of a large number of parallel synchronous requests. For example, in the 1mops_write bench with parameters --fibers=6000 --ops=1000000 --transaction=1, performance increases by 13-18 times on small clusters of 2-4 nodes and 2 times on large clusters of 31 nodes. Closes #9917 NO_DOC=performance improvement NO_TEST=performance improvement (cherry picked from commit 4a866f64d64c610a3c8441835fee3d8dda5eca71)
-
Astronomax authored
Two new vclock methods have been added: `vclock_nth_element` and `vclock_count_ge`. * `vclock_nth_element` takes n and returns whatever element would occur in nth position if vclock were sorted. This method is very useful for synchronous replication because it can be used to find out the lsn of the last confirmed transaction - it's simply the result of calling this method with argument {vclock_size - replication_synchro_quorum} (provided that vclock_size >= replication synchro quorum, otherwise it is obvious that no transaction has yet been confirmed). * `vclock_count_ge` takes lsn and returns the number of components whose value is greater than or equal to lsn. This can be useful to understand how many replicas have already received a transaction with a given lsn. Part of #9917 NO_CHANGELOG=Will be added in another commit NO_DOC=internal (cherry picked from commit 58f3c93b660499e85f08a4f63373040bcae28732)
-
- Oct 04, 2024
-
-
Andrey Saranchin authored
According to the C standard, passing `NULL` to `memcpy` is UB, even if it copies nothing (number of bytes to copy is 0). The commit fixes such situation in memtx MVCC. Closes tarantool/security#129 NO_TEST=fix UB NO_CHANGELOG=fix UB NO_DOC=fix UB (cherry picked from commit 24d38cef5adff900bea2484235762678ac1c5234)
-
- Oct 02, 2024
-
-
Vladimir Davydov authored
Since commit e19bca5a74e8 ("box: check fiber slice in generic implementation of index count"), Vinyl's version of `index.count()` checks the fiber slice. As a result, the test may fail if it runs under a heavy load: ``` | @@ -94,6 +94,7 @@ | end | end; | | --- | + | - error: fiber slice is exceeded | | ... | -- Verify that at any moment max index is corresponding to amount of tuples, | -- which means that changes apply order is correct ``` Let's set the max fiber slice to a big value to avoid that. NO_DOC=test fix NO_CHANGELOG=test fix (cherry picked from commit b5fb66437a22ea65ed27c2ed14636e8036b079d3)
-
- Sep 30, 2024
-
-
Alexander Turenko authored
The new version of the publishing action leads to the following error: NOWRAP ``` Error: File not found: '/home/runner/work/_actions/JamesIves/github-pages-deploy-action/v4.6.6/lib/main.js' ``` NOWRAP Let's revert it to v4.6.4 (because v4.6.5 change was reverted in v4.6.6). See also https://github.com/JamesIves/github-pages-deploy-action/issues/1697 NO_DOC=no code changes NO_CHANGELOG=see NO_DOC NO_TEST=see NO_DOC (cherry picked from commit 7cf41a1bb9b3f0a2eac82c296a764688f5d8b35c)
-
Alexander Turenko authored
The `ubuntu-latest` image is now `ubuntu-24.04`, see [1]. The job fails on this image with the following error: NOWRAP ``` CMake Error at /usr/local/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:233 (message): Could NOT find Readline (missing: READLINE_INCLUDE_DIR READLINE_LIBRARY) ``` NOWRAP It seems, the libreadline-dev package is missing. Let's install it. Also, update a version of the publishing action to the latest at the moment. I didn't perform any check, but I guess that a new version of NodeJS is needed and the latest action version has better support of it. [1]: https://github.blog/changelog/2024-09-25-actions-new-images-and-ubuntu-latest-changes/ NO_DOC=no code changes NO_CHANGELOG=see NO_DOC NO_TEST=see NO_DOC (cherry picked from commit 3e09e9f293925195ddebb13a4fcfd64762e52b89)
-
- Sep 25, 2024
-
-
Vladimir Davydov authored
Vinyl doesn't support altering the primary index of a non-empty space, but the check forbidding this isn't entirely reliable - the DDL function may yield to wait for pending WAL writes to finish after ensuring that the space doesn't contain any tuples. If a new tuples is inserted into the space in the meantime, the DDL operation will proceed rebuilding the primary index and trigger a crash because the code is written on the assumption that it's rebuilding a secondary index: ``` ./src/box/vinyl.c:1572: vy_check_is_unique_secondary_one: Assertion `lsm->index_id > 0' failed. ``` Let's fix this by moving the check after syncing on WAL. Closes #10603 NO_DOC=bug fix (cherry picked from commit 955537b57c2aade58b7ca42501a9bbe50dd91f26)
-
- Sep 24, 2024
-
-
Col-Waltz authored
Curl option BUILD_MISC_DOCS builds misc man pages and set ON by default. Other documentation building options such as ENABLE_CURL_MANUAL and BUILD_LIBCURL_DOCS was set OFF in BuildLibCurl.cmake. I suppose this option has to be added in commit 7192bf66 ("third_party: update libcurl from 8.7.0 to 8.8.0+patches") and set OFF. Follows up #9885 NO_TEST=does not change tarantool behavior NO_DOC=does not change tarantool behavior (cherry picked from commit 1080995fa4083c4de6aa19e964b98f0ffb7e34c5)
-
- Sep 23, 2024
-
-
Vladimir Davydov authored
`index.count()` may hang for too long in Vinyl if a substantial consecutive hunk of the space is stored in memory. Let's add a fiber slice check to it to prevent it from blocking the TX thread for too long. Closes #10553 NO_DOC=bug fix (cherry picked from commit e19bca5a74e83d2521fe770f2a93c3e3d3ad4801)
-
Vladimir Davydov authored
The tuple cache doesn't store historical data. It stores only the newest tuple versions, including prepared but not yet confirmed (committed but not written to WAL) tuples. This means that transactions sent to a read view shouldn't add any new chains to the cache because such a chain may bypass a tuple invisible from the read view. A transaction may be sent to a read view in two cases: 1. If some other transactions updates data read by it. 2. If the transaction is operating in the 'read-confirmed' isolation mode and skips an unconfirmed tuple while scanning the memory level. This was added in commit 588170a7 ("vinyl: implement transaction isolation levels"). The second point should be checked by the read iterator itself, and it is indeed for the standard case when we scan the memory level before reading the disk. However, there's the second case: if some other tuples are inserted into the memory level while the read iterator was waiting for a disk read to complete, it rescans the memory level and may skip a new unconfirmed tuple that wasn't there the first time we scanned the memory level. Currently, if this happens, it won't send itself to a read view and may corrupt the cache by inserting a chain that skips over the unconfirmed tuple. Fix this by adding the missing check. While we are at it, let's simplify the code a bit by moving the check inside `vy_read_iterator_scan_mem()`. It's okay because sending to a read view a transaction that's already in the read view is handled correctly by `vy_tx_send_to_read_view()`. Closes #10558 NO_DOC=bug fix (cherry picked from commit a3feee322e76a1e10ab874e63f17f97b6457b59d)
-
- Sep 20, 2024
-
-
Vladimir Davydov authored
`vy_slice_stream_next()` clears the return value on failure. This isn't expected by `vy_write_iterator_merge_step()`, which doesn't update the source position in the `vy_wirte_iterator::src_heap` in this case. As a result, an attempt to remove `end_of_key_src` from the heap in `vy_write_iterator_build_history()` may crash as follows: ``` # 1 0x572a2ecc21a6 in crash_collect+256 # 2 0x572a2ecc2be2 in crash_signal_cb+100 # 3 0x7cfef6645320 in __sigaction+80 # 4 0x572a2eab16de in tuple_format+16 # 5 0x572a2eab1a25 in vy_stmt_is_key+24 # 6 0x572a2eab1be8 in vy_stmt_compare+89 # 7 0x572a2eab1e37 in vy_entry_compare+74 # 8 0x572a2eab2913 in heap_less+88 # 9 0x572a2eab21e3 in vy_source_heap_sift_up+255 # 10 0x572a2eab20b9 in vy_source_heap_update_node+54 # 11 0x572a2eab25c1 in vy_source_heap_delete+249 # 12 0x572a2eab4134 in vy_write_iterator_build_history+1497 # 13 0x572a2eab4995 in vy_write_iterator_build_read_views+193 # 14 0x572a2eab4ce6 in vy_write_iterator_next+380 # 15 0x572a2eadd20b in vy_task_write_run+1132 # 16 0x572a2eade6cf in vy_task_compaction_execute+124 # 17 0x572a2eadfa8d in vy_task_f+445 # 18 0x572a2e9ea143 in fiber_cxx_invoke(int (*)(__va_list_tag*), __va_list_tag*)+34 # 19 0x572a2eccee7c in fiber_loop+219 # 20 0x572a2f0aef18 in coro_init+120 ``` Normally, a function shouldn't update the return value on failure so let's fix `vy_slice_stream_next()`. Closes #10555 NO_DOC=bug fix (cherry picked from commit f1144c533b6c52c324ffe1cc4fcaeab1f2f6cd9f)
-
Vladimir Davydov authored
ERRINJ_VY_RUN_OPEN and ERRINJ_VY_STMT_ALLOC are countdown injections. Let's name them appropriately and use the helper macro. Also, let's raise the ER_INJECTION error code for them to make it clear that they aren't real errors. NO_DOC=internal NO_CHANGELOG=internal (cherry picked from commit 21fe14582c948f560720fa285ed3e21483d11dc2)
-
Vladimir Davydov authored
We shouldn't decrement the counter if it's negative - otherwise it may wrap around and mistakenly trigger the error injection. NO_DOC=internal NO_TEST=internal NO_CHANGELOG=internal (cherry picked from commit d11d4576b0d0cbfc03dc1a3570573b7bbf1126b5)
-
- Sep 18, 2024
-
-
Sergey Bronnikov authored
There is an option tz in `datetime.parse()`, it was added in commit 3c403661 ("datetime, lua: date parsing functions"). The option is not documented, and the commit message says that option `tz` is "Not yet implemented in this commit.". The patch added tests and a doc request for this option. The behaviour of the option `tz` is the same as with option `tzoffset`: - if timezone was not set in a parsed string then it is set to a value specified by `tz` - if timezone was set in a parsed string then option `tz` is ignored ``` tarantool> date.parse("1970-01-01T01:00:00 MSK", { tz = 'Europe/Paris' }) --- - 1970-01-01T01:00:00 MSK - 23 ... tarantool> date.parse("1970-01-01T01:00:00", { tz = 'Europe/Paris' }) --- - 1970-01-01T01:00:00 Europe/Paris - 19 ... ``` Follows up #6731 Fixes #10420 @TarantoolBot document Title: Introduce option `tz` in `datetime.parse()` The option `tz` is added in a function `datetime.parse()`. The option set timezone to a passed value if it was not set in a parsed string. (cherry picked from commit c6bab23a6dc4f819167cbc78eb93859847a389ea)
-
Sergey Bronnikov authored
The patch fixes a behaviour, when `datetime.parse()` ignores `tzoffset` option if custom format is used. Fixes #8333 Relates to #10420 NO_DOC=bugfix (cherry picked from commit 04811e032f29afe0fa6206ef2c7a0f8434861830)
-
Sergey Bronnikov authored
The patch refactors a function `datetime_parse_full()` - overriding of timezone is not a part of datetime string parsing and this part was removed. Needed for #8333 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring (cherry picked from commit d7d3063fbd5a74563fde539f2c74852a1e04c1cd)
-
- Sep 17, 2024
-
-
Sergey Bronnikov authored
The patch forbids using non-integer values in datetime's `:set()` for `year`, `month`, `day`, `hour`, `min`, `sec`, `usec`, `msec`, `nsec` and `tzoffset` keys. `timestamp` can be double, and integer values allowed in timestamp if `nsec`, `usec`, or `msecs` provided. An error will be raised when a value of incorrect type is passed. Fixes #10391 @TarantoolBot document Title: Update types of datetime values passed to SQL's `CAST();` `CAST` can accept only integer values for `year`, `month`, `day`, `hour`, `min`, `sec`, `usec`, `msec`, `nsec` and `tzoffset`. `timestamp` can be integer or double. (cherry picked from commit f57be571b5e4cc8d57c7e97c15b52df37ad6f12c)
-
Sergey Bronnikov authored
The patch forbids using non-integer values in datetime's `:set()` for `year`, `month`, `day`, `hour`, `min`, `sec`, `usec`, `msec` and `nsec` keys. The type of `tzoffset` can be integer or string, `timestamp` can be double, and integer values allowed in timestamp if `nsec`, `usec`, or `msecs` provided. An error will be raised when a value of incorrect type is passed. Part of #10391 @TarantoolBot document Title: Update types of values passed to `:set()` and parse() `:set()` can accept only integer values for `year`, `month`, `day`, `hour`, `min`, `sec`, `usec`, `msec` and `nsec`. The type of `tzoffset` can be integer or string, `timestamp` can be integer or double. `tzoffset` passed to `datetime.parse()` can be integer or string. (cherry picked from commit 6e77907baa3cbeebc79241cc0046a539a09e3f2c)
-
Sergey Bronnikov authored
The patch forbids using non-integer values in datetime constructor `datetime.new()` for `year`, `month`, `day`, `hour`, `min`, `sec`, `usec`, `msec` and `nsec` keys. The type of `tzoffset` can be integer or string, `timestamp` can be double, and integer values allowed in timestamp if `nsec`, `usec`, or `msecs` provided. An error will be raised when a value of incorrect type is passed. Part of #10391 @TarantoolBot document Title: Update types of values passed to `datetime.new()` `datetime.new()` can accept only integer values for `year`, `month`, `day`, `hour`, `min`, `sec`, `usec`, `msec` and `nsec`. The type of `tzoffset` can be integer or string, `timestamp` can be integer or double. (cherry picked from commit cc9010a2b11477b2f16f2b2e168a6b9dcca2fb20)
-
Andrey Saranchin authored
During the latest rework of DDL in MVCC, the new helper `memtx_tx_history_rollback_empty_stmt` was introduced - it is used for statements without stories (such statements can appear, for example, when DDL removes all stories). By mistake, we forgot to unreference the new tuple and reference the old one there - the commit fixes this embarrassing mistake. Follow-up #10146 NO_CHANGELOG=bugfix for unreleased patch NO_DOC=bugfix (cherry picked from commit 32797f703079664abfe9b7e6112aee1039a52337)
-
Andrey Saranchin authored
DDL with memtx mvcc enabled used to crash a lot until the previous commits have fixed it. To make sure it's stable now, the commit introduces a stress test that executes various DDL and DML operations concurrently. The test doesn't check serialization of transactions, the only goal is to make sure that Tarantool does not crash. Along the way, the commit introduces a new error injection that disables yield while building an index. The problem is any concurrent DML with DDL building an index will abort it if MVCC is enabled, so the error injection is needed to make index build successful during the stress test. Follow-up #10146 NO_CHANGELOG=test NO_DOC=test (cherry picked from commit 260b10bc3616d9eeeea4f245dc523cab5494f711)
-
Andrey Saranchin authored
Currently, we delete MVCC trackers from index when it's being deleted. It doesn't actually break anything but we can do it right on DDL since all concurrent transactions are aborted and trackers are not needed anymore. Also, having a saved key belonging to old key def (it's saved by gap inplace item) is not a good idea and can lead to sudden bugs in future - let's just delete all the trackers right on DDL while we are here. Follow-up #10146 NO_TEST=refactoring NO_CHANGELOG=refactoring NO_DOC=refactoring (cherry picked from commit 8879489834f39fdb46093c1988fbf4f1601d51ff)
-
Andrey Saranchin authored
When DDL happens, we remove statements of concurrent transactions from MVCC. When removing statements, we set their `engine_savepoint` to `NULL` so that they won't be rolled back because we've already handled them. However, we remove statements only from stories, and not all statements can be accessed in this way. For example, when we have several delete statements of one story, and one of them gets prepared, others are unlinked. It leads to use-after-free, but it's read-only and doesn't affect anything, so only ASAN can catch it. It happens when the statement is being rolled back in `memtx_engine_rollback_statement`: we check if `space->upgrade` is not `NULL` (space can be already deleted) but this check affects instruction flow only if `stmt->new_tuple != NULL` and in our case that's not so. Anyway, let's iterate over all statements of all transactions and remove savepoints for ones related to the space that is being invalidated. It takes more time, but anyway, we are doing DDL that is heavy, so it doesn't really matter. Along the way, the commit removes helper `memtx_tx_history_remove_stmt` and all its helpers because they are not needed anymore. This helper unlinks added story from history chain, links all its delete statements to the previous story, if any, unlinks the statement from related stories and sets `engine_savepoint` to `NULL`. Since we already do all of this things except for unlinking statements from stories, let's simply call `memtx_tx_story_unlink_added[deleted]_by` instead. This change makes the code much more straightforward. Closes #10146 NO_DOC=bugfix (cherry picked from commit ac112b73192ad96271a02ee85dba3e9737fdaa9d)
-
Andrey Saranchin authored
Currently, when DDL is being committed, we delete all the stories and rollback prepared delete statements. The problem is such rollback is likely to fail because of assertion. When transaction is rolled back, all the statements are rolled back in reversed order, but when rollback happened because of DDL, order is not specified and some invariants are violated. Let's simply unlink delete statement instead of rollback. Rollback does two things: unlink delete statement and abort readers (including gaps) of prepared stories. The commit actually drops the second part - it's safe because after the previous commits we delete stories right after aborting all concurrent transactions so there is no need to abort anything anymore. Part of #10146 Closes #10474 NO_CHANGELOG=later NO_DOC=bugfix (cherry picked from commit 6a11224c85c1be28e7d1570cd4ba01efc033c34f)
-
Andrey Saranchin authored
Currently, every DDL transaction has such properties: 1. DDL and DML operations can be mixed inside one transaction, order is not restricted. 2. DDL has an effect right after the operation, not after it's prepared or committed. 3. After the first DDL operation, transaction is not allowed to yield. 4. If DDL yields (format check, index build), operation must be the first in its transaction. If DDL yields, space cache is updated strictly after yields are over. In other words, transaction cannot yield after DDL changes became visible. Keeping these properties in mind, we can recognize several flows in the way memtx MVCC handles DDL now. Here is the approach: 1. All transactions concurrent with DDL are aborted when DDL gets prepared. Memtx stories are not deleted here. 2. If DDL gets committed, all memtx stories of the old space are deleted. It means that only stories created before DDL are deleted. Such design is bad. Firstly, if transaction does a DDL and than a DML, stories belonging to the new and the old schema will be mixed in the indexes. Starting a DML transaction after DDL is prepared but before it's committed leads to the same problem. If stories of different schemas are mixed, Tarantool is most likely to crash since MVCC does not handle this case at all. Secondly, transactions started after DDL is prepared but before it's committed can read stories belonging to the old schema. In this case, after DDL is committed and the stories are deleted, we cannot do proper mvcc for such transactions anymore because we've just deleted the tuple it has read. Here is the new approach that current commit implements: 1. Abort all concurrent transactions and delete all memtx stories on every replace in space cache (right on DDL operation and on its rollback). In this case, every time a space and its schema is replaced with a new one, all the mvcc objects belonging to the old schema are deleted. In order not to break isolation of concurrent transactions, we have to abort them all right before deleting stories - it makes sense because we don't support old schema anymore and all its readers should be aborted. 2. When DDL gets committed, nothing happens since nothing has been changed. 3. Abort all concurrent transactions when DDL is being prepared. It is needed for DDL that does not update any space. For example, update in space `_cluster` or `_schema`. It worked this way before the commit so it is needed to provide the old behavior in such cases. Note that if DDL updates any space, all concurrent transactions are aborted right on DDL operation, when replace in space cache happens, and since DDL cannot yield, we have nothing to abort here so it's noop in such cases. Since `memtx_tx_on_space_delete` was supposed to be called on commit, it inserts prepared tuples to indexes. Since we want it to be called on actual DDL, the commit makes it insert all tuples visible to the transaction doing DDL. Also, if we delete all the stories on DDL, we should correctly rollback all non-committed DML statements - DML statements of the transaction doing DDL if they happened before DDL and all transactions prepared before DDL - if WAL write fails, we will need to roll them back without stories. For this purpose, firstly, let's remove `engine_savepoint` only for statements of aborted transactions in `memtx_tx_history_remove_stmt` (it is needed not to roll them back because they are already handled) so that statements of transaction doing DDL can be rolled back. And, secondly, let's add a new helper `memtx_tx_history_rollback_empty_stmt` that handles rollback of statements without stories (see helper's description for elaboration). Along the way, let's clean read lists of transactions aborted on DDL - they are not needed anymore and potentially keeping them can lead to use-after-free in future. Moreover, we should remove transactions aborted by DDL from `read_view_txs` list so that they won't affect memtx story GC until they are deleted - cleaning of transactions does it as well. Part of #10146 Part of #10474 Closes #10171 Closes #10096 Closes #10097 NO_CHANGELOG=later NO_DOC=bugfix (cherry picked from commit 959027de5553078dbe442aacb52e01e9b46c542c)
-
Andrey Saranchin authored
The helper consists of `unlink_top_common` and `unlink_top_on_space_delete_light` helpers. It would make sense if `unlink_top_common` would be used in any other place, but it's used only here actually. Let's inline this helpers - it will be easier to read this code and it will allow to simply patch this function in future commits. NO_TEST=refactoring NO_CHANGELOG=refactoring NO_DOC=refactoring (cherry picked from commit 7b2243828937413db9b1c3ffcf8abd8fd92daf02)
-
Andrey Saranchin authored
When space is being altered, `memtx_tx_space_on_delete` is called - it deletes all the stories associated with the old schema. However, before deleting a story, its `reader_list` member is not unlinked from the list so other nodes can still access this memory. The commit fixes this problem and adds an assertion that checks if story is always unlinked from reader list when is being deleted. Part of #10146 NO_CHANGELOG=later NO_DOC=bugfix (cherry picked from commit a32f56dfbb4b56b410ac376fce079613cac0ccb6)
-
Andrey Saranchin authored
Now background build of index uses index iterator that collects conflicts during iteration if MVCC is enabled. Thus, trigger `memtx_build_on_replace` is not needed - if someone writes to prefix we already scanned, it will lead to transaction conflict. Moreover, `memtx_ddl_state` that is needed for rollback is allocated on stack of function called from DDL transaction, so if conflicted transaction rolls back later that DDL is over (and it's possible only with MVCC enabled), segmentation fault will happen. So let's simply don't set the trigger is MVCC is enabled. Closes #10147 NO_CHANGELOG=later NO_DOC=bugfix (cherry picked from commit 9fe60c5754cf77686404fc7ee3d24af32b6c486c)
-