- Apr 09, 2019
-
-
Vladislav Shpilevoy authored
Dissemination components broadcasts events about member status updates. When any member attribute is updated (incarnation, status, UUID, address), the member stands into an event queue. Members from the queue are encoded into each round step message with a higher priority and before anti-entropy section. It means, then even if a cluster consists of hundreds of members and one of them was updated on one of instances, this update will be disseminated regardless of whether this memeber is encoded into anti-entropy section or not. It drastically speeds events dissemination up, according to the SWIM paper, and is noticed in the tests. Part of #3234
-
Vladislav Shpilevoy authored
Before dissemination component it was enough in the tests to either drop all packets to/from a certain member, or do not drop at all. But after dissemination it will be time to test more granulated packet loss table: not 0/100, but 5/10/20/50/.../100 packet loss rate. Part of #3234
-
Vladislav Shpilevoy authored
The test checks that if a member has failed in a big cluster, it is eventually deleted from all instances. But it takes too much real time despite usage of virtual time. This is because member total deletion takes O(N + ack_timeout * 5) time. N so as to wait until every member pinged the failed one at least once, + 3 * ack_timeout to learn that it is dead, and + 2 * ack_timeout to drop it. Of course, it is an upper border, and usually it is faster but not much. For example, on the cluster of size 50 it takes easily 55 virtual seconds. On the contrary, to just learn that a member is dead on every instance takes O(log(N)) according to the SWIM paper. On the same test with 50 instances cluster it takes ~15 virtual seconds to disseminate 'dead' status of the failed member on every instance. And even without dissemination component, with anti-entropy only. Leaping ahead, for the subsequent patches it is tested that with the dissemination component it takes already ~6 virtual seconds. In the summary, without losing test coverage it is much faster to turn off SWIM GC and wait until the failed member looks dead on all instances. Part of #3234
-
Vladislav Shpilevoy authored
At this moment SWIM protocol stores array of members only in one place: inside the anti-entropy component. Its decoding is a simple loop taking the member definitions one by one and upserting them into the member table. But the dissemination also has something kinda like members array: an array of events. The trick is that an event is basically the same as a member +/- a couple of optional fields. Events are also decoded into the member definition structure. It means that anti-entropy decoder can be easily reused. Part of #3234
-
Vladislav Shpilevoy authored
Each member stored in components dissemination and anti-entropy should carry a unique identifier, a status, and an address. Those attributes are UUID, IP, Port, enum swim_member_status, incarnation. Now they are sent only in scope of anti-entropy, but forthcoming dissemination component also would like to use these attributes for each event. This commit makes the vital attributes and their code more reusable by encapsulation of them into a binary passport structure. Part of #3234
-
- Apr 08, 2019
-
-
Konstantin Osipov authored
Rename vy_set() and vy_set_with_colmask() to vy_tx_set() and vy_tx_set_with_colmask() These methods really belong to vy_tx module, so move them there.
-
Serge Petrenko authored
Add the type of operation which is being executed to before_replace and on_replace triggers. Closes #4099 @TarantoolBot document Title: new parameter for space before_replace and on_replace triggers Now before_replace and on_replace triggers accept an additional parameter: the type of operation that is being executed. (INSERT/REPLACE/DELETE/UPDATE/UPSERT) For example, a trigger function may now look like this: ``` function before_replace_trig(old, new, space_name, op_type) if op_type == 'INSERT' then return old else return new end end ``` And will restrict all INSERTs, but allow REPLACEs, UPSERTs, DELETEs and UPDATEs.
-
Roman Tokarev authored
-
- Apr 07, 2019
-
-
Alexander Turenko authored
Add more logging into wait_fullmesh() and return immediately with false when 'stopped' status is observed. The purpose of the change is to provide more information in case of a master-master replication bootstrap failure.
-
Vladimir Davydov authored
Apart from speeding up statement comparisons and hence index lookups, this is also a prerequisite for multikey indexes, which will reuse tuple comparison hints as offsets in indexed arrays. Albeit huge, this patch is pretty straightforward - all it does is replace struct tuple with struct vy_entry (which is tuple + hint pair) practically everywhere in the code. Now statements are stored and compared without hints only in a few places, primarily at the very top level. Hints are also computed at the top level so it should be pretty easy to replace them with multikey offsets when the time comes.
-
Vladimir Davydov authored
This patch adds a helper struct vy_entry, which unites a statement with a hint. We will use this struct to store hinted statements in vinyl data structures, such as cache or memory tree. Note, it's defined in a separate file to minimize dependencies.
-
Vladimir Davydov authored
This patch adds vy_set and vy_set_with_colmask functions. For now they simply forward all arguments to vy_tx_set, but once comparison hints are introduced, they will also compute a hint for the inserted statement. Later, with the appearance of multikey indexes, they will also extract multikey offsets.
-
Vladimir Davydov authored
It's a trivial one-line function, which can be folded without hurting readability, i.e. it only obfuscates the code. Let's kill it.
-
Vladimir Davydov authored
For aesthetic purposes. No functional changes.
-
Vladimir Davydov authored
In the next patch I'm planning to introduce the concept of vy_entry, which will encapsulate a statement stored in a container. Let's rename vy_cache_entry to vy_cache_node so as not to mix the two concepts.
-
Vladimir Davydov authored
So as not to include heavy key_def.h when we only need hint_t.
-
Kirill Shcherbatov authored
The msgpack dependency has been updated because the new version introduces the new method mp_stack_top for the mp_stack class which we will use to store a pointer for a multikey frame to initialize a field_map in case of multikey index. As the library API has changed, the code has been modified correspondingly. @locker: add missing frame update in vy_stmt_new_surrogate_delete. Needed for #1012
-
Serge Petrenko authored
Improve row printing to log. Since say only has 16k buffer, there is no point in printing the whole packet, which can have arbitrary length, in one go. So, print the header row by row, 16 bytes in a row, and format output to match `xxd` output: ``` [001] 2019-04-05 18:22:46.679 [11859] iproto V> Got a corrupted row: [001] 2019-04-05 18:22:46.679 [11859] iproto V> 00000000: A3 02 D6 5A E4 D9 E7 68 A1 53 8D 53 60 5F 20 3F [001] 2019-04-05 18:22:46.679 [11859] iproto V> 00000010: D8 E2 D6 E2 A3 02 D6 5A E4 D9 E7 68 A1 53 8D 53 ``` Now we can get rid of malloc, and use a preallocated tt_static_buf instead. Also, replace a big macro with a small macro and a helper function. Followup to f645119f
-
Vladimir Davydov authored
This reverts commit 8be593ce. Now, as the use-after-free bug in space_truncate() implementation has been fixed, we can enable this test again. Follow-up #4093
-
Vladimir Davydov authored
space_truncate allocates a statement on the stack which is grossly incorrect as the stack may be purged once the function returns while box_process_rw expects the statement to be valid until the end of the transaction. By happy accident, it worked fine until commit 1f7b0d65 ("Require for single statement not autocommit in case of ddl"), which made it possible to run this function from a transaction and hence increased the probability of hitting the use-after-free bug. The fix is trivial: allocate a truncation statement on the region. Fixes commit 353bcdc5 ("Rework space truncation"). Closes #4093
-
Alexander Turenko authored
This commit enables pretest_clean test-run option on 'core = tarantool' test suites with Lua tests and 'core = app' test suites. Consider #4094 for an example of a problem that is eliminated by this option. For 'core = tarantool': this option drops non-system spaces, drops data in system spaces and global variables to the initial state, unloads packages except build-in ones. For 'core = app': this option deletes xlog and snap files before run a test. test-run doesn't remove global variables that are listed in the 'protected_globals' global variable. Use it for, say, functions that are defined in an instance file and called from tests. Consider test-run/README.md for the information how exactly the option works. Removed unused cfg_filter() function from test/engine/box.lua. Fixes #4094.
-
- Apr 05, 2019
-
-
Alexander Turenko authored
This reverts commit 14a87bb7. The test cases generate corrupted xlog files (see #4093) and don't allow other tests to proceed successfully, so we need to temporary disable these cases. They should be enabled back in the scope of #4093.
-
Alexander Turenko authored
* Added default timeout for wait_cond() (60 sec). * Updated pyyaml version in requirements.txt. * Fixed reporting of non-default server fail at start. * Stop 'proxy' when a new non-default instance fails. * Added user-defined protected globals for pretest_clean.
-
Nikita Pettik authored
In SQL type of constant literal (e.g. 1, 2.5, 'abc') is assigned right after parsing and saving into struct Expr. Occasionally, type is re-assigned before emitting opcodes to store literal into VDBE memory. What is more, for floating point number type is changed to "integer". This patch fixes this obvious misbehaviour.
-
Vladimir Davydov authored
Using the const qualifier for complex structures like tuple is bad. We already have to cast it to drop the const qualifier now and then, e.g. to increment/decrement the reference counter. We are planning to wrap struct tuple in a helper struct (aka entry) to store it in vinyl containers along with a comparison hint (cache, memory tree, etc). We will be passing this struct by value so we won't be able to retain const qualifier, because in contrast to a const pointer, one must initialize a const struct upon definition. That said, it's time to drop const qualifier of struct tuple everywhere, like we have already done in case of struct key_def and tuple_format.
-
Mergen Imeev authored
These tables won't be used anymore and should be deleted. Note, this patch breaks backward compatibility between 2.1.1 and 2.1.2, but that's okay as 2.1.1 was beta and we didn't recommend anyone to use it. Part of #2843 Follow up #4069
-
Mergen Imeev authored
Currently, the memory for index_id is not allocated in VDBE code in the sql_code_drop_table() and sql_drop_index() functions. This may lead to SEGMENTATION FAULT. Needed for #2843
-
Kirill Shcherbatov authored
Because sql_normalize_name used to be called twice - to estimate the size of the name buffer and to process data querying the UCaseMap object each time performance in SQL felt by 15%. This patch should eliminate some of the negative effects of using ICU for name normalization. Thanks @avtikhon for a bechmark Follow up e7558062
-
- Apr 04, 2019
-
-
Mergen Imeev authored
This patch changes the name of the function used in the port_sql methods and the name of the member of the result of the execution. Follow up #3505
-
Vladislav Shpilevoy authored
It takes random time since SWIM uses random members selection for each round step. Appeared, that in one of tests choosen time was too small sometimes. Now it is fixed by just increasing timeout, but one of forthcoming patches reworks that test in order to make it faster. Follow up for f510dc6f
-
Serge Petrenko authored
Add row hex printing to log on verbose level. This would be useful during investigation of errors related to invalid msgpack packet arrival. Here's some example output: ``` 2019-04-01 16:04:51.629 [31304] iproto V> Corrupted row is: 3D A5 41 55 99 18 A8 08 C2 40 2B BA 50 12 F6 C6 0B 27 66 1C 2019-04-01 16:04:51.629 [31304] iproto xrow.c:133 E> ER_INVALID_MSGPACK: Invalid MsgPack - packet header ``` Related to #4040
-
Vladislav Shpilevoy authored
Failure detection components allows to find which members are already dead. It works via pings attached to each round message. A member, received ping, should respond with an ack. If it is not done within a timeout, then the ping is considered unacknowledged. When a number of pings grows to a threshold, the member is declared DEAD. After more unacked pings it is dropped from the member table. Unless the GC was turned off. Part of #3234
-
- Apr 03, 2019
-
-
Alexander Turenko authored
The function allows to create a tuple with specific tuple format in C code using a Lua table, another tuple, or objects on a Lua stack. Needed for #3276, #3398, #4025
-
Kirill Shcherbatov authored
Due to the fact that the allocation of offset_slot in the case of multikey indexes will become more complicated and will be necessary for intermediate nodes of the tuple_field tree, we must move this logic to the tuple_format_add_field that performs an intermediate nodes allocation for a JSON path. Needed for #1257
-
Kirill Shcherbatov authored
Introduced a new procedure json_path_multikey_offset. This helper scans the JSON path string and returns the offset of the first character [*] (the array index placeholder). We need this function in the scope of the multikey index patchset to extract the number of keys to be inserted into the index using JSON subpath that has json_path_multikey_offset() length. Needed for #1257
-
Kirill Shcherbatov authored
This patch is inspired by commit 829c811c ("tuple_compare: cleanup key_def virtual func setter") and attempts to simplify setting appropriate tuple_extract_key pointer for plain and json indexes in key_def_set_extract_func routine. Being split to plain and json blocks this code becomes easier to understand and extend. In further patches we need to introduce is_multikey branch and without this refactoring required amendments turn the key_def_set_extract_func code into a mess. Needed for #1257
-
Alexander Turenko authored
A problem appears when tests are run in the following order on one test-run worker: - sql-tap/select6.test.lua - sql-tap/gh-4077-iproto-execute-no-bind.test.lua The latter one fails with the following error: > Space 'T' already exists This test does not perform require('sqltester'), where *.{xlog,snap} files from a previous test are removed, and so the test can observe a non-clean state of a database. Now test-run has pretest_clean option that do exactly what sqltester had been doing: when the option enabled test-run removes *.{xlog,snap} files before run a 'core = app' test. The only difference that now these files will be removed for every test in sql-tap test suite, not only for ones that use sqltester. Follows up #4077.
-
Alexander Turenko authored
The test was introduced in a99d7a0c ('Feature request for a new collation'). It succeeds when tarantool is linked with a fresh libicu version, but fails with older ones. At least: * dev-libs/icu-63.1-r1 (Gentoo) -- passes; * libicu.x86_64 50.1.2-15.el7 (CentOS 7.4) -- fails. We need to handle different behaviours of libicu versions somehow, maybe add only stable subset of collations. This will also fix the test and then it will be enabled back. Related to #4007.
-
- Apr 02, 2019
-
-
Vladislav Shpilevoy authored
SWIM's fake file descriptors were implemented to test SWIM with virtual time and fully controlled network with immediate packet delivery. One of their features - API to block a file descriptor and test various failures about it. But blocked fake fd looks the same as closed fd, and it can confuse new test's author. Now if an fd is not unblocked at the end of a test, it leads to a crash. This commit fixes that via adding explicit difference between blocked and closed fd. Part of #3234
-
Vladislav Shpilevoy authored
At least for testing it is necessary to be able to get status of a member. Now it is always 'alive', but forthcoming failure-detection component would change it. Part of #3234
-