- Mar 19, 2019
-
-
Alexander Turenko authored
Fixes #4027.
-
Alexander Turenko authored
SHA-0 is considered weak for years and was removed in OpenSSL since 1.1.0. These Lua functions did not work since 15ed10e4, but give 'Digest method "sha" is not supported' error for any input. Removed them to don't confuse a user with a Tab completion. Follow up #1722. Fixes #4028.
-
Ivan Koptelov authored
Without multi-directional iterators (to be introduced in #3309) ORDER BY + LIMIT with different specified ordered leads to wrong result. So for now (till #3309 is resolved) different sort orders are forbidden in ORDER BY + LIMIT. Close #4038 @TarantoolBot document Title: different sorting orders in ORDER BY are forbidden now The error will be raised if different sorting orders are encountered.
-
Mergen Imeev authored
After this patch all SQL parser errors will be set via diag_set(). They were saved in field zErrMsg of struct Parse before this patch. Part of #3965
-
- Mar 18, 2019
-
-
Alexander Turenko authored
Fixed _func system space clean when pretest_clean is set to True in suite.ini.
-
Vladimir Davydov authored
If tt_static_buf is too small to store the request string, 'pos' will become greater than 'end', leading to snprintf(pos, end - pos) crash, as it doesn't allow the buffer size to be negative. Use SNPRINT instead.
-
Cyrill Gorcunov authored
Fixes commit 553dc562 ("lib/core/fiber: Relax stack memory usage on recycle").
-
Vladimir Davydov authored
So that we notice in case anything goes wrong.
-
Cyrill Gorcunov authored
We want to detect a situation where task in fiber is too eager for stack memory and relax rss usage in such case. For this sake upon stack creation we put 8 marks near 64K bound (such params allows us to fill ~1/4 of a page, which seem reasonable but we might change this params with time). Once stack get recycled we investigate the marks and if they were overwritten we drop all pages behind to relax memory usage (if OS supports madvise syscall). Another important moment is that we're marking the whole stack as not present thus if fiber never stepped over 64K limit the marks will be in tact and it means the fibers are light ones there won't be much #pf in future. Later we plan to implement an intelligent fiber scheduling considering how many memory fibers consume in average. @locker: - fix watermark page alignment for grow-up stack - improve MADV_DONTNEED check - clean up code and elaborate comments - add test case to unit/fiber - fix unit/guard test Follow-up #3418
-
Cyrill Gorcunov authored
The default 64K stack size used for years become too small for modern distors (Fedora 29 and etc) where third party libraries (such as ncurses) started to use 64K for own buffers and we get SIGSGV early without reaching interactive console phase. Thus we increase default size up to 512K which should fit for common case. Later we will make this value configurable to address arbitrary stack sizes without a need to rebuild the whole code. Closes #3418
-
- Mar 15, 2019
-
-
Vladimir Davydov authored
See commit 65061cb4 ("test: don't leave open read views after vinyl/iterator") for more details. Follow-up #3862
-
Vladimir Davydov authored
The patch fixes the following test failure: | --- vinyl/errinj.result Fri Mar 15 19:19:19 2019 | +++ vinyl/errinj.reject Fri Mar 15 19:21:37 2019 | @@ -854,7 +854,8 @@ | ... | ret | --- | -- - [3, 3] | +- - [2, 2] | + - [3, 3] | ... | s:drop() | --- The error occurrs, because the test doesn't make sure that a read from the scondary index actually occurs and the fiber does stall on a read from the primary index. Use index stats to assure that. Closes #3863
-
Alexander V. Tikhonov authored
Fixed Mojave Mac build with setting MACOSX_DEPLOYMENT_TARGET environment variable for LuaJIT's Makefile. This variable specifies the minimum version of OS X on which the target binaries are to be deployed. The reason why we need to set MACOSX_DEPLOYMENT_TARGET to at least 10.6 is that 10.4 SDK (which is set by default in LuaJIT's Makefile) is not longer included in Mac OS X Mojave 10.14. See also https://github.com/LuaJIT/LuaJIT/issues/484 We already set -Wl,-macosx_version_min,10.6 (-macosx_version_min and MACOSX_DEPLOYMENT_TARGET are synonymous), but it affects only a linker. We possibly should remove -macosx_version_min, because it superseded by MACOSX_DEPLOYMENT_TARGET, but it should be done with verification that the original problem really fixed by the new way to set a deployment target. This is not part of this patch. Removed virtualenv usage in CI for OS X (so pip just installs packages into a system), because OS X Mojave 10.14 does not offer virtualenv by default. Fixed #3797
-
Vladimir Davydov authored
The patch fixes the following test failure: | --- vinyl/deferred_delete.result Fri Mar 15 18:17:05 2019 | +++ vinyl/deferred_delete.reject Fri Mar 15 18:18:18 2019 | @@ -577,7 +577,7 @@ | ... | sk:stat().rows -- ditto | --- | -- 5 | +- 25 | ... | s:drop() | --- The failure was introduced by commit 6dd0d2fb ("vinyl: do not apply run_count_per_level to the last level"). Due to the commit compaction of the secondary index may happen before compaction of the primary index, in which case deferred DELETE statements won't make it to the secondary index in time against the test expectation. Fix this by making the first run big enough to prevent major compaction from kicking in. Follow-up #3657 Closes #4047
-
Nikita Pettik authored
According to ANSI[1] specification, LIKE arguments must be of string-like type (VARCHAR, CHAR, TEXT etc). If one of arguments is NULL, then the result of LIKE function is NULL as well. This patch makes LIKE follow these rules. ANSI 2013 Part 2: Foundation; Chapter 8.5 <like predicate> Closes #3954
-
Nikita Pettik authored
We don't rely on this debug facility anymore, so let's remove it.
-
Nikita Pettik authored
We are going to always throw an error if value of BLOB type gets to LIKE arguments, so code under is macro is not needed anymore. Part of #3954
-
Vladimir Davydov authored
The watermark is updated every second anyway, however not updating it when the limit is reset results in vinyl/quota test failure: | --- vinyl/quota.result Thu Mar 14 16:03:54 2019 | +++ vinyl/quota.reject Fri Mar 15 16:32:44 2019 | @@ -146,7 +146,7 @@ | for i = 1, count do s:replace{i, pad} end -- does not trigger dump | --- | ... | -box.stat.vinyl().memory.level0 > count * pad:len() | +box.stat.vinyl().memory.level0 > count * pad:len() or box.stat.vinyl() | --- | - true | ... Closes #3864
-
Mergen Imeev authored
This patch reworks SQL syntax errors. After this patch, these error will be set as Tarantool errors. Part of #3965
-
Kirill Yukhin authored
Fixed GC issue.
-
Vladimir Davydov authored
An open iterator may disrupt the following test run, because it may prevent dump/compaction from purging stale rows. In particular, iterators left by vinyl/iterator result in the following test failure: | --- vinyl/deferred_delete.result Mon Feb 11 19:14:01 2019 | +++ vinyl/deferred_delete.reject Fri Mar 15 16:21:11 2019 | @@ -155,7 +155,7 @@ | ... | pk:stat().rows -- 5 new REPLACEs | --- | -- 5 | +- 10 | ... | i1:stat().rows -- 10 old REPLACE + 5 new REPLACEs + 10 deferred DELETEs | --- Fix this by calling the Lua garbage collector to delete all dangling iterators in the end of vinyl/iterator test. Closes #3862
-
- Mar 14, 2019
-
-
Serge Petrenko authored
The function evio_timeout_update() failed to update the starting time point, which lead to timeouts happening much faster than they should if there were consecutive calls to the function. This lead, for example, to applier timing out while reading a several megabyte-size row in 0.2 seconds even if replication_timeout was set to 15 seconds. Closes #4042
-
- Mar 13, 2019
-
-
Alexander Turenko authored
Run a unit test from a var directory. It is needed to, say, allow a test to write a log file to a gitignored directory. The only behaviour change observed on tarantool's tests is that unit/swim.test and unit/swim_proto.test write a log.txt file to a test var directory ./test/xxx_unit instead of ./test.
-
Sergei Voronezhskii authored
Fixed issues: - box-py/iproto.test.py 1) Fixed receive_response() to wait for whole response. 2) Clean up _cluster space. - replication-py/cluster.test.py 1) Clean up _cluster space. - replication-py/multi.test.py 1) Removed vclock checking because it fails if previous test make some DML and vclock was incremented. Looks like it was used for debug and is not part of this test case. 2) Fixed typo in 'Synchronize' block. The following test sequences did fail due to unexpected IDs in _cluster space: - [box-py/iproto.test.py, null] - [box-py/bootstrap.test.py, null] - [replication-py/cluster.test.py, null] - [replication-py/multi.test.py, null] Part of #3232
-
Sergei Voronezhskii authored
Also enabled it for luajit-tap. Part of #3232
-
Vladimir Davydov authored
It had been used only in Vinyl's vy_stmt_new_surrogate_from_key, which was deleted by the previous patches, so we can drop it as well.
-
Vladimir Davydov authored
There are three places where we use this expensive functions while we could get along with a cheaper one: - Deferred DELETE space on_replace trigger. Here we can use simple vy_stmt_new_delete, because the trigger is already passed a surrogate DELETE statement. - Secondary index build on_replace trigger. Here we can extract the secondary key, set its type to DELETE and insert it into the index. We don't need all the other indexed fields. - Secondary index build recovery procedure. Similarly to the previous case, we can use extracted here rather than building a surrogate DELETE statement.
-
Vladimir Davydov authored
This heavy function isn't needed anymore, as we can now insert key statements into the memory level.
-
Vladimir Davydov authored
In contrast to a primary index, which stores full tuples, secondary indexes only store extended (secondary + primary) keys on disk. To make them look like tuples, we fill missing fields with nulls (aka tuple surrogate). This isn't going to work nicely with multikey indexes though: how would you make a surrogate array from a key? We could special-case multikey index handling, but that would look cumbersome. So this patch removes nulls from secondary tuples restored from disk altogether. To achieve that, it's enough to use key_format for them - then the comparators will detect that it's actually a key, not a tuple and use the appropriate primitive.
-
Vladimir Davydov authored
It's actually only needed to initialize disk streams so let's pass it to vy_write_iterator_new_slice() instead.
-
Vladimir Davydov authored
By convention we have two methods in each write iterator stream implementation (including the write iterator itself as it implements the interface too): 'stop' and 'close'. The 'stop' method is called in a worker thread. It reverses the effect of 'start'. We need it unreference all tuples referenced during the iteration (we must do it in the worker thread, where the tuples were referenced in the first place so as not to unreference tuple formats, see vy_tuple_delete). The 'close' method is called from the tx thread to unreference tuple formats if necessary and release memory. For the write iterator itself we follow this convention. However, for individual sources, for vy_slice_stream source to be more exact, we do not - the write iterator calls both 'stop' and 'close' from its own 'stop method. Let's cleanup this mess and make the write iterator follow the convention. We'll need it in the next patch.
-
Vladimir Davydov authored
Use the format of the given statement instead. Passing format is a legacy from the time when we have a separate format for UPSERTs. Nowadays it only obfuscates the code.
-
Vladimir Davydov authored
A Vinyl statement may be either a key or a tuple. We must use different functions for the two kinds when working with a bloom filter. Let's introduce helpers incorporating that logic. Notes: - Currently, we never add keys to bloom filters, but after the next patch we will, so this patch adds tuple_bloom_builder_add_key helper. - According to the function protocol, tuple_bloom_builder_add may fail with out-of-memory, but we never checked that. Fix that while we are at it.
-
Vladimir Davydov authored
No functional changes, just move a piece of code, so as not to mix it in the next patch.
-
Vladimir Davydov authored
Tuple bloom filter is an array of bloom filters, each of which reflects lookups by all possible partial keys. To optimize the overall bloom filter size, we need to know how many unique elements there are for each partial key. To achieve that, we require the caller to pass the number of key parts that have been hashed for the given tuple. Here's how it looks in Vinyl: uint32_t hashed_parts = writer->last_stmt == NULL ? 0 : tuple_common_key_parts(stmt, writer->last_stmt, writer->key_def); tuple_bloom_builder_add(writer->bloom, stmt, writer->key_def, hashed_parts); Actually, there's no need in such a requirement as instead we can calculate the hash value for the given tuple, compare it with the hash of the tuple added last time, and add the new hash only if the two values differ. This should be accurate enough while allowing us to get rid of the cumbersome tuple_common_key_parts helper. Note, such a check will only work if tuples are added in the order defined by the key definition, but that already holds - anyway, one wouldn't be able to use tuple_common_key_parts either if it wasn't true. While we are at it, refresh the obsolete comment to tuple_bloom_builder.
-
Vladimir Davydov authored
To differentiate between key and tuple statements in comparators, we set IPROTO_SELECT type for key statements. As a result, we can't use key statements in the run iterator directly although secondary index runs do store statements in key format. Instead we create surrogate tuples filling missing fields with NULLs. This won't play nicely with multikey indexes so we need to teach iterators to deal with statements in key format. The first step in this direction is dropping IPROTO_SELECT in favor of identifying key statements by format.
-
Vladimir Davydov authored
Currently, it's called vy_stmt_new_select, but soon a key statement will be allowed to have any type, not just IPROTO_SELECT. So let's rename it to vy_key_new.
-
Vladimir Davydov authored
Store tuple_format_vtab, max_tuple_size, and key_format there. This will allow us to determine a statement type (key or tuple) by checking its format against key_format.
-
Vladimir Davydov authored
A vinyl statement (vy_stmt struct) may represent either a tuple or a key. We differentiate between the two kinds by statement type - we use SELECT for keys and other types for tuples. This was done that way so that we could pass both tuples and keys to a read iterator as a search key. To avoid branching in comparators when the types of compared statements are known in advance, we provide several comparators, each of which expects certain statement types, e.g. a tuple and a key. Actually, such a micro optimization looks like an overkill, because a typical comparator is called by function pointer and has a lot of comparisons in the code, see tuple_compare_slowpath for instance. Eliminating one branch will hardly make the code perform better. At the same time, it makes the code more difficult to write. Besides, once we remove nils from statements read from disk (aka surrogate tuples), which will ease implementation of multikey indexes, the number of places where types of compared statements are known will diminish drastically. That said, let's remove optimized comparators and always use vy_stmt_compare, which checks types of compared statements and calls the appropriate comparator.
-
Vladimir Davydov authored
We advance replica->gc state only when an xlog file is fully recovered, see recovery_close_log and relay_on_close_log_f. It may turn out that an xlog file is fully recovered, but isn't closed properly by relay (i.e. recovery_close_log isn't called), because the replica closes connection for some reason (e.g. timeout). If this happens, the old xlog file won't be removed when the replica reconnects, because we don't advance replica->gc state on reconnect, so the useless xlog file won't be removed until the next xlog file is relayed. This results in occasional replication/gc.test.lua failures. Fix this by updating replica->gc on reconnect with the current replica vclock. Closes #4034
-