- Aug 28, 2019
-
-
Konstantin Osipov authored
A follow up on gh-4305, fix failing args.test.py
-
Konstantin Osipov authored
-
- Aug 27, 2019
-
-
Max Melentiev authored
To make it possible to develop and test related features on systems without systemd. WITH_SYSTEMD cmake flag is used to generate systemd related files: unit, generator script, etc. To keep this behavior and make it possible to use NOTIFY_SOCKET without other systemd-related stuff, I added WITH_NOTIFY_SOCKET cmake flag. It also required some changes to support other OS: SOCK_CLOEXEC (not available on macOS) flag for socket() is replaced with `fcntl(fd, F_SETFD, FD_CLOEXEC)` which has the same effect. MSG_NOSIGNAL flag for sendmsg is also not available on macOS. However it has SO_NOSIGPIPE flag for setsockopt which disables SIGPIPE. So it requires different solution for different OS. Inspired by https://nwat.xyz/blog/2014/01/16/porting-msg_more-and-msg_nosigpipe-to-osx/ Have to reduce send-buffer size to 4MB because larger values are not supported on macOS by default. This value should be enough for all systems because notification messages are usually less than 1KB. Fixes #4436
-
Alexander Turenko authored
System-wide dynamic libraries usually (always?) have NEEDED and RUNPATH tags in a dynamic section (as `readelf -d /usr/lib/lib<...>.so` shows), so when we link, say, with libssl.so, which depends on libz.so, a linker does not complain against unresolved symbols that can be found in Z library (if it is installed within a system). Things are different when we linking with a static library. Say, when we linking with libssl.a, which contains an unresolved symbol from Z library, a linker reports an error. It is not possible to store an information where to find unresolved symbols (NEEDED / RUNPATH) in a static library (AFAIK). We depend on three libraries that are depend on Z library: libcurl, libssl and libcrypto (two latter are part of OpenSSL). When one of those libraries is linked statically we should link with libz.so or libz.a (depending on BUILD_STATIC flag). The patch doing exactly this. The patch changes OPENSSL_LIBRARIES variable to fix the issue with static linking of OpenSSL libraries. It also changes CURL_LIBRARIES in the same way, however this does not alter any visible behaviour, because OPENSSL_LIBRARIES is added to CURL_LIBRARIES. The latter change was made to unify the way to choose libraries to link with: it is pure refactoring part. Fixes #4437.
-
Vladislav Shpilevoy authored
Tuple fields can be named, accessed by name, indexed by name, but till this commit field names couldn't be used in update operations. Now it is possible. This patch is a teaser of updates by JSON path. Part of #1261
-
Vladislav Shpilevoy authored
-
Vladislav Shpilevoy authored
Rope will not be used by tuple_update directly in next patches.
-
Vladislav Shpilevoy authored
Rope library stores alloc, split and free functions by pointer, that is a huge slog at performance, as any other virtual function call. There is no reason, why the rope library may not become a template, except historical ones. The patch not only removes virtual functions, but also makes rope deletion almost no-op in case if free() function is not defined. A second motivation point of that patch is that original rope structure was too big. Again, because it stored several pointers to functions. In forthcoming patches on #1261 multiple ropes can be created per each update, so it makes sense to reduce size of this structure.
-
Vladislav Shpilevoy authored
They are always region_aligned_alloc and region pointer. Lets use them always inside tuple_update.c, with no necessity to pass explicitly. The patch is motivated by forthcoming updates by JSON path, which will strongly complicate and perhaps slow down the tuple_update.c code. The present patch as well as some next ones should smooth these problems.
-
Alexander Turenko authored
A manual action is needed after pulling of this commit to actually use the downstream repository instead of the upstream one: sed -i -e 's@https://github.com/curl/curl.git@https://github.com/tarantool/curl.git@' .git/config It is part of our processes to use forked repositories for submodules. Suggested by Konstantin Osipov.
-
Alexander V. Tikhonov authored
Cleaned up the .gitlab-ci.yml file from duplicating code - added templates that storres the needed configuration values for different jobs. Also moved the static_build from 'deploy' tag to 'deploy_test' tag to be sure that tests will not be run under high load.
-
Serge Petrenko authored
This problem is similar to the one fixed in commit 3c6c1cc9 (lua:fix decimal comparison with nil) We should handle box.NULL the same way. Closes #4454
-
Alexander Turenko authored
* Added "fragile" option with a list of tests that are not intended to be run in parallel with others. The format of the list is the same as for "disabled" option (#187, PR #188). * Ensured that non-parallel test suites and fragile tests will be run only when all parallel ones will be finished (PR #188). * Fixed reporting in case of lack of temporary result (PR #172).
-
- Aug 26, 2019
-
-
Alexander Turenko authored
CentOS 6 and FreeBSD 12 implementations of getpwuid() rewind setpwent()-{getpwent()}-endpwent() loop to a start that leads to a hang during pwd.getpwall() invoke. The same is true for a getgrgid() call during setgrent()-{getgrent()}-endgrent() loop. The commit modifies pwd module to avoid getpwuid() calls during passwd database traversal and to avoid getgrgid() calls when traversing groups. The commit also fixes the important regression on CentOS 6 after f5d8331e ('lua: workaround pwd.getpwall() issue on Fedora 29'): tarantool hungs during startup due to added getpwall() call. This made tarantool unusable on CentOS 6 at all. Aside of that the commit fixes another pwd.getgrall() problem: the function gaves password entries instead of group entries. Fixes #4428. Fixes #4447. Part of #4271.
-
- Aug 23, 2019
-
-
Alexander Turenko authored
app-tap.tarantoolctl.test.lua fails after 17df9edf ('tarantoolctl: allow to start instances with delayed box.cfg{}'). The commit fixes the test case that did check that an error is reported if box.cfg() was not called in an instance script. Follows up #4435. Fixes #4448.
-
- Aug 22, 2019
-
-
Yaroslav Dynnikov authored
This change is related to #4391. The objective was to collect additional information about modules, but it's hard to do without changing API. This patch will allow to monkey-patch report generation and achieve the same results without interfering the daemon behavior.
-
Max Melentiev authored
There is a problem with calculating .msg_namelen field of msghdr struct. Instead of .msg_name = &sa, .msg_namelen = sizeof(sa.sun_family) + strlen(sd_unix_path), it must set as .msg_namelen = sizeof(sa) // larger value than current invalid one It works on linux but when I tried to enable this feature for macOS it didn't (maybe because of different order of fields in the struct). Instead of fixing calculation, I've replaced original sendmsg call with sendto, because it's a convenient shortcut which simplifies code and can prevent such mistakes. Required for #4436
-
Max Melentiev authored
`tarantoolctl start` patches box.cfg two times: 1) before the init script to set default values and enforce some others, 2) after the init script to prevent changing a pid_file in runtime. The second patching fails if an init file does not call box.cfg{} before it's finished. This can take a place in apps with managed instances which receive configuration from external server. This patch moves the second patching into the box.cfg wrapper created during the first patching. So the second patching is performed only after box.cfg{} was invoked, so it does not fail anymore. However there is relatively minor flaw for applications that invoke box.cfg{} after init script is finished: `tarantoolctl start` goes to background only when box.cfg{} is called. Though this is not the case for daemon management systems like systemd, as they handle backgrounding on their side Fixes #4435 @TarantoolBot document Title: tarantoolctl allows to start instances without a box.cfg{} tarantoolctl now works for instances without box.cfg{} or with delayed box.cfg{} call. This can be managed instances which receive configuration from external server. For such instances `tarantoolctl start` goes to background when box.cfg{} is called, so it will wait until options for box.cfg are received. However this is not the case for daemon management systems like systemd, as they handle backgrounding on their side.
-
Nikita Pettik authored
This function implements common way of precise comparison between unsigned integer and floating point values (doubles). Currently, it is used in tuple comparators, but we need the same thing in SQL. Hence, let's move it to header containing set of utilities.
-
Nikita Pettik authored
To compare floating point values and integers in SQL functions compare_uint_float() and compare_int_float() are used. Unfortunately, they contain bug connected with checking border case: that's not correct to cast UINT64_MAX (2^64 - 1) to double. Proper way is to use exp2(2^64) or predefined floating point constant. To not bother fixing function which in turn may contain other tricky places, let's use instead already verified double_compare_uint64(). So that we have unified way of integer<->float comparison.
-
Alexander Turenko authored
`apt-get update <...>` fails on Debian Buster on docker_bootstrap goal (see #4331 for the similar issue). Added a description how to change dependencies in .travis.mk.
-
Nikita Pettik authored
Closes #4422 @TarantoolBot document Title: Introduce <WITH ENGINE> clause for CREATE TABLE statement To allow user to specify engine as per table option, CREATE TABLE statement has been extended with optional <WITH ENGINE = engine_name> clause. This clause comes at the end of CREATE TABLE statement. For instance: CREATE TABLE t_vinyl (id INT PRIMARY KEY) WITH ENGINE = 'vinyl'; Name of engine is considered to be string literal ergo should be enclosed in single quotation marks and be lower-cased. Note that engine specified in WITH ENGINE clause overwrites default engine, which is set via 'pragma sql_default_engine'.
-
Nikita Pettik authored
Error logging in engine_find() seems to be redundant: error message is displayed twice (since its callers always push error on the top). Let's remove this duplication.
-
Nikita Pettik authored
Name duplicates are allowed for savepoints (both in our SQL implementation and in ANSI specification). ANSI SQL states that previous savepoint should be deleted. What is more, our doc confirms this fact and says that "...it is released before the new savepoint is set." Unfortunately, it's not true - currently old savepoint remains in the list. For instance: SAVEPOINT t; SAVEPOINT t; RELEASE SAVEPOINT t; RELEASE SAVEPOINT t; -- no error is raised Let's fix this and remove old savepoint from the list.
-
Nikita Pettik authored
This allows us to completely remove SQL specific struct Savepoint and use instead original struct txn_savepoint.
-
Nikita Pettik authored
This procedure is processed in several steps. Firstly, we add name to struct txn_savepoint since we should be capable of operating on named savepoints (which in turn is SQL feature). Still, anonymous (in the sense of name absence) savepoints are also valid. Then, we add list (as implementation of stailq) of savepoints to struct txn: it allows us to find savepoint by its name. Finally, we patch rollback to/release savepoint routines: for rollback tail of the list containing savepoints is cut (but subject of rollback routine remains in the list); for release routine we cut tail including node being released.
-
Nikita Pettik authored
We are going to merge struct psql_txn with struct txn as a part of SQL integration into NoSQL, so let's move counter of deferred foreign key violations directly to struct txn.
-
Serge Petrenko authored
Closes #4413 @TarantoolBot document Title: update operations on decimal fields. tuple:update and space:update now support deicmal operands for arithmetic operations ('+' and '-'). The syntax is as usual: ``` d = box.tuple.new(decimal.new('1')) --- ... d:update{{'+', 1, decimal.new('0.5')}} --- - [1.5] ... ``` Insertion ('!') and assignment ('=') are also supported: ``` a = decimal.new('1') --- ... b = decimal.new('1e10') --- ... c = decimal.new('1e-10') --- ... d = box.tuple.new{5, a, 6, b, 7, c, "string"} --- ... d --- - [5, 1, 6, 10000000000, 7, 0.0000000001, 'string'] ... d:update{{'!', 3, dec.new('1234.5678')}} --- - [5, 1, 1234.5678, 6, 10000000000, 7, 0.0000000001, 'string'] ... d:update{{'=', -1, dec.new('0.12345678910111213')}} --- - [5, 1, 6, 10000000000, 7, 0.0000000001, 0.12345678910111213] When performing an arithmetic operation ('+', '-'), where either the updated field or the operand is decimal, the result will be decimal. When both the updated field and the operand are decimal, the result will, of course, be decimal. ... ```
-
Serge Petrenko authored
Closes #4333 @TarantoolBot document Title: Document decimal field type. Decimals may now be stored in spaces. A corresponding field type is introduced: 'decimal'. Decimal values are also allowed in 'scalar', 'any' and 'number' fields. 'decimal' field type is appropriate for both memtx HASH and TREE indices, as well as for vinyl TREE index. ``` To create an index 'pk' over a decimal field, say ``` tarantool> box.space.test:create_index('pk', {parts={1, 'decimal'}}) --- - unique: true parts: - type: decimal is_nullable: false fieldno: 1 id: 0 space_id: 512 type: TREE name: pk ... ``` Now you can insert some decimal values: ``` tarantool> for i = 1,10 do > box.space.test:insert{decimal.new((i-5)/10)} > end --- ... ``` tarantool> box.space.test:select{} --- - - [-0.4] - [-0.3] - [-0.2] - [-0.1] - [0] - [0.1] - [0.2] - [0.3] - [0.4] - [0.5] ... ``` Decimals may alse be inserted into `scalar` and `number` fields. In this case all the number values are sorted correctly: ``` tarantool> box.schema.space.create('test') tarantool> box.space.test:create_index('pk', {parts={1, 'number'}}) tarantool> box.space.test:insert{-1.0001, 'number'} --- - [-1.0001, 'number'] ... tarantool> box.space.test:insert{decimal.new(-1.00001), 'decimal'} --- - [-1.00001, 'decimal'] ... tarantool> box.space.test:insert{-1, 'number'} --- - [-1, 'number'] ... tarantool> box.space.test:insert{decimal.new(-0.999), 'decimal'} --- - [-0.999, 'decimal'] ... tarantool> box.space.test:insert{-0.998, 'number'} --- - [-0.998, 'number'] ... tarantool> box.space.test:insert{-0.9, 'number'} --- - [-0.9, 'number'] ... tarantool> box.space.test:insert{-0.95, 'number'} --- - [-0.95, 'number'] ... tarantool> box.space.test:insert{decimal.new(-0.92), 'decimal'} --- - [-0.92, 'decimal'] ... tarantool> box.space.test:insert{decimal.new(-0.971), 'decimal'} --- - [-0.971, 'decimal'] ... tarantool> box.space.test:select{} --- - - [-1.0001, 'number'] - [-1.00001, 'decimal'] - [-1, 'number'] - [-0.999, 'decimal'] - [-0.998, 'number'] - [-0.971, 'decimal'] - [-0.95, 'number'] - [-0.92, 'decimal'] - [-0.9, 'number'] ... ``` Uniqueness is also preserved between decimals and other number types: ``` tarantool> box.space.test:insert{-0.92} --- - error: Duplicate key exists in unique index 'pk' in space 'test' ... tarantool> box.space.test:insert{decimal.new(-0.9)} --- - error: Duplicate key exists in unique index 'pk' in space 'test' ... ``` You can also set decimal fields in space format: ``` tarantool> _ = box.schema.space.create('test') --- ... tarantool> _ = box.space.test:create_index('pk') --- ... tarantool> box.space.test:format{{name='id', type='unsigned'}, {name='balance', type='decimal'}} --- ... tarantool> box.space.test:insert{1} --- - error: Tuple field 2 required by space format is missing ... tarantool> box.space.test:insert{1, 'string'} --- - error: 'Tuple field 2 type does not match one required by operation: expected decimal' ... tarantool> box.space.test:insert{1, 1.2345} --- - error: 'Tuple field 2 type does not match one required by operation: expected decimal' ... tarantool> box.space.test:insert{1, decimal.new('1337.420')} --- - [1, 1337.420] ... ```
-
Serge Petrenko authored
Update decNumber library, add methods to convert decimals to uint64_t and int64_t, add unit tests. Also replace decimal_round() function with decimal_round_with_mode() to allow setting rounding mode. We need to round with mode DEC_ROUND_DOWN in to_int64 conversions in order to be consistent with double to int conversions. It will be needed to compute hints for decimal fields. Prerequisite #4333
-
Serge Petrenko authored
This patch adds the methods necessary to encode and decode decimals to MsgPack. MsgPack EXT type (MP_EXT) together with a new extension type MP_DECIMAL is used as a record header. The decimal MsgPack representation looks like this: +--------+-------------------+------------+===============+ | MP_EXT | length (optional) | MP_DECIMAL | PackedDecimal | +--------+-------------------+------------+===============+ The whole record may be encoded and decoded with mp_encode_decimal() and mp_decode_decimal(). This is equivalent to performing mp_encode_extl()/mp_decode_extl() on the first 3 fields and decimal_pack/unpack() on the PackedDecimal field. It is also possible to decode and encode decimals to msgpack from lua, which means you can insert decimals into spaces, but only into unindexed fields for now. Follow up #692 Part of #4333
-
- Aug 21, 2019
-
-
Serge Petrenko authored
Prior to this patch format checking was broken for 'i' (integer) and 'N' (big-endian integer). pickle.pack() rejected negative integers with these formats. Fix this
-
Serge Petrenko authored
When a number having a positive exponent is encoded, the internal decPackedFromNumber function returns a negative scale, which differs from the scale, returned by decimal_scale(). This leads to errors in decoding. Account for negative scale in decimal_pack() and decimal_unpack(). Follow-up #692
-
Serge Petrenko authored
Previously decimal comparison with nil failed with following error: `expected decimal, number or string as 2 argument`. Fix this. Throw a more verbose error in case of '>', '<', '>=', '<=' and fix equality check. Follow-up #692
-
Mergen Imeev authored
Hold libcurl-7.65.3. This version is not affected by the following issues: * #4180 ('httpc: redirects are broken with libcurl-7.30 and older'); * #4389 ('libcurl memory leak'); * #4397 ('HTTPS seem to be unstable'). After this patch libcurl will be statically linked when ENABLE_BUNDLED_LIBCURL option is set. This option is set by default. Closes #4318 @TarantoolBot document Title: Tarantool dependency list was changed * Added build dependencies: autoconf, automake, libtool, zlib-devel (zlib1g-dev on Debian). * Added runtime dependencies: zlib (zlib1g on Debian). * Removed build dependencies: libcurl-devel (libcurl4-openssl-dev on Debian). * Removed runtime dependencies: curl. The reason is that now we use compiled-in libcurl: so we don't depend on a system libcurl, but inherit its dependencies.
-
Alexander Turenko authored
This is a workaround for systemd-nss issue: https://github.com/systemd/systemd/issues/9585 The following error is observed on app-tap/pwd.test.lua on Fedora 29 (glibc-2.28-26.fc29, systemd-239-12.git8bca462.fc29) when tarantool is linked with libcurl w/o GSS-API support: | builtin/pwd.lua:169: getpwall failed [errno 2]: No such file or directory Such tarantool build lacks of libselinux.so.1 transitive dependency (tarantool -> libcurl.so.4 -> libgssapi_krb5.so.2 -> libkrb5support.so.0 -> libselinux.so.1) and strace shows the following calls when pwd.getpwall() is invoked first time: | openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 7A | <...> | access("/etc/selinux/config", F_OK) = -1 ENOENT (No such file or directory) It looks like a part of libselinux initialization code and is invoked during execution of a last ffi.C.getpwent() call that returns `nil` as a result and left errno set to ENOENT. Our pwd module set errno to zero before getpwent() call and expects that it will be preserved if no unrecoverable errors occur. It seems that this expectation is not meet due to the systemd-nss issue linked above. Second and next getpwall() calls will succeed, so the commit adds an extra getpwall() during pwd module load. This workaround is disabled on FreeBSD due to another issue: #4428 ('getpwall() hangs on FreeBSD 12'). See also the previous related commit: efccac69 ('lua: fix error handling in getpwall and getgrall'). Follows up #3766. Part of #4318.
-
Alexander V. Tikhonov authored
Added static build using Dockerfile on Centos 7 for release commit criteria only. Added the cleanup for cmake generating CMakeCache.txt files and CMakeFiles directories to avoid of cmake localy created setup failing inside the docker after the whole tarantool path was copied into it. Added testing into the static build, running only when RUN_TESTS environment variable set to non empty value, used in gitlab-ci job to run the testing after the build. Closes #3668
-
- Aug 20, 2019
-
-
Vladimir Davydov authored
We remove an LSM tree from the scheduler queues as soon as it is dropped, even though the tree may hang around for a while after that, e.g. because it is pinned by an iterator. As a result, once an index is dropped, it won't be dumped anymore - its memory level will simply disappear without a trace. This is okay for now, but to implement snapshot iterators we must make sure that an index will stay valid as long as there's an iterator that references it. That said, let's delay removal of an index from the scheduler queues until it is about to be destroyed.
-
Vladimir Davydov authored
There's no reason to use a special method instead of the generic space_execute_dml for applying rows received from a master during the initial join stage. Moreover, using the special method results in not running space.before_replace trigger, which makes it impossible to, for example, update space engine on a replica, see the on_schema_init test of the replication test suite. So this patch removes the special method altogether and makes the code that used it switch to space_execute_dml. Closes #4417
-
Vladimir Davydov authored
We must enable SMALL_DELAYED_FREE_MODE to safely use a memtx snapshot iterator. Currently, we do that in checkpoint related callbacks, but if we want to reuse snapshot iterators for other purposes, e.g. feeding a read view to a newly joined replica, we better hide this code behind snapshot iterator constructors.
-