- Apr 07, 2020
-
-
Nikita Pettik authored
This patch introduces support of stacked errors in IProto protocol and in net.box module. Closes #1148 @TarantoolBot document Title: Stacked error diagnostic area Starting from now errors can be organized into lists. To achieve this Lua table representing error object is extended with .prev field and e:set_prev(err) method. .prev field returns previous error if any exist. e:set_prev(err) method expects err to be error object or nil and sets err as previous error of e. For instance: ``` e1 = box.error.new({code = 111, reason = "cause"}) e2 = box.error.new({code = 111, reason = "cause of cause"}) e1:set_prev(e2) assert(e1.prev == e2) -- true ``` Cycles are not allowed for error lists: ``` e2:set_prev(e1) - error: 'builtin/error.lua: Cycles are not allowed' ``` Nil is valid input to :set_prev() method: ``` e1:set_prev(nil) assert(e1.prev == nil) -- true ``` Note that error can be 'previous' only to the one error at once: ``` e1:set_prev(e2) e3:set_prev(e2) assert(e1.prev == nil) -- true assert(e3.prev == e2) -- true ``` Setting previous error does not erase its own previous members: ``` -- e1 -> e2 -> e3 -> e4 e1:set_prev(e2) e2:set_prev(e3) e3:set_prev(e4) e2:set_prev(e5) -- Now there are two lists: e1->e2->e5 and e3->e4 assert(e1.prev == e2) -- true assert(e2.prev == e5) -- true assert(e3.prev == e4) -- true ``` Alternatively: ``` e1:set_prev(e2) e2:set_prev(e3) e3:set_prev(e4) e5:set_prev(e3) -- Now there are two lists: e1->e2 and e5->e3->e4 assert(e1.prev == e2) -- true assert(e2.prev == nil) -- true assert(e5.prev == e3) -- true assert(e3.prev == e4) -- true `` Stacked diagnostics is also supported by IProto protocol. Now responses containing errors always (even if there's only one error to be returned) include new IProto key: IPROTO_ERROR_STACK (0x51). So, body corresponding to error response now looks like: ``` MAP{IPROTO_ERROR : string, IPROTO_ERROR_STACK : ARRAY[MAP{ERROR_CODE : uint, ERROR_MESSAGE : string}, MAP{...}, ...]} ``` where IPROTO_ERROR is 0x31 key, IPROTO_ERROR_STACK is 0x52, ERROR_CODE is 0x01 and ERROR_MESSAGE is 0x02. Instances of older versions (without support of stacked errors in protocol) simply ignore unknown keys and still rely only on IPROTO_ERROR key.
-
Nikita Pettik authored
This patch makes box.error() always promote error to the diagnostic area despite of passed arguments. Closes #4829 @TarantoolBot document Title: always promote error created via box.error() to diag box.error() is able to accept two types of argument: either pair of code and reason (box.error{code = 555, reason = 'Arbitrary message'}) or error object (box.error(err)). In the first case error is promoted to diagnostic area, meanwhile in the latter - it is not: ``` e1 = box.error.new({code = 111, reason = "cause"}) box.error({code = 111, reason = "err"}) - error: err box.error.last() - err box.error(e1) - error: cause box.error.last() - err ``` From now box.error(e1) sets error to diagnostic area as well: ``` box.error(e1) - error: cause box.error.last() - cause ```
-
Kirill Shcherbatov authored
Refactor iproto_reply_error and iproto_write_error with a new mpstream-based helper mpstream_iproto_encode_error that encodes error object for iproto protocol on a given stream object. Previously each routine implemented an own error encoding, but with the increasing complexity of encode operation with following patches we need a uniform way to do it. The iproto_write_error routine starts using region location to use region-based mpstream. It is not a problem itself, because errors reporting is not really performance-critical path. Needed for #1148
-
Nikita Pettik authored
-
Nikita Pettik authored
Since we've introduced stacked diagnostic in previous commit, let's use it in the code implementing functional indexes. Part of #1148
-
Nikita Pettik authored
In terms of implementation, now struct error objects can be organized into double-linked lists. To achieve this pointers to the next and previous elements (cause and effect correspondingly) have been added to struct error. It is worth mentioning that already existing rlist and stailq list implementations are not suitable: rlist is cycled list, as a result it is impossible to start iteration over the list from random list entry and finish it at the logical end of the list; stailq is single-linked list leaving no possibility to remove elements from the middle of the list. As a part of C interface, box_error_add() has been introduced. In contrast to box_error_set() it does not replace last raised error, but instead it adds error to the list of diagnostic errors having already been set. If error is to be deleted (its reference counter hits 0 value) it is unlinked from the list it belongs to and destroyed. Meanwhile, error destruction leads to decrement of reference counter of its previous error and so on. To organize errors into lists in Lua, table representing error object in Lua now has .prev field (corresponding to 'previous' error) and method :set_prev(e). The latter accepts error object (i.e. created via box.error.new() or box.error.last()) and nil value. Both field .prev and :set_prev() method are implemented as ffi functions. Also note that cycles are not allowed while organizing errors into lists: e1 -> e2 -> e3; e3:set_prev(e1) -- would lead to error. Part of #1148
-
Tim Curtis authored
Found that 'openssl-1.1.0h' is EOL and its external link was removed. Fixed static build based on Dockerfile with the new link to the external openssl version 'openssl-1.1.1f'. Close #4830
-
- Apr 06, 2020
-
-
Vladislav Shpilevoy authored
-
- Apr 02, 2020
-
-
Leonid authored
@Changelog Fixed the tarantoolctl rocks remove flag --force Forwarding of the --force flag to tarantoolctl rocks module was added. (Command: tarantoolctl rocks remove --force) Fixes: #3632
-
Leonid authored
@Changelog Fixed the tarantoolctl rocks search flag --all Forwarding of the --all flag to tarantoolctl rocks module was added. (Command: tarantoolctl rocks search --all) Fixes: #4529
-
Alexander V. Tikhonov authored
Fixed static build with '-DBUILD_STATIC=ON' option: - installed liblzma-dev library for libunwind static, due to found that static libunwind library uses undefined lzma functions: nm -a /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a | grep lzma U lzma_index_buffer_decode U lzma_index_end U lzma_index_size U lzma_index_uncompressed_size U lzma_stream_buffer_decode U lzma_stream_footer_decode while dynamic libunwind correctly sees liblzma installed: ldd /usr/lib/x86_64-linux-gnu/libunwind-x86_64.so | grep lzma liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8fd1c23000) so to fix it the static library of lzma was needed. - added lzma library to unwind library for Tarantool build at file: cmake/compiler.cmake due to fail: /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o): In function `xz_uncompressed_size': ./src/elfxx.c:194: undefined reference to `lzma_stream_footer_decode' ./src/elfxx.c:201: undefined reference to `lzma_index_buffer_decode' ./src/elfxx.c:205: undefined reference to `lzma_index_size' ./src/elfxx.c:210: undefined reference to `lzma_index_end' ./src/elfxx.c:207: undefined reference to `lzma_index_uncompressed_size' ./src/elfxx.c:210: undefined reference to `lzma_index_end' /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o): In function `_Uelf64_extract_minidebuginfo': ./src/elfxx.c:278: undefined reference to `lzma_stream_buffer_decode' collect2: error: ld returned 1 exit status test/unit/CMakeFiles/luaL_iterator.test.dir/build.make:134: recipe for target 'test/unit/luaL_iterator.test' failed make[2]: *** [test/unit/luaL_iterator.test] Error 1 - added dl library to gomp library for test/unit tests binaries builds at file: cmake/BuildMisc.cmake due to fail: /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d): more undefined references to `dlsym' follow /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function `gomp_target_init': (.text+0x9cc): undefined reference to `dlerror' collect2: error: ld returned 1 exit status - added dl library to icu library for test/unit tests binaries builds at file: cmake/FindICU.cmake due to fail: /usr/x86_64-linux-gnu/libicuuc.a(putil.ao): In function `uprv_dl_open_60': (.text+0x1ce2): undefined reference to `dlopen' /usr/x86_64-linux-gnu/libicuuc.a(putil.ao): In function `uprv_dlsym_func_60': (.text+0x1d3d): undefined reference to `dlsym' /usr/x86_64-linux-gnu/libicuuc.a(putil.ao): In function `uprv_dl_close_60': (.text+0x1d21): undefined reference to `dlclose' collect2: error: ld returned 1 exit status Added static build to gitlab-ci in release check criteria named as static_build job. Previously named static_build job renamed to static_docker_build, due to it checks the build at Dockerfile. Also moved static build make targets from .gitlab.mk to .travis.mk to store it in common place with the other test/build make targets. Moved environement from .gitlab-ci.yml file into make targets to make this targets true building in static w/o additional setup. Close #4551
-
Alexander V. Tikhonov authored
Found that wget may fail on downloading the file from flaky available servers with 500 HTTP error, like it was seen on icu4c sources downloading, please check the issue: https://sourceforge.net/p/forge/site-support/20071/ Found that curl successfully downloads needed files even in such situations. Decided to use curl instead of wget tool to avoid of such errors feather. Also found that sourceforge site too often has issues and responds with 500 HTTP error. Decided to use the link from github instead of sourceforge to download the icu4c sources, as suggested on icu4c web site.
-
Alexander V. Tikhonov authored
Added the command for 'build' directory cleanup. It purges all artefacts produced for curl build, including the old configuration in build/curl.
-
Alexander V. Tikhonov authored
Set Dockerfile WORKDIR from the very start of Tarantool sources builds to make the Dockerfile code more readable and removed all duplicating calls to Tarantool sources directory changes. Check: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#workdir
-
Alexander V. Tikhonov authored
Removed LD_LIBRARY_PATH environment from curl build, due to the path is empty in the docker image and no other commands from the Dockerfile filled it, seemed that it was not needed.
-
Alexander V. Tikhonov authored
Added '-j' option to make for tools buildings, it decreased build time in 4 times.
-
Alexander V. Tikhonov authored
The change enables memory leaks detection to existing ASAN testing routine and introduces suppression files with the corresponding exception list: * address sanitizer for compile-time: asan/asan.supp * memory leak sanitizer for run-time: asan/lsan.supp Furthermore, added engine and replication suites for ASAN testing routine. Additionally to the tests blacklisted within #4359, 'box/on_shutdown.test.lua' is also disabled since it fails the introduced leak check. All blacklisted tests have to be enabled within #4360. Close #2058
-
Alexander V. Tikhonov authored
All local connection timeout settings not related to the testing scenario are removed within this change. Instead of removed values the default one from src/box/lua/load_cfg.lua will be used. The approach with a single default value helps to avoid flaky test results regarding different timeout values and makes the future maintainence easier. The change is required for LSAN and ASAN testing machinery since it introduces a little overhead and cause failures for tests with excessively strict time limits. Needed for #2058
-
Alexander V. Tikhonov authored
Switched OSX 10.14 test jobs to MAC mini hosts. Renamed Gitlab make rule target 'test_%_no_deps' to 'test_%', which is used to pass all appropriate target calls to Travis make rule target.
-
- Mar 29, 2020
-
-
Alexander Turenko authored
Added popen build-in module to exclusions of pretest_clean script. It'll allow to test the module from a test of 'core = tarantool' type. Part of #4031
-
- Mar 27, 2020
-
-
Nikita Pettik authored
To achieve this let's refactor luaT_error_create() to return error object instead of setting it via box_error_set(). luaT_error_create() is used both to handle box.error() and box.error.new() invocations, and box.error() is still expected to set error to diagnostic area. So, luaT_error_call() which implements box.error() processing at the end calls diag_set_error(). It is worth mentioning that net.box module relied on the fact that box.error.new() set error to diagnostic area: otherwise request errors don't get to diagnostic area on client side. Needed for #1148 Closes #4778 @TarantoolBot document Title: Don't promote error created via box.error.new to diagnostic area Now box.error.new() only creates error object, but doesn't set it to Tarantool's diagnostic area: ``` box.error.clear() e = box.error.new({code = 111, reason = "cause"}) assert(box.error.last() == nil) --- - true ... ``` To set error in diagnostic area explicitly box.error.set() has been introduced. It accepts error object which is set as last system error (i.e. becomes available via box.error.last()). Finally, box.error.new() does not longer accept error object as an argument (this was undocumented feature). Note that patch does not affect box.error(), which still pushes error to diagnostic area. This fact is reflected in docs: ''' Emulate a request error, with text based on one of the pre-defined Tarantool errors... '''
-
- Mar 26, 2020
-
-
Vladislav Shpilevoy authored
File descriptor was set to -1 regardless of whether the object was closed properly. As a result, in case of an error the descriptor would leak. GC finalizer of a descriptor is left intact not to overcomplicate it.
-
Vladislav Shpilevoy authored
swim object created a new fiber in its GC function, because C function swim_delete() yields, and can't be called from an ffi.gc hook. It is not needed since the fiber module has a single worker exactly for such cases. The patch uses it. Follow up #4727
-
Vladislav Shpilevoy authored
Fio.open() returned a file descriptor, which was not closed automatically after all its links were nullified. In other words, GC didn't close the descriptor. This was not really useful, because after fio.open() an exception may appear, and user needed to workaround this to manually call fio_object:close(). Also this was not consistent with io.open(). Now fio.open() object closes the descriptor automatically when GCed. Closes #4727 @TarantoolBot document Title: fio descriptor is closed automatically by GC fio.open() returns a descriptor which can be closed manually by calling :close() method, or it will be closed automatically, when it has no references, and GC deletes it. :close() method existed always, auto GC was added just now. Keep in mind, that the number of file descriptors is limited, and they can end earlier than GC will be triggered to collect not used descriptors. It is always better to close them manually as soon as possible.
-
Vladislav Shpilevoy authored
fiber._internal.schedule_task() is an API for a singleton fiber worker object. It serves for not urgent delayed execution of functions. Main purpose - schedule execution of a function, which is going to yield, from a context, where a yield is not allowed. Such as an FFI object's GC callback. It will be used by SWIM and by fio, whose destruction yields, but they need to use GC finalizer, where a yield is not allowed. Part of #4727
-
Alexander V. Tikhonov authored
Fixed OSX host setup for Tarantool build: - set brew installation from Homebrew repository instructions; - set in use Python 2 latest commit from tapped local formula, since Python 2 is EOL, also removed extra pip installation with get-pip script, because tapped formula installs pip itself. python@2 was deleted from homebrew/core in commit 028f11f9e: python@2: delete (https://github.com/Homebrew/homebrew-core/issues/49796) EOL 1 January 2020. Tapped formula created from the latest formula before its removal: git -C "$(brew --repo homebrew/core)" show 028f11f9e^:Formula/python@2.rb - added upgrade packages call to avoid of fails on already installed packages, but with previous version; - fixed the gitlab-ci configuration for sudo on testing hosts and removed pip option '--user' to avoid of use the users paths with special setup for it. Fixed OSX host setup for Tarantool test: - set maximum processes limit value to 2500 for testing process; - new Mac machines are going to be added into CI and usernames on them are long according to internal policies. It makes a home directory to be long too and so a path to a unix socket created during testing can be longer then UNIX_PATH_MAX=108 constant which is described as issue https://github.com/tarantool/tarantool/issues/4634 To avoid of it the short working directory for testing set by option: --vardir /tmp/tnt
-
Nikita Pettik authored
box.error.set(err) sets err to instance's diagnostics area. Argument err is supposed to be instance of error object. This method is required since we are going to avoid adding created via box.error.new() errors to Tarantool's diagnostic area. Needed for #1148 Part of #4778
-
Nikita Pettik authored
We are going to introduce more tests related to error module, so let's move all error-related tests from box/misc.test.lua to a separate test file (box/error.test.lua). Needed for #1148
-
Kirill Shcherbatov authored
Let's rename diag_add_error() to diag_set_error() because it actually replaces an error object in diagnostic area with a new one and this name is not representative. Moreover, we are going to introduce a new diag_add_error() which will place error at the top of stack diagnostic area. Needed for #1148
-
Kirill Shcherbatov authored
Part of #1148
-
Cyrill Gorcunov authored
Testing via plain C interface with a shell is not stable, the shell might simply be misconfigured or not found and we will simply stuck forever (the signal handling in libev is tricky and requires at least idle cycles or similar to pass event processing). Thus lets rather run a program we know is presenting in the system (popen-child executable). Fixes #4811 Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
Cyrill Gorcunov authored
In case of direct execute without using a shell there is no need to require a caller to allocate redundant space, lets pass executable name in first argument. Since this is yet testing api we're allowed to change without breaking aything. Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
- Mar 25, 2020
-
-
Sergey Bronnikov authored
-
- Mar 23, 2020
-
-
Timur Safin authored
SO_LINGER makes no much sense for unix-sockets, and Microsoft WSL is returning EINVAL if setsockopts called for SO_LINGER over unix sockets: [004] 2020-03-11 18:42:29.592 [29182] main/102/app sio.c:169 !> SystemError setsockopt(SO_LINGER), called on fd 16, aka [004] 2020-03-11 18:42:29.592 [29182] main/102/app F> can't initialize storage: setsockopt(SO_LINGER), called on fd 16, [004] 2020-03-11 18:42:29.592 [29182] main/102/app F> can't initialize storage: setsockopt(SO_LINGER), called on fd 16, And it's sort of correct here, but the problem is Linux is simply silently ignoring it, which passes tests. After much debates we decided to work-around this case via CMAKE define. NB! In a future (April/May 2020), when WSL2 with full Linux kernel would be released we should disable this check. Acked-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
- Mar 20, 2020
-
-
Nikita Pettik authored
vy_build_insert_tuple() processes insertion into secondary indexes being created. It contains yield points during which in-memory level of LSM tree may change (for example rotate owing to triggered dump). So after yield point it is required to fetch from LSM struct pointer to mem again to operate on valid metadata. This patch updates pointer to mem after mentioned yield point. Closes #4810
-
Kirill Yukhin authored
Cleaned up the tests code according to the Lua style guide: - made scoped variables local - added os.exit call at the end of the test chunk - adjusted the messages in test:ok calls
-
Cyrill Gorcunov authored
Currently the journal provides only one method -- write, which implies a callback to trigger upon write completion (in contrary with 1.10 series where all commits were processing in synchronous way). Lets make difference between sync and async writes more notable: provide journal::write_async method which runs completion function once entry is written, in turn journal:write handle transaction in synchronous way. Redesing notes: 1) The callback for async write set once in journal creation. There is no need to carry callback in every journal entry. This allows us to save some memory; 2) txn_commit and txn_commit_async call txn_rollback where appropriate; 3) no need to call journal_entry_complete on sync writes anymore; 4) wal_write_in_wal_mode_none is too long, renamed to wal_write_none; 5) wal engine use async writes internally but it is transparent to callers. Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
Cyrill Gorcunov authored
This makes code easier to read and allows to reuse txn allocation in sync\async writes. Acked-by:
Konstantin Osipov <kostja.osipov@gmail.com> Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
Cyrill Gorcunov authored
Otherwise we won't be able to make a rollback in case of journal_entry_new allocation failure. Acked-by:
Konstantin Osipov <kostja.osipov@gmail.com> Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
Cyrill Gorcunov authored
We gonna diverge sync and async code flow thus lets make txn_commit to not use txn_commit_async. Fixes #4776 Acked-by:
Konstantin Osipov <kostja.osipov@gmail.com> Signed-off-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-