- Dec 01, 2021
-
-
Vladimir Davydov authored
1. The net.box state machine is supposed to set E_NO_CONNECTION on error, but netbox_transport_connect returns an error returned by coio_connect and coio_read as is. Fix this. 2. netbox_transport_communicate sets the error message to strerror(errno) on coio_read or coio_write error. This works now, because iostream can only be backed by plain sockets, but once we implement encryption, errno won't make any sense. We should use error::errmsg instead. 3. While we are at it, fix luaT_netbox_transport_make_request: it's wrong to pass errmsg directly to box_error_raise - we should use a format specifier (%s). Fixes commit c13b3a31 ("net.box: rewrite state machine (transport) in C").
-
Vladimir Davydov authored
Currently, we append strerror(errno) when logging those errors while error::errmsg only contains the message passed to the error constructor, which usually points at the system call that failed, e.g. sio_connect() sets it to connect, called on fd 50, aka 127.0.0.1:33728 Errno can shed light at the reason of failure so we'd better append it to errmsg too: connect, called on fd 50, aka 127.0.0.1:33728: Connection refused In particular, this improves error messages returned by the net.box state machine, which was recently switched to diag, see commit c13b3a31 ("net.box: rewrite state machine (transport) in C"). After this patch, we don't need to append strerror(errno) to SystemError message manually anymore so this patch also walks over the whole code base and removes strerror from diag_set calls - there are just two modules that do that, fio and say. Also, this patch slightly changes logged error messages: - A colon (:) is printed after SystemError, because SystemError::log doesn't override Exception::log anymore. This makes SystemError logging consistent with other error logging. - For the same reason we now print the actual error type of errors derived from SystemError (SocketError, OutOfMemory and TimedOut) instead of SystemError to the log (this should be fine), and don't print strerror(errno) for OutOfMemory and TimedOut (it's useless anyway).
-
- Nov 30, 2021
-
-
Vladislav Shpilevoy authored
tt_sprintf, tt_snprintf, and tt_vsnprintf now have compile-time checks for their arguments fitting the format string.
-
Vladislav Shpilevoy authored
Caught by coverity.
-
Vladislav Shpilevoy authored
Error messages tried to printf uint64_t term as %llu, but apparently it does not always work. Needs explicit cast to unsigned long long. Caught by coverity.
-
- Nov 29, 2021
-
-
Yan Shtunder authored
Transactions should be committed after they reach quorum of "real" cluster members. Therefore, anonymous replicas shouldn't participate in the quorum. Closes #5418
-
Yan Shtunder authored
The truncate method could be called from within a transaction. The flag of GROUP_LOCAL was set in truncate method after statement row had been being checked on the GROUP_LOCAL. Accordingly, after a local transaction NOP row was not appended. Closes #6123
-
Vladimir Davydov authored
Fixes commit c13b3a31 ("net.box: rewrite state machine (transport) in C"). Found by Coverity. While we are at it, fix malformed indentation.
-
Vladimir Davydov authored
Fixes commit c13b3a31 ("net.box: rewrite state machine (transport) in C"). Found by Coverity.
-
- Nov 26, 2021
-
-
Yaroslav Lobankov authored
Problem: For now, the integration testing runs only on a push of changes/tags to the 'master' and release branches. This approach is not good enough because we become aware of integration issues after the changes were merged already to the target branch. So this patch adds the facility to run the integration testing on a dev branch to catch integration issues before the changes are merged to the target branch. Now it can be done via naming the branch as `*-full-ci` and pushing it to the main repository or setting the 'full-ci' label on the pull request.
-
Alexander Turenko authored
This commit actually reverts 2.10.0-beta1-136-gaa2a6bc83 ('ci: temporarily disable deploy for CentOS 7 for aarch64'). I built packages necessary for testing for CentOS 7 on aarch64 and placed them into packpack backports [1]. The source packages are from CentOS 8 [2] and are the following: * Cython-0.29.14-4.module_el8.3.0+441+3b561464.src.rpm * python-gevent-1.2.2-4.el8.src.rpm * PyYAML-5.3.1-1.module_el8.3.0+441+3b561464.src.rpm * python-six-1.12.0-9.module_el8.3.0+441+3b561464.src.rpm [1]: https://packagecloud.io/packpack/backports [2]: https://vault.centos.org/8.3.2011/AppStream/Source/SPackages/ If you're curious about steps to re-build the packages for CentOS 7, read below. ---- Build steps are the following: ```sh $ ssh <aarch64 machine> $ docker run -it -v <...>:/src -v <...>:/res centos:7 ``` Install tools: ```sh $ yum install yum-utils rpm-build ``` (Here I also installed epel-release, but I doubt that it is necessary.) Build and install Cython: ```sh $ rpmbuild --rebuild /src/Cpython*.src.rpm $ vim /root/rpmbuild/SPECS/Cython.spec <..fix changelog dates..> $ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec <...> error: Failed build dependencies: gcc is needed by Cython-0.29.14-4.el7.aarch64 python3-devel is needed by Cython-0.29.14-4.el7.aarch64 python3-setuptools is needed by Cython-0.29.14-4.el7.aarch64 python3-rpm-macros is needed by Cython-0.29.14-4.el7.aarch64 $ yum install gcc python3-devel python3-rpm-macros python3-setuptools $ rpmbuild -bb /root/rpmbuild/SPECS/Cython.spec # repeat, yep $ ls -1 /root/rpmbuild/RPMS/aarch64/ Cython-debuginfo-0.29.14-4.el7.aarch64.rpm python3-Cython-0.29.14-4.el7.aarch64.rpm $ cp /root/rpmbuild/RPMS/aarch64/* /res/ $ rpm -i /root/rpmbuild/RPMS/aarch64/* ``` Build python-gevent: ```sh $ rpmbuild --rebuild /src/python-gevent*.src.rpm <...> error: Failed build dependencies: c-ares-devel is needed by python-gevent-1.2.2-4.el7.aarch64 libev-devel is needed by python-gevent-1.2.2-4.el7.aarch64 python2-devel is needed by python-gevent-1.2.2-4.el7.aarch64 $ yum install c-ares-devel libev-devel # but not python2-devel $ vim /root/rpmbuild/SPECS/python-gevent.spec <..leave '%bcond_with python2'..> $ rpmbuild -bb /root/rpmbuild/SPECS/python-gevent.spec $ ls -1 /root/rpmbuild/RPMS/aarch64/*gevent* /root/rpmbuild/RPMS/aarch64/python-gevent-debuginfo-1.2.2-4.el7.aarch64.rpm /root/rpmbuild/RPMS/aarch64/python3-gevent-1.2.2-4.el7.aarch64.rpm $ cp /root/rpmbuild/RPMS/aarch64/*gevent* /res/ ``` Build PyYAML: ```sh $ rpmbuild --rebuild /src/PyYAML*.src.rpm <...> error: Failed build dependencies: libyaml-devel is needed by PyYAML-5.3.1-1.el7.aarch64 $ yum install libyaml-devel $ rpmbuild --rebuild /src/PyYAML*.src.rpm # again, yep $ ls -1 /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}* /root/rpmbuild/RPMS/aarch64/PyYAML-debuginfo-5.3.1-1.el7.aarch64.rpm /root/rpmbuild/RPMS/aarch64/python3-pyyaml-5.3.1-1.el7.aarch64.rpm $ cp /root/rpmbuild/RPMS/aarch64/*{pyyaml,PyYAML}* /res/ ``` Build python-six: ```sh $ rpmbuild --rebuild /src/python-six*.src.rpm error: Failed build dependencies: python3-wheel is needed by python-six-1.12.0-9.el7.noarch python3-pytest is needed by python-six-1.12.0-9.el7.noarch python3-tkinter is needed by python-six-1.12.0-9.el7.noarch $ yum install python3-wheel python3-pytest python3-tkinter $ rpmbuild --rebuild /src/python-six*.src.rpm # again, yep error: Failed build dependencies: python3-pytest is needed by python-six-1.12.0-9.el7.noarch $ vim /root/rpmbuild/SPECS/python-six.spec <..change '%bcond_without tests' to '%bcond_with tests'..> $ rpmbuild -bb /root/rpmbuild/SPECS/python-six.spec $ ls -1 /root/rpmbuild/RPMS/noarch/ python3-six-1.12.0-9.el7.noarch.rpm $ cp /root/rpmbuild/RPMS/noarch/* /res/ ```
-
Georgiy Lebedev authored
If 'box.cfg' fails with error, 'log.cfg' still gets updated: instead, update 'log.cfg' after verifying 'box.cfg', and rollback if update of 'log.cfg' fails. Closes #6086
-
Georgiy Lebedev authored
box_api_cfg_set_log_{level, format} are part of dynamic_cfg (see box/lua/load_cfg.lua) and are expected to throw exceptions (see reload_cfg ib.). Needed for #6086
-
Georgiy Lebedev authored
On big instances core produces too much logs related to shapshotting, which feels redudant: reduce snapshot verbosity. Closes #6220
-
Georgiy Lebedev authored
Make say_ratelimit_check emit warnings about suppressed messages only if the correspoding log level is greater or equal then WARN. Add say_crit_ratelimited and say_info_ratelimited macros for convenience. Needed for #6220
-
- Nov 24, 2021
-
-
Igor Munkin authored
* memprof: add info about trace start to symtab * memprof: group allocations on traces by traceno * memprof: refactor location parsing * test: separate memprof Lua API tests into subtests Closes #5814
-
Mergen Imeev authored
This patch introduces MAP to SQL. After this patch, all SQL operations and built-in functions should work correctly with MAP values. However, there is currently no way to create MAP values using only SQL tools. Part of #4763 @TarantoolBot document Title: Field type MAP in SQL Properties of type MAP in SQL: 1) a value of type MAP can be implicitly and explicitly cast only to ANY; 2) only a value of type ANY with primitive type MAP can be explicitly cast to MAP; 3) a value of any other type cannot be implicitly cast to MAP; 4) a value of type MAP cannot participate in arithmetic, bitwise, comparison, and concatination operations.
-
Mergen Imeev authored
This patch introduces ARRAY to SQL. After this patch, all SQL operations and built-in functions should work correctly with ARRAY values. However, there is currently no way to create ARRAY values using only SQL tools. Part of #4762 @TarantoolBot document Title: Field type ARRAY in SQL Properties of type ARRAY in SQL: 1) a value of type ARRAY can be implicitly and explicitly cast only to ANY; 2) only a value of type ANY with primitive type ARRAY can be explicitly cast to ARRAY; 3) a value of any other type cannot be implicitly cast to ARRAY; 4) a value of type ARRAY cannot participate in arithmetic, bitwise, comparison, and concatination operations.
-
Mergen Imeev authored
This patch introduces the mem_snprintf() function, which writes the string representation of a MEM to buf.
-
Mergen Imeev authored
This patch removes quotes from the representation of UUID values in the error description. This is because UUID values are printed without quotes elsewhere.
-
Vladimir Davydov authored
After this commit only three workflow are run on pull request or push to a developer branch: - luacheck - release - debug_coverage To run all other tests, one should either name the branch `*-full-ci` and push it to the main repository or set the 'full-ci' label on the pull request. It is also possible to disable all tests on push by naming branch as `*-notest' or setting the 'notest' label on the pull request. **Caveats**: - Unfortunately, currently it doesn't seem to be possible to run workflows automatically when a particular label is set - the best we can do is run workflows when *any* label is set. So labeling a PR that has the 'full-ci' label set will trigger all workflows! - For the same reason, removing the 'notest' label doesn't trigger ci. One has to synchronize the PR afterwards. We could trigger ci on the 'unlabel' event, but this would trigger tests when any label is removed, not necessarily 'notest'. Since 'notest' is supposed to be used only by developers, who can sync the branch, this should be acceptable. While we are at it: - Remove the check disabling certain workflow runs on forks - it's pointless, because forks don't have ci. Anyway, we don't bother disabling most of our workflows on forks, even those that we run on self-hosted machines, so that would only be consistent. - Remove the condition from the coverity workflow - coverity doesn't run on push or PR so it doesn't make any sense. - Remove the condition from the 'source' workflow. Instead trigger it only when a tag is pushed. This is needed to avoid showing it as a skipped workflow in PRs and commits. Closes #6605
-
Vladimir Davydov authored
Line-by-line (mostly) migration of the net.box state machine implementation from Lua to C. The C implementation uses the iostream abstration, which will allow us to to easily support complex network protocols under IPROTO in future. Closes #6291
-
Vladimir Davydov authored
Currently, - socket.tcp_server sets AI_PASSIVE. - socket.tcp_connect sets no flags, but passes 'tcp' for protocol. - coio_connect sets AI_ADDRCONFIG|AI_PASSIVE|AI_NUMERICSERV. - evio_service sets AI_ADDRCONFIG|AI_PASSIVE. This leads to different getaddrinfo results on some platforms (freebsd) and, as a result, test breakages (net.box_connect_timeout_gh-2054) after we switch net.box from socket.tcp_connect to coio_connect. Let's use compatible flags: - AI_ADDRCONFIG for clients (coio_connect and tcp_connect). (AI_PASSIVE doesn't make sense for connect.) (AI_NUMERICSERV isn't really needed in our case.) - AI_ADDRCONFIG|AI_PASSIVE for servers (evio_service, tcp_server). Let's also set ai_protocol to 'tcp' in both tcp_connect and tcp_server for consistency, because both these functions are only supposed to work with the tcp protocol, judging by their names.
-
Vladimir Davydov authored
It's too low-level. Use conn:_request instead. While we are at it, let's explicitly check the error in net.box_long-poll_input_gh-3400.test.lua to make it more robust.
-
Vladimir Davydov authored
We will use iostream in net.box when we convert the state machine to C. Rework netbox_communicate() so that it's easy to migrate to iostream.
-
Vladimir Davydov authored
xrow_decode_auth() expects the password to be set so omitting it while preparing a request is incorrect. Net.box state machine (in Lua) sets the password to an empty string if it's nil so the code handling NULL password in netbox_encode_auth() is actually dead. Let's remove it.
-
Vladimir Davydov authored
The callback passed to the worker fiber (in create_transport()) references the connection object. Since a fiber is a GC root, it means that the worker fiber pins the connection object. To avoid this, we use a weak callback reference. To prevent the callback from being GC-ed prematurely, we need to store a strong reference to it somewhere. Currently, we have some GC magic in create_transport(): we store a strong reference to the callback in the bound parameters of transport.stop(). This works fine, but when we migrate the transport implementation to C, we won't be able to override transport.stop() so let's store a strong reference in the connection object instead. While we are at it, let's drop create_transport method from net.box module exports, because it's not used anywhere in Tarantool code or tests, neither is it documented.
-
Vladimir Davydov authored
A GC callback must not yield. Fixes commit 72adeda6 ("net.box: implement connection closing without dropping requests"); Closes #6617
-
Vladimir Davydov authored
The callback isn't supposed to normally raise exceptions (it runs client code in pcall). If something abnormal happens, like a memory allocation error, better break the loop and stop the state machine. Also, raise an exception if we failed to decode an IPROTO_EVENT packet - this means that something is wrong with the other end and we better stop the connection (this is what we do if we fail to decode other packets).
-
Vladimir Davydov authored
Currently, some errors raise exception (e.g. memory allocation, fiber is cancelled) while others are handled by the state machine code. Let's pass all errors in exceptions and drop error_sm. This simplifies the state machine code a little bit.
-
Vladimir Davydov authored
We can implement it in Lua, because the transport invokes the callback whenever the state changes. This will simplify conversion of the state machine to C.
-
Vladimir Davydov authored
Pass protocol version and features to handshake callback along with greeting. The fewer callbacks we have, the easiser it'll be to convert the code to C. While we are at it, add a missing comment for the event callback.
-
Vladimir Davydov authored
There's no point in using the callback for this. It'd only complicate moving the state machine to C.
-
Vladimir Davydov authored
Not used since console support was dropped. Drop it to simplify migration of the net.box state machine to C. @TarantoolBot document Title: Update the net.box state machine diagram New diagram from the net.box source code: ``` initial -> auth -> fetch_schema <-> active (any state, on error) -> error_reconnect -> auth -> ... \ -> error (any state, but 'error') -> closed ```
-
Vladimir Davydov authored
net.box console support was deprecated more than three years ago, in commit bd06e32a ("netbox: deprecate console support"). Let's drop it, because it'll greatly simply rewriting the net.box state machine in C. @TarantoolBot document Title: Delete documentation entry for net.box console net.box console support, which was deprecated in 1.10, was dropped. Now, one has to use `require('console').connect(host, port)` instead of `require('net.box').connect(host, port, {console = true})`.
-
Vladimir Davydov authored
establish_connection is used by console to check greeting: if the protocol is 'Binary', it wraps the connected socket in a net.box connection; if it's 'Lua console', it wraps the socket in a console handler. Presence of these methods complicates migration of the net.box state machine to C. Also, it'll be difficult to support establish_connection when we switch net.box to the iostream abstraction, because a greeting should be received via iostream, not fd, probably after some protocol-specific negoitiation (e.g. TLS), while iostream isn't (and won't be) available in Lua and so can't be easily passed around from establish_connection to Lua and then back to net.box state machine. Since Lua console will always use plain sockets (it doesn't require encryption as it's supposed to run on the local host), let's first try to establish a net.box connection and if it fails because of unsupported protocol, try to establish a Lua console connection. It means that the socket will be connected twice for Lua console, which is suboptimal, but it should be fine, because it's not a hot path.
-
- Nov 23, 2021
-
-
Igor Munkin authored
* Fix frame traversal for __gc handler frames.
-
Yaroslav Lobankov authored
According to a huge amount of commit pushes by developers we should cancel all outdated workflow runs (previously scheduled and not relevant due to new changes) to make CI more efficient. GitHub Actions provides the 'concurrency' feature [1] as a method of reaching that and this patch introduces its using. How does it work? Basically, an update of a developer branch cancels the previously scheduled workflow run for this branch. However, the 'master' branch, release branch (1.10, 2.8, etc.), and tag workflow runs are never canceled. [1] https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#concurrency Closes tarantool/tarantool-qa#100
-
- Nov 22, 2021
-
-
Oleg Babin authored
Before this patch test build failed with following error: ``` /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:2666:26: error: unexpected type name '_CompressedPair': expected expression struct _ALIGNAS_TYPE(_CompressedPair) _Storage { ``` This patch seems to be fix this issue. The root of the problem is in "trivia/util.h" module. It defines alignas macros: `#define alignas(_n) __attribute__((aligned(_n)))`. And it seems causes some issues in stdlib internals. To fix this issue let's unconditionally include stdalign.h for C++. According [1] this header should internally define __alignas_is_defined macro. [1] https://en.cppreference.com/w/cpp/language/alignas Part of #6576
-
Mergen Imeev authored
After this patch, all functions that take arguments of one of two or more types have defined a default type, which is used when the argument type cannot be determined. Closes #6483 @TarantoolBot document Title: Default types of SQL built-in functions In case a function takes an argument of one of two or more types, this function has defined the default type for this argument. This type is used when the type of the argument cannot be determined, for example, in the case of a bound variable. Rules for determining default types as follows: 1) When there is only one possible type, it is default. 2) When possible types are INTEGER, DOUBLE or DECIMAL, DECIMAL is default. 3) When possible types are STRING or VARBINARY, STRING is default. 4) When possible data types are any other scalar data type, SCALAR is default. 5) Otherwise, there is no default type.
-