- Aug 18, 2023
-
-
Timur Safin authored
Implemented readline history and autocomplete by reusing readline facilities of Tarantool console. They used to be being hidden once Lua 'console' module is loaded. With c432e9e9 (lua: don't use public module name as internal one), now they are available as 'console.lib'. Closes #7738 NO_TEST=covered by refactored console_debugger_session_test.lua @TarantoolBot document Title: proper readline support in readline Similar to tarantool interactive console, tdbg now uses readline for its shell. It enables handier input editing, command history and so on.
-
Gleb Kashkin authored
`console_debugger_session_test.lua` can be simplified and made more stable with interactive_tarantool helper. With the new `read_untill_prompt()` helper function, dbgr prompt can be used as end-of-output marker instead of '...' in yaml-outputting main console. This way, popen results are unambiguous and no retries are required. At the same time, dbgr prompt is now expected with every result, thus there is no need for an extra check. But `continue` command usually reaches the end of the script and exits debugger, thus '<END_OF_EXECUTION>' marker was introduced. Empty stderr and header check were moved to interactive_tarantool helper. This patch breaks the test without debugger readline support patch, that is next in the patchset. Part of #7738 NO_CHANGELOG=test refactoring NO_DOC=test refactoring
-
Gleb Kashkin authored
The following changes were applied to `interactive_tarantool` helper to adapt it for debugger tests: * compared commands are now stripped from tabs and color codes too * now each command independent of control symbols is being stripped before comparison in `execute_command()` * created internal new function that is called from both `new()` and `new_debugger()` * now user defined prompt can be set up from `new` and `new_debugger()` * now there are several less typos in comments Part of #7738 NO_CHANGELOG=test helper update NO_DOC=test helper update NO_TEST=test helper update
-
- Aug 17, 2023
-
-
Vladimir Davydov authored
Runtime tuple formats are reusable, which means that a tuple format returned by runtime_tuple_format_new may not be brand new, but actually be used by a Lua object. As a result, if we call any function that may trigger Lua GC between runtime_tuple_format_new and tuple_format_ref, the tuple format may be deleted, leading to a use-after-free bug. This is what happens in lbox_tuple_format_new. Fix this issue by moving the runtime_tuple_format_new call after the Lua object allocation. Closes #8889 NO_DOC=bug fix NO_TEST=difficult to reproduce, found by ASAN
-
- Aug 16, 2023
-
-
Vladimir Davydov authored
Closes #8801 @TarantoolBot document Title: Document `box.session.new` The new function creates a new session given a table of options: - `type`: string, optional. Default: "binary". [Type][session-type] of the new session. Currently, only "binary" is supported, which creates a new IPROTO session. - `fd`: number, mandatory. File descriptor number (fd) to be used for the new connection input-output. The fd must refer to a [socket] and be switched to the [non-blocking mode][socket-nonblock] but this isn't enforced, i.e. the user may pass an invalid fd, in which case the connection won't work as expected. - `user`: string, optional. Default: "guest". Name of the user to authenticate the new session as. Note, this doesn't prevent the other end to perform explicit [authentication]. - `storage`: table, optional. Default: empty table. Initial value of the [session-local storage][session-storage]. On success, `box.session.new` takes ownership of the `fd` and returns nothing. On failure, an error is raised. Possible errors: - Invalid option value type. - `fd` isn't specified or has an invalid value. - `box.cfg` wasn't called. - `user` doesn't exist. Example: The code below creates a TCP server that accepts all incoming IPROTO connections on port 3301, authenticates them as 'admin' and sets the session-local storage to `{foo = 'bar'}`. ```lua box.cfg() require('socket').tcp_server('localhost', 3301, function(s) box.session.new({ type = 'binary', fd = s:fd(), user = 'admin', storage = {foo = 'bar'}, }) s:detach() end) ``` Notes: - `box.cfg` must be called before using `box.session.new` to start IPROTO threads. Setting [`box.cfg.listen`][box-cfg-listen] isn't required though. - The socket object must be detached after passing its fd to `box.session.new`, otherwise the fd would be closed on Lua garbage collection. [authentication]: https://www.tarantool.io/en/doc/latest/dev_guide/internals/iproto/authentication/ [box-cfg-listen]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-basic-listen [session-storage]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/storage/ [session-type]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_session/type/ [socket]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/ [socket-nonblock]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/#socket-nonblock
-
Vladimir Davydov authored
Sometimes we need to submit a remote call over cbus without waiting for the result or even yielding. The cbus_call_timeout function isn't apt for that because it yields even if timeout is 0 and sets diag. Let's introduce cbus_call_async that exits immediately after pushing the call message to the remote thread. NO_DOC=internal NO_CHANGELOG=internal
-
Alexander Turenko authored
Part of #8862 NO_DOC=this API is not marked as public, at least now
-
Alexander Turenko authored
A set of testing helpers was added in commit 06ca83c9 ("test/config: add several application script tests"). In particular, they allow to run a tarantool instance with some default config file and write only necessary config options in a test case. In fact, if no options were set in the test, the config file was not written. It is fixed in this commit and used in the next commit. Part of #8862 NO_DOC=testing helper change NO_CHANGELOG=see NO_DOC
-
Vladimir Davydov authored
Closes #8928 @TarantoolBot document Title: Document `from_fd` net.box module function The function takes a file descriptor number (fd) as its first argument and a table of connection options as its second, optional argument. It creates and returns a new connection object. The fd should point to a socket and be switched to the non-blocking mode, but this isn't enforced. If the fd is invalid, the connection may not work as expected. The functions takes the same options as [`connect`][net-box-connect]. The function takes the ownership of the fd, i.e. the fd must not be used or closed after this function successfully returns. Example: The code below connects to a Tarantool instance at port 3301 using the [`socket`][socket] module, then wraps the socket fd in a `net.box` connection object. ```Lua local net = require('net.box') local socket = require('socket') -- Connect a socket then wrap it in a net.box connection object. local s = socket.tcp_connect('localhost', 3301) assert(s ~= nil) local conn = net.from_fd(s:fd(), {fetch_schema = false}) s:detach() conn:call('box.schema.user.info') conn:close() ``` [net-box-connect]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/net_box/#net-box-connect [socket]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/
-
Vladimir Davydov authored
Those are really nice helpers. Let's move them to the utils module so that we can use them everywhere. Also, let's add some unit tests. NO_DOC=refactoring NO_CHANGELOG=refactoring
-
Vladimir Davydov authored
The check_param and check_param_table Lua helpers are defined in box/lua/schema.lua but used across the whole code base. The problem is we can't use them in files that are loaded before box/lua/schema.lua, like box/lua/session.lua. Let's move them to a separate source file lua/utils.lua to overcome this limitation. Also, let's add some tests. NO_DOC=refactoring NO_CHANGELOG=refactoring
-
Igor Munkin authored
* ci: support coveralls * cmake: add code coverage support * test: run flake8 static analysis via CMake * test: fix E741 errors by pycodestyle * test: fix E722 errors by pycodestyle * test: fix E711 errors by pycodestyle * test: fix E502 errors by pycodestyle * test: fix E501 errors by pycodestyle * test: fix E305 errors by pycodestyle * test: fix E303 errors by pycodestyle * test: fix E302 errors by pycodestyle * test: fix E301 errors by pycodestyle * test: fix E275 errors by pycodestyle * test: fix E251 errors by pycodestyle * test: fix E231 errors by pycodestyle * test: fix E203 errors by pycodestyle * test: fix E201 and E202 errors by pycodestyle * test: suppress E131 errors by pycodestyle * test: fix E128 errors by pycodestyle * test: fix E122 errors by pycodestyle * gdb: fix Python <assert> statement usage NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump NO_CHANGELOG=LuaJIT submodule bump
-
Sergey Bronnikov authored
NO_DOC=seed corpus NO_CHANGELOG=seed corpus NO_TEST=seed corpus
-
Sergey Bronnikov authored
NO_DOC=testing NO_CHANGELOG=testing
-
Sergey Bronnikov authored
NO_DOC=seed corpus NO_CHANGELOG=seed corpus NO_TEST=seed corpus
-
Sergey Bronnikov authored
Examples of IPROTO decoding issues: #3900, #1928, #6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing
-
- Aug 15, 2023
-
-
Ilya Verbin authored
Now the default value of a tuple field can be computed as a result of a function call. Closes #8609 @TarantoolBot document Title: Document functional default field values Product: Tarantool Since: 3.0 Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/format/ Depends on: tarantool/doc#3520 The format clause contains, for each field, a definition within braces: `{name='...',type='...'[,default=...][,default_func=...]}`, where: * (Optional) The `default_func` string value specifies the name of a function, that is used to generate a default value for the field. If `default_func` is set, the `default` value is used as the function argument. See [field default functions](https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#field-default-functions). --- Root document: https://www.tarantool.io/en/doc/latest/concepts/data_model/value_store/#constraint-functions **Field default functions** Stored Lua function can be used to generate a default value for the tuple field. It can take one optional parameter, and must return exactly one value. To create a field default function, use func.create with function body. The function must not yield. Example: ```lua box.schema.func.create('random_point', { language = 'Lua', body = 'function(param) return math.random(param.min, param.max) end' }) box.schema.space.create('test') box.space.test:create_index('pk') box.space.test:format({ {name = 'id', type = 'integer'}, {name = 'latitude', type = 'number', default_func = 'random_point', default = {min = -90, max = 90}}, {name = 'longitude', type = 'number', default_func = 'random_point', default = {min = -180, max = 180}} }) ``` ```lua tarantool> math.randomseed(os.time()) --- ... tarantool> box.space.test:insert{1} --- - [1, 56, 38] ... ```
-
Ilya Verbin authored
This structure will be extended in the next commit. Part of #8609 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Ilya Verbin authored
This call is redundant and does nothing. Constraints are destroyed in space_cleanup_constraints(), which is called from space_create() and space_delete(). Standalone tuples can't have initialized constraints. NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Ilya Verbin authored
User id instead of function id was erroneously used there. NO_DOC=bugfix NO_CHANGELOG=minor
-
Ilya Verbin authored
The checks in box.schema.index.create() and box.schema.index.alter() were case sensitive, also it was possible to insert incorrect index options directly into `box.space._index`. Fixed by adding checks to memtx_space_check_index_def() and vinyl_space_check_index_def(). Closes #8937 NO_DOC=bugfix
-
Ilya Verbin authored
And remove unused index_def_check_xc(). As index_def_dup() never returns NULL anymore, change index_create() and index_read_view_create() return type to `void` and update their callers. NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
-
Ilya Verbin authored
The `__sanitizer_start_switch_fiber()` function takes a pointer as the first argument to store the current fake stack if there is one (it is necessary when stack-use-after-return detection is enabled). When leaving a fiber definitely, NULL must be passed so that the fake stack is destroyed. Before this patch, NULL was passed for dead fibers, however this is wrong for dead fibers that are recycled and resumed. In such cases ASAN destroys the fake stack, and the fiber crashes trying to use it in `fiber_yield()` upon return from `coro_transfer()`. Closes tarantool/tarantool-qa#321 NO_DOC=bugfix NO_TEST=tested by test-release-asan workflow
-
Vladislav Shpilevoy authored
_cluster on-replace trigger parses old and new tuples into replica_def objects. One path handles the case of new_def != NULL. The other assumes that old_def != NULL if new_def was NULL. This is correct, because replace wouldn't happen if both tuples are NULL. It would mean nothing changed. Nonetheless coverity complained here that the old tuple could be NULL even if the new one was NULL. The patch silences this warning by adding an assert. NO_DOC=refactor NO_CHANGELOG=refactor NO_TEST=not testable
-
Vladislav Shpilevoy authored
box_set_instance_name() has a scoped guard to try to restore the replication back to normal if the connection to master was recreated to make the applier request a new name. The guard calls box_restart_replication() which might throw exceptions. But in this case it apparently can only happen in theory with bootstrap strategy legacy + high connection quorum + failing to gather this quorum. Also iostream creation might fail. Iostream creation failure is not testable, and the legacy bootstrap strategy is being deprecated at some point. So the patch simply adds try-catch without a test as it is too much effort for too little gain to bother. This should also calm down the coverity reporter. NO_DOC=bugfix/refactor NO_CHANGELOG=hardly possible to observe this bug NO_TEST=too hard to repro
-
- Aug 14, 2023
-
-
Sergey Bronnikov authored
Fuzzing test for LuaJIT generates random Lua programs and executes them. We want to build a fuzzing test that will produce Lua programs that will not contain semantic errors and will trigger as much as possible components in LuaJIT. This proposed patch introduces metrics that gathered after running the test. LuaJIT metrics gathered using LuaJIT getmetrics module [1]. All gathered metrics test will output after running with a finite number of runs or finite duration of time (options `-runs` and `-max_total_time`) or after sending SIGUSR1 to a test process. ``` $ ./build/test/fuzz/luaL_loadbuffer/luaL_loadbuffer_fuzzer -runs=1000 <snipped> Done 1000 runs in 1 second(s) Total number of samples: 1000 Total number of samples with errors: 438 (43%) Total number of samples with recorded traces: 87 (8%) Total number of samples with snap restores: 30 (3%) Total number of samples with abort traces: 55 (5%) ``` 1. https://www.tarantool.io/en/doc/latest/reference/tooling/luajit_getmetrics/#getmetrics-c-api NO_CHANGELOG=testing NO_DOC=testing
-
Vladimir Davydov authored
The function can't be called on an unconfigured instance because it needs IPROTO threads up and running. Let's raise an error to avoid a crash. Since we have two other places where we need to raise the same error (box.session.su and box.__index), let's introduce the new code ER_UNCONFIGURED for this error. Closes #8975 NO_DOC=bug fix
-
Alexander Turenko authored
The `popen_new()` function may be called with inherit_fds = NULL, nr_inherit_fds = 0. The lua/popen.c code actually does this. It leads to the following memcpy() call. ``` memcpy(dst, NULL, 0); ``` According to the C11 standard (n1256, 7.21.1), the pointer argument should have a valid value, which means pointing to some area in the program's address space, not NULL. The standard doesn't make an exception for a zero size array. Personally I doubt that any memcpy() implementation attempts to dereference the source pointer in case of a zero size, but it is my assumption, while the standard is the standard. The problem is found by Coverity. Follows up #8926 NO_DOC=it is a bug NO_CHANGELOG=this code is not released yet NO_TEST=verified by existing test cases, which call popen.new() without the inherit_fds option
-
- Aug 11, 2023
-
-
Vladimir Davydov authored
Closes #8802 @TarantoolBot document Title: Deprecate box.session.push The `box.session.push` Lua function and `box_session_push` C API function are deprecated starting from Tarantool 3.0. Calling any of these functions for the first time results in printing a warning message to the log. The new compat module option `box_session_push_deprecation` was introduced to control whether the functions are still available. With the old behavior, which is the default in Tarantool 3.0, `box.session.push` is still available. With the new behavior, any attempt to use it raises an exception. (Please create https://tarantool.io/compat/box_session_push_deprecation) We are planning to switch the compat option to the new behavior starting from Tarantool 4.0 with the ability to revert to the old behavior. Starting from Tarantool 5.0 we are planning to drop `box.session.push` completely.
-
Alexander Turenko authored
It lists the environment variables that are taken into account by the config module. Part of #8862 NO_DOC=reflected in https://github.com/tarantool/doc/issues/3544
-
Alexander Turenko authored
The next commit needs to output a lot of environment variables, their types in the config schema, defaults and so on. It looks much more readable when formatted in the tabular form. This commits adds a simple tabular formatter for this purpose. Part of #8862 NO_DOC=the new module is internal NO_CHANGELOG=see NO_DOC
-
Alexander Turenko authored
Sometimes we need to know somethings about ancestor nodes during processing of a current node. Now, the `<schema node>.computed.annotations` table represents all the annotations from the root of the schema down to the given schema node merged into a flat table. If the same named annotation met several times on the path, the descendant value is preferred. This computed annotations table is available everywhere, where the schema node is returned or passed as a callback's argument. For example, in values of the `<schema>:pairs()` iterator and in the `validate()` schema node's callback. It will be used in one of the next commits this way: ```lua for _, w in instance_config:pairs() do if w.schema.computed.annotations.enterprise_edition then <...> else <...> end end ``` Part of #8862 NO_DOC=the schema module is internal, at least now NO_CHANGELOG=see NO_DOC
-
- Aug 10, 2023
-
-
Ilya Verbin authored
Pin in cache the collation identifiers that are referenced by space format and/or indexes, so that they can't be deleted. Closes #4544 NO_DOC=bugfix
-
Ilya Verbin authored
It was possible to delete a collation, which is in use by a key_def. Part of #4544 NO_DOC=bugfix NO_CHANGELOG=next commit
-
Magomed Kostoev authored
Since number type was introduced we can not assume if tuples are equal by comparison then their sizes are equal too. So the place the assumption is used is fixed. Closes #8899 NO_DOC=bugfix
-
Vladimir Davydov authored
Getting rid of C++ exception is our long term goal. This commit removes the exceptions from the IPROTO code as follows: - Make iproto_set_msg_max return -1 instead of raising an exception on error. Update box_set_net_msg_max accordingly. - Make box_process_auth return -1 instead of raising an exception on error. Move it along with box_process_vote (which never fails) to the extern "C" namespace. - Remove try/catch from iproto_connection_on_input, tx_process_misc, tx_process_connect. Use a label instead. - Panic instead of raising an exception on an error starting an IPROTO thread in iproto_init. It should be fine because the function is called only at startup and shouldn't normally fail. - Remove try/catch from iproto_do_cfg_f. It turns out that this function never fails so we don't even need to use an error label. - Drop iproto_do_cfg_crit and move the assertion to iproto_do_cfg because this function should never fail and so we don't need the wrapper. After this commit, the only exception-handling piece of code left in IPROTO is tx_process_replication. Dropping it would entail patching replication, relay, and recovery code so it was postponed. NO_DOC=code cleanup NO_TEST=code cleanup NO_CHANGELOG=code cleanup
-
Vladimir Davydov authored
The buffer is unlimited so allocations from it never fail. Even if decide to add a hard limit some day, it's better to do it somewhere in one place to simplify testing so the dropped code isn't going to be needed. NO_DOC=code cleanup NO_TEST=code cleanup NO_CHANGELOG=code cleanup
-
Vladimir Davydov authored
The mempools are unlimited so allocations from them never fail. Even if we decide to add a hard limit some day, it's better to do it somewhere in one place to simplify testing so the dropped code isn't going to be needed. NO_DOC=code cleanup NO_TEST=code cleanup NO_CHANGELOG=code cleanup
-
Vladimir Davydov authored
malloc isn't supposed to fail so let's use xmalloc helper in rmean_new and get rid of allocation failure paths. NO_DOC=code cleanup NO_TEST=code cleanup NO_CHANGELOG=code cleanup
-
- Aug 09, 2023
-
-
Vladimir Davydov authored
The new function is a wrapper around the socketpair system call. It takes the same arguments as the socket constructor and returns two socket objects representing the two ends of the newly created socket pair on success. It may be useful for establishing a communication channel between related processes. Closes #8927 @TarantoolBot document Title: Document new socket functions Two socket module functions and one socket object method were added to Tarantool 3.0: - `socket.from_fd(fd)`: constructs a socket object from the file descriptor number. Returns the new socket object on success. Never fails. Note, the function doesn't perform any checks on the given file descriptor so technically it's possible to pass a closed file descriptor or a file descriptor that refers to a file, in which case the new socket object methods may not work as expected. - `socket:detach()`: like `socket:close()` but doesn't close the socket file descriptor, only switches the socket object to the closed state. Returns nothing. If the socket was already detached or closed, raises an exception. Along with `socket.from_fd`, this method may be used for transferring file descriptor ownership from one socket to another: ```Lua local socket = require('socket') local s1 = socket('AF_INET', 'SOCK_STREAM', 'tcp') local s2 = socket.from_fd(s1:fd()) s1:detach() ``` - `socket.socketpair(domain, type, proto)`: a wrapper around the [`socketpair`][1] system call. Returns two socket objects representing the two ends of the new socket pair on success. On failure, returns nil and sets [`errno`][2]. Example: ```Lua local errno = require('errno') local socket = require('socket') local s1, s1 = socket.socketpair('AF_UNIX', 'SOCK_STREAM', 0) if not s1 then error('socketpair: ' .. errno.strerror()) end s1:send('foo') assert(s2:recv() == 'foo') s1:close() s2:close() ``` [1]: https://man7.org/linux/man-pages/man2/socketpair.2.html [2]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/errno/
-