- Jun 26, 2018
-
-
Georgy Kirichenko authored
Allow define access privileges for all spaces, functions and sequences. Read and write privileges are supported for spaces, execute privilege for sequences. Privilege granting and revoking might be done through old api without object identification: box.schema.user.grant("guest", "read", "space") Prerequisite #945
-
Konstantin Osipov authored
Use constants, rename methods, invert acl-object compatibility matrix. In scope of gh-945.
-
imarkov authored
Introduce constraints on object_type-privilege pairs. These constraints limit senseless grants/revokes, i.e., sequence - execute, all space related privileges(insert, delete, update), function - alter, all space related privileges, role - all privileges except create, drop, alter, execute Prerequisite #945
-
- Jun 25, 2018
-
-
Konstantin Osipov authored
-
Vladimir Davydov authored
If called on a unix socket, bind(2) creates a new file, see unix(7). When we stop a unix tcp server, we should remove that file. Currently, we do it from the tcp server fiber, after the server loop is broken, which happens when the socket is closed, see tcp_server_loop(). This opens a time window for another tcp server to reuse the same path: main fiber tcp server loop ---------- --------------- -- Start a tcp server. s = socket.tcp_server('unix/', sock_path, ...) -- Stop the server. s:close() socket_readable? => no, break loop -- Start a new tcp server. Use the same path as before. -- This function succeeds, because the socket is closed -- so tcp_server_bind_addr() will clean up by itself. s = socket.tcp_server('unix/', sock_path, ...) tcp_server_bind tcp_server_bind_addr socket_bind => EADDRINUSE tcp_connect => ECONNREFUSED -- Remove dead unix socket. fio.unlink(addr.port) socket_bind => success -- Deletes unix socket used -- by the new server. fio.unlink(addr.port) In particular, the race results in sporadic failures of app-tap/console test, which restarts a tcp server using the same file path. To fix this issue, let's close the socket after removing the socket file. This is absolutely legit on any UNIX system, and this eliminates the race shown above, because a new server that tries to bind on the same path as the one already used by a dying server will not receive ECONNREFUSED until the socket fd is closed and hence the file is removed. A note about the app-tap/console test. After this patch is applied, socket.close() takes a little longer for unix tcp server, because it yields twice, once for removing the socket file and once for closing the socket file descriptor. As a result, on_disconnect() trigger left from the previous test case has time to run after session.type() check. Actually, those triggers have already been tested and we should have cleared them before proceeding to the next test case. So instead of adding two new on_disconnect checks to the test plan, let's clear the triggers before session.type() test case and remove 3 on_connect and 5 on_auth checks from the test plan. Closes #3168
-
Konstantin Osipov authored
-
Vladislav Shpilevoy authored
Consider this packet: msgpack = require('msgpack') data = msgpack.encode(18400000000000000000)..'aaaaaaa' Tarantool interprets 18400000000000000000 as size of a coming iproto request, and tries with no any checks to allocate buffer of such size. It calculates needed capacity like this: capacity = start_value; while (capacity < size) capacity *= 2; Here it is possible that on i-th iteration 'capacity' < 'size', but 'capacity * 2' overflows 64 bits and becomes < 'size' again, so this loop never ends and occupies 100% CPU. Strictly speaking overflow has undefined behavior. On the original system it led to nullifying 'capacity'. Such size is improbable as a real packet gabarits, but can appear as a result of parsing of some invalid packet, first bytes of which accidentally appears to be valid MessagePack uint. This is how the bug emerged on the real system. Lets restrict the maximal packet size to 2GB. Closes #3464
-
- Jun 24, 2018
-
-
Mergen Imeev authored
Due to limitation of reference counters for tuple being only 65535 it was possible to reach this limitation. This patch increases capacity of reference counters to 4 billions. Closes #3224
-
- Jun 14, 2018
-
-
Vladislav Shpilevoy authored
Before the patch box.session.sync() is global for the session and is updated on each new iproto request. When the connection is multiplexed, box.session.sync() can be changed with no finishing a current request, if a new one arrives. The patch makes box.session.push() local for the request, protecting it from multiplexing mess. Box.session.sync() after the patch can be safely used inside a request. Closes #3450 @TarantoolBot document Title: box.session.sync() became request local Box.session.sync() was global for a session, so it was unusable when the connection behind the session is multiplexed. Now box.session.sync() is request local and can be safely used inside the request processor.
-
Vladislav Shpilevoy authored
Replace it with more specific structures and pointers in order to prepare to add `net` storage. This allows to make the code working with fiber storage simpler, remove useless wrappers and casts, and in the next patch - remove broken session.sync and add fiber sync. Note that under no circumstances fiber.h is allowed to include application-specific headers like session.h or txn.h. One only is allowed to announce a struct and add opaque pointer to it.
-
Vladimir Davydov authored
-
Vladimir Davydov authored
Since tuples stored in temporary spaces are never written to disk, we can always delete them immediately, even when a snapshot is in progress. Closes #3432
-
Vladimir Davydov authored
-
Vladimir Davydov authored
Since commit 8f63d5d9 ("vinyl: fail transaction immediately if it does not fit in memory"), vinyl won't trigger memory dump if the size of memory needed by a transaction is greater than the memory limit, instead it will fail the transaction immediately. This broke the aforementioned script, which relied on this to trigger system-wide memory dump. Fix it by reworking the dump trigger logic used by the script: now it tries to insert two tuples, box.cfg.vinyl_memory / 2 size each, instead of one. Closes #3449
-
- Jun 09, 2018
-
-
Konstantin Osipov authored
-
- Jun 08, 2018
-
-
Vladislav Shpilevoy authored
-
Alexander Turenko authored
-
Alexander Turenko authored
It fixes the following errors during tarantool installation from packages on debian / ubuntu: ``` Unpacking tarantool (1.9.1.23.gacbd91c-1) ... dpkg: error processing archive /var/cache/apt/archives/tarantool_1.9.1.23.gacbd91c-1_amd64.deb (--unpack): trying to overwrite '/lib/systemd/system/tarantool.service', which is also in package tarantool-common 1.9.1.23.gacbd91c-1 ``` The problem is that tarantool.service file was shipped with tarantool-common and tarantool packages both. It is the regression after 8925b862. The way to avoid installing / enabling the service file within tarantool package is to pass `--name` option to dh_systemd_enable, but do not pass the service file name. In that case dh_systemd_enable does not found the service file and does not enforce existence of the file. Hope there is less hacky way to do so, but I don't found one at the moment.
-
Vladimir Davydov authored
Currently, we start a hot standby fiber even if not in hot standby mode (see recovery_follow_local). And we scan the wal directory twice - first time in recovery_follow_local(), second time in recovery_finalize(). Let's factor out recover_remaining_wals() from those functions and call it explicitly. And let's call follow_local() and stop_local() only if in hot standby mode. Needed for #461
-
Vladimir Davydov authored
In order to find out if the current instance fell too much behind its peers in the cluster and so needs to be rebootstrapped, we need to connect it to remote peers before proceeding to local recovery. The problem is box.cfg.replication may have an entry corresponding to the instance itself so before connecting we have to start listening to incoming connections. Since an instance is supposed to sent its uuid in the greeting message, we also have to initialize INSTANCE_UUID early, before we start local recovery. So this patch makes memtx engine constructor not only scan the snapshot directory, but also read the header of the most recent snapshot to initialize INSTANCE_UUID. Needed for #461
-
Konstantin Osipov authored
-
Georgy Kirichenko authored
Use volatile asm modifier to prevent unwanted and awkward optimizations causing segfault while backtracing
-
Vladislav Shpilevoy authored
Netbox has two major ways to execute a request: sync and async. During execution of any a server can send multiplie responses via IPROTO_CHUNK. And the execution ways differ in how to handle the chunks (called messages or pushes). For a sync request a one can specify on_push callback and its on_push_ctx argument called on each message. When a request is async a user has a future object only, and can not specify any callbacks. To get the pushed messages a one must iterate over future object like this: for i, message in future:pairs(one_iteration_timeout) do ... end Or ignore messages just calling future:wait_result(). Anyway messages are not deleted, so a one can iterate over future object again and again. Follow up #2677
-
Vladislav Shpilevoy authored
Box.session.push() allows to send a message to a client with no finishing a main request. Tarantool after this patch supports pushes over binary protocol. IProto message is encoded using a new header code - IPROTO_CHUNK. Push works as follows: a user calls box.session.push(message). The message is encoded into currently active obuf in TX thread, and then Kharon notifies IProto thread about new data. Originally Kharon is the ferryman of Hades who carries souls of the newly deceased across the rivers Styx and Acheron that divided the world of the living from the world of the dead. In Tarantool Kharon is a message and does the similar work. It notifies IProto thread about new data in an output buffer carrying pushed messages to IProto. Styx here is cpipe, and the boat is cbus message. One connection has single Kharon for all pushes. But Kharon can not be in two places at the time. So once he got away from TX to IProto, new messages can not send Kharon. They just set a special flag. When Kharon is back to TX and sees the flag is set, he immediately takes the road back to IProto. Herewith a user is not blocked to write to obuf when Kharon is busy. The user just updates obuf and set the flag if not set. There is no waiting for Kharon arrival back. Closes #2677
-
imarkov authored
Add lua bindings of PRIV_XXX constants. This patch helps to avoid using numerical constants of privilleges in schema.lua code. Relates #945
-
Vladimir Davydov authored
So as not to duplicate the same code over and over again.
-
Vladimir Davydov authored
An extra new line looks ugly in the log: 2018-06-06 15:22:22.682 [9807] main/101/interactive C> Tarantool 1.10.1-58-gd2272132 2018-06-06 15:22:22.682 [9807] main/101/interactive C> log level 5 2018-06-06 15:22:22.682 [9807] main/101/interactive I> mapping 268435456 bytes for memtx tuple arena... 2018-06-06 15:22:22.683 [9807] main/101/interactive I> mapping 134217728 bytes for vinyl tuple arena... 2018-06-06 15:22:22.692 [9807] main/101/interactive I> recovery start 2018-06-06 15:22:22.692 [9807] main/101/interactive I> recovering from `./00000000000000000006.snap' 2018-06-06 15:22:22.721 [9807] main/106/applier/ I> remote master is 1.10.1 at 0.0.0.0:44441 2018-06-06 15:22:22.723 [9807] main/106/applier/ C> leaving orphan mode 2018-06-06 15:22:22.723 [9807] main/101/interactive C> replica set sync complete, quorum of 1 replicas formed 2018-06-06 15:22:22.723 [9807] main/101/interactive I> ready to accept requests
-
Vladimir Davydov authored
Neither recovery_new() nor recover_remaining_wals() need to modify it.
-
Vladimir Davydov authored
-
Ilya Markov authored
System views are used instead of direct reads of corresponding system spaces to explore all accessible objects such as spaces, functions, users and e.g. An operation with an inaccessible object produces a 'not found' error even if the object exists. In scope of #3250 Includes up fixes from Georgy
-
- Jun 07, 2018
-
-
Alexander Turenko authored
-
Alexander Turenko authored
* added --verbose to show output of successful TAP13 test (#73) * allow to call create_cluster(), drop_cluster() multiple times (#83) * support configurations (*.cfg files) in core = app tests * added return_listen_uri = <boolean> option for create_cluster() * save and print at fail tarantool log for core = app tests (#87)
-
Vladimir Davydov authored
This patch implements space_vtab::build_index callback for vinyl spaces. Now instead of returning an error in case the space is not empty, the callback will actually try to build a new index. The build procedure consists of four steps: 1. Prepare the LSM tree for building. This implies writing a special record to vylog, VY_LOG_PREPARE_LSM, and adding the new index to the vinyl scheduler so that it can be dumped during build. We need to log the new LSM tree so that we can keep track of run files created for it during build and remove them if build procedure fails. 2. Inserting tuples stored in the space into the new LSM tree. Since there may concurrent DML requests, we install a trigger to forward them to the new index. 3. Dumping the index to disk so that we don't have to rebuild it after recovery. 4. Committing the new LSM tree in vylog (VY_LOG_CREATE_LSM). Steps 1-3 are done from the space_vtab::build_index callback while step 4 is done after WAL write, from index_vtab::commit_create. While step 3 is being performed, new DML requests may be executed for the altered space. Those requests will be reflected in the new index thanks to the on_replace trigger, however they won't be recovered during WAL recovery as they will appear in WAL before the ALTER record that created the index. To recover them, we replay all statements stored in the primary key's memory level when replaying the ALTER record during WAL recovery. Closes #1653
-
Vladimir Davydov authored
Currently, we assume that no two runs of the same range intersect by LSN. This holds, because LSNs grow strictly monotonically, and no transaction may be split between two runs (as we pin each affected vy_mem until the transaction is complete). We ensure this with an assertion in vy_task_dump_complete. However, tuples inserted during index build will have arbitrary (not monotonically growing) LSNs. This is OK as for each particular key, two statements will still have different LSNs, but this may break the assertion in vy_task_dump_complete in case dump occurs while build is in progress. To avoid that, let's relax this limitation and assume that a dumped run may intersect by LSN with runs dumped before. Moreover, let's assume that it may have max LSN less than the max LSN stored on disk so that we should update vy_lsm::dump_lsn only if the dumped run has newer data. Needed for #1653
-
Konstantin Osipov authored
-
Vladimir Davydov authored
The fact that we may yield after we added a new slice created by dump, but before we removed the dumped in-memory index from the LSM tree complicates read iterator logic, as it has to detect such a case and filter out sources that contain duplicates. This logic relies on the fact that no two slices of the same range intersect by LSN. For the sake of ALTER we have to relax this limitation, as statements inserted during index build can have arbitrary (not monotonically growing) LSNs, so the no-LSN-intersection property won't be fulfilled for whole slices, only for individual keys. Since there shouldn't be more than 1000 ranges in the same LSM tree, yielding doesn't make much sense as iteration over the whole range tree should be pretty fast. Besides, dump isn't done frequently. That said, let's remove yielding altogether. Needed for #1653
-
Alexander Turenko authored
It is necessary for build on Ubuntu Bionic. Debian bugreport: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=881481 Debhelper commit: https://github.com/Debian/debhelper/commit/740c628a1e571acded7e2aac5d6e7058e61da37f
-
lifemaker authored
-
Vladimir Davydov authored
We need to check unique constraint when building a new index. So let's factor out this helper function and pass space_name, index_name, and read view to it explicitly (because index_name_by_id isn't going to work for an index that is under construction and there's no tx when we are building a new index). Suggested by @Gerold103. Needed for #1653
-
Vladislav Shpilevoy authored
It is quite simple - just use stdout file descriptor as the destination for push messages. It is needed to make remote and local console be similar.
-