- Apr 09, 2018
-
-
Arseny Antonov authored
* Added new coveralls options to sync repo * Pass travis job to coverage docker
-
- Apr 07, 2018
-
-
Vladimir Davydov authored
-
Vladimir Davydov authored
-
Vladimir Davydov authored
If a fiber waiting for a read task to complete is cancelled, it will leave the read iterator immediately, leaving the read task pending. If the index is dropped before the read task is complete, the task will attempt to dereference a deleted run upon completion: 0 0x560b4007dbbc in print_backtrace+9 1 0x560b3ff80a1d in _ZL12sig_fatal_cbiP9siginfo_tPv+1e7 2 0x7f52b09190c0 in __restore_rt+0 3 0x7f52af6ea30a in bzero+5a 4 0x560b3ffc7a99 in mempool_free+2a 5 0x560b3ffcaeb7 in vy_page_read_cb_free+47 6 0x560b400806a2 in cbus_call_done+3f 7 0x560b400805ea in cmsg_deliver+30 8 0x560b40080e4b in cbus_process+51 9 0x560b4003046b in _ZL10tx_prio_cbP7ev_loopP10ev_watcheri+2b 10 0x560b4023d86e in ev_invoke_pending+ca 11 0x560b4023e772 in ev_run+5a0 12 0x560b3ff822dc in main+5ed 13 0x7f52af6862b1 in __libc_start_main+f1 14 0x560b3ff801da in _start+2a 15 (nil) in +2a Fix this by elevating the run reference counter per each read task. Note, currently we use vy_run::refs not only as a reference counter, but also as a counter of slices created for the run - see how we compare it to vy_run::compacted_slice_count in vy_task_compact_complete(). This isn't going to work anymore, obviously. Now we need to count slices created per each run in a separate counter, vy_run::slice_count. Anyway, it was a rather dubious hack to abuse reference counter for counting slices and it's good to finally get rid of it.
-
Vladimir Davydov authored
We use ERRINJ_DOUBLE for all other timeout injections. This makes them more flexible as we can inject an arbitrary timeout in tests, not just enable some hard-coded timeout. Besides, it makes tests easier to follow. So let's use ERRINJ_DOUBLE for ERRINJ_VY_READ_PAGE_TIMEOUT too.
-
Vladimir Davydov authored
If a space has no indexes, index_find() will return NULL, which will be happily dereferenced by on_replace_dd_sequence(). Looks like this bug goes back to the time when we made index_find() exception-free and introduced index_find_xc() wrapper. Fix it and add a test case.
-
- Apr 06, 2018
-
-
Vladimir Davydov authored
space_vtab::commit_alter is implemented only by memtx, which uses it to set bsize for a new space. However, it isn't necessary to use commit_alter for this - instead we can set bsize in prepare_alter and reset it in drop_primary_key, along with memtx_space::replace. Let's do it and zap space_vtab::commit_alter altogether, because the callback is confusing: judging by its name it should be called after WAL write, but it is called before.
-
Vladimir Davydov authored
When the last index of a memtx space is dropped, we need to delete all tuples stored in the space. We do it in space_vtab::commit_alter, but this is wrong, because this function is not rolled back and so we may get use-after-free error if we fail to write a DDL operation to WAL. To avoid that, let's delete tuples in index_vtab::commit_drop, which is called after WAL write. There's a nuance here: index_vtab::commit_drop is called if an index is rebuilt (because essentially it is drop + create) so we must elevate the reference counter of every tuple added to the new index during rebuild and, respectively, drop all the references in index_vtab::abort_create, which is called if index creation is aborted for some reason. This also means that now we iterate over all tuples twice when a primary key is rebuilt - first to build the new index, then to unreference all tuples stored in the old index. This is OK as we can make the last step asynchronous, which will also speed up the more common case of space drop. Closes #3289
-
Vladislav Shpilevoy authored
-
Vladislav Shpilevoy authored
When a schema version change is detected, there is no reason to cancel and retry already sent requests. They can be already executed on a server, and their retrying leads to multiple execution. A request must be retried only if a server responded with WRONG_SCHEMA_VERSION error exactly to this request. Closes #3325
-
Vladislav Shpilevoy authored
In Lua to get the only first value in a sequence it is enough to place it into (). Follow up #3323, #3322
-
Konstantin Osipov authored
-
Konstantin Osipov authored
Introducing a formal, trackable process for server enhancement. Before working on a complex feature, please write an RFC document, describing what and how you see changed, and get it approved. All historical RFCs are kept in doc/rfc.
-
- Apr 05, 2018
-
-
Vladimir Davydov authored
Currently, space_vtab::commit_alter is called before WAL write so we can use it for preparing new indexes in vinyl. However, this is going to change soon, because actually space_vtab::commit_alter should be called after WAL write, like index_vtab::commit_drop or commit_create. Calling it before WAL write may result in use-after-free in memtx (see #3289). Besides, using this function for iterating over space indexes just feels wrong, as we have index methods invoked by AlterSpaceOp descendants for this. So let's move all the operations performed by vinyl_space_commit_alter somewhere else. Fortunately, it's easy to do without damaging code readability or efficiency: - Update of vy_lsm::pk can be done by vinyl_space_swap_index and vinyl_build_secondary_key. - vy_lsm->check_is_unique can be computed by vinyl_engine_create_space and then set by vinyl_space_swap_index. - Call to key_def_update_optionality is implied by key_def_swap, which is already called by vinyl_space_swap_index, hence it can be removed. Needed for #3289
-
Vladimir Davydov authored
Basically, index_begin_build() followed by index_end_build() is a no-op. There's absolutely no point in calling it for primary indexes after initial recovery has completed.
-
Vladislav Shpilevoy authored
_request() is a wrapper for perform_request, that detects schema changes, and waits until it is reloaded. Lets use _request() instead of direct perform_request() for execute(). The reason why the _request() was not used earlier was my attempt to avoid multiple return values in _request(), that leads to minor fixes in non-execute methods like index.select or eval/call_16, which return the _request() directly. But diplicating schema reloading logic for execute() is worse. Closes #3323 Closes #3322
-
Vladislav Shpilevoy authored
This API is deprecated in 1.7.4, so it must not be supported in new methods like execute().
-
Vladimir Davydov authored
We read tuples from disk hence we should use disk_format, not mem_format. Fix it. While we are at it, let's also update the outdated comment to vy_run_rebuild_index.
-
Vladimir Davydov authored
The rtree is empty when this function is called (in fact, it is called right after creating the index), there's no need to purge it.
-
Vladimir Davydov authored
To allow extending key definition for non-empty vinyl spaces, this patch performs the following steps: - Revert commit c31dd19a ("vinyl: forbid vinyl index key definition alter") that forbade any key def alter. It isn't needed anymore. - Update key_def and cmp_def in vinyl_space_swap_index(). We simply swap the definitions between the old and new indexes in memory. Since all vinyl objects reference either vy_lsm::key_def or vy_lsm::cmp_def or both, and the change is compatible (does not change the order for existing tuples), this should work just fine. - Update key definition in vylog on ALTER. For this, we introduce a new vylog record type, VY_LOG_MODIFY_LSM, which updates key definition. To be able to replay it on recovery in case we failed to flush it before restart, we also store the LSN of the WAL record that triggered the ALTER. It also adds the following test cases: - Modify key definition of primary and secondary indexes of a non-empty space (engine/ddl). - Modify key definition before snapshot and relay it to a newly joined replica (engine/replica_join). - Make sure key definition is updated in vylog on ALTER (vinyl/layout).
-
Vladimir Davydov authored
There's no point in writing this record to snapshot, because we can store LSN of the last index dump right in VY_LOG_CREATE_LSM record.
-
Vladimir Davydov authored
So as to draw the line between LSN of index creation and LSN of last index modification, which is introduced by later in the series.
-
Nikita Pettik authored
As a part of SQL data dictionary integration, 'DROP INDEX' and 'DROP TABLE' statements proccessing has been refactored in order to avoid using SQL specific internal structures. However, triggers still aren't transfered to server, so to drop them it is required to lookup SQL table and emit apporpriate opcodes. Also, added comments and fixed codestyle for functions processing 'DROP' routine. Part of #3222.
-
Nikita Pettik authored
Some of legacy functions seem to be useless, since they serve as wrappers around others; the rest rely on capabilities which are no longer relevant. This patch provides slight refactoring of such functions or removes them at all.
-
Vladimir Davydov authored
There are two functions in vy_stmt.c that blindly copy tuple field map, vy_stmt_dup() and vy_stmt_replace_from_upsert(). Both these functions take a tuple format to use for the new statement and require this format to be the same as the source tuple format in terms of fields definition, otherwise they'll just crash. The only reason why we did that is that back when these functions were written we used a separate format for UPSERT statements so we needed this extra argument for creating a REPLACE from UPSERT. Now it's not needed, and we can use the source tuple format instead. Moreover, passing the current tuple format to any of those functions is even harmful, because tuple format can be extended by ALTER, in which case these functions will crash if called on a statement created before ALTER. That being said, let's drop the tuple format argument.
-
Vladimir Davydov authored
space->format and cmp_def must be compatible, i.e. space->format has is_nullable flag set for a field iff it is set for all key parts indexing this field. Therefore there's no point to set is_nullable for disk_format as it must have been initialized by tuple_format_create(). Remove the pointless loop. Also, while we are at it, fix the minor memory leak - disk_format is referenced twice for the primary key.
-
Vladimir Davydov authored
A piece of code left from the inglorious past, which doesn't even have a forward declaration, let alone used anywhere. Remove it.
-
Vladimir Davydov authored
We allocate index formats in the only place, vy_lsm_new, so there's no point in this debug-only check anymore.
-
Vladimir Davydov authored
We create new formats for all indexes of the new space in vinyl_space_commit_alter() while we don't actually need to do this, because the new formats have already been created by vy_lsm_new() - all we need to do is reuse them somehow. This patch does the trick: it implements the swap_index() space virtual method for vinyl so that it swaps tuple formats between the old and new spaces.
-
Vladimir Davydov authored
This function is called by MoveIndex and ModifyIndex ALTER operations, i.e. when the index definition is not changed at all or is extended. Making this method virtual will allow to avoid reallocation of vinyl formats in vinyl_space_commit_alter().
-
Vladimir Davydov authored
If the new cmp_def of a secondary index is compatible with the old one after the primary key parts have changed, we don't need to rebuild it, we just need to update its definition.
-
Nikita Pettik authored
Some pragmas turn out to be useless and can't be applied to Tarantool's SQL. Moreover, couple of them lead to assertion faults. List or removed pragmas: - synchronous - empty_result_callbacks - WAL_autocheckpoint - WAL_checkpoint - secure_delete - database_list - data_version - compile_options - application_id - user_version - pragmas connected with HAS_CODEC macro - pragmas connected with heap memory managment
-
Vladimir Davydov authored
If the primary key is modified, we schedule rebuild of all non-unique (including nullable) secondary TREE indexes. This is valid for memtx, but is not quite right for vinyl. For vinyl we have to rebuild all secondary indexes, because they are all non-clustered (i.e. point to tuples via primary key parts). This doesn't result in any bugs for now, because rebuild of vinyl indexes is not supported, but hopefully this is going to change soon. So let's introduce a new virtual index method, index_vtab::depends_on_pk, which returns true iff the index needs to be updated if the primary key changes, and define this new method for vinyl and memtx TREE indexes.
-
vanyail authored
This change removes REINDEX from keywords and also disables tests which were using this syntax. This patch affects only parser, REINDEX implementation still exists, because it is planned to be fixed and enabled in the scope of #3195. Note that REINDEX keyword is still in '%fallback' directive in 'parse.y'. Reserved keywords included there to avoid situation when a keyword has no usages within 'parse.y' file. Fixes #2174
-
Vladimir Davydov authored
The new method is called after successful update of index definition. It is passed the signature of the WAL record that committed the operation. It will be used by Vinyl to update key definition in vylog.
-
Bulat Niatshin authored
Fix non-working 'INDEXED BY' clause for SELECT statement, which caused syntax-error during execution. Closes #2996
-
Vladislav Shpilevoy authored
Type of an operation for struct request can be passed directly with no "proxying" by special codes.
-
Vladislav Shpilevoy authored
1) Code is completely unreadable and complicated by strange optimizations; 2) It still supposes, that there can be multiple statements in a single string; 3) It pushes first letter of affinity together with column names - why? It is not used anywhere. Lets remove prepared statements list, affinity first letters pushing.
-
Vladislav Shpilevoy authored
Remove unused error codes, use enum instead of defines. Later the prefix SQLITE_ must be removed - see #3315.
-
Vladislav Shpilevoy authored
And rework sqlite3CompleteInsertion. Before the patch it * accepts actually unused pTab; * array of records to insert in different indexes, but inserts only one; * has sqlite code style. Code style fix is obvious. PTab is used only to check, that it is not a view and to check, that a first index is primary: 1) caller functions already guarantee the pTab is not a view; 2) regardless of a first index nature: primary or not - it is not used here. It is useless check. With the same success we can check this in each function, that uses struct Table. Array of records to insert makes no sense, since insertion is being done in a primary index only. It is enough to pass a register with a primary index tuple.
-