Skip to content
Snippets Groups Projects
  1. Apr 02, 2019
    • Stanislav Zudin's avatar
      Feature request for a new collation · a99d7a0c
      Stanislav Zudin authored
      Adds a new set of default collations.
      The collation 'unicode_ky_s1' supports the difference
      between Cyrillic letters 'Е' and 'Ё'. The standard case insensitive
      collation ('unicode_ci') doesn't distinguish these letters.
      Adds tests for a new collations.
      
      Closes #4007
      a99d7a0c
    • Kirill Shcherbatov's avatar
      vinyl: fix invalid tuple in surrogate delete · 35eacc55
      Kirill Shcherbatov authored
      If the map contains non-string keys that cannot be used in the
      JSON index, an incorrect tuple is created in
      vy_stmt_new_surrogate_delete_raw during the construction of the
      surrogate tuple. This occurs because the number of items that
      the map contains has already been copied, while such invalid
      key-value pairs are not copied.
      The problem is resolved through the creation of dummy nil:nil
      pairs in such cases.
      35eacc55
    • Vladimir Davydov's avatar
      vinyl: rename vy_read_view_merge argument from hint to prev_tuple · 077fba80
      Vladimir Davydov authored
      So as not to confuse it with a tuple comparison hint.
      077fba80
    • Vladimir Davydov's avatar
      space: fix space_group_id return value · 5c30b215
      Vladimir Davydov authored
      Should be uint32_t obviously, not bool. This didn't affect anything,
      because there are only two replication groups right now - 0 and 1.
      
      Spotted by @GeorgyKirichenko.
      
      Fixes commit f64f4619 ("Introduce replica local spaces").
      5c30b215
    • Nikita Pettik's avatar
      sql: don't change type of function's retval after codegen · e93932b5
      Nikita Pettik authored
      Proper type of function's returning value is set during names resolution
      (i.e. promotion from struct FuncDef to struct Expr, see
      resolveExprStep()). Accidentally, it was set again during byte-code
      generation for VDBE. What is more, in some cases it was set to a wrong
      type. For instance, built-in function randomblob() returns value to be
      encoded as MP_BIN, so its returning type is SCALAR. However, it was
      reset to INTEGER (as its first argument). This patch simply removes this
      second type promotion.
      e93932b5
    • Nikita Pettik's avatar
      sql: make type in column-meta be consistent with NoSQL names · 160579d8
      Nikita Pettik authored
      Column meta-information which is sent alongside execution result via
      IProto protocol, contains string representation of column type.
      In some cases, name of type is different from actual type of field. For
      instance, if column has type SCALAR, string representation in
      meta-information will be "BLOB"; for NUMBER NoSQL type - it will be
      "NUMERIC"; for STRING - "TEXT". Instead of this mess, let's always return
      exact name of underlying NoSQL type.
      160579d8
    • Nikita Pettik's avatar
      sql: introduce ALTER TABLE ADD CONSTRAINT UNIQUE/PRIMARY KEY · 91d57dce
      Nikita Pettik authored
      Table (aka space) can be created without indexes at least from Lua-land
      (note that according ANSI SQL table may lack PK). Since there were no
      facilities to create primary key constraint on already existing table,
      lets extend ADD CONSTRAINT statement with UNIQUE and PRIMARY KEY
      clauses. In this case, UNIQUE works exactly in the same way as CREATE
      UNIQUE INDEX ... statement does.  In Tarantool primary index is an index
      with id == 0, so during execution of ADD CONSTRAINT PRIMARY KEY we check
      that there is no any entries in _index space and create that one.
      Otherwise, error is raised.
      
      Part of #3097
      Follow-up #3914
      
      @TarantoolBot document
      Title: ALTER TABLE ADD CONSTRAINT UNIQUE/PK
      
      Currently, tables which lack primary key can take part neither in DDL
      nor DQL routines. Attempt to do this leads to error. Such tables
      (without PK) may appear as spaces created from Lua NoSQL interface.
      To improve NoSQL-SQL interoperability, we are introducing way to add
      primary key to already existing table:
      
      ALTER TABLE <table name> ADD CONSTRAINT <constraint name> PRIMARY KEY(<column list>)
      
      And alongside with this another one alias to CREATE UNIQUE INDEX:
      
      ALTER TABLE <table name> ADD CONSTRAINT <constraint name> UNIQUE(<column list>)
      
      Note that Tarantool PK constraint should be explicitly added before
      any other unique constraints or secondary indexes. Otherwise, error is
      raised: "can not add a secondary key before primary".
      91d57dce
    • Nikita Pettik's avatar
      sql: fix error message for improperly created index · 1410d729
      Nikita Pettik authored
      Table can be created without any indexes (for instance, from Lua-land).
      On the other hand, bytecode generated for CREATE INDEX statement
      attempts at finding entry in _index space with given space id.
      In case it is not found error "wrong space id" is raised. On the other
      hand, it is obvious that such message is appeared when table doesn't
      have any created indexes yet. Hence, lets fix this message to indicate
      that primary key should be created before any secondary indexes.
      
      Closes #3914
      1410d729
    • Nikita Pettik's avatar
      sql: refactor getNewIid() function · b1d9ff98
      Nikita Pettik authored
      This commit includes no functional changes. Lets simply rewrite
      getNewIid() function according to Tarantool codestyle.
      
      Part of #3914
      b1d9ff98
    • Nikita Pettik's avatar
      sql: rework ALTER TABLE grammar · 3c2146e4
      Nikita Pettik authored
      Since ALTER TABLE ADD CONSTRAINT is going to be used to add various
      constraint types (foreign key, unique etc), let's a bit refactor it to
      make it more extendable.
      
      Needed for #3097
      3c2146e4
    • Nikita Pettik's avatar
      sql: introduce structs assembling DDL arguments during parsing · 1701c134
      Nikita Pettik authored
      Parser's rules implementing DDL have a lot in common. For instance,
      to drop any entity it is enough to know its name and name of table it is
      related to.
      Thus, it was suggested to arrange arguments of DDL rules into
      hierarchical structure. The root of chain always includes name of table
      to be altered: all existing entities are related to some table.  Then
      comes one of drop/create/rename rules. Indeed, each DDL operation can be
      classified in these terms, at least until we introduce ALTER TABLE ALTER
      CONSTRAINT statement. Drop is represented by single structure (as it was
      mentioned); rename can be applied only to table; create can be applied
      to CONSTRAINT (indexes are considered as constraints) and TRIGGER, which
      in turn are different in arguments required to create those objects. And
      so forth.
      
      What is more, we are going to introduce ALTER TABLE ADD CONSTRAINT
      UNIQUE With new hierarchy we can extend ALTER TABLE statement with ease:
      basic structures (alter -> create entity -> create constraint) are the
      same for .. FOREIGN KEY/UNIQUE, but the last one will be different.
      
      Finally, during refactoring we've managed to rework syntax for named
      constraint, so that now only real constraints (i.e. UNIQUE, PRIMARY KEY,
      CHECK and FOREIGN KEY) can have names; others column attributes like
      DEFAULT or COLLATE can't.
      
      Needed for #3097
      Closes #3820
      1701c134
    • Nikita Pettik's avatar
      sql: remove remains of CREATE TABLE AS SELECT stmt · 512aa15b
      Nikita Pettik authored
      This statement is completely broken and to be re-implemented in
      scope of #3223 issue. Current patch removes remains of this feature.
      512aa15b
    • Nikita Pettik's avatar
      sql: simplify VIEW creation procedure · a474ec11
      Nikita Pettik authored
      Common table and view are created in a similar two-step way: firstly
      called sqlStartTable() to allocate and initialize struct space_def
      (which will hold meta-information about space to be created) and provide
      set of checks (e.g. test if space already exists, if name is valid
      identifier etc); then called sqlEndTable() which emits byte-code to make
      insertion to _space, _index, _fk_constraint and so on. On the other
      hand, VIEW can feature neither indexes nor constraints. As a result, it
      makes no sense to invoke whole procedure of table creation. In other
      words, we can short it up to emitting byte-code for single insertion to
      _space system space.
      a474ec11
    • Nikita Pettik's avatar
      sql: don't compute string for VIEW twice · e6adec5e
      Nikita Pettik authored
      String containing 'CREATE VIEW' statement is calculated in
      sql_create_view() function and saved to space_def.opts.sql.
      After that, right before code generation, we compute it again and
      pass to sql_encode_table_opts() as a separate parameters.
      Lets avoid that second and redundant calculation and use one
      from space opts everywhere.
      e6adec5e
    • Nikita Pettik's avatar
      sql: disallow creation of FK referencing space without PK · 0230fa17
      Nikita Pettik authored
      In our SQL implementation we can omit list of referenced columns in
      FOREIGN KEY constraint:
      
      ... FOREIGN KEY (id) REFERENCES parent;
      
      In this case columns composing primary key are used. Hence, it makes
      it impossible to use this variant of FK statement if space doesn't have
      primary key. Let's now raise an error in this case.
      0230fa17
    • Nikita Pettik's avatar
      sql: disallow creation of index on space without format · 96b9aa84
      Nikita Pettik authored
      During index creation from SQL we should resolve columns names, which in
      turn impossible for spaces without format. Now it leads to crash. Let's
      fix it by raising corresponding diag error.
      96b9aa84
    • Nikita Pettik's avatar
      Fix creation of FK constraint in case of no child's PK · dae66131
      Nikita Pettik authored
      on_replace_dd_fk_constraint() assumes that child's primary index
      exists: it is used to verify emptiness of space invoking index_size().
      This commit fixes this obvious bug which could lead to crash.
      dae66131
    • Nikita Pettik's avatar
      Drop foreign keys before indexes in space:drop() · c3621949
      Nikita Pettik authored
      Implicitly involved in foreign key constraint index can't be dropped
      before constraint itself. So, to drop self-referenced table, we should
      firstly drop foreign key constraints, and only then indexes.
      c3621949
    • Mergen Imeev's avatar
      sql: parameter binding for box.execute() · 9cff3978
      Mergen Imeev authored
      This patch defines parameters binding for SQL statements executed
      through box.execute().
      
      Closes #3401
      9cff3978
    • Mergen Imeev's avatar
      sql: remove box.sql.execute() · 9ebc41dc
      Mergen Imeev authored
      This patch replaces box.sql.execute() by box.execute() in tests
      and removes box.sql.execute() from code.
      
      Closes #3505
      9ebc41dc
    • Mergen Imeev's avatar
      sql: create box.execute() · c3d2f127
      Mergen Imeev authored
      This patch creates the method dump_lua() for port_sql and uses it
      in the new function box.execute(). The function box.execute()
      replaces box.sql.execute() in the next patch.
      
      Part of #3505
      c3d2f127
    • Kirill Shcherbatov's avatar
      sql: export sql_bind structure and API · 18d4d37b
      Kirill Shcherbatov authored
      Exported sql_bind structure, sql_bind_decode, sql_bind_column
      and sql_bind routines in separate module bind.h. We need SQL
      bindings in further pathes with check constraints. Also, bind
      encapsulated in execute.c prevent us from implementation of a Lua
      part of forthcoming box.execute().
      
      Needed for #3691, #3505
      18d4d37b
    • Mergen Imeev's avatar
      iproto: create port_sql · fc1df2c5
      Mergen Imeev authored
      This patch creates port_sql implementation for the port. This will
      allow us to dump sql responses to obuf or to Lua. Also this patch
      defines methods dump_msgpack() and destroy() of port_sql.
      
      Part of #3505
      fc1df2c5
    • Mergen Imeev's avatar
      lua: remove exceptions from function luaL_tofield() · 161d6d04
      Mergen Imeev authored
      Before this commit, the luaL_tofield() function threw a Lua
      exception when an error occurred. This behavior has been changed
      in this patch. Now it sets diag_set() and returns -1 in case of
      an error.
      
      Needed for #3505
      161d6d04
    • Mergen Imeev's avatar
      sql: remove field zErrMsg from struct Parse · 600fdd77
      Mergen Imeev authored
      This field become unused and should be removed.
      
      Part of #3965
      600fdd77
  2. Apr 01, 2019
    • Ivan Koptelov's avatar
      sql: fix flaky collation test · ecd110c7
      Ivan Koptelov authored
      Sometimes this test failed because collation 'c' was not dropped
      automatically. Now it is always dropped manually, so the problem
      is no more.
      ecd110c7
    • Ivan Koptelov's avatar
      sql: rename instr to position & add collation usage · 896f396c
      Ivan Koptelov authored
      Before this patch we had instr() SQL function.
      After the patch it is renamed to position() for a better ANSI
      compatibility. Also a few things have been changed: arguments
      order, allowed arguments types and usage of collations.
      
      Note: after the patch position() syntax is still different
      from ANSI. The desirable syntax is POSITION(substring IN string).
      It is not possible to implement right now, because in our SQL
      grammar we lack expr types. We have only basic 'expr' type and
      it is not possible to write unambiguous rule for POSITION IN.
      To solve this we have to refactor grammar and add something
      like 'string_expr' (as it is done in other DBs grammars)
      
      Workaround for #3933
      
      @TarantoolBot document
      Title: instr() is replaced with position()
      Name and order of the arguments has changed for a better ANSI
      compatibility:
      Before: instr(haystack, needle).
      After: position(needle, haystack).
      
      Type checking became more strict: before it was
      possible to call the function with INTEGER arguments
      or with arguments of different types. Now both arguments
      must have the same type and be either text or binary
      strings.
      
      Before the patch collations were not taken into
      consideration during the search. Now it is fixed, and
      both implicit (column) collations and explicit
      (using COLLATE expression) are used. Single collation which
      would be used in function is determined using ANSI
      “Type combination” rules.
      896f396c
    • Ivan Koptelov's avatar
      sql: add better collation determination in functions · 86e80544
      Ivan Koptelov authored
      Before the patch determination of collation in SQL functions
      was too narrow: the arguments were scanned
      from left to right, till the argument with collation was
      encountered, then its collation was used.
      Now every arguments collation is considered. The right collation
      which would be used in function is determined using ANSI
      compatibility rules ("type combination" rules).
      86e80544
    • Vladimir Davydov's avatar
      vinyl: factor out procedure looking up LSM tree range intersection · f04e567c
      Vladimir Davydov authored
      It's an independent piece of code that is definitely worth moving
      from vy_scheduler to vy_lsm internals anyway. Besides, having it
      wrapped up in a separate function will make it easier to patch.
      f04e567c
  3. Feb 13, 2019
    • Mergen Imeev's avatar
      sql: remove box.sql.debug() · 952803fb
      Mergen Imeev authored
      Due to removal of box.sql.execute(), it makes sense to completely
      remove box.sql. This patch moves the SQL statistics to box.stat
      and removes box.sql.debug().
      
      Part or #3505
      952803fb
    • Mergen Imeev's avatar
      sql: fix error code for SQL errors in execute.c · a11e821f
      Mergen Imeev authored
      Currently, functions sql_execute() and sql_prepare_and_execute()
      set the ER_SQL_EXECUTE code for all errors that occur during the
      execution of a SQL command. This is considered incorrect because
      some of these errors may have their own error code. After this
      patch, only errors without an error code will have the error code
      ER_SQL_EXECUTE.
      
      Part of #3505
      a11e821f
    • Mergen Imeev's avatar
      sql: add column name to SQL change counter · 0959062a
      Mergen Imeev authored
      Currently, if the count_changes pragma is enabled, the INSERT,
      REPLACE and UPDATE statements will return the number of changes at
      execution time. This patch sets an INTEGER type for this result.
      
      Follow up #3832
      Needed from #3505
      0959062a
  4. Mar 29, 2019
    • Vladislav Shpilevoy's avatar
      swim: follow-ups for SWIM anti-entropy · 5f494092
      Vladislav Shpilevoy authored
      * fix some obvious errors in swim test utils;
      
      * fix a bug with NULL URI garbage on recfg;
      
      * fix typos in a comment and in a log message in swim.c;
      
      * do not start any timers in swim_cfg. Indeed, round timer should
        start only when at least one new member is added except self,
        and it is already done in swim_new_member;
      
      * log not only round begin, but each round step - it helps in
        debug, but does not affect production anyway because the logs
        are verbose;
      
      * in SWIM's event loop log new watch value instead of the old
        one - turned out, that new is more useful for debug;
      
      * log 'process <name> component' inside swim_process_<name>()
        functions. It is needed for failure detection, where a log of
        kind 'process failure detection' says nothing - much better to
        say 'process ping from', or 'process ack';
      
      * in swim tests instead of swim_cluster_wait_...(max_steps) use
        swim_cluster_wait_...(timeout). Step count restriction appeared
        to be useful for anti-entropy being equal to number of round
        steps, but it is not so once failure detection appears. Replies
        for failure detection requests does not depend on SWIM
        heartbeat and affect step count in a not trivial way - it makes
        test writing, debugging and supporting much harder.
      
      Follow-up for 03b9a6e9
      5f494092
  5. Mar 28, 2019
    • Vladimir Davydov's avatar
      Revert "test: skip ddl test for vinyl on travis" · cedb2983
      Vladimir Davydov authored
      This reverts commit c2de45c4.
      
      Not needed anymore as the DDL bug it was supposed to work around has
      been finally fixed.
      
      Follow-up #3420
      cedb2983
    • Vladimir Davydov's avatar
      vinyl: abort affected transactions when space is removed from cache · d3e12369
      Vladimir Davydov authored
      A DDL operation creates a new struct space container, moving unaffected
      indexes from the old container, then destroying it. The problem is there
      may be a DML request for this space, which was passed the old container
      in the argument list and then yielded on disk read. When it resumes, it
      may try to dereference the old space container, which may have already
      been destroyed. This will most certainly result in a crash.
      
      To address this problem, we introduce a new space callback, invalidate,
      which is called for the old space on space_cache_replace(). In case of
      vinyl, this callback aborts all transactions involving the space. To
      prevent a DML request from dereferencing a destroyed space, we also make
      the iterator code check the current transaction state after each yield
      and return an error if it was aborted. This should make any DML request
      bail out early without dereferencing the space anymore.
      
      Closes #3420
      d3e12369
    • Vladimir Davydov's avatar
      vinyl: make tx_manager_abort_writers_for_ddl more thorough · 4f62ec21
      Vladimir Davydov authored
      We need to abort all transactions writing to an altered space when
      a new index is built. Currently, we use the write set to look up such
      transactions, but it isn't quite correct, because a transaction could
      yield on disk read before inserting a statement into the write set.
      To address this problem, this patch adds vy_tx->last_stmt_space, which
      points to the space affected by the last prepared transaction. Now,
      tx_manager_abort_writers_for_ddl will look not only at the write set,
      but also at this variable to check if it needs to abort a transaction.
      
      Needed for #3420
      4f62ec21
    • Vladimir Davydov's avatar
      vinyl: simplify read iterator restoration behavior · 0621ea44
      Vladimir Davydov authored
      The 'restore' method, which is implemented by txw, cache, and memory
      sources, restores an iterator position after a yield in case the source
      has changed. It is passed the last key and it is supposed to position
      the iterator to the statement following the last key provided it had to
      reposition the iterator at all. For all kinds of sources, it first
      checks the source version and bails out early if it hasn't changed since
      the last time the iterator was used. If it has changed, it either looks
      up the statement following the given last key or tries to advance the
      iterator statement by statement (in case of a cache iterator, to avoid
      extra cache lookups).
      
      Currently, the method returns 1 if the iterator position has actually
      changed as a result of the call, 0 otherwise. That is, it returns 0 even
      if the version has changed and it had to perform a full lookup, but
      landed on the same statement. This is confusing, because in this case
      the caller has to check if it has to advance the iterator even though it
      doesn't need to, as the iterator is already positioned at the next key -
      see vy_read_iterator_scan_* family of functions.
      
      Let's simplify the restoration rules - make the 'restore' method return
      1 if it had to restore the iterator position irrespective of whether the
      position has actually changed or not. This means that in case the method
      returns 1, the caller knows that the iterator is already positioned
      properly and doesn't need to be advanced. If it returns 0, it means that
      the iterator is positioned at the same statement as before and we need
      to check if we need to advance it. This simplifies the iterator code by
      removing position checking, which would turn real nasty once multikey
      indexes are introduced. Note, comments to 'restore' methods already
      conform to this behavior (they weren't quite correct before this patch).
      
      There's a catch though - vy_read_iterator_restore_mem relies on the old
      behavior to skip the cache source in case the current key gets updated
      during yield. However, it's easy to fix that without introducing any
      extra overhead - we just need to check if the cache source is at the
      front of the iterator and clear its history if it is. BTW it turned out
      that this behavior wasn't covered by tests - when I removed the line
      invalidating the cache source from vy_read_iterator_restore_mem, all
      tests passed as usual. So while we are at it, let's also add a test case
      covering this piece of code.
      0621ea44
    • Vladimir Davydov's avatar
      vinyl: run iterator: refactor seek method · 54c23c9d
      Vladimir Davydov authored
      A few changes to make the function more straightforward:
      
       - Move bloom checking and LSN filtering out of 'do_seek' helper. Make
         the helper do just one simple task - lookup the first one in a series
         of statements matching the given search criteria.
       - Fold iterator type and key substitution in 'seek' method, similarly
         to how we did it for other iterators.
       - Cleanup EQ checks. Use the original iterator type and key where
         appropriate to remove extra checks in calling methods. Don't check EQ
         in 'seek' method in case it was checked by 'do_seek'.
       - Add some comments.
      54c23c9d
    • Vladimir Davydov's avatar
      vinyl: run iterator: zap search_ended flag · 7b244c5d
      Vladimir Davydov authored
      It's equivalent to (itr->search_started && itr->curr_stmt == NULL).
      7b244c5d
    • Vladimir Davydov's avatar
      vinyl: cache iterator: consolidate curr_stmt updates · 01a3e783
      Vladimir Davydov authored
      Currently, vy_cache_iterator->curr_stmt is updated by top-level iterator
      functions - next, skip, restore - which results in code duplication and
      spreads core logic among multiple places. To reduce the amount of code
      and make it generally easier to follow, this patch moves the updates to
      low level functions - step, seek. It also makes the seek method return
      the stop flag, which makes it similar to step, thus making the code more
      consistent.
      01a3e783
Loading