Skip to content
Snippets Groups Projects
  1. Jul 24, 2019
    • Cyrill Gorcunov's avatar
      third_party/serpent: Add serpent repo · 6d568296
      Cyrill Gorcunov authored
      We will use it for Lua console output format serialization
      
      Part-of #3834
      6d568296
    • Maria Khaydich's avatar
      Initial box.cfg call logs changes now · 0c772aae
      Maria Khaydich authored
      In contrast to subsequent calls, the initial call to box.cfg didn't log
      configuration changes to the default state. As a result, by looking at
      a log file we coudn't tell which configuration was being used.
      
      Closes #4236
      0c772aae
    • Mergen Imeev's avatar
      sql: increase row_count when adding CHECK constraint · 1b62e98e
      Mergen Imeev authored
      If CHECK constraint is added using ALTER TABLE statement, row_count
      should be incremented. Note that row_count does not increase if CHECK
      constraint is created during the execution of CREATE TABLE statement.
      For example:
      
      box.execute('CREATE TABLE t1(id INTEGER PRIMARY KEY);')
      box.execute('ALTER TABLE t1 ADD CONSTRAINT ck1 CHECK(id > 0);')
      
      Should return:
      - row_count: 1
      
      However, prior to this patch it was always equal to zero. Let's fix
      this passing appropriate flag to OP_SInsert opcode.
      
      Closes #4363
      1b62e98e
    • Serge Petrenko's avatar
      build: enable bundled libyaml for all systems. · 47b91e90
      Serge Petrenko authored
      After we fixed bundled libyaml to correctly print 4-byte Unicode
      characters, it is no longer compatible with the upstream version, so
      enable building with bundled libyaml for every platform.
      This way the tests will pass.
      
      Follow-up #4090
      47b91e90
    • Mergen Imeev's avatar
      sql: do not increase row-count if INSERT OR IGNORE fails · 1e509a03
      Mergen Imeev authored
      If INSERT statement is executed with IGNORE error action (i.e.
      INSERT OR IGNORE ...), it will return number of rows inserted.
      For example:
      
      CREATE TABLE t (i INT PRIMARY KEY, a INT check (a > 0));
      INSERT OR IGNORE INTO t VALUES (1, 1), (2, -1), (3, 2);
      
      Should return:
      ---
      - row_count: 2
      ...
      
      However it was three before this patch. So, let's account number
      of successful insertions in case of INSERT OR IGNORE.
      
      Follow-up #4188
      1e509a03
    • Mergen Imeev's avatar
      sql: remove VDBE from TXN · 85ce885e
      Mergen Imeev authored
      VDBE was added to TXN because the generated identifiers were added
      to VDBE in the sequence_next() function. Since they are now stored
      in the VDBE itself, it is not necessary to have it in TXN.
      
      Follow-up #4188
      85ce885e
    • Mergen Imeev's avatar
      sql: skip autoinc IDs generated inside SQL trigger · 7415cb01
      Mergen Imeev authored
      Currently, if an INSERT is executed inside SQL trigger and it
      results in generated autoincrement identifiers, ones will be
      displayed as a result of the statement. This is wrong, since we
      are not able to divide IDs obtained into those that belong to the
      table mentioned in the statement and those that do not belong to
      this table. This has been fixed by adding a new argument to
      OP_IdxInsert. In case the argument is not 0, recently generated
      identifier is retrieved and saved into the list, which is held in
      VDBE itself. Note that from now we don't save autoincremented
      value to VDBE right in sequence_next() - this operation is moved
      to OP_IdxInsert. So that, VDBE can be removed from struct txn.
      
      For example:
      box.execute('CREATE TABLE t1 (i INT PRIMARY KEY AUTOINCREMENT);')
      box.execute('CREATE TABLE t2 (i INT PRIMARY KEY AUTOINCREMENT);')
      box.execute('CREATE TRIGGER r AFTER INSERT ON t1 FOR EACH ROW '..
                  'BEGIN INSERT INTO t2 VALUES (null); END')
      box.execute('INSERT INTO t2 VALUES (100);')
      box.execute('INSERT INTO t1 VALUES (NULL), (NULL), (NULL);')
      
      Result should be:
      ---
      - autoincrement_ids:
        - 1
        - 2
        - 3
        row_count: 3
      ...
      
      Closes #4188
      7415cb01
    • Mergen Imeev's avatar
      sql: remove unnecessary AUTOINCREMENT ID generation · a03ec36e
      Mergen Imeev authored
      Currently, if we perform something like
      CREATE TABLE t1 (
              s1 INTEGER PRIMARY KEY AUTOINCREMENT,
              s2 INTEGER,
              CHECK (s1 <> 19)
      );
      INSERT INTO t1 VALUES (18, NULL);
      INSERT INTO t1 (s2) VALUES (NULL);
      
      we generate a new identifier in VDBE, but in any other case we
      generate it in BOX. That was needed since the CHECK did not work
      properly. This is not necessary now, because CHECK was moved to
      BOX due to issue #3691. After this patch all new identifiers will
      be generated in BOX.
      
      Part of #4188
      a03ec36e
    • Nikita Pettik's avatar
      sql: allow to specify UNSIGNED column type · a4386277
      Nikita Pettik authored
      Since all preparations concerning internal handling of unsigned values
      have been done, now nothing prevents us from using UNSIGNED type in SQL.
      This patch allows to specify UNSIGNED as a column type and adds CAST
      rules, which are the same as for casual INTEGER, but with additional
      check - result must be positive. Otherwise, error is raised.
      
      Closes #4015
      a4386277
    • Nikita Pettik's avatar
      sql: introduce extended range for INTEGER type · 41477ada
      Nikita Pettik authored
      This patch allows to operate on integer values in range [2^63, 2^64 - 1]
      It means that:
       - One can use literals from 9223372036854775808 to 18446744073709551615
       - One can pass values from mentioned range to bindings
       - One can insert and select values from mentioned range
      
      Support of built-in functions and operators has been introduced in
      previous patches.
      
      Closes #3810
      Part of #4015
      41477ada
    • Nikita Pettik's avatar
      sql: make built-in functions operate on unsigned values · cf43cd8c
      Nikita Pettik authored
      As a part of introduction unsigned type in SQL, let's patch all built-in
      function to make them accept and operate on unsigned value, i.e. values
      which come with MEM_UInt VDBE memory type.
      
      Part of #3810
      Part of #4015
      cf43cd8c
    • Nikita Pettik's avatar
      sql: refactor arithmetic operations to support unsigned ints · be389cb0
      Nikita Pettik authored
      Let's patch internal VDBE routines which add, subtract, multiply, divide
      and calculate the remainder of division to allow them take operands of
      unsigned type. In this respect, each operator now accepts signs of both
      operands and return sign of result.
      
      Part of #3810
      Part of #4015
      be389cb0
    • Nikita Pettik's avatar
      sql: separate VDBE memory holding positive and negative ints · 5d6c09b3
      Nikita Pettik authored
      As it was stated in the previous commit message, we are going to support
      operations on unsigned values. Since unsigned and signed integers have
      different memory representations, to provide correct results of
      arithmetic operations we should be able to tell whether value is signed
      or not.
      This patch introduces new type of value placed in VDBE memory cell -
      MEM_UInt. This flag means that value is integer and greater than zero,
      hence can be fitted in range [0, 2^64 - 1]. Such approach would make
      further replacing MEM_* flags with MP_ format types quite easy: during
      decoding and encoding msgpack we assume that negative integers have
      MP_INT type and positive - MP_UINT. We also add and refactor several
      auxiliary helpers to operate on integers. Note that current changes
      don't add ability to operate on unsigned integers - it is still
      unavailable.
      
      Needed for #3810
      Needed for #4015
      5d6c09b3
    • Nikita Pettik's avatar
      sql: remove sqlColumnDefault() function · e92802fc
      Nikita Pettik authored
      This routine implements obsolete mechanism to retrieve default column
      value. Since it is not used anymore, let's remove it. Note that related
      functions valieFromExpr()/valueFromFunction() are not erased since they
      look pretty useful and may be involved later.
      e92802fc
    • Nikita Pettik's avatar
      sql: refactor VDBE opcode OP_OffsetLimit · c98f971e
      Nikita Pettik authored
      OP_OffsetLimit instruction calculates sum of OFFSET and LIMIT values
      when they are present. This sum serves as a counter of entries to be
      inserted to temporary space during VDBE execution. Consider query like:
      
      SELECT * FROM t ORDER BY x LIMIT 5 OFFSET 2;
      
      To perform ordering alongside with applying limit and offset
      restrictions, first 7 (5 + 2) entries are inserted into temporary space.
      They are sorted and then first two tuples are skipped according to offset
      clause. The rest tuples from temporary space get to result set.
      
      When sum of LIMIT and OFFSET values is big enough to cause integer
      overflow, we can't apply this approach. Previously, counter was simply
      set to -1 which means that all entries from base table will be transferred
      to ephemeral space. As a result, LIMIT clause was ignored and the result
      of query would be incorrect. Motivation for this obviously wrong step was
      that to perform query with such huge limit and offset values too many time
      is required (like years). What is more, ephemeral spaces support
      auto-generated IDs in the range up to 2^32, so there's even no opportunity
      to process such queries in theory. Nevertheless, to make code cleaner
      let's fix this tricky place and just raise an overflow error if result
      of addition exceeds integer range.
      
      This patch fixes obsolete comments saying that in case of overflow
      execution won't stop; now limit and offset counter are always >= 0, so
      removed redundant branching.
      c98f971e
    • Nikita Pettik's avatar
      sql: refactor sql_atoi64() · 8c6edcf5
      Nikita Pettik authored
      We are going to allow using unsigned values in SQL and extend range of
      INTEGER type. Hence, we should be able to parse and operate on integers
      in range of [2^63, 2^64 - 1]. Current mechanism which involves
      sql_atoi64() function doesn't allow this.
      
      Let's refactor this function: firstly, get rid of manual parsing and use
      strtoll() and strtoull() functions from standard library. Then, let's
      return sign of parsed literal. In case of success now function returns 0,
      -1 otherwise.
      
      This patch also inlines sql_dec_or_hex_to_i64() to place of its only
      usage: it makes code cleaner and more straightforward.
      
      Needed for #3810
      Needed for #4015
      8c6edcf5
    • Serge Petrenko's avatar
      decimal: fix decimal.round() when scale == 0 · b0e5fa65
      Serge Petrenko authored
      Fixes decimal.round() with zero scale, fixes an error with
      decimal.round() when rounding leads to a number with the same
      precision, for example, decimal.round(9.9, 0) -> 10.
      
      Follow-up #692
      b0e5fa65
    • Kirill Yukhin's avatar
      Bump libyaml version · 7f4029c7
      Kirill Yukhin authored
      7f4029c7
    • Serge Petrenko's avatar
      decimal: add methods trim and rescale · 8221cced
      Serge Petrenko authored
      This patch adds 2 methods to decimal library and lua module:
      trim will remove all trailing fractional zeros and rescale will
      either perform rounding or append excess fractional zeros.
      
      Closes #4372
      
      @TarantoolBot document
      Title: document 2 new functions in decimal Lua module
      
      2 new functions are added to the decimal module:
      `decimal.trim()` and `decimal.rescale()`
      
      `decimal.trim()` removes any trailing fractional zeros from the
      number:
      ```
      tarantool> a = dec.new('123.45570000')
      ---
      ...
      tarantool> decimal.trim(a)
      ---
      - '123.4557'
      ...
      ```
      `decimal.rescale()` will round the number to a given scale, if it is
      less than the number scale. Otherwise it will append trailing fractional
      zeros, so that the resulting number scale will be the same as the given
      one.
      ```
      tarantool> a = dec.new(123.45)
      ---
      ...
      tarantool> dec.rescale(a,1)
      ---
      - '123.5'
      ...
      tarantool> dec.rescale(a, 4)
      ---
      - '123.4500'
      ...
      ```
      8221cced
  2. Jul 22, 2019
  3. Jul 19, 2019
    • Alexander V. Tikhonov's avatar
      test: need to run all test suites on OSX · 73319ae8
      Alexander V. Tikhonov authored
      Enabled complete testing for OSX, temporary skipped the
      failed suite replication/ and tests engine/replica_join
      with issue #4370 and small/quota from repository
      https://github.com/tarantool/small.git with issue #4357
      to enable it back. Also fixed the OSX ulimit setup
      different for VBOX and Travis-ci images.
      
      Closes #4358
      73319ae8
    • Kirill Yukhin's avatar
      Bump libsmall version · 59cc9033
      Kirill Yukhin authored
      59cc9033
    • Ivan Koptelov's avatar
      Extend range of printable unicode characters · cdf37876
      Ivan Koptelov authored
      Before the patch unicode characters encoded with 4 bytes
      were always treated as non-printable and displayed as byte
      sequences (with 'binary' tag).
      With the patch, range of printable characters is extended and
      include characters encoded with 4 bytes.
      Currently it is: (old printable range) U (icu printable range).
      Corresponding changes are also made in tarantool/libyaml.
      
      Closes: #4090
      cdf37876
    • Alexander V. Tikhonov's avatar
      test: need ASAN testing in commit process · 55f7586a
      Alexander V. Tikhonov authored
      Added ASAN tesing in commit process, used clang-8 for
      ASAN build under debian-buster image. Added for testing
      only the passing test suites, the rest of the tests
      not used and will be enabled durring issue #4360. Also
      fixed job for testing LTO with clang on debian-buster
      in travis-ci, changed it to the same as in gitlab-ci,
      changed default clang to clang-8.
      
      Closes #4359
      55f7586a
    • Serge Petrenko's avatar
      test: fix another net.box failure · 1a2addb8
      Serge Petrenko authored
      This last error
      ```
      [035]  ...
      [035]  disconnected_cnt
      [035]  ---
      [035] -- 1
      [035] +- 2
      [035]  ...
      [035]  conn:close()
      [035]  ---
      [035]  ...
      [035]  disconnected_cnt
      [035]  ---
      [035] -- 2
      [035] +- 3
      [035]  ...
      [035]  test_run:cmd('stop server connecter')
      [035]  ---
      [035]
      ```
      Happens because net.box is able to connect to tarantool before it has
      finished bootstrap. When connecting, net.box tries to fetch schema
      executing a couple of selects, but fails to pass access check since
      grants aren't applied yet. This is described in detail in
      https://github.com/tarantool/tarantool/issues/2763#issuecomment-499046998
      So, alter the test so that it tolerates multiple connection failures.
      
      Closes #4273
      1a2addb8
  4. Jul 18, 2019
    • Kirill Shcherbatov's avatar
      box: refactor key_validate_parts to return key_end · e43eeb53
      Kirill Shcherbatov authored
      The key_validate_parts helper is refactored to return a pointer
      to the end of a given key argument in case of success.
      This is required to effectively validate a sequence of keys in
      scope of functional multikey indexes.
      
      Needed for #1260
      e43eeb53
    • Kirill Shcherbatov's avatar
      box: introduce key_def->is_multikey flag · 2c7c3e44
      Kirill Shcherbatov authored
      Previously only key definitions that have JSON paths were able
      to define multikey index. We used to check multikey_path != NULL
      test to determine whether given key definition is multikey.
      In further patches with functional indexes this rule becomes
      outdated. Functional index extracted key definition may be
      multikey, but has no JSON paths.
      So an explicit is_multikey flag was introduced.
      
      Needed for #1260
      2c7c3e44
    • Kirill Yukhin's avatar
      Bump libyaml version · 53fbb587
      Kirill Yukhin authored
      53fbb587
    • Mergen Imeev's avatar
      box: increase connection timeout in "net.box.test.lua" · 79f876ad
      Mergen Imeev authored
      The "box/net.box.test.lua" test contains a check that the error
      received contains a 'timed out'. But in cases when testing was
      conducted on a slow computer or in the case of a very large load,
      it is possible that the connection time-out will be reached
      earlier than the mentioned error. In this case, the error "Invalid
      argument" will be returned. To prevent this from happening,
      this patch will increase the connection timeout.
      
      Closes #4341
      79f876ad
    • Mergen Imeev's avatar
      sql: use common registers instead of temp. for constraints data · 0fbfeed9
      Mergen Imeev authored
      Prior to this patch, data needed to form tuple to be inserted to
      _fk_constraint and _ck_constraint system spaces (to create
      corresponding constraints) was stored in the range of temporary
      register. After insertion, temporary registers are released. On
      the other hand, this data is required for providing clean-up in
      case of creation fail (i.e. removing already created constraints
      within one CREATE TABLE statement). Hence, instead of using
      temporary registers let's use ordinary ones.
      
      Closes #4183
      0fbfeed9
    • Mergen Imeev's avatar
      sql: clean-up in case constraint creation failed · c06a3d55
      Mergen Imeev authored
      This patch makes VDBE to perform a clean-up if the creation of a
      constraint fails because of the creation of two or more
      constraints of the same type with the same name and in the same
      CREATE TABLE statement.
      
      For example:
      CREATE TABLE t1(
      	id INT PRIMARY KEY,
      	CONSTRAINT ck1 CHECK(id > 1),
      	CONSTRAINT ck1 CHECK(id < 10)
      );
      
      Part of #4183
      c06a3d55
    • Mergen Imeev's avatar
      sql: add OP_SetDiag opcode in VDBE · 852bb4d4
      Mergen Imeev authored
      To separate the error setting and execution halting, a new opcode
      OP_SetDiag was created. The only functionality of the opcode is
      the execution of diag_set(). It is important to note that
      OP_SetDiag does not set is_aborted to true, so we can continue
      working with other opcodes, if necessary. This function allows us
      to perform cleanup in some special cases, for example, when
      creating a constraint failed because of the creation of two or
      more constraints with the same name in the same CREATE TABLE
      statement.
      
      Since now diag_set() is executed in OP_SetDiag, this functionality
      has been removed from OP_Halt.
      
      Needed for #4183
      852bb4d4
  5. Jul 17, 2019
    • Vladimir Davydov's avatar
      auth: fix empty password authentication · c185a387
      Vladimir Davydov authored
      We are supposed to authenticate guest user without a password. This
      used to work before commit 076a8420 ("Permit empty passwords in
      net.box"), when guest didn't have any password. Now it has an empty
      password and the check in authenticate turns out to be broken, which
      breaks assumptions made by certain connectors. This patch fixes the
      check.
      
      Closes #4327
      c185a387
  6. Jul 16, 2019
  7. Jul 15, 2019
    • Alexander V. Tikhonov's avatar
      test: vinyl/recover fails on dump counter check · 2d3f299e
      Alexander V. Tikhonov authored
      On high loaded host the test vinyl/recover failed on waiting loop
      for the dumper counter check. It expected that the value should be
      equal to "2" exactly, while on the high loaded host the dump could
      be already run more times and the counter value found to be "3" or
      even bigger values. To fix it the counter value check was changed
      from the exact value to the range bigger or equal of the expecting
      value. The error message before the fix was:
      
      [003] --- vinyl/recover.result	Mon Jul 15 10:46:00 2019
      [003] +++ vinyl/recover.reject	Mon Jul 15 10:58:10 2019
      [003] @@ -517,7 +517,7 @@
      [003]  ...
      [003]  test_run:wait_cond(function() return pk:stat().disk.dump.count == 2 end)
      [003]  ---
      [003] -- true
      [003] +- false
      [003]  ...
      [003]  sk:stat().disk.dump.count -- 1
      [003]  ---
      
      Closes #4345
      2d3f299e
    • Vladimir Davydov's avatar
      ddl: allow to execute non-yielding DDL statements in transactions · f266559b
      Vladimir Davydov authored
      The patch is pretty straightforward - all it does is moves checks for
      single statement transactions from alter.cc to txn_enable_yield_for_ddl
      so that now any DDL request may be executed in a transaction unless it
      builds an index or checks the format of a non-empty space (those are the
      only two operations that may yield).
      
      There's two things that must be noted explicitly. The first is removal
      of an assertion from priv_grant. The assertion ensured that a revoked
      privilege was in the cache. The problem is the cache is built from the
      contents of the space, see user_reload_privs. On rollback, we first
      revert the content of the space to the original state, and only then
      start invoking rollback triggers, which call priv_grant. As a result, we
      will revert the cache to the original state right after the first
      trigger is invoked and the following triggers will have no effect on it.
      Thus we have to remove this assertion.
      
      The second subtlety lays in vinyl_index_commit_modify. Before the commit
      we assumed that if statement lsn is <= vy_lsm::commit_lsn, then it must
      be local recovery from WAL. Now it's not true, because there may be
      several operations for the same index in a transaction, and they all
      will receive the same signature in on_commit trigger. We could, of
      course, try to assign different signatures to them, but that would look
      cumbersome - better simply allow lsn <= vy_lsm::commit_lsn after local
      recovery, there's actually nothing wrong about that.
      
      Closes #4083
      
      @TarantoolBot document
      Title: Transactional DDL
      
      Now it's possible to group non-yielding DDL statements into
      transactions, e.g.
      
      ```Lua
      box.begin()
      box.schema.space.create('my_space')
      box.space.my_space:create_index('primary')
      box.commit() -- or box.rollback()
      ```
      
      Most DDL statements don't yield and hence can be run from transactions.
      There are just two exceptions: creation of a new index and changing the
      format of a non-empty space. Those are long operations that may yield
      so as not to block the event loop for too long. Those statements can't
      be executed from transactions (to be more exact, such a statement must
      go first in any transaction).
      
      Also, just like in case of DML transactions in memtx, it's forbidden to
      explicitly yield in a DDL transaction by calling fiber.sleep or any
      other yielding function. If this happens, the transaction will be
      aborted and an attempt to commit it will fail.
      f266559b
    • Vladimir Davydov's avatar
      ddl: don't use space_index from AlterSpaceOp::commit,rollback · 626c5fd0
      Vladimir Davydov authored
      If there are multiple DDL operations in the same transactions, which is
      impossible now, but will be implemented soon, AlterSpaceOp::commit and
      rollback methods must not access space index map. To understand that,
      consider the following example:
      
        - on_replace: AlterSpaceOp1 creates index I1 for space S1
        - on_replace: AlterSpaceOp2 moves index I1 from space S1 to space S2
        - on_commit:  AlterSpaceOp1 commits creation of index I1
      
      AlterSpaceOp1 can't lookup I1 in S1 by id, because the index was moved
      from S1 to S2 by AlterSpaceOp2. If AlterSpaceOp1 attempts to look it up,
      it will access a wrong index.
      
      Fix that by caching pointers to old and new indexes in AlterSpaceOp on
      construct/prepare instead of using space_index() on commit/rollback to
      access them.
      626c5fd0
    • Vladimir Davydov's avatar
      memtx: fix txn_on_yield for DDL transactions · 0ae5a2d7
      Vladimir Davydov authored
      Memtx engine doesn't allow yielding inside a transaction. To achieve
      that, it installs fiber->on_yield trigger that aborts the current
      transaction (rolls it back, but leaves it be so that commit fails).
      
      There's an exception though - DDL statements are allowed to yield.
      This is required so as not to block the event loop while a new index
      is built or a space format is checked. Currently, we handle this
      exception by checking space id and omitting installation of the
      trigger for system spaces. This isn't entirely correct, because we
      may yield after a DDL statement is complete, in which case the
      transaction won't be aborted though it should:
      
        box.begin()
        box.space.my_space:create_index('my_index')
        fiber.sleep(0) -- doesn't abort the transaction!
      
      This patch fixes the problem by making the memtx engine install the
      on_yield trigger unconditionally, for all kinds of transactions, and
      instead explicitly disabling the trigger for yielding DDL operations.
      
      In order not to spread the yield-in-transaction logic between memtx
      and txn code, let's move all fiber_on_yield related stuff to txn,
      export a method to disable yields, and use the method in memtx.
      0ae5a2d7
  8. Jul 13, 2019
    • Nikita Pettik's avatar
      sql: introduce ADD CONSTRAINT CHECK statement · 260f6328
      Nikita Pettik authored
      This patch extends parser's grammar to allow to create CHECK constraints
      on already existent tables via SQL facilities.
      
      Closes #3097
      
      @TarantoolBot document
      Title: Document ADD CONSTRAINT CHECK statement
      
      Now it is possible to add CHECK constraints to already existent table
      via SQL means. To achieve this one must use following syntax:
      
      ALTER TABLE <table> ADD CONSTRAINT <name> CHECK (<expr>);
      260f6328
    • Kirill Yukhin's avatar
      Fix broken build · 307133d3
      Kirill Yukhin authored
      The argument in func_c_new() is used in Debug mode only.
      Mark it w/ MAYBE_UNUSED.
      307133d3
Loading