Skip to content
Snippets Groups Projects
  1. Nov 09, 2018
    • Mergen Imeev's avatar
      sql: refresh result-file of sql/iproto.test.lua · cfa7da04
      Mergen Imeev authored
      Test sql/iproto.test.lua fails due to its result-file being a bit
      outdated.
      
      Follow up #2618
      cfa7da04
    • Vladislav Shpilevoy's avatar
      sql: remove psql_txn from Vdbe · 50e014e5
      Vladislav Shpilevoy authored
      It makes no sense to store it here. SQL transaction
      specific things shall be taken from global txn object,
      as any transaction specific things.
      
      Follow up #2618
      50e014e5
    • Mergen Imeev's avatar
      sql: return all generated ids via IPROTO · ef67de41
      Mergen Imeev authored
      According to documentation some JDBC functions have an ability to
      return all ids that were generated in executed INSERT statement.
      This patch gives a way to implement such functionality. After
      this patch all ids autogenerated during VDBE execution will be
      saved and returned through IPROTO.
      
      Closes #2618
      ef67de41
    • Vladislav Shpilevoy's avatar
      box: factor fiber_gc out of txn_commit · 77fa1736
      Vladislav Shpilevoy authored
      Now txn_commit is judge, jury and executioner. It both
      commits and rollbacks data, and collects it calling fiber_gc,
      which destroys the region.
      
      But SQL wants to use some transactional data after commit. It is
      autogenerated identifiers - a list of sequence values generated
      for autoincrement columns and explicit sequence:next() calls.
      
      It is possible to store the list on malloced mem inside Vdbe, but
      it complicates deallocation. Much more convenient to store all
      transactional data on the transaction memory region, so it would
      be freed together with fiber_gc.
      
      After this patch applied, Vdbe takes care of txn memory
      deallocation in a finalizer routine. Between commit and
      finalization transactional data can be serialized wherever.
      
      Needed for #2618
      77fa1736
  2. Nov 03, 2018
  3. Nov 02, 2018
    • Mergen Imeev's avatar
      box: wrong is_nullable for multiple indexes · 52b84d2e
      Mergen Imeev authored
      If field isn't defined by space format, than in case of multiple
      indexes field option is_nullable was the same as it was for last
      index that defines it. This is wrong as it should be 'true' only
      if it is 'true' for all indexes that defines it.
      
      Closes #3744.
      52b84d2e
    • Nikita Pettik's avatar
      sql: return result-set type via IProto · 8997657a
      Nikita Pettik authored
      Lets evaluate an expression type during processing of expression's AST
      and code generation. It allows to calculate resulting columns data types
      and export them as IProto meta alongside with columns' names.
      Also, correct types are also returned for binding parameters as well.
      
      Note that NULL literal has type "BOOLEAN". It was made on purpose -
      different DBs interpret NULL's type in different ways: some of them
      use INT; others - VARCHAR; still others - UNKNOWN. We've decided that
      NULL is rather of type "BOOLEAN", since NULL is kind if subset of
      "BOOLEAN" values: any comparison with NULL results in neither TRUE nor
      FALSE, but in NULL.
      
      Part of #2620
      8997657a
    • Georgy Kirichenko's avatar
      sql: enforce implicit type conversions · 53b0ef87
      Georgy Kirichenko authored
      Most DBs (at least PostgreSQL, Oracle and DB2) allow to process
      following queries:
      
      CREATE TABLE t1 (id INT PRIMARY KEY);
      INSERT INTO t1 VALUES (1.123), ('2');
      
      In this particular case, 1.123 should be simply truncated to 1,
      and '2' - converted to literal number 2.
      
      After passing real type to Tarantool (instead of <SCALAR>), example
      above would fail without conversions. Thus, lets add implicit
      conversions inside VDBE to make this example be legal.
      However, still some types conversions must be prohibited. For instance,
      <BLOB> can't be converted to integer or floating point numerical,
      and vice versa.
      53b0ef87
    • Nikita Pettik's avatar
      sql: discard numeric conversion by unary plus · de53867d
      Nikita Pettik authored
      In SQLite unary plus behaves as implicit conversion to numeric type.
      Consider following example:
      
      CREATE TABLE t1 (id INT PRIMARY KEY, a TEXT, b BLOB);
      INSERT INTO t1 VALUES (1, '99', '99');
      
      SELECT * FROM t1 WHERE a = b; (*)
      SELECT * FROM t1 WHERE +a = +b; (**)
      
      Since BLOB and TEXT are incompatible, result of (*) would be empty set.
      However, comparison in second query (**) would be of <NUMERIC> types,
      and result set would consist of one tuple [1, '99', '99'].
      
      Lets discard this conversion produced by unary plus, since it implicitly
      affects result set of query and no one other DB support such behaviour.
      Instead, simply use type of operand it is related to.
      Note, that unary minus doesn't affect types in any way.
      de53867d
    • Georgy Kirichenko's avatar
      sql: pass true types of columns to Tarantool · 7752cdfd
      Georgy Kirichenko authored
      As a main part of introducing strict typing in SQL it is required to
      prohibit typeless columns in parser's grammar. Originally, SQLite simply
      assigns typeless columns to BLOB affinity. Moreover, due to historical
      reasons, all columns were stored with <SCALAR> type in Tarantool core
      (except for <INTEGER> when it comes to primary key).  Column type should
      be defined on table creation. Allowed data types are: <TEXT>, <VARCHAR>,
      <CHAR>, <BLOB>, <INT[EGER]>, <REAL>, <FLOAT>, <NUMERIC>, <DECIMAL>,
      <DOUBLE> <DATE> and <DATETIME>. However, still any declared data type is
      converted to one of <BLOB>, <TEXT>, <REAL> or <INTEGER> affinities.
      While affinity reaches space format, it is (again) converted to
      Tarantool's field type. To be more precise, table of conversions:
      
      +----------+----------+------------+
      | SQL TYPE | AFFINITY | FIELD TYPE |
      +----------+----------+------------+
      | FLOAT    | REAL     | NUMBER     |
      | REAL     | REAL     | NUMBER     |
      | DOUBLE   | REAL     | NUMBER     |
      | NUMERIC  | REAL     | NUMBER     |
      | DECIMAL  | REAL     | NUMBER     |
      | INTEGER  | INTEGER  | INTEGER    |
      | TEXT     | TEXT     | STRING     |
      | VARCHAR  | TEXT     | STRING     |
      | CHAR     | TEXT     | STRING     |
      | BLOB     | BLOB     | SCALAR     |
      | DATETIME | REAL     | NUMBER     |
      | DATE     | REAL     | NUMBER     |
      | TIME     | REAL     | NUMBER     |
      +----------+----------+------------+
      
      <VARCHAR> and <CHAR> types should be specified with length
      (e.g. name VARCHAR(10)), but this length currently is not used when
      types are processed. Only purpose is to support ANSI syntax.
      The same for <NUMERIC> and <DECIMAL> - it is allowed to specify scale
      and precision, but they don't affect the way they are stored in memory.
      
      Note that patch is not self-sufficient: a lot of tests still fail due to
      wrong types conversions. Fix for that comes as next two patches.
      
      Closes #3018
      Closes #3104
      Closes #2494
      Closes #3459
      7752cdfd
    • Georgy Kirichenko's avatar
      sql: annotate SQL functions with return type · 2b22b913
      Georgy Kirichenko authored
      Any SQL function should have declared type of returning value.
      Lets temporary add affinity to function definition as a type of
      returning value.
      2b22b913
    • Georgy Kirichenko's avatar
      sql: split conflict action and affinity for Expr · 684cb8d6
      Georgy Kirichenko authored
      Lets introduce separate field in struct Expr to store conflict action of
      RAISE() function, instead of messing it with affinity.
      684cb8d6
  4. Nov 01, 2018
    • Nikita Pettik's avatar
      sql: make DROP TABLE delete entry from _sequence_data · 0cccd5bd
      Nikita Pettik authored
      Before this patch, _sequence_data system space wasn't taken into
      consideration when space was dropped. However, entries in this space
      may appear after recovery. For example:
      
      CREATE TABLE t (id INT PRIMARY KEY AUTOINCREMENT);
      INSERT INTO t VALUES (NULL);
      box.snapshot()
      os.exit()
      ...
      box.cfg{}
      DROP TABLE t;
      
      Last DROP statement didn't generate VDBE code to delete entry from
      _sequence_data, but it had to do so. This patch simply modifies code
      generation to remove entry from _sequence_data space before space is
      dropped.
      
      Closes #3712
      0cccd5bd
    • Nikita Pettik's avatar
      sql: remove nKey from struct BtCursor · aab97195
      Nikita Pettik authored
      nKey member (indicating number of parts in key) now is used only for
      COUNT routine. On the other hand, it is always equal to 0 (as well as
      key is really NULL), since functions which are relied on this argument
      (tarantoolSqlite3Count and tarantoolSqlite3EphemeralCount) are called to
      perform OP_Count opcode. In its turn, OP_Count is emitted only for
      queries like <SELECT COUNT(*) FROM tab>, i.e. simple queries without
      filter conditions. Hence, nothing prevents us from removing this field
      at all.
      aab97195
    • Mergen Imeev's avatar
      sql: assertion in autoincrement column · fc16a650
      Mergen Imeev authored
      If query is like "INSERT INTO table SELECT ..." then it is
      completely fine that the result of SELECT isn't integer. These
      values wouldn’t be checked properly if it was inserted in
      column with autoincrement. This patch fixes mentioned check.
      
      Closes #3670
      fc16a650
    • Mergen Imeev's avatar
      sql: clean-up on failed CREATE TABLE · 77d73e72
      Mergen Imeev authored
      In case statement "CREATE TABLE ..." fails it can left some
      records in system spaces that shouldn't be there. These records
      won't be left behind after this patch.
      
      @TarantoolBot document
      Title: Clean up after failure of CREATE TABLE
      Usually CREATE TABLE creates no less than two objects which are
      space and index. If creation of index (or any object after space)
      failed, created space (and other created objects) won't be deleted
      though operation failed. Now these objects will be deleted
      properly.
      
      Closes #3592
      77d73e72
    • Georgy Kirichenko's avatar
      Show names of Lua functions in backtraces · 5b00d646
      Georgy Kirichenko authored
      Trace corresponding Lua state as well as normal C stack frames while
      fiber backtracing. This might be useful for debugging purposes.
      
      Fixes: #3538
      5b00d646
    • Georgy Kirichenko's avatar
      Use fiber lua state for triggers if possible · 7d040848
      Georgy Kirichenko authored
      Lua trigger invocation reuses a fiber lua state if exists instead of
      creating of new one for each new invocation. This is needed for a lua
      stack reconstruction during backtracing.
      
      Relates: #3538
      7d040848
    • Georgy Kirichenko's avatar
      Proper unwind for currently executing fiber · 98b2cdfa
      Georgy Kirichenko authored
      Each yielded fiber has a preserved coro state stored in a corresponding
      variable however an executing fiber has a volatile state placed in CPU
      registers (stack pointer, instruction pointer and non-volatile registers)
      and corresponding context-storing variable value is invalid.
      For already yielded fiber we have to use a special asm-written handler to make
      a temporary switch to the preserved state and capture executing context what
      is not needed for executing fiber.
      After the patch for the executing fiber NULL is passed to the backtrace
      function as coro context and then backtrace function could decide should
      it use special context-switching handler or might just use unw_getcontext
      from the unwind library.
      98b2cdfa
    • Georgy Kirichenko's avatar
      Do not inline coro unwind function · 85878bcf
      Georgy Kirichenko authored
      Do not inline coro_unwcontext because the unwind handler expects
      for a separate stack frame.
      85878bcf
    • Vladimir Davydov's avatar
      Merge branch '1.10' into 1.10-features · ef749ac3
      Vladimir Davydov authored
      ef749ac3
    • Vladimir Davydov's avatar
      httpc: fix compilation with libcurl >= 7.62.0 · 02da15f7
      Vladimir Davydov authored
      Starting from libcurl 7.62.0, CURL_SSL_CACERT is defined as a macro
      alias to CURLE_PEER_FAILED_VERIFICATION, see
      
        https://github.com/curl/curl/commit/3f3b26d6feb0667714902e836af608094235fca2
      
      This breaks compilation:
      
        httpc.c:337:7: error: duplicate case value 'CURLE_PEER_FAILED_VERIFICATION'
                case CURLE_PEER_FAILED_VERIFICATION:
                     ^
        httpc.c:336:7: note: previous case defined here
                case CURLE_SSL_CACERT:
                     ^
        curl.h:589:26: note: expanded from macro 'CURLE_SSL_CACERT'
        #define CURLE_SSL_CACERT CURLE_PEER_FAILED_VERIFICATION
                                 ^
      
      Fix this by using CURLE_SSL_CACERT only if libcurl version is less
      than 7.62.0.
      
      Note, we can't use CURL_AT_LEAST_VERSION to check libcurl version,
      because it isn't available in libcurl shipped with CentOS 6.
      02da15f7
    • Vladimir Davydov's avatar
      httpc: fix curl version check in httpc_set_keepalive · 74e20755
      Vladimir Davydov authored
      Obviously, the version check as it is now won't work once libcurl 8.0.0
      is released. Use LIBCURL_VERSION_NUM to correctly check libcurl version.
      
      Note, we can't use CURL_AT_LEAST_VERSION to check libcurl version,
      because it isn't available in libcurl shipped with CentOS 6.
      
      Fixex commit 7e62ac79 ("Add HTTP client based on libcurl").
      74e20755
    • Serge Petrenko's avatar
      Introduce a function to reencode scramble · bc43ffb0
      Serge Petrenko authored
      Introduce a new function, scramble_reencode(), which allows one cluster
      instance to reencode a scramble received from  a client using the salt
      from another cluster instance. This is needed for proxy to mimic client
      connections when connecting to remote instances.
      
      Needed for #2625
      bc43ffb0
    • Vladimir Davydov's avatar
      test: fix engine/ddl sporadic hang · 38845d6e
      Vladimir Davydov authored
      In Vinyl DDL aborts all affected writers before modifying a space so we
      must use pcall() to avoid hang.
      
      Closes #3786
      38845d6e
  5. Oct 30, 2018
    • Serge Petrenko's avatar
      box: autogrant CREATE,ALTER,DROP to users with READ+WRITE · 269e36d8
      Serge Petrenko authored
      This patch adds an upgrade script to grant CREATE, ALTER, DROP
      privileges to users which have READ+WRITE on respective objects.
      This is needed after removing 1.7 compatibility mode for privileges.
      
      Closes #3539
      269e36d8
    • Serge Petrenko's avatar
      box: remove compatibility mode for privileges · 3e7d3070
      Serge Petrenko authored
      Before version 1.7.7 there were no CREATE or ALTER privileges.
      READ+WRITE permitted object creation and altering.
      In 1.10 CREATE and ALTER privileges were introduced together with a
      compatibility mode in access_check_ddl() which assumed user had CREATE
      and ALTER if it had READ and WRITE on a respective object.
      Now its time for us to remove this compatibility mode.
      
      Part of #3539
      3e7d3070
  6. Oct 29, 2018
    • Vladimir Davydov's avatar
      replication: keep header when request is modified by before_replace · 480c55b6
      Vladimir Davydov authored
      When space.before_replace trigger modifies the result of a remote
      operation, we clear the request header so that it gets rebuilt on
      commit. This is incorrect, because as a result we don't bump the
      master's component of the replica's vclock, which leads to the request
      being applied again when the replica reconnects. The issue manifests
      itself in sporadic replication/before_replace test failures.
      
      Fix it by updating the request header rather than clearing it so that
      replica id and lsn get preserved.
      
      Closes #3722
      480c55b6
    • Serge Petrenko's avatar
      hot_standby: reflect amount of recovered rows in box.info · 85299d97
      Serge Petrenko authored
      To be able to switch to hot_standby instance with minimal downtime, we
      need to know how far is it behind the primary instance, i.e. up to what
      vclock we have recovered. Previously this was impossible because
      box.info.vclock always referenced replicaset.vclock, which isn't updated
      during hot_standby.
      
      Introduce a pointer to relevant vclock: either recovery vclock (during
      local recovery) or replicaset.vclock (at all other times) and use it in
      box.info.vclock, box.info.lsn and box.info.signature.
      
      @locker: renamed last_row_vclock to box_vclock and constified it.
      
      Closes #3002
      85299d97
    • Alexander Turenko's avatar
      test: fix unix socket conflict in socket.test.lua · f950eb31
      Alexander Turenko authored
      Increased socket.readable / socket.wait timeouts.
      
      Rewritten port choosing: repeat bind+listen until succeeds, exclude
      incorrect port 65536 from the range.
      
      All these changes are needed to run the test in parallel on several
      test-run workers to investigate flaky failures of the test / of the
      suite. Some of these changes can also eliminate possible flaky failures.
      f950eb31
  7. Oct 26, 2018
    • Kirill Yukhin's avatar
      sql: check read access while executing SQL query · e5834e43
      Kirill Yukhin authored
      Since SQL front-end is not using box API,
      no checkes for read access are performed by VDBE engine.
      Add check to IteratorOpen op-code to make sure that read
      privilege exists for given space.
      Note, that there's is no need to perform DML/DDL checkes as
      they're performed by Tarantool's core.
      
      @TarantoolBot document
      Title: Document behaviour of SQL in presence of
      read access restrictions. Need to clarify, that
      if there's no read access to the space, then not
      only SELECT statements will fail, but also those DML
      which implies reading from spaces indirectly, e.g.:
        UPDATE t1 SET a=2 WHERE b=3;
      
      Closes #2362
      e5834e43
    • Kirill Yukhin's avatar
      sql: use space_by_name in SQL · 5b7cc294
      Kirill Yukhin authored
      Since hash, which maps space name to space pointer was introduced in
      previous patch, use it in SQL front-end as it is heavily needed.
      
      @locker: removed a couple of useless assertions and variable renames
      5b7cc294
    • Kirill Yukhin's avatar
      schema: add space name cache · 030d28f5
      Kirill Yukhin authored
      Since SQL is heavily using name -> space mapping, introduce (instead of
      scanning _space space) dedicated cache, which maps space name to space
      pointer.
      
      @locker: code cleanup
      030d28f5
    • Vladimir Davydov's avatar
      alter: install space commit/rollback triggers before preparing sql view · d792fd26
      Vladimir Davydov authored
      sql_compile_view() may fail, in which case the space will never be
      deleted from (in case of space creation) or inserted back into (in case
      of space drop) the space cache, because commit/rollback triggers, which
      are supposed to do the job, are only installed after preparing a view.
      Fix this by installing triggers before sql_compile_view().
      
      No need to write a test as without this commit sql/view test will
      crash after applying the next commit (the one that introduces space
      name cache).
      
      Fixes commit dc358cb0 ("sql: rework VIEW internals").
      d792fd26
    • Vladimir Davydov's avatar
      Merge branch '1.10-features' into 2.1 · 9975ed06
      Vladimir Davydov authored
      9975ed06
    • Vladimir Davydov's avatar
      Merge branch '1.10' into 1.10-features · 9dcc7849
      Vladimir Davydov authored
      9dcc7849
Loading