Skip to content
Snippets Groups Projects
  1. Oct 31, 2023
    • Mergen Imeev's avatar
      sql: second lookup for constraint names · a9d07e90
      Mergen Imeev authored
      This patch add second lookup for constraint name in the DROP CONSTRAINT
      statement.
      
      Part of #4467
      
      @TarantoolBot document
      Title: Second lookup for names in SQL
      
      Since all names are now case sensitive, a second lookup is now supported
      to maintain backward compatibility. A second lookup by name will be
      performed if the object was not found by name in the first lookup and if
      the name is not enclosed in double quotation marks. The name for the
      second lookup is normalized using old rules.
      a9d07e90
    • Mergen Imeev's avatar
      sql: second lookup for index names · 69fc9e0b
      Mergen Imeev authored
      This patch add second lookup for index name in the following cases:
      - index in INDEXED BY clause;
      - index in DROP INDEX statement;
      - index in PRAGMA statement.
      
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      69fc9e0b
    • Mergen Imeev's avatar
      sql: second lookup for function names · ba5df3f2
      Mergen Imeev authored
      This patch adds second lookup for function names in expressions.
      
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      ba5df3f2
    • Mergen Imeev's avatar
      sql: second lookup for column names · 44a055d7
      Mergen Imeev authored
      This patch add second lookup for column name in the following cases:
      - columns in tuple foreign key creation clause;
      - columns in field foreign key creation clause;
      - columns in primary key creation clause;
      - columns in unique constraint creation clause;
      - columns in expressions;
      - columns in UPDATE TABLE statement;
      - columns in INSERT INTO statement;
      - columns in CREATE INDEX statement;
      
      Also, second lookup for table name in expressions also now supported.
      
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      44a055d7
    • Mergen Imeev's avatar
      sql: second lookup for space names · 66234c63
      Mergen Imeev authored
      This patch add second lookup for space name in the following cases:
      - foreign table in tuple foreign key creation clause;
      - foreign table in field foreign key creation clause;
      - table in DROP TABLE statement;
      - view in DROP VIEW statement;
      - table in SELECT FROM statement;
      - table in DELETE FROM statement;
      - table in TRUNCATE TABLE statement;
      - table in UPDATE TABLE statement;
      - table in INSERT INTO statement;
      - table in CREATE INDEX statement;
      - table in DROP INDEX statement;
      - table in PRAGMA statement;
      - table in SHOW CREATE TABLE statement;
      - table in CREATE TRIGGER statement;
      - table in ALTER TABLE statements;
      
      Table names in expressions will be supported in subsequent commits.
      
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      66234c63
    • Mergen Imeev's avatar
      sql: second lookup for collation names · 2b164f62
      Mergen Imeev authored
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      2b164f62
    • Mergen Imeev's avatar
      sql: second lookup for savepoint names · ca271f86
      Mergen Imeev authored
      This patch introduces a second lookup for the savepoint name in the
      "RELEASE savepoint_name" and "ROLLBACK TO savepoint_name" statements.
      
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      ca271f86
    • Mergen Imeev's avatar
      sql: make SQL case-sensitive · aa68c46a
      Mergen Imeev authored
      This patch makes identifiers in SQL case sensitive. Prior to this patch,
      these identifiers were normalized unless they were enclosed in double
      quotes.
      
      Part of #4467
      
      NO_DOC=will be added later
      NO_CHANGELOG=will be added later
      aa68c46a
    • Mergen Imeev's avatar
      sql: introduce sql_coll_id_by_expr() · a6104077
      Mergen Imeev authored
      This patch introduces the sql_coll_id_by_expr() function. This function
      is used to search for a collation based on a given expression.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      a6104077
    • Mergen Imeev's avatar
      sql: introduce sql_coll_id_by_token() · cca8dfad
      Mergen Imeev authored
      This patch introduces the sql_coll_id_by_token() function. This function
      is used to search for a collation based on a given token.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      cca8dfad
    • Mergen Imeev's avatar
      sql: introduce sql_fieldno_by_item() · 0c4315f0
      Mergen Imeev authored
      This patch introduces the sql_fieldno_by_item() function. This function
      is used to search for a space field based on a given element of the
      struct ExprList.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      0c4315f0
    • Mergen Imeev's avatar
      sql: introduce sql_fieldno_by_expr() · 66e62448
      Mergen Imeev authored
      This patch introduces the sql_fieldno_by_expr() function. This function
      is used to search for a space field based on a given expression.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      66e62448
    • Mergen Imeev's avatar
      sql: introduce sql_fieldno_by_id() · 51c4cce3
      Mergen Imeev authored
      This patch introduces the sql_fieldno_by_id() function. This function is
      used to search for a space field based on a given element of the
      struct IdList.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      51c4cce3
    • Mergen Imeev's avatar
      sql: introduce sql_index_id_by_src() · a887a1f4
      Mergen Imeev authored
      This patch introduces the sql_index_id_by_src() function. This function
      is used to search for an index based on a given element of the
      struct SrcList.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      a887a1f4
    • Mergen Imeev's avatar
      sql: introduce sql_space_by_src() · 45ca4a3a
      Mergen Imeev authored
      This patch introduces the sql_space_by_src() function. This function is
      used to search for a space based on a given element of the
      struct SrcList. The main purpose of this function is to introduce a
      second lookup for spaces, so this function is not used where a second
      lookup is not required.
      
      Needed for #4467
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      45ca4a3a
    • Mergen Imeev's avatar
      sql: check space for SHOW CREATE TABLE · b3377ccf
      Mergen Imeev authored
      Before this patch, the existence of an space for a
      "SHOW CREATE TABLE table_name" statement was checked at runtime. This is
      quite inconvenient for #4467, so this will now be checked when creating
      the VDBE.
      
      Needed for #4467
      
      NO_DOC=no user-visible changes
      NO_TEST=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      b3377ccf
    • Mergen Imeev's avatar
      sql: check index for DROP INDEX before runtime · fa0de2bf
      Mergen Imeev authored
      Before this patch, the existence of an index for a DROP INDEX statement
      was checked at runtime. This is quite inconvenient for #4467, so this
      will now be checked when creating the VDBE.
      
      Needed for #4467
      
      NO_DOC=no user-visible changes
      NO_TEST=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      fa0de2bf
    • Mergen Imeev's avatar
      sql: refactor pragma.c · b44fc36f
      Mergen Imeev authored
      This patch refactors functions in pragma.c and fixes an issue with
      incorrectly ordered arguments in sql_pragma_index_info().
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      b44fc36f
    • Mergen Imeev's avatar
      sql: remove unused struct Expr flags · b88ce35e
      Mergen Imeev authored
      This patch removes the EP_InfixFunc, EP_DblQuoted and EP_Alias flags as
      they are not used.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      b88ce35e
    • Mergen Imeev's avatar
      sql: remove unused functions · 380dd1a1
      Mergen Imeev authored
      This patch removes the sql_id_eq_str_expr() and
      vdbe_emit_stat_space_clear() functions as they are no longer used.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      380dd1a1
  2. Oct 27, 2023
    • Georgiy Lebedev's avatar
      box: support tuple formats in IPROTO `call` and `eval` request arguments · d79e7a3c
      Georgiy Lebedev authored
      Since tuple format cannot be created in IPROTO thread, we only save the
      tuple format data and later create a tuple format map from it when
      processing `call` and `eval` requests.
      
      Closes #8633
      
      @TarantoolBot document
      Title: Tuple formats in IPROTO
      
      New MsgPack extension was added, `MP_TUPLE`, which is used exclusively
      in IPROTO. It has the following structure:
      +----------+-----------------+----------+
      |MP_EXT = 7|      MP_UINT    | MP_ARRAY |
      +----------+-----------------+----------+
                  ^                 ^
                  format identifier tuple data
      
      New IPROTO features were added:
      * `IPROTO_FEATURE_DML_TUPLE_EXTENSION = 7` — tuples in `IPROTO_DATA` are
         encoded as `MP_TUPLE` and tuple format is sent in response to DML
         requests.
      * `IPROTO_FEATURE_CALL_RET_TUPLE_EXTENSION = 8` —tuples in `IPROTO_DATA`
        are encoded as `MP_TUPLE` and tuple format is sent in response to `call`
        and `eval` requests.
      * `IPROTO_FEATURE_CALL_ARG_TUPLE_EXTENSION = 9` — tuples in `IPROTO_TUPLE`
        of `call` and `eval` requests are encoded as `MP_TUPLE` with their
        formats encoded as described below.
      
      The `IPROTO_FEATURE_DML_TUPLE_EXTENSION` feature can be enabled in
      `net.box` by setting the `fetch_schema` option to `false` when creating a
      new connection.
      
      A new IPROTO key for bodies of responses with data and bodies of `call` and
      `eval` requests: `IPROTO_TUPLE_FORMATS = 0x60` — a `MP_MAP` consisting of
      `MP_UINT` keys (format identifiers) and `MP_STRING` values (tuple format
      clauses).
      
      A new backward compatibility option was added (and a new page,
      `https://tarantool.io/compat/box_tuple_extension`, needs to be created for
      it): `BOX_TUPLE_EXTENSION_BRIEF` — it controls the
      `IPROTO_FEATURE_CALL_RET_TUPLE_EXTENSION` and
      `IPROTO_FEATURE_CALL_ARG_TUPLE_EXTENSION` feature bits. It was added for
      clients that assume to receive `MP_ARRAY`s in `call` and `eval` requests
      arguments and response data.
      
      Please refer to the design document for [details](https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#7d86000c69cb4f8087a289f2caad73a0).
      d79e7a3c
    • Georgiy Lebedev's avatar
      box: add MsgPack encoding/decoding context to MsgPack port · 4d66f1eb
      Georgiy Lebedev authored
      Extend the MsgPack port to store a pointer to a MsgPack context (the
      lifetime of a MsgPack port is static, so we simply allocate it on the
      stack), avoiding copying. This context can later be used for dumping
      MsgPack to Lua, and possibly acquired by a MsgPack object when the
      'MsgPack object' mode of `port_dump_lua` is used.
      
      Needed for #8147
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      4d66f1eb
    • Georgiy Lebedev's avatar
      box: support tuple formats in IPROTO responses · cf484f07
      Georgiy Lebedev authored
      Tuple formats are sent in a separate IPROTO_TUPLE_FORMATS response body
      field, which is namely a mapping from format identifiers to serialized
      tuple formats.
      
      The algorithm for parsing IPROTO response body is the following:
      1. Parse tuple formats and build a tuple format index.
      2. Parse data passing the tuple format index in the decoding context.
      
      Add new backward compatibility option `box_tuple_extension` which controls
      the server's `IPROTO_FEATURE_CALL_RET_TUPLE_EXTENSION` bit: needed for
      users that don't expect the new tuple extension to be returned in call and
      eval responses instead of basic MsgPack types. This option will
      also be used for `IPROTO_FEATURE_CALL_ARG_TUPLE_EXTENSION` bit with the
      same purpose.
      
      Closes #8147
      
      NO_DOC=<added in a later commit together with documentation of tuple
              formats in IPROTO `call` and `eval` arguments>
      cf484f07
    • Georgiy Lebedev's avatar
      merger: handle `MP_TUPLE` extension in merge buffer source `next` method · 561b5559
      Georgiy Lebedev authored
      In scope of tarantool/tarantool#8147, a new context-dependent extension for
      box tuples, `MP_TUPLE`, is introduced. The buffer source uses a buffer with
      raw MsgPack, which does not allow for passing the context required for
      decoding `MP_TUPLE`, so, in order to decode it, we need to manually skip
      the extension header and the tuple format identifier to get to the tuple
      data and create a tuple. We can ignore the tuple format identifier (and the
      tuple format that was originally sent for this tuple), since the format is
      provided by the merger itself.
      
      Needed for #8147
      
      NO_CHANGELOG=<internal change>
      NO_TEST=<tested by integration tests>
      NO_DOC=<internal change>
      561b5559
    • Georgiy Lebedev's avatar
      lua: add MsgPack encoding/decoding context to MsgPack objects · 9f023095
      Georgiy Lebedev authored
      Extend the MsgPack object creation interface to push a MsgPack context
      required for decoding tuples coming from IPROTO, the context ownership is
      acquired by the new object to avoid copying (thus, a virtual `move`
      function is added to the MsgPack context).
      
      When a MsgPack context is owned by a MsgPack object, the lifetime of the
      former is controlled by the latter, hence a virtual `destroy` function is
      added to the MsgPack context.
      
      Needed for #8147
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      9f023095
    • Georgiy Lebedev's avatar
      box: introduce `MP_TUPLE` extension · 11d5f1e9
      Georgiy Lebedev authored
      Add new MsgPack extension for tuples MP_TUPLE with the following structure:
      +------+-----------------+----------+
      |MP_EXT|      MP_UINT    | MP_ARRAY |
      +------+-----------------+----------+
              ^                 ^
              format identifier tuple data
      
      Needed for #8147
      
      NO_CHANGELOG=<internal change>
      NO_DOC=<internal change>
      11d5f1e9
    • Georgiy Lebedev's avatar
      core: extend `port_dump_lua` with 'Msgpack object' mode · 11dad290
      Georgiy Lebedev authored
      When data is coming from IPROTO and is passed to Lua, it is convenient to
      pass raw MsgPack data (see `box.schema.func::takes_raw_args` option).
      Currently, the port's MsgPack is retrieved directly via `port_get_msgpack`
      and then a MsgPack object is constructed via `luamp_push`. In the future,
      ports can have a MsgPack decoding context that will also be needed to be
      passed to the MsgPack object constructor, so we encapsulate the MsgPack
      object construction completely into a port method by introducing a new
      operation mode to `port_dump_lua`.
      
      This new operation mode is essentially a no-op for a MsgPack port (which is
      the case for data coming from IPROTO), and is completely inefficient for
      other types of ports (since an intermediate conversion to MsgPack occurs),
      but we do not expect users to be calling Lua stored procedures with the
      `takes_raw_args` locally. To handle the latter case we introduce a
      `port_dump_lua_mp_object_mode_slow` helper function which facilitates
      retrieving MsgPack data from ports via region allocation, creating a
      Msgpack object and freeing the MsgPack data.
      
      Needed for #8147
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      11dad290
    • Georgiy Lebedev's avatar
      core: refactor `port_dump_lua` operation modes · d1f1faa1
      Georgiy Lebedev authored
      Currently, `port_dump_lua` has 2 modes of operation: 'flat' and 'tabular',
      which are controlled via a boolean flag. We plan to extend it with another
      mode of operation, 'Msgpack object', so we need to replace the flag with a
      enumeration. Plus, this refactoring should improve readability.
      
      Needed for #8147
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      d1f1faa1
    • Georgiy Lebedev's avatar
      net.box: refactor netbox method encoders arguments · e3f2bea0
      Georgiy Lebedev authored
      In scope of #8633 call and eval encoding will require an additional
      argument (whether to encode tuples as extensions and send formats), and
      method encoders already have 3 arguments (MsgPack stream, IPROTO sync and
      IPROTO stream identifier): to simplify further extension of method encoders
      move arguments to a netbox_method_encode_ctx struct.
      
      Needed for #8633
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      e3f2bea0
    • Georgiy Lebedev's avatar
      lua: refactor MsgPack extension encoder · a1a13b55
      Georgiy Lebedev authored
      The MsgPack extension encoder has a strange interface: it returns the type
      it has encoded the value to which at the same time indicates whether it has
      succeeded in encoding the value — instead, let's return the type in an
      output parameter and return the encoding status as a boolean value.
      
      Needed for #8147
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      a1a13b55
    • Georgiy Lebedev's avatar
      core: introduce MsgPack encoding/decoding context structure · 9b05dbfc
      Georgiy Lebedev authored
      The MsgPack context structure is the essential data structure for
      collecting and retrieving various meta information during MsgPack
      encoding/decoding. This structure will be a base for a box-specific MsgPack
      context (and, possibly, others), so we make it extendable by adding a
      padding (to avoid dynamic allocations).
      
      Refactor existing MsgPack encoding interfaces to use the MsgPack context
      structure:
      1. extend the port dumping interface, so that we can collect meta
      information about dumped tuples' formats;
      2. extend the port MsgPack retrieval interface, so that we can also
      retrieve meta information about tuple formats for decoding tuples coming
      from IPROTO (needed when decoding is postponed to a MsgPack object), the
      context ownership is acquired by the caller to avoid copying;
      3. extend the MsgPack encoding/decoding interfaces to accept a MsgPack
      context to collect meta information during encoding or to use this meta
      information for decoding — the context is passed down up to the extension
      encoder/decoder, which is the point where future box tuple extension will
      be processed.
      
      Needed for #8147
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      9b05dbfc
    • Georgiy Lebedev's avatar
      box: introduce tuple format map · 2324fbcf
      Georgiy Lebedev authored
      
      The tuple format map is the essential data structure for collecting tuple
      formats during box tuple encoding and for looking up tuple formats during
      box tuple decoding.
      
      Needed for #8147
      
      NO_CHANGELOG=<internal data structure>
      NO_DOC=<internal data structure>
      
      Co-authored-by: default avatarAleksandr Lyapunov <alyapunov@tarantool.org>
      2324fbcf
    • Nikita Zheleztsov's avatar
      config: fix hanging alert about missing name · c2c82860
      Nikita Zheleztsov authored
      At the last itaration of review it was decided to throw an alerts,
      when no UUID was passed to config and name is not set in _cluster.
      
      This leads to alerts, thrown during replicaset bootstrap. However,
      _cluster:on_replace trigger wasn't updated for that change, it
      asssumed that when insert of a new replica is done, no alerts was
      thrown.
      
      Let's fix the behavior of on_replace trigger so that it deletes
      alerts as soon as replica joined with name.
      
      Follow-up #8978
      
      NO_DOC=bugfix
      NO_TEST=<already tested>
      NO_CHANGELOG=bugfix
      c2c82860
    • Nikolay Shirokovskiy's avatar
      small: bump version · f399cc7b
      Nikolay Shirokovskiy authored
      New commits:
      
      - test: fix test build for Debian 9 and alike
      - test: fix unused variable warning in matras test
      
      NO_TEST=submodule bump
      NO_CHANGELOG=submodule bump
      NO_DOC=submodule bump
      f399cc7b
    • Nikita Zheleztsov's avatar
      config: set names automatically on schema upgrade · 24d7c8f3
      Nikita Zheleztsov authored
      It's impossible to set names on Tarantool below 3.0.0, as all DDL
      is forbidden before schema upgrade.
      
      Let's make names NoOp on schema below Tarantool 3.0.0 and set names
      automatically only when schema upgrade is done.
      
      Follow-up #8978
      
      NO_DOC=tarantool/doc#3661
      24d7c8f3
    • Nikita Zheleztsov's avatar
      lua: move mkversion into a separate module · 29daaf3e
      Nikita Zheleztsov authored
      This commit moves all code, related to working with versions and which
      was used in box/lua/upgrade.lua, into a separate module and exports it
      to Lua API as 'internal.version'
      
      This is needed, as in the following commit we set names automatically
      only when schema version is more than 3.0.0. This module is used their
      in order to avoid code duplication.
      
      Follow-up #8978
      
      NO_DOC=internal
      NO_TEST=<already tested>
      NO_CHANGELOG=internal
      29daaf3e
    • Nikita Zheleztsov's avatar
      config: automatically set missing names · d3ce4b78
      Nikita Zheleztsov authored
      For now it's impossible to use config module in order to recover
      from snaps, which don't have names set in them. Calling box.cfg
      with names on recovery fails with MISMATCH error, which is caused
      by difficult implementation of setting names on first box.cfg,
      as names can be set only on rw instance.
      
      This commit doesn't call box.cfg with names, if these names are
      missing from the snapshot file. Instead it creates a fiber, which
      will set names, when it's possible to do so. Only master sets the
      names for the whole cluster.
      
      Closes #8978
      
      NO_DOC=tarantool/doc#3661
      NO_CHANGELOG=following commits
      d3ce4b78
    • Nikita Zheleztsov's avatar
      box: fix replica hang after applying already set name · 9988ac58
      Nikita Zheleztsov authored
      When the name is manually set on master by replace in _cluster space,
      calling box.cfg on replica with the same name causes its hang. The
      problem is the fact, that resubscribe is initiated and waiting for
      APPLIER_REGISTERED status is started. As applier knows, that no
      registration should be done, this never happens.
      
      Let's don't initiate registration, when instance name is already set.
      
      Needed for #8978
      
      NO_DOC=bugfix
      NO_CHANGELOG=not released yet
      9988ac58
    • Nikita Zheleztsov's avatar
      config: introduce droppable alerts · fa97cc0c
      Nikita Zheleztsov authored
      For now it's impossible to drop created alert in any way except
      manual searching for _alerts table. However, we need to drop alerts
      on missing names, when the names are set.
      
      Let's introduce simple key-value alerts in order to easily drop them
      by key.
      
      Needed for #8978
      
      NO_DOC=internal
      NO_TEST=following commit
      NO_CHANGELOG=internal
      fa97cc0c
    • Nikita Zheleztsov's avatar
      config: validate names and UUIDs before box.cfg · b04f9c5f
      Nikita Zheleztsov authored
      Currently only instance_uuid is validated before recovery process.
      All names and replicaset_uuid are checked only when recovery is done,
      which can take a long time. It can be frustrating to users, which
      have been waiting for several hours only to get name mismatch error.
      
      Let's read the small part of snapshot file before calling box.cfg
      in order to figure out, whether the names and uuids, passed to
      configuration match the ones, saved inside the snapshot.
      
      During config reload there's no sense in reading snapshot file, as
      data is already saved inside spaces, let's read them. We still check
      that names in config and names in spaces don't contradict during
      config reload.
      
      This commit also introduces methods, for getting names, which are
      not set in snap (or memory), this'll be used in consequent commits
      to set names automatically.
      
      Needed for #8978
      
      NO_DOC=tarantool/doc#3661
      b04f9c5f
Loading