Skip to content
Snippets Groups Projects
  1. Dec 11, 2024
    • Arseniy Volynets's avatar
      fix: wrong sql cache byte counter · 5b97eaa0
      Arseniy Volynets authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      - If you prepare and execute statement with params
      in the projection and then unprepare the
      statement, byte counter may show the wrong value
      or even overflow.
      - The problem is that when we compile sql
      statement, we set parameter type to 'any'.
      But when we execute the statement we set parameter
      type to actual type. Then we use this type in
      calculation of estimated of sql cache entry size.
      This leads to different estimated sizes of cache
      entry during prepare and during unprepare after
      statement was executed
      - Fix this by resetting type to 'any' after
      executing the statement
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      5b97eaa0
    • Arseniy Volynets's avatar
      refactor: add session id to prepare funcs · 66297e1a
      Arseniy Volynets authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      BREAKING CHANGE!:
      1. add session id argument to sql_prepare_ext
      2. introduce sql_unprepare_ext function.
      This function removes prepared stmt using
      given session id.
      
      In picodata SQL, we may prepare stmt in one
      session and unprepare it in some
      other session, which does not know in
      what session the statement was prepared. Now
      sql_prepare_ext returns not only statement id,
      but also a session id. This way statement can
      be unprepared from other session using
      sql_unprepare_ext.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      66297e1a
    • godzie44's avatar
      feat(wal_ext): add support for WAL extensions. · deec091f
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      WAL extensions allows to add auxiliary information to each
      write-ahead log record. WAL extensions configured by
      `box.cfg.wal_ext` option. Currently, there is only one builtin
      extension: `new_old`. `new_old` extension add information about
      new and old tuples for ddl operations.
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      deec091f
    • Denis Smirnov's avatar
      refactor: replace sql_prepare with sql_prepare_ext symbol · 4533d5b5
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      BREAKING CHANGE!:
      1. remove sql_prepare from the export list;
      2. introduce sql_prepare_ext.
      
      The sql_prepare symbol previously included the tarantool port as
      an output parameter. However, this structure was inconvenient for
      libraries using the C API, as they primarily required just the
      statement ID. To address this issue, the sql_prepare symbol was
      replaced with the sql_prepare_ext symbol.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      4533d5b5
    • Denis Smirnov's avatar
      feat: share prepared statements among sessions · 672a182a
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Previously, users with multiple connections to tarantool instance
      couldn't share prepared statements across sessions. They had to
      manually call prepare in each session before execution.
      
      This commit automates this process for the exported version of
      SQL prepared statement execution (sql_execute_prepared_ext symbol).
      Original Lua execution keeps the old behavior for backward
      compatibility.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      672a182a
    • Denis Smirnov's avatar
      refactor: reorder arguments in SQL execution functions · 74d98841
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Previously, sql_prepare_and_execute and sql_execute_prepared
      functions didn't follow a convention to keep output parameters
      (the port to be exact) at the end of the argument list.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      74d98841
    • Denis Smirnov's avatar
      refactor: change exported C API for SQL · 9cb8e19f
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      BREAKING CHANGE!:
      1. sql_bind_list_decode - removed
      2. sql_execute_prepared_ext - new arguments
      3. sql_prepare_and_execute_ext - exported
      
      There were several reasons to refactor the API.
      1. sql_bind_list_decode (decodes message pack parameters into
         internal C bind structure) is very difficult to use without
         memory leaks (as it allocates results on fiber()->gc).
      2. sql_execute_prepared_ext missed vdbe step limit in parameters
         and used the default value.
      3. Sometimes SQL queries don't fit into prepared statement cache
         and the user still wants to execute them via a slow pass with
         full compilations from the query text. That was the reason to
         export sql_prepare_and_execute_ext symbol.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      9cb8e19f
    • Dmitry Ivanov's avatar
      fix(sql): remove dead code from sql_trigger_begin · eea36eed
      Dmitry Ivanov authored
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      
      The following commit introduced a tautological if expression:
      
      ```gitcommit
      sql: introduce structs assembling DDL arguments during parsing (ba56b145fafaa3)
      ```
      
      Due to the changes in commit
      
      ```gitcommit
      sql: refactor memory allocation system (cb777136dd7a0c)
      ```
      
      the allocations in sql expressions became infallible, which means
      that we may safely fix static analysis warnings by dropping the
      tautological comparison altogether.
      
      Original patch by Feodor Alexandrov.
      eea36eed
    • Egor Ivkov's avatar
      fix: missing check for inconsistent vclock values in mp_decode_vclock_ignore0 · e8075c0f
      Egor Ivkov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      e8075c0f
    • Egor Ivkov's avatar
      fix: missing check for mp type in xrow_decode_raft · 218d5a28
      Egor Ivkov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      218d5a28
    • godzie44's avatar
      fix: Do not abort incoming transaction if MVCC mode is not enabled. · af2cd107
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Currently, DDL transaction may abort other in-progress
      transactions (this behavior added in #8f4be322 - avoid sharing
      (ergo phantom reads) metadata object for different
      transactions in MVCC mode).
      This behavior makes no sense if the MVCC is
      disabled (cause these conflicts never happen).
      
      Closes #47
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      NO_TEST=internal
      af2cd107
    • Denis Smirnov's avatar
      feat: reserve function id range · c15bc3f7
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Current commit is a part of the SQL procedures track for Picodata.
      It introduces several features:
      - the reserved id range in the _func space, that can be only consumed
        implicitly;
      - an additional "id" option for function creation with Lua (to set
        the function identifier implicitly);
      - helper function to generate function ids either in reserved or default
        ranges
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      c15bc3f7
    • Vladimir Davydov's avatar
      xlog: introduce xlog_remove_file function for removing xlog files · 82b31466
      Vladimir Davydov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      This commit introduces the xlog_remove_file() function that removes
      a file by name and logs the error on failure. We use this function
      everywhere we delete xlog files so that there's a single place where we
      call unlink(). We also factor out the core functionality to a callback
      function that can be overridden. This will help us implement thorough
      file deletion.
      
      Needed for tarantool/tarantool-ee#540
      
      Cherry-picked from d139f245
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      82b31466
    • Egor Ivkov's avatar
      feat: report actual error in console.connect · 11c1f3a8
      Egor Ivkov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      11c1f3a8
    • Dmitry Rodionov's avatar
      refactor: rename schema_object_type to box_schema_object_type · f452ba18
      Dmitry Rodionov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      This patch makes schema_object_type to be exported.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      f452ba18
    • Dmitry Rodionov's avatar
      feat: export box_access_check_ddl · 89cf3837
      Dmitry Rodionov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      89cf3837
    • Georgy Moshkin's avatar
      box: fix memory leak when dropping temporary spaces · 75794401
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_CHANGELOG=bug fix for unreleased feature
      NO_DOC=bug fix for unreleased feature
      75794401
    • Kurdakov Alexander's avatar
      feat: support of overriding sql executor · 6ef85168
      Kurdakov Alexander authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Add to console lua api function that overrides sql executor
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      6ef85168
    • Georgy Moshkin's avatar
      iproto: request handlers set the fiber name to proc name · 7ee8aee5
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      7ee8aee5
    • Georgy Moshkin's avatar
      feat: add support for static stored procedures using ".foo" syntax · d6fdfe0c
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      d6fdfe0c
    • Dmitry Rodionov's avatar
      refactor: rename priv_type to box_privilege_type and expose it · a9069652
      Dmitry Rodionov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Rename the members as well. Keep uint16_t in box_check_acess_space
      as it is for other exported functions
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      a9069652
    • Dmitry Rodionov's avatar
      feat: export box_access_check_space · 1d6d76f8
      Dmitry Rodionov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      
      Co-authored-by: default avatarYaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
      1d6d76f8
    • Georgy Moshkin's avatar
      box: introduce box_read_view_* ffi API functions · c098944a
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduce API for opening a read view over the specified spaces and
      indexes and creating iterators over the index read views.
      
      NO_DOC=picodata patch
      c098944a
    • Nikolay Shirokovskiy's avatar
      box: introduce memtx_sort_threads config parameter · 0137bb35
      Nikolay Shirokovskiy authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Closes #3389
      Closes #7689
      Closes #4646
      
      @TarantoolBot document
      Title: new box.cfg parameter memtx_sort_threads
      
      The parameter sets the number of threads used to sort keys of secondary
      indexes on loading memtx database. The parameter cannot be changed
      dynamically (as it does not make sense).
      
      Maximum value is 256, minimum is 1. Default is to use all available cores.
      
      Usage example:
      ```
      box.cfg{memtx_sort_threads=4}
      ```
      0137bb35
    • Georgy Moshkin's avatar
      box: fully temporary spaces · 910d2bb0
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduce fully temporary spaces: same as data-temporary space but with
      temporary metadata. Basically temporary spaces now do not exist on
      restart and do not exist on replicas. They can also be created, altered
      and deleted when box.cfg.read_only = true.
      
      To avoid conflicts with spaces created on replicas, the temporary
      space ids by default start in a special range starting at
      BOX_SPACE_ID_TEMPORARY_MIN.
      
      Temporary spaces currently do not support several features e.g.
      foreign key references (to and from), functional indexes, sql sequences,
      sql triggers, etc. This may change in the future.
      
      Implementing temporary spaces requires temporary tuples to be
      inserted into system spaces: tuples which are neither replicated or
      persisted. This mostly done in on_replace_dd_* triggers by dropping the
      txn->stmt->row.
      
      Closes #8323
      
      @TarantoolBot document
      Title: Introduce fully temporary spaces with temporary metadata
      
      Temporary spaces are now data-temporary spaces with temporary metadata.
      Created by specifying { type = "temporary" } in the options.
      Temporary spaces will not exist upon server restart and will not
      exist on replicas. They can also be created in read-only mode.
      910d2bb0
    • Georgy Moshkin's avatar
      box: factor out new_tuple def construction · d0ca64a2
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      A tiny preparatory commit for meta-temporary spaces
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      d0ca64a2
    • Georgy Moshkin's avatar
      box: extract txn_update_row_counts function · bd96ecee
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Move code that handles txn row counters into a separate function in
      preparation of meta-temporary spaces introduction.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      bd96ecee
    • Georgy Moshkin's avatar
      core: rename temporary spaces to data-temporary · c11e7a10
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Everywhere where we refer to temporary spaces we now say data-temporary.
      This is because temporary spaces were never truly temporary because
      their definitions would still be persisted and replicated and they
      couldn't be created on read-only replicas. In a following commit we will
      introduce a new fully temporary type of spaces, which will be just
      called 'temporary', so this commit signifies this terminology change.
      
      NO_DOC=renaming
      NO_CHANGELOG=renaming
      NO_TEST=renaming
      c11e7a10
    • Georgy Moshkin's avatar
      box: introduce space type · bbd533ef
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduces a new field `type` to the space definition. Currently it can
      only be "normal" or "data-temporary". It is backwards compatible with
      temporary=true.
      
      @TarantoolBot document
      Title: Introduce space field type
      
      A new space definition field "type" can now be used to specify the type
      of the space. Usage: box.schema.create_space("s", { type = "normal" }).
      Currently only 2 types are supported: "normal" & "data-temporary", which
      is equivalent to { temporary = true }. Old-style { temporary = true } is
      still supported, but only one option either 'temporary' or 'type' may be
      specified at the same time.
      
      Space type "temporary" will be introduced in a later commit.
      In the future options "local", "synchronous", etc. may also be
      supported.
      
      NO_TEST=will be tested in the following commit
      bbd533ef
    • Aleksandr Lyapunov's avatar
      sql: refactor update_view_references a bit · f6c88e6b
      Aleksandr Lyapunov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      The function update_view_references is called when an SQL view
      is created or dropped. The goal of this function is to modify
      (increment or decrement) view_ref_count member of spaces that
      the view references.
      
      There were a several issues that deserves to be refactored:
      * By design in case of error it left the job partially done, so
        some space references were modified while some other - not.
        Although there was no bug since special steps were made in case
        of error, this pattern is inconvenient and should be avoided.
      * In case of error the failing space name was returned via special
        argument which is not flexible and even requires allocation.
      * Another argument - suppress_error - has actually never
        suppressed any error because the only case when an error could
        occur is creation of a view, which used suppress_error = false.
      * Fail of that function was not actually covered with tests.
      
      So this commit:
      * Makes the function to do all or nothing.
      * Forces the function to set diag by itself in case of error.
      * Removes suppress_error argument while adding several asserts.\
      * Adds a small test that fulfills coverage.
      
      NO_DOC=refactoring
      NO_CHANGELOG=reafactoring
      f6c88e6b
    • Aleksandr Lyapunov's avatar
      sql: don't store the first NULL element in list · e173f491
      Aleksandr Lyapunov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      By design a newly created SrcList object contains one element
      with NULL name. That was confusing and led to strange NULL checks
      in a list that could not contain NULL names.
      
      Fix it by clearing the list before usage.
      
      NO_DOC=refactoring
      NO_CHANGELOG=reafactoring
      NO_TEST=refactoring
      e173f491
    • Aleksandr Lyapunov's avatar
      sql: don't catch OOM in sql_select_expand_from_tables · 65ab1d1b
      Aleksandr Lyapunov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Since we panic on OOM now, no OOM error handling is needed now.
      Fix both internals of the function and how it is used in alter.
      
      NO_DOC=refactoring
      NO_CHANGELOG=reafactoring
      NO_TEST=refactoring
      65ab1d1b
    • Ilya Verbin's avatar
      box: support default field values in the space format · b2f58221
      Ilya Verbin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Now a field can be assigned a default value in the space format. When a new
      tuple is inserted into a space, and some of the fields contain null values,
      those fields will be filled with their respective default values.
      
      Closes #8157
      
      @TarantoolBot document
      Title: Document default field values
      Product: Tarantool
      Since: 3.0
      Root document: https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_space/format/
      
      The format clause contains, for each field, a definition within braces:
      `{name='...',type='...'[,is_nullable=...][,default=...]}`, where:
      
      * the optional `default` value contains a default value for the field.
        Its type must be compatible with the field type. If default value is set,
        it is applied regardless of whether `is_nullable` is true or false.
      
      Example:
      
      ```lua
      tarantool> box.space.tester:format{
               > {name = 'id', type = 'unsigned'},
               > {name = 'name', type = 'string', default = 'Noname'},
               > {name = 'pass', type = 'string'},
               > {name = 'shell', type = 'string', default = '/bin/sh'}}
      ---
      ...
      
      tarantool> box.space.tester:insert{1000, nil, 'qwerty'}
      ---
      - [1000, 'Noname', 'qwerty', '/bin/sh']
      ...
      ```
      b2f58221
    • Ilya Verbin's avatar
      box: introduce tuple_builder class · a9a20579
      Ilya Verbin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      It encapsulates the logic that helps to build a new MsgPack array by
      concatenating tuple fields from various locations. The idea is to
      postpone memory allocation and copying until the finalization.
      
      Needed for #8157
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      a9a20579
    • Ilya Verbin's avatar
      sql: rename field_def::default_value to field_def::sql_default_value · 2cf79575
      Ilya Verbin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      In order to avoid collision with the upcoming core default_value.
      
      Also rename tuple_field::default_value_expr to
      tuple_field::sql_default_value_expr.
      
      Part of #8157
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      2cf79575
    • Maksim Kaitmazian's avatar
      build: add exports for simple query flow · 3c989fc4
      Maksim Kaitmazian authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      PG protocol run queries by calling dispatch_query, which stores the
      response in port_c contained in box_function_ctx structure.
      
      NO_DOC=exports
      NO_CHANGELOG=exports
      NO_TEST=exports
      3c989fc4
    • Dmitry Ivanov's avatar
      fix: buffer overflow in auth_md5's bytes_to_hex() · 81959875
      Dmitry Ivanov authored
      NO_DOC=internal
      NO_CHANGELOG=internal
      NO_TEST=internal
      81959875
    • Denis Smirnov's avatar
      fix: wrong argument type in box_auth_data_prepare() · e89b633d
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      box_auth_data_prepare() method declared to return a tuple while in
      reality it returned a region allocated message pack string. Fixed.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      e89b633d
    • Denis Smirnov's avatar
      feat: extend C box API with a new auth method · 70dc777d
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      1. Current commit introduces 'box_auth_data_prepare()' to prepare
         a data string for any supported authentication methods.
      2. The user name argument is refactored in the auth methods: the
         null-terminated string is replaced with an address range approach.
         Now Rust users don't need to re-allocate username with CString.
      3. Password length type was set to uint32_t (previously it was size_t,
         int, uint32_t for different functions). Tarantool uses murmur3a,
         so all the hashed strings should be up to 32 bit long.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      70dc777d
    • Denis Smirnov's avatar
      feat: extend C box API with new user methods · b201f939
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduce new methods:
      
      1. box_user_id_by_name - get the user identifier by name;
      2. box_effective_user_id - get current effective user
         identifier;
      3. box_session_user_id - get current session user identifier;
      4. box_session_su - change current session user;
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      b201f939
Loading