Skip to content
Snippets Groups Projects
  1. Mar 24, 2022
    • Nick Volynkin's avatar
      test-run: bump to new version again · d9178b1b
      Nick Volynkin authored
      Reverting changes from 3cad9398,
      originally introduced in 0b46154f
      
      NO_DOC=testing
      NO_TEST=testing
      NO_CHANGELOG=testing
      d9178b1b
    • Aleksandr Lyapunov's avatar
      box: add changelog for gh 6436 · c60a8205
      Aleksandr Lyapunov authored
      @TarantoolBot document
      Title: Document constraints and foreign keys
      
      See details here:
      https://docs.google.com/document/d/1EtqfFSIMi6fDqj2y-9WgpunkHsszU65p6LfNly5E9Ag/
      
      NO_TEST=no code changes
      c60a8205
    • Aleksandr Lyapunov's avatar
      box: implement complex foreign keys · 1150adf2
      Aleksandr Lyapunov authored
      Implement complext foreign keys addition to field foreign keys.
      They are quite similar to field foreign keys, the difference is:
      * The are set in space options instead of format field definition.
      * Several fields may be specified in relation.
      * By design field foreign keys are more optimal.
      
      One can set up foreign keys in space options:
      box.schema.space.create(.. {.., foreign_key=<foreign_key>})
      where foreign_key can be of one of the following forms:
       foreign_key={space=..,field=..}
       foreign_key={<name1>={space=..,field=..}, ..}
      where field must be a table with local -> foreing fields mapping:
       field={local_field1=foreign_field1, ..}
      
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      1150adf2
    • Aleksandr Lyapunov's avatar
      box: implement field foreign keys · d950fdde
      Aleksandr Lyapunov authored
      Foreign key is a special type of constraint that makes a relation
      between spaces. When declared in some space, each tuple in that
      space refers to some tuple in another, foreign space. Reference is
      defined in foreign key definition as a correspondence of field of
      that spaces, local and remote.
      
      Foreign key preserves that reference between tuples and consists
      of two checks:
      1. When a tuple is added to space with foreign space constraint,
      it must be checked that there is a corresponding tuple in foreign
      space, with the same values in fields according to foreign key
      definitiion.
      2. When a tuple is deleted from space that is a foreign space for
      some other space, it must be checked that no tuple references the
      deleted one.
      
      This commit introduces field foreign keys that link spaces by
      just one field. They are declared in tuple format in one of the
      following forms:
       space:format{..,{name=.., foreign_key=<fkey>},..}
       space:format{..,{name=.., foreign_key={<name>=<fkey>}},..}
      Where fkey has a form of a table:
       {space=<foreign space id/name>, field=<foreign field id/name>}
      
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      d950fdde
    • Aleksandr Lyapunov's avatar
      box: add pin/unping infrastructure for spaces · b00a4579
      Aleksandr Lyapunov authored
      There are cases when we need to be sure that a space by given
      id or name is not deleted; and if it is replaced (in space cache),
      there's need to track pointer to new space. Like ib constraints:
      they must hold a pointer to struct space while it's very hard to
      determine whether there'a constraint that points to given space.
      
      Implement space pin/unpin for this purpose. You can pin a space to
      declare that the space is require to exist. To have to unpin it
      when the space is not needed anymore.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      b00a4579
    • Aleksandr Lyapunov's avatar
      box: move space_cache to a separate file · 9b1c1e8d
      Aleksandr Lyapunov authored
      I'm going to extend space cache API so it should be separated.
      One function went to space.h/c.
      No logical changes.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      9b1c1e8d
    • Aleksandr Lyapunov's avatar
      box: use field names in tuple constraint function · 1d35d866
      Aleksandr Lyapunov authored
      The previous commit adds tuple constraint lua functions that check
      format of entire tuple. The problem was that in those functions
      tuple could be accessed only by field indexes.
      
      Add an ability to use field names too.
      
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      1d35d866
    • Aleksandr Lyapunov's avatar
      box: implement tuple constraints · 53f5d4e7
      Aleksandr Lyapunov authored
      Implement whole tuple constraints in addition to field constraints.
      They are quite similar to field constraints, the difference is:
       * The are set in space options instead of format field definition.
       * Entire tuple is passed to check function.
       * By design field constraints are a bit more optimal.
      
      One can set up constraint in space options, with one or several
      functions that must be present in _func space:
      box.schema.space.create(.. {.. constraint='func1'})
      box.schema.space.create(.. {.. constraint={name1='func1'})
      box.schema.space.create(.. {.. constraint={name1='f1', name2='f2'})
      
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      53f5d4e7
    • Aleksandr Lyapunov's avatar
      box: introduce a pair of tuple_format_new helpers · 4b8dc6b7
      Aleksandr Lyapunov authored
      tuple_format_new has lots of arguments, all of them necessary
      indeed. But a small analysss showed that almost always there are
      only two kinds of usage of that function: with lots of zeros as
      arguments and lots of values taken from space_def.
      
      Make two versions of tuple_format_new:
      simple_tuple_format_new, with all those zeros omitted, and
      space_tuple_format_new, that takes space_def as an argument.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      4b8dc6b7
    • Aleksandr Lyapunov's avatar
      box: implement field constraints · ed9b982d
      Aleksandr Lyapunov authored
      Introduce field constraints - limitaions for particular fields.
      Each constraint must refer to a function in _func space. For the
      first step we expect lua functions with body there.
      
      Field constraint checks are very close to field type checks, so
      it's natural to implement them in tuple formats. On the other hand
      tuple formats belong to tuple library, that does not include
      functions (func.h/c etc), so constraints are split into two parts:
      - a part in tuple library that implements arbitrary constraints
       with pointers to functions that actually check constraints.
      - a part in box library which uses the part above, sets particular
       check functions and handles alter etc.
      
      There are two not-so-obvious problems that are solved here:
       - Functions in _func space must be preserved while used by such
       constraints. Func pinning is used for this purpose.
       - During initial recovery constraits are created before _func
       space recovery, so we have to pospone constraint initialization.
      
      One can set up constraint for any field in tuple format with one
      or several functions that must be present in _func space:
      space:format{name='a', constraint='func1'}
      space:format{name='a', constraint={name1='func1'}}
      space:format{name='a', constraint={name1='func1', name2='func2'}}
      
      So constraint(s) can be set by one function name or by lua table
      with function name values. Each consraints has a name that can be
      specified directly (with string key in table) or imlicitly set to
      the name of function.
      
      The check function receives two arguments: the checking value and
      the name of the constraint. Also the name of the failed constraint
      is present in raised exception.
      
      The only way to pass the constraint is to return true from its
      function. All other values and exception are treated as failure
      (exeptions are also logged).
      
      NO_DOC=see later commits
      NO_CHANGELOG=see later commits
      ed9b982d
    • Aleksandr Lyapunov's avatar
      box: implement ability to add a string to C port. · 44a49408
      Aleksandr Lyapunov authored
      Now C port allows to add a tuple or raw msgpack to it.
      
      Add another function that encodes and appends given string.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      44a49408
    • Aleksandr Lyapunov's avatar
      salad: introduce group alloc · af7667d1
      Aleksandr Lyapunov authored
      gpr_alloc is a small library that is designed for simplification
      of allocation of several objects in one memory block. It could be
      anything, but special attention is given to string objects, that
      are arrays of chars.
      
      Typical usage consist of two phases: gathering total needed size
      of memory block and creation of objects in given block.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      af7667d1
    • Aleksandr Lyapunov's avatar
      box: intoduce engine recovery state · f7435757
      Aleksandr Lyapunov authored
      There's several important stages of recovery of database: loading
      from snapshot, then loading from WAL(s) and then normal operation.
      
      Introduce a global recovery state that shows this stage.
      
      Note that there's already a recovery state in memtx engine which is
      very close but still different to the new introduced state. That
      state in memtx is a private property of memtx that internally shows
      initialization status of memtx spaces and indexes. Memtx can set the
      value for convenience' sake, for example it can jump directly to
      MEMTX_OK before snapshot loading in case of force recovert.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      f7435757
    • Aleksandr Lyapunov's avatar
      box: add OPT_CUSTOM to opt_def · 2a88b535
      Aleksandr Lyapunov authored
      opt_def is an option parser from msgpack by given scheme.
      The scheme consists of set of predefined types that the parser
      can handle (something like int, enum, str). The problen is that
      those predefined types must be generic, but there are cases when
      a specific unusual must be parsed.
      
      This patch introduces OPT_CUSTOM type that uses arbitrary function
      callback and can be used for any non-generic option value.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      2a88b535
    • Aleksandr Lyapunov's avatar
      box: add pin/unping infrastructure for func cache · ffc9cee4
      Aleksandr Lyapunov authored
      There are cases when we need to be sure that a function is not
      deleted and/or removed from func cache. For example constraints:
      they must hold a pointer to struct func while it's very hard to
      determine whether there'a constraint that points to given func.
      
      Implement func pin/unpin for this purpose. You can pin a func to
      declare that the func must not be deleted. To have to unpin it
      when the func is not needed anymore.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      ffc9cee4
    • Aleksandr Lyapunov's avatar
      lib: refactor assoc library a bit · 259a7584
      Aleksandr Lyapunov authored
      - Use uint32_t for string length. Actually internally it cannot
      take more that INT_MAX length, so uin32_t is enough. This change
      makes the hash table a bit more compact.
      - Rename mh_strnptr_find_inp -> mh_strnptr_find_str. I beleive it
      makes it more understandable.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      259a7584
    • Aleksandr Lyapunov's avatar
      box: move func_cache to a separate file · 3cad9398
      Aleksandr Lyapunov authored
      I'm going to extend func cache API so it should be separated.
      A couple of comments added.
      No logical changes.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      3cad9398
    • Aleksandr Lyapunov's avatar
      box: trashify port data after destruction · bdd821bf
      Aleksandr Lyapunov authored
      Fill port with a corrupted data in debug (TRASH it) in order to
      detect double port destruction early.
      
      Add a comment for func_call function that describes what states
      of ports are expected before and after func_call.
      
      Fix port usage in lua using ffi - allocate a full port structure
      instead of a (bit smaller) structure port_c. That makes any port
      structure to have fixed determined size which in turn makes safe
      to cast and use different port types.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      bdd821bf
    • Aleksandr Lyapunov's avatar
      sql: initialize field def properly · eb09d3c7
      Aleksandr Lyapunov authored
      There must be a normal code flow: if you want to initialize some
      structure you should set members you want to and set with defaults
      the rest members. Such a code flow would be durable when some new
      members are added to the struct.
      
      Make field_def structure to comply those rules.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      eb09d3c7
    • Aleksandr Lyapunov's avatar
      box: free funcs_by_name along with other tables in schema_free · 2c0aedca
      Aleksandr Lyapunov authored
      It seems that is was simply forgotten.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring
      2c0aedca
    • Aleksandr Lyapunov's avatar
      all: insignificant changes · 1b9bd0f4
      Aleksandr Lyapunov authored
      Some non-important fixes that are not related to any issue.
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      1b9bd0f4
    • Aleksandr Lyapunov's avatar
      box: improve fselect · 22898127
      Aleksandr Lyapunov authored
      fselect is designed to make selects results in console better
      readable for human eyes. The usage is expected to be rare and
      by developers only so there's no performance requirements.
      
      Now the API of fselect is:
      s:fselect(<key>, <select+fselect option map>, <fselect option map>)
      
      But sometimes there are cases when only some columns are needed
      for analysis, while all other columns consume space on screen.
      
      So make an option in fselect that allows to choose which columns
      to output (like some other databases allows).
      
      Add an option 'columns' in fselect options along with other its
      options with list of columns (by number or name). Allow lua tables
      and comma-separated-strings for that.
      
      If options argument in fselect is string, interpret is as columns
      options.
      
      NO_DOC=There's no doc about fselect yet, I'll append these change
      to the existing doc ticket
      NO_CHANGELOG=minor changes in minor feature
      22898127
  2. Mar 23, 2022
  3. Mar 21, 2022
    • Kirill Yukhin's avatar
      github-ci: add server start timeout · f1d750f0
      Kirill Yukhin authored
      Added 290 sec. timeout for starting tarantool server.
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      f1d750f0
    • Kirill Yukhin's avatar
      test-run: bump to new version · 0b46154f
      Kirill Yukhin authored
      Bump test-run to new version with the following improvements:
      - Add timeout for starting tarantool server (tarantool/test-run#302)
      - Kill hanging processes of not started servers (tarantool/test-run#332)
      - Rerun all failed tests, not only marked as fragile
        (tarantool/test-run#329)
      
      NO_DOC=testing
      NO_TEST=testing
      NO_CHANGELOG=testing
      0b46154f
    • Kirill Yukhin's avatar
      luacheck: exclude test-run tests · 8d6a154f
      Kirill Yukhin authored
      Running of luacheck against test-run submodule is redundant.
      Add dedicated bypass to .luacheckrc.
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      8d6a154f
  4. Mar 18, 2022
    • Georgiy Lebedev's avatar
      libunwind: add new submodule · 7dc9fe44
      Georgiy Lebedev authored
      Investigation of GNU libunwind problems on the aarch64-linux-gnu
      platform drive us to the conclusion that libunwind-1.2.1 provided by
      major distribution packages is broken. Not to mention that its test
      suite fails with SEGFAULTs.
      
      Last but not least, some distributions, e.g. CentOS 8 (see #4611) do
      not provide a libunwind package.
      
      Hence, bundle libunwind: bundling is enabled by default on all
      platforms, except for macOS — a system package can be used if its
      version is greater or equal than 1.3.0 (minimal version that does not
      seem to be broken on aarch64-linux-gnu).
      
      * Add new submodule: bump it to current master.
      * Refactor libunwind package search logic out of compiler.cmake.
      * Add CMake script for building bundled libunwind.
      * Add CMake script for extracting version of libunwind.
      * Re-enable backtrace for all RHEL distributions by default.
      * Remove libunwind from static build.
      
      Needed for #4002
      Closes #4611
      
      NO_DOC=build system
      NO_TEST=build system
      7dc9fe44
    • Yaroslav Lobankov's avatar
      ci: use 'concurrency' feature in fuzzing.yml · dbaf03dc
      Yaroslav Lobankov authored
      It looks like we just missed the fuzzing.yml workflow when worked on
      adding this feature to our CI process in #6446.
      
      Follows up tarantool/tarantool-qa#100
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      dbaf03dc
    • Yaroslav Lobankov's avatar
      ci: remove unused 'repository_dispatch' trigger · 88b46b4c
      Yaroslav Lobankov authored
      It looks like this trigger was added beforehand for some purposes
      but never used. So there is no sense to keep it in our workflows.
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      88b46b4c
    • Yaroslav Lobankov's avatar
      ci: remove logic for 'full-ci' branch suffix · 1a0b5764
      Yaroslav Lobankov authored
      We have recently moved all development from local branches to forks and
      disabled running workflows in forks. So there is no sense to keep logic
      for running full testing in forks if a branch has the 'full-ci' suffix.
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      1a0b5764
    • Yaroslav Lobankov's avatar
      ci: not run workflows in forks · c27e6c74
      Yaroslav Lobankov authored
      We have recently moved all development from local branches to forks.
      Most workflows cannot be run in forks because many of them need access
      to private runners and/or secrets.
      
      Workflows in forks can be disabled manually, but it's hard to remember
      and it needs an extra action from a developer. Instead, we can detect
      the repo in workflows and run them if the repo is tarantool/tarantool.
      
      Closes #6913
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      c27e6c74
    • AnastasMIPT's avatar
      log: move info messages to info logging level · 1ab3c9d5
      AnastasMIPT authored
      Information messages have been moved from the critical logging level
      to info to avoid false alarms when reading logs.
      
      NO_DOC=commit does not add any new visible functionality
      
      NO_TEST=testing will require more resources than the result it will bring
      1ab3c9d5
    • Sergey Ostanevich's avatar
      netbox: support autocomplete in stream and future · d331b283
      Sergey Ostanevich authored
      Added support to netbox's `stream` and `future` objects using the
      __autocomplete metamethod.
      
      Closes #6305
      NO_DOC=internal
      d331b283
    • Sergey Ostanevich's avatar
      console: introduce __autocomplete metamethod · 7817fc13
      Sergey Ostanevich authored
      This patch updates the logic in lua_rl_complete() where __index from
      a metatable apparently expected to be a table, not a function. It is
      rather simple for a table object, but some of them are userdata, so
      an __autocomplete method is introduced to be sure it returns a table
      with all names suitable for tabcompletion.
      
      Part of #6305
      NO_DOC=internal
      NO_CHANGELOG=part of a fix
      7817fc13
  5. Mar 17, 2022
    • Vladimir Davydov's avatar
      vinyl: disable deferred deletes by default · b9f6d385
      Vladimir Davydov authored
      When the deferred DELETE optimization was introduced, it was enabled for
      all Vinyl spaces. Though the optimization  should improve write
      performance, at the same time it may result in significant degradation
      of reads. Let's disable this optimization by default and add a space
      option to enable it.
      
      Closes #4501
      
      @TarantoolBot document
      Title: Document defer_deletes space option
      
      The new option is a boolean flag that only makes sense for Vinyl spaces
      that have secondary indexes. Setting the flag results in deferring
      generation of DELETE statements for secondary indexes till compaction of
      the primary index. It should speed up writes, because it eliminates a
      lookup in the space for REPLACE and DELETE operations. At the same time,
      it may result in degradation of reads from secondary indexes, because it
      entails an additional lookup in the primary index per each "phantom"
      tuple (a tuple that was deleted in the primary index, but not yet in the
      secondary index, and hence should be skipped on read).
      
      Example:
      
      ```lua
      box.schema.space.create('test', {
          engine = 'vinyl',
          defer_deletes = true,
      })
      ```
      
      If omitted on space creation, this option is set to the value of
      `box.cfg.vinyl_defer_deletes`, which is false by default.
      b9f6d385
    • vr009's avatar
      lua: fix SIGINT handling · a8b3b267
      vr009 authored
      Tarantool console quits if you type Ctrl+C.
      This patch fixes console behavior on sending SIGINT.
      Console discards the current input on typing Ctrl+C
      and invites user to the new line.
      
      In daemon mode the process will exit after receiving SIGINT.
      
      Test gh-2717 should be skipped on release build, cause it
      uses error injection which is enabled only on debug mode.
      
      Fixes #2717
      
      @TarantoolBot document
      Title: Use Ctrl+C to discard the input in console
      
      The signal SIGINT discards the input in console mode.
      When tarantool executes with -e flag or runs as a daemon, SIGINT
      kills the process and tarantool complains about it in log.
      a8b3b267
    • EvgenyMekhanik's avatar
      Fix crash in space on_replace triggers. · e4e65e40
      EvgenyMekhanik authored
      During DDL operations triggers of old space and new created space
      are swapped. This leads to crash in case if this swap occurs from
      space on_replace triggers. This patch banned all DDL operations
      from on_replace triggers.
      
      Closes #6920
      
      @TarantoolBot document
      Title: Ban DDL operations from space on_replace triggers
      
      Previously user can set function for space on_replace trigger,
      which performs DDL operation. This may leads to tarantool crash,
      that's why this patch bans DDL operations from on_replace triggers.
      All this operations fails with error: "Space on_replace trigger
      does not support DDL operations".
      e4e65e40
  6. Mar 16, 2022
  7. Mar 15, 2022
    • AnastasMIPT's avatar
      net.box: fix exception thrown on time-out error · c463e56b
      AnastasMIPT authored
      netbox_perform_request throws a general ClientError exception on
      time-out error: it should throw the more suitable and POSIX-compliant
      TimedOut exception which sets errno to ETIMEDOUT.
      
      Closes #6144
      
      NO_DOC=There is no mention of what kind of error is returned on timeout
      in the net.box documentation.
      c463e56b
    • mechanik20051988's avatar
      memtx: fixed incorrect calculation of the number of compressed tuples · 38b28e2e
      mechanik20051988 authored
      There was no zeroing of the number of compressed tuples in the case of
      space truncation. Also there was no update of compressed tuples count
      in case of using memtx mvcc engine. This patch fix these problems.
      
      Follow up #2695
      
      NO_CHANGELOG=bugfix
      NO_DOC=bugfix
      NO_TEST=test in ee version
      38b28e2e
Loading