Skip to content
Snippets Groups Projects
  1. May 04, 2022
    • Vladimir Davydov's avatar
      alter: fix space upgrade check · 34da4ce9
      Vladimir Davydov authored
      space_upgrade_check_alter() must be called after the new space
      definition is updated.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      34da4ce9
  2. Apr 29, 2022
  3. Apr 28, 2022
    • Denis Smirnov's avatar
      sql: fix -Wnull-pointer-subtraction warning · 0289433a
      Denis Smirnov authored
      
      clang 13 includes a check for subtraction from NULL pointer which
      is considered UB: historically, we had an alignment checking macro
      which is affected by this. It seems like the intention of the
      macro's author was to implicitly cast the pointer being checked to
      uintptr_t without including stddef.h — replace this subtraction
      with an explicit cast.
      
      There is no way to set SQL_4_BYTE_ALIGNED_MALLOC, so the corresponding
      part of the code was removed. Now there are only 8 byte alignment
      assertions.
      
      NO_CHANGELOG=UB fix
      NO_DOC=UB fix
      NO_TEST=UB fix
      
      Co-authored-by: default avatarGeorgiy Lebedev <curiousgeorgiy@gmail.com>
      0289433a
    • Denis Smirnov's avatar
      sql: fix string dequoting · 76e09548
      Denis Smirnov authored
      
      Previously,
      
      select "t1"."a" from (select "a" from "t") as "t1";
      
      returned a result column name `t1` instead of `t1.a` because of
      incorrect work of a dequoting function. The reason was that
      previously sqlDequote() function finished its work when found the
      first closing quote.
      
      Old logic worked for simple selects where the column name doesn't
      contain an explicit scan name ("a" -> a).
      But for the sub-queries results sqlDequote() finished its work right
      after the scan name ("t1"."a" -> t1). Now the function continues its
      deqouting till it gets the null terminator at the end of the string.
      
      Closes #7063
      
      NO_DOC=don't change any public API, only a bug fix
      
      Co-authored-by: default avatarMergen Imeev <imeevma@gmail.com>
      76e09548
    • Vladimir Davydov's avatar
      errinj: add SPACE_UPGRADE_DELAY error injection · d772aef9
      Vladimir Davydov authored
      It will be used to stall background space upgrade in tests.
      
      While we are at it, move ERRINJ_TX_DELAY_PRIO_ENDPOINT to restore
      the sorted order of the error injection list.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      d772aef9
    • Vladimir Davydov's avatar
      space_upgrade: pass format to space_upgrade_new · 5269f2ea
      Vladimir Davydov authored
      We need the new format to apply the upgrade function to a tuple.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      5269f2ea
    • Vladimir Davydov's avatar
      test: check space upgrade stubs · 764d18ab
      Vladimir Davydov authored
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=test
      NO_CHANGELOG=test
      764d18ab
    • Vladimir Davydov's avatar
      test: add skip_if_enterprise luatest helper · d2b216e8
      Vladimir Davydov authored
      The helper skips a running test with luatest.skip_if in case executed by
      Tarantool Enterprise. It's better than checking the package directly in
      the test, because luatest.skip_if prints the reason why the test is
      skipped.
      
      NO_DOC=test
      NO_CHANGELOG=test
      d2b216e8
    • Vladimir Davydov's avatar
      func_cache: add space upgrade func holder type · 98bb3bd4
      Vladimir Davydov authored
      Online space upgrade applies a function to all tuples in a space.
      The function should be pinned so we need a new holder type.
      
      Follow-up commit 38b25832 ("box: add space upgrade stubs").
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      98bb3bd4
    • Vladimir Davydov's avatar
      opt_def: make opt_def::enum_strs const char *const * · 624076ba
      Vladimir Davydov authored
      Enum keys are usually stored in a const array of const strings.
      Fix opt_def::enum_strs type accordingly.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      624076ba
    • Vladimir Davydov's avatar
      field_def: pass char ** instead of char * to field_def_array_decode · 78b0d83a
      Vladimir Davydov authored
      So that the caller can figure out the end of fields array without
      calling mp_next(). Needed for decoding space upgrade format.
      
      While we are at it, let's also
       - Replace MP_ARRAY assertion with diag_set. This makes the function
         easier to use when the MsgPack format isn't guaranteed to be
         MP_ARRAY.
       - Rearrange arguments so that [out] arguments are grouped together.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      78b0d83a
    • Yaroslav Lobankov's avatar
      ci: fix jepsen testing · 45bc9281
      Yaroslav Lobankov authored
      Fix the following error:
      
          {"badRequest": {"message": "The requested availability zone is not
          available", "code": 400}}
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      45bc9281
    • Yaroslav Lobankov's avatar
      cmake: drop --vardir option when running tests · 96d7642e
      Yaroslav Lobankov authored
      This option very often makes tests fail due to the issue with the max
      length of socket paths (107 chars). Let's rely on the VARDIR environment
      variable or another default value (/tmp/t) when VARDIR is not defined.
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      96d7642e
    • Yaroslav Lobankov's avatar
      luatest: use /tmp/t as default socket dir · c90a227e
      Yaroslav Lobankov authored
      This patch changes the length of default socket paths for luatest tests
      (when the VARDIR environment variable is not defined). It should reduce
      the chance for the issue with the max length of socket paths (107 chars)
      to happen.
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      c90a227e
    • Yaroslav Lobankov's avatar
      ci: add VARDIR to PRESERVE_ENVVARS env variable · 7ae22101
      Yaroslav Lobankov authored
      This change adds VARDIR to the PRESERVE_ENVVARS environment variable
      to deliver it to 'packpack' docker containers while running packaging
      workflows.
      
      NO_DOC=ci
      NO_CHANGELOG=ci
      NO_TEST=ci
      7ae22101
    • Yaroslav Lobankov's avatar
      ci: change value for VARDIR env variable to /tmp/t · f483eac0
      Yaroslav Lobankov authored
      This patch changes the length of socket paths for tests by 2 chars.
      It should reduce the chance for the issue with the max length of socket
      paths (107 chars) to happen.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      f483eac0
    • Yaroslav Lobankov's avatar
      test-run: bump to new version · 60abb06d
      Yaroslav Lobankov authored
      Bump test-run to new version with the following changes:
      
      - Change default value for `vardir` option [1]
      - Read default value for `vardir` option from env [1]
      
      [1] tarantool/test-run#338
      
      NO_DOC=testing stuff
      NO_TEST=testing stuff
      NO_CHANGELOG=testing stuff
      60abb06d
  4. Apr 26, 2022
  5. Apr 25, 2022
    • Mons Anderson's avatar
      clock: return signed ints from time64 functions · fea8c8e4
      Mons Anderson authored
      Changed return value type in Lua for the following functions from
      `uint64_t` to `int64_t`:
      
      * clock.time64
      * clock.realtime64
      * clock.monotonic64
      * clock.process64
      * clock.thread64
      * fiber.time64
      * fiber.clock64
      
      Changed return value type in Module API for the following functions from
      `uint64_t` to `int64_t`:
      
      * fiber_time64
      * fiber_clock64
      
      Also enhanced clock.test.lua tap test with more testcases
      
      Due to unsigned values returned from time64 family function time
      calculations may lead to undesired value overflows. Having two 64-bit
      timestamps I cannot just subtract them without prior comparison. It is
      used often, for example when calculating time to a deadline (in queues
      or for expiration).
      
      Let's measure time, that pass between two sequential clock.time calls:
      
      ```lua
      print(clock.time() - clock.time()) -- -9.5367431640625e-07
      ```
      
      It's ok, with double type time return, but what about 64-bit integers?
      
      ```
      print(clock.time64() - clock.time64()) -- 18446744073709550616
      ```
      
      It returns weird integer overflow instead of expected `-1000`
      
      This is counterintuitive and gets conflicted with every time functions
      from system headers (which are always signed).
      
      If we consider some hypothetical code, that waits some time for a
      deadline, then instead of exiting loop, this code will go to "infinite"
      (actually 585 years) wait in the case when the deadline was passed.
      
      ```lua
      do
      local ts = clock.time64() + 1e9 -- some_timestamp_in_future, ex: now+1s
      -- ...
      local remaining = clock.time64() - ts
      -- ...
      if remaining > 0 then
          fiber.sleep( tonumber(remaining/1e9) )
      end
      print "1 second passed"
      end
      ```
      
      This is hard to notice (in real tests deadlines are always in the
      future), but in a real environment and under high load such cases occur.
      
      1. Since Lua functions will return ffi.typeof `ctype<int64_t>` instead
         of old `ctype<uint64_t>` any code, that uses _strict_ ffi type check
         will fail. But I hardly believe such code ever exists.
      2. Low-level Module API also changed, and code, which compiles with
         `-Werror` may also fail. This could be an issue and in such a case we
         may remove the update of module API change. Still, I consider this
         change reasonable, since all C-level API uses signed time values.
      
      @TarantoolBot document
      Title: Return type of time64 functions is changed
      
      These C functions in the public API used to return `uint64_t`, now they
      return `int64_t`: `clock_realtime64`, `clock_monotonic64`,
      `clock_process64`, `clock_thread64`, `fiber_time64()`,
      `fiber_clock64()`.
      
      These Lua functions used to return cdata of type `uint64_t`, now they
      return cdata of type `int64_t`: `fiber.time64()`, `fiber.clock64()`.
      fea8c8e4
    • Valeria Khokha's avatar
      doc: proofread changelogs/unreleased · 55cd16b5
      Valeria Khokha authored
      
      Fix wording, punctuation, and formatting.
      Fix dashes, trailing whitespaces, indentation, and file names.
      
      Needed for: #6430
      
      NO_DOC=changelog
      NO_TEST=changelog
      
      Co-authored-by: default avatarValeria Khokha <valeria.khokha@gmail.com>
      Co-authored-by: default avatarKseniia Antonova <xuniq.is.here@gmail.com>
      55cd16b5
    • Maxim Kokryashkin's avatar
      luajit: bump new version · c6b038b0
      Maxim Kokryashkin authored
      LuaJIT submodule is bumped to introduce the following changes:
      * GC64: disable sysprof support
      * build: make -DLUAJIT_DISABLE_SYSPROF work
      * test: disable sysprof C API tests with backtrace
      * tools: introduce parsers for sysprof
      * sysprof: introduce Lua API
      * memprof: add profile common section
      * core: introduce lua and platform profiler
      * memprof: move symtab to a separate module
      * core: separate the profiling timer from lj_profile
      * vm: save topframe info into global_State
      
      Within this changeset a parser for binary data dumped via the sampling
      profiler to Tarantool binary. It is a set of the following Lua modules:
      * sysprof/parse.lua: decode the sampling profiler event stream
      * sysprof/collapse.lua: collapse stacks obtained while profiling
      * sysprof.lua: Lua script and module to display data
      
      Besides, all new public Lua C API introduced within this changeset is
      added to extra/exports.
      
      Closes #781
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      c6b038b0
    • artembo's avatar
      ci: build Tarantool with enabled gc64 · 4af8e8d6
      artembo authored
      To make tarantool-gc64 available in repository we need to provide
      a product name in a package itself and in the repository.
      
      To build Tarantool with gc64 enabled, the flag -DLUAJIT_ENABLE_GC64
      needs to be passed to cmake flags. For this purposes GC64 env var
      was added to all workflows for Debian, Ubuntu, Centos and Fedora.
      
      This variable is processed to rename PRODUCT_NAME. If GC64 is
      eanbled, PRODUCT_NAME is `tarantool-gc64`, otherwise it stays
      `tarantool`, which goes to repository name and the package itself.
      Thus, Tarantool with gc64 enabled can be installed by
      `<packaen_manager> install -y tarantool-gc64`
      in the one single repository.
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      Closes: tarantool/tarantool-qa#161
      4af8e8d6
    • Sergey Bronnikov's avatar
      third_party: update libcurl from 7.76.0 to 7.80.0 · a353ec91
      Sergey Bronnikov authored
      Changelog: https://curl.se/changes.html#7_80_0
      
      NO_DOC=libcurl submodule bump
      NO_CHANGELOG=libcurl submodule bump
      NO_TEST=libcurl submodule bump
      
      Closes: #6029
      Closes: https://github.com/tarantool/security/issues/10
      a353ec91
    • Nikita Pettik's avatar
      box: add OS stubs for flight recorder requests · 247515e9
      Nikita Pettik authored
      This patch introduces two stubs: one to log incoming requests and
      another one to log outgoing responses. Log itself takes place in TX
      thread, in function family `tx_process_*`. At the start of TX processing
      we are able to use raw msgpack from input buffer to log it as it was
      received. In the same way we can log raw msgpack for responses (before
      sending message to IProto thread).
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      247515e9
    • Nikita Pettik's avatar
      iproto: pass to tx_end_msg() start of response · 211a3bfb
      Nikita Pettik authored
      It may turn out to be useful having output buffer savepoint pointing to
      the start of response. Using it one can restore the whole encoded IProto
      response. For instance, to dump it to auxiliary logs.
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      211a3bfb
    • Nikita Pettik's avatar
      box: add OS stubs for flight recorder metrics · ba47b766
      Nikita Pettik authored
      Let's add no-op interface functions and box cfg default parameters.
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      ba47b766
    • Nikita Pettik's avatar
      box: add OS stubs for flight recorder logger · e96d7847
      Nikita Pettik authored
      Let's add no-op interface functions and box cfg default parameters.
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      e96d7847
    • Nikita Pettik's avatar
      say: introduce on_log callback · 0483ace2
      Nikita Pettik authored
      It can be used for instance to duplicate logs somewhere. In particular
      case it is going to be used for flight recorder.
      
      NO_DOC=<No user visible changes>
      NO_TEST=<No functional changes>
      NO_CHANGELOG=<No functional changes>
      0483ace2
    • Nikita Pettik's avatar
      lua/prbuf: introduce entry:data() method · de90e54b
      Nikita Pettik authored
      prbuf entry contains pointer to raw memory (char *) and its length.
      Let's introduce :data() methods which would convert this memory chunk
      to Lua string.
      
      NO_DOC=<Feature for internal usage>
      NO_CHANGELOG=<Feature for internal usage>
      de90e54b
    • Ilya Verbin's avatar
      box: fix complex foreign keys with nullable fields · 6e7fa12a
      Ilya Verbin authored
      If a foreign key consists of multiple fields with `is_nullable=true`
      attribute, it's OK to insert a tuple with all-null key, however a
      combination of null and non-null fields is not allowed.
      
      Closes #7046
      
      NO_DOC=bugfix
      6e7fa12a
    • Boris Stepanenko's avatar
      raft: leader fencing on quorum loss · ed46feb2
      Boris Stepanenko authored
      Losing quorum didn't modify RAFT leader behaviour, this led to easy
      split brain situation, when current leader gets disconnected from the
      rest of the replicaset. Old leader remained RW while new leader could be
      elected in the rest of the replicaset.
      
      Now when current leader looses connection to replica (connection being
      alive relay and applier for that replica) leader checks that he still
      has at least quorum of alive connections to replicas. If less than
      quorum of alive connection remain: leader resigns its role becoming
      follower without starting new term. This makes old leader RO. Old leader
      won't start new elections because of pre-vote, added in 421a0968.
      
      Fenced leader will "freeze" its limbo, and won't CONFIRM nor ROLLBACK
      any synchronous transactions until it regains leadership. If another
      leader is elected while old one is fenced, once old leader is reconnected
      to new one "frozen" transaction will be rolled back.
      
      When quorum of connections is regained and new leader is not elected
      old leader or another node will start elections and elect new leader.
      
      Closes #6661
      
      @TarantoolBot document
      Title: RAFT leader fencing
      
      RAFT documentation must include that RAFT leader will resign its
      leadership if it has less than replication_synchro_quruom of alive
      connections to replicas (alive connection is connected relay and applier)
      if fencing is on. This applies to election_mode 'candidate' or 'manual'.
      Resigning leadership is becoming follower in current RAFT term, this
      leads to resigned leader becoming read-only.
      
      Introduce a new `box.cfg` option - `election_fencing_enabled`. If set to
      `true` fencing is on (default behaviour). If set to `false` fencing is off
      and leader doesn't resign it's leadership when looses quorum. If enabled
      on the current leader, when it doesn't have quorum of alive connections -
      leader will immediately resign its leadership.
      ed46feb2
    • Timur Safin's avatar
      datetime: timezone abbreviations changelog · 2539b02a
      Timur Safin authored
      Extended datetime literal parser(s) with basic support for timezone
      abbreviations. Not all names could be directly supported at the moment,
      only non-ambiguous.
      
      NO_DOC=will be documented with Olson patchset
      NO_TEST=changelog only
      2539b02a
    • Timur Safin's avatar
      datetime: raise error for ambiguous, nyi timezones · a3d64a37
      Timur Safin authored
      Use different return values for indication that recognized
      timezone suffix is ambiguous or not yet handled (before tzdata
      integration to come in next patchset).
      
      i.e. we could always parse '2020-02-02T01:03 MSK' as MSK is
      static +03:00, but it's ambiguous if there is something like
      '2020-02-02T01:03 AT' as there are multiple ATs (e.g.
      'Alaska Time' and 'Atlantic Time').
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      a3d64a37
    • Timur Safin's avatar
      datetime: output original timezone name · 98750d7d
      Timur Safin authored
      Use tzindex field wherever we need to output datetime
      value to string (with or without format in `:format()` or
      `tostring()` correspondingly).
      
      NO_DOC=yet
      NO_CHANGELOG=yet
      98750d7d
    • Timur Safin's avatar
      datetime: parse timezone abbreations · 36cac6f8
      Timur Safin authored
      Use our own list of generated timezone names for symbol table
      initialization, and then use this symbol table for datetime suffix
      parsing without format.
      
      Also, use newly integrated timezone names lookup functions in the
      `tnt_strptime` if `%Z` flag was ordered. Return corresponding
      `tm_gmtoff` (datetime->tzoffset) and `tm_tzindex`
      (datetime->tzindex) for later consumption.
      
      At the moment we could test only non-ambiguous abbreviations,
      which might be converted to tzoffset in a deterministic way.
      
      i.e. 'MSK' is always +0300, or 'MSD' is always +0400, but 'AT'
      may be either 'Alaska Time' (-0900) or 'Atlantic Time' (-0400/-0300)
      thus there is no (yet) any simple way to translate to their offset.
      
      NO_DOC=yet
      NO_CHANGELOG=yet
      36cac6f8
    • Timur Safin's avatar
      datetime, tools: generate timezone abbreviations list · 9e645d64
      Timur Safin authored
      We use gen-zone-abbrevs.pl script to generate file describing timezones
      and their fixed ids. Please see `perldoc gen-zone-abbrevs.pl` for more
      detailed usage description.
      
      Generator uses `zone-abbrevs.txt` as an input list of all known to
      the date timezone abbreviations. We use this page as original source
      https://www.timeanddate.com/time/zones/
      
      Names encoding schema used by generator script supposed to be stable
      enough for input set of properly sorted zone names, including:
      - military zone names;
      - zone abbreviations;
      - human-readable IANA zone names.
      
      It generates _future compatible_, strictly ordered, sequential
      list.
      
      NO_DOC=yet
      NO_CHANGELOG=yet
      NO_TEST=yet
      9e645d64
    • Timur Safin's avatar
      infra: .editorconfig for perl files · b22073e1
      Timur Safin authored
      NO_DOC=infra
      NO_CHANGELOG=infra
      NO_TEST=infra
      b22073e1
    • Georgiy Lebedev's avatar
      box: log dangerous `select` calls · 7aa02f30
      Georgiy Lebedev authored
      Empty or nil `select` calls on user spaces tend to be
      dangerous: we see Tarantool crashing with OOM, or Tarantool becoming
      unacceptably slow and its transaction thread utilizing 100% CPU usage.
      Also, debugging of such killer-requests is often too painful and
      exhausting.
      
      To mitigate this, create a critical log entry containing the current Lua
      stack traceback upon such function calls — a user can explicitly
      request a full scan though by passing `fullscan = true` to `select`'s
      `options` table argument in which case the log entry will not be created.
      These log entries are also rate limited.
      
      Closes #6539
      
      @TarantoolBot document
      Title: new behaviour of empty or nil `select` calls on user spaces
      
      A critical log entry containing the current stack
      traceback is created upon empty or nil `select` calls on user spaces.
      
      A user can explicitly request a full scan though by passing
      `fullscan = true` to `select`'s `options` table argument in which a case a
      log entry will not be created.
      7aa02f30
    • Georgiy Lebedev's avatar
      lua: introduce `Ratelimit` class · 9b7467d1
      Georgiy Lebedev authored
      Introduce a `Ratelimit` class equivalent to `src/lib/core/ratelimit.h` to
      `log` module for internal usage: it acts as a wrapper around the log API.
      
      In scope of #6539 we need to make critical log entries: add S_CRIT alias
      for this purpose and a `log_crit` method to `Ratelimit`.
      
      Needed for #6539
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      NO_TEST=internal
      9b7467d1
Loading