Skip to content
Snippets Groups Projects
  1. Oct 26, 2022
    • Vladimir Davydov's avatar
      msgpack: fix crash on decode of 0xc1 · e48a8f4b
      Vladimir Davydov authored
      0xc1 isn't a valid MsgPack header, but it was allowed by mp_check.
      As a result, msgpack.decode crashed while trying to decode it.
      This commit updates the msgpuck library to fix this issue.
      
      Closes #7818
      
      NO_DOC=bug fix
      
      (cherry picked from commit ced405af)
      e48a8f4b
  2. Oct 25, 2022
    • Serge Petrenko's avatar
      security: make os.getenv safe · ec3eb525
      Serge Petrenko authored
      Closes #7797
      
      NO_DOC=security fix
      NO_TEST=security fix
      
      (cherry picked from commit dd7d46af)
      ec3eb525
    • Serge Petrenko's avatar
      security: check size boundaries for getenv() returns · 829b65f8
      Serge Petrenko authored
      getenv() return values cannot be trusted, because an attacker might set
      them. For instance, we shouldn't expect, that getenv() returns a value
      of some sane size.
      
      Another problem is that getenv() returns a pointer to one of
      `char **environ` members, which might change upon next setenv().
      
      Introduce a wrapper, getenv_safe(), which returns the value only when
      it fits in a buffer of a specified size, and copies the value onto the
      buffer. Use this wrapper everywhere in our code.
      
      Below's a slightly decorated output of `grep -rwn getenv ./src --include
      *.c --include *.h --include *.cc --include *.cpp --include *.hpp
      --exclude *.lua.c` as of 2022-10-14.
      `-` marks invalid occurences (comments, for example),
      `*` marks the places that are already guarded before this patch,
      `X` mars the places guarded in this patch, and
      `^` marks places fixed in the next commit:
      
      NO_WRAP
      ```
      * ./src/lib/core/coio_file.c:509:	const char *tmpdir = getenv("TMPDIR");
      X ./src/lib/core/errinj.c:75: const char *env_value = getenv(inj->name);
      - ./src/proc_title.c:202: * that might try to hang onto a getenv() result.)
      - ./src/proc_title.c:241:	* is mandatory to flush internal libc caches on getenv/setenv
      X ./src/systemd.c:54: sd_unix_path = getenv("NOTIFY_SOCKET");
      * ./src/box/module_cache.c:300: const char *tmpdir = getenv("TMPDIR");
      X ./src/box/sql/os_unix.c:1441: azDirs[0] = getenv("SQL_TMPDIR");
      X ./src/box/sql/os_unix.c:1446: azDirs[1] = getenv("TMPDIR");
      * ./src/box/lua/console.c:394: const char *envvar = getenv("TT_CONSOLE_HIDE_SHOW_PROMPT");
      ^ ./src/box/lua/console.lua:771: local home_dir = os.getenv('HOME')
      ^ ./src/box/lua/load_cfg.lua:1007: local raw_value = os.getenv(env_var_name)
      X ./src/lua/init.c:575: const char *path = getenv(envname);
      X ./src/lua/init.c:592: const char *home = getenv("HOME");
      * ./src/find_path.c:77: snprintf(buf, sizeof(buf) - 1, "%s", getenv("_"));
      ```
      NO_WRAP
      
      Part-of #7797
      
      NO_DOC=security
      
      (cherry picked from commit b86395ff)
      829b65f8
    • Mergen Imeev's avatar
      sql: fix another cursor invalidation · 1b357d93
      Mergen Imeev authored
      This patch fixes the issue described in issue #5310 when the tuple
      format has more fields than the space format. This solution is more
      general than the solution in 89057a21.
      
      Follow-up #5310
      Closes #4666
      
      NO_DOC=bugfix
      
      (cherry picked from commit 5a38c5c9)
      1b357d93
  3. Oct 21, 2022
    • Georgiy Lebedev's avatar
      build: use relative paths in diagnostics and debugging information · ac645aa4
      Georgiy Lebedev authored
      Since our diagnostics use the `__FILE__` macro, they provide absolute
      paths, which is kind of redundant and inconsistent: replace them with
      relative ones.
      
      As for debugging information, replacing absolute paths with relative ones
      also requires an extra command to tell the debugger where to find the
      source files, which is not convenient for developers: provide a new
      `DEV_BUILD` option (turned off by default), which replaces absolute paths
      with relative ones in debugging information if turned off.
      
      Strip the prefix map flags from compiler flags exported to tarantool via
      `src/trvia/config.h`.
      
      Closes #7808
      
      NO_DOC=<verbosity>
      NO_TEST=<verbosity>
      
      (cherry picked from commit 256da010)
      ac645aa4
    • Georgiy Lebedev's avatar
      test-run: bump new version · bfa153bb
      Georgiy Lebedev authored
      Fixes bug with determination of debug build (tarantool/test-run#352).
      
      NO_CHANGELOG=<internal submodule version update>
      NO_TEST=<submodule version update>
      NO_DOC=<submodule version update>
      
      (cherry picked from commit e554e481)
      bfa153bb
    • Georgiy Lebedev's avatar
      build: refactor setting hardening compiler flags · 0b47b16e
      Georgiy Lebedev authored
      Setting hardening compiler flags is used in three places: default build,
      static build and enterprise build — refactor it into a separate module.
      
      Follow-up e6abe1c9
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      
      (cherry picked from commit dd51a2fa)
      0b47b16e
    • Georgiy Lebedev's avatar
      build: refactor passing compiler flags to dependencies · 2a61c0a7
      Georgiy Lebedev authored
      e6abe1c9 passes compiler flags to dependencies via a `<project>_build` macro
      parameter, which is, firstly, inconvenient, and, secondly, as a result, not
      all dependencies got the required compiler flags passed: use global
      variables instead and pass these flags to skipped dependencies.
      
      Follow-up e6abe1c9
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      
      (cherry picked from commit c6794757)
      2a61c0a7
    • Georgiy Lebedev's avatar
      build: fix unsetting of variables in `add_compile_flags` macro · d5e9d14c
      Georgiy Lebedev authored
      `add_compile_flags` macro relied on undefined behaviour: instead of
      unsetting the `_lang` and `_flag` variables, it tried to unset their
      values — fix this.
      
      NO_CHANGELOG=build
      NO_DOC=build
      NO_TEST=build
      
      (cherry picked from commit 4aa9f154)
      d5e9d14c
  4. Oct 20, 2022
  5. Oct 19, 2022
    • Mergen Imeev's avatar
      box: fix format of _vfunc · ee06f892
      Mergen Imeev authored
      The _vfunc system space is the sysview for the _func system space.
      However, the _vfunc format is different from the _func format. This
      patch makes the _vfunc format the same as the _func format.
      
      Closes #7822
      
      NO_DOC=bugfix
      
      (cherry picked from commit 707da125)
      ee06f892
  6. Oct 18, 2022
    • Timur Safin's avatar
      datetime: datetimes subtractions ignored timezone · 8aa13474
      Timur Safin authored
      We used to ignore timezone difference (in `tzoffset`) for
      datetime subtraction operation:
      
      ```
      tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
      ---
      - +0 seconds
      ...
      
      tarantool> datetime.new{tz='MSK'}.timestamp -
                 datetime.new{tz='UTC'}.timestamp
      ---
      - -10800
      ...
      ```
      
      Now we accumulate tzoffset difference in the minute component
      of a resultant interval:
      
      ```
      tarantool> datetime.new{tz='MSK'} - datetime.new{tz='UTC'}
      ---
      - -180 minutes
      ...
      ```
      
      Closes #7698
      
      NO_DOC=bugfix
      
      (cherry picked from commit 0daed8d5)
      Unverified
      8aa13474
    • Timur Safin's avatar
      datetime: fix interval arithmetic for DST · 9f8b05ab
      Timur Safin authored
      We did not take into consideration the fact that
      as result of date/time arithmetic we could get
      in a different timezone, if DST boundary has been
      crossed during operation.
      
      ```
      tarantool> datetime.new{year=2008, month=1, day=1,
      			tz='Europe/Moscow'} +
      	   datetime.interval.new{month=6}
      ---
      - 2008-07-01T01:00:00 Europe/Moscow
      ...
      ```
      
      Now we resolve tzoffset at the end of operation if
      tzindex is not 0.
      
      Fixes #7700
      
      NO_DOC=bugfix
      
      (cherry picked from commit 6ca07285)
      Unverified
      9f8b05ab
    • Yaroslav Lobankov's avatar
      ci: use `coveralls-lcov` instead of GitHub action · abba4f80
      Yaroslav Lobankov authored
      In PR #7791 the coverage stuff is slightly updated. The changes are
      related to excluding some directories where the coverage will not be
      calculated. The patch itself is straightforward enough, but I can see
      the problems with uploading the results to coveralls.io:
      
      NO_WRAP
          Using lcov file: ./coverage.info
          FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
      
          <--- Last few GCs --->
      
          [2683200:0x327c000]    27836 ms: Scavenge 2033.4 (2038.7) -> 2033.2 (2044.2) MB, 6.8 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
          [2683200:0x327c000]    27854 ms: Scavenge 2036.7 (2044.2) -> 2037.1 (2044.4) MB, 15.6 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
          [2683200:0x327c000]    27871 ms: Scavenge 2037.2 (2044.4) -> 2036.7 (2055.9) MB, 17.1 / 0.0 ms  (average mu = 0.258, current mu = 0.243) allocation failure
      
          <--- JS stacktrace --->
      
          ==== JS stack trace =========================================
      
              0: ExitFrame [pc: 0x140dc19]
          Security context: 0x2481918808d1 <JSObject>
              1: encode(aka encode) [0x23ec5e514a11] [/opt/actions-runner/_work/_actions/coverallsapp/github-action/v1.1.2/node_modules/qs/lib/utils.js:~118] [pc=0x193ecd3d5d82](this=0x35d34e4804b1 <undefined>,0x1e5895380119 <Very long string[60387428]>)
              2: stringify(aka stringify) [0x23ec5e5174c9] [/opt/actions-runner/_work/_actions/coverallsapp/github-action/v1.1.2/n...
      
           1: 0xa1a640 node::Abort() [/opt/actions-runner/externals/node12/bin/node]
           2: 0xa1aa4c node::OnFatalError(char const*, char const*) [/opt/actions-runner/externals/node12/bin/node]
           3: 0xb9a62e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/opt/actions-runner/externals/node12/bin/node]
           4: 0xb9a9a9 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/opt/actions-runner/externals/node12/bin/node]
           5: 0xd57c25  [/opt/actions-runner/externals/node12/bin/node]
           6: 0xd582b6 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/opt/actions-runner/externals/node12/bin/node]
           7: 0xd64b75 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/opt/actions-runner/externals/node12/bin/node]
           8: 0xd65a25 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/opt/actions-runner/externals/node12/bin/node]
           9: 0xd670cf v8::internal::Heap::HandleGCRequest() [/opt/actions-runner/externals/node12/bin/node]
          10: 0xd15b35 v8::internal::StackGuard::HandleInterrupts() [/opt/actions-runner/externals/node12/bin/node]
          11: 0x1071176 v8::internal::Runtime_StackGuard(int, unsigned long*, v8::internal::Isolate*) [/opt/actions-runner/externals/node12/bin/node]
          12: 0x140dc19  [/opt/actions-runner/externals/node12/bin/node]
      NO_WRAP
      
      I have tried to use the latest version of the action (1.1.3), but I have
      got the same issue.
      
      It looks like it is a `coverallsapp/github-action` issue due to issue
      in JavaScript due to inefficient amount of memory. The corresponding
      bug [1] was filed against the repo.
      
      So I suggest switching to the console utility `coveralls-lcov`. I did
      some internal testing and it works fine.
      
      [1] https://github.com/coverallsapp/github-action/issues/133
      
      Fixes tarantool/tarantool-qa#278
      
      NO_DOC=ci
      NO_TEST=ci
      NO_CHANGELOG=ci
      
      (cherry picked from commit 761574e3)
      Unverified
      abba4f80
  7. Oct 14, 2022
  8. Oct 13, 2022
  9. Oct 12, 2022
  10. Oct 11, 2022
    • Mergen Imeev's avatar
      sql: change rules used to determine NULLIF() type · 5c6afe47
      Mergen Imeev authored
      This patch introduces new rules to determine type of NULLIF() built-in
      function.
      
      Closes #6990
      
      @TarantoolBot document
      Title: New rules to determine type of result of NULLIF
      
      The type of the result of NULLIF() function now matches the type of the
      first argument.
      
      (cherry picked from commit 805cbaa7)
      5c6afe47
    • Mergen Imeev's avatar
      sql: change rules used to determine CASE type · 5585825a
      Mergen Imeev authored
      This patch introduces new rules to determine type of CASE operation.
      
      Part of #6990
      
      @TarantoolBot document
      Title: New rules to determine type of result of CASE
      
      New rules are applied to determine the type of the CASE operation. If
      all values are NULL with no type, or if a bind variable exists among
      the possible results, then the type of CASE is ANY. Otherwise, all NULL
      values with no type are ignored, and the type of CASE is determined
      using the following rules:
      1) if all values of the same type, then type of CASE is this type;
      2) otherwise, if any of the possible results is of one of the
      incomparable types, then the type of CASE is ANY;
      3) otherwise, if any of the possible results is of one of the
      non-numeric types, then the type of CASE is SCALAR;
      4) otherwise, if any of the possible results is of type NUMBER, then the
      type of CASE is NUMBER;
      5) otherwise, if any of the possible results is of type DECIMAL, then
      the type of CASE is DECIMAL;
      6) otherwise, if any of the possible results is of type DOUBLE, then the
      type of CASE is DOUBLE;
      7) otherwise the type of CASE is INTEGER.
      
      (cherry picked from commit 90f64460)
      5585825a
  11. Oct 07, 2022
  12. Oct 06, 2022
  13. Oct 05, 2022
  14. Sep 30, 2022
  15. Sep 29, 2022
    • Serge Petrenko's avatar
      gc: replace vclockset_psearch with _match in wal_collect_garbage_f · d6fc95f6
      Serge Petrenko authored
      When using vclockset_psearch, the resulting vclock may be incomparable
      to the search key. For example, with a vclock set { } (empty vclock),
      {0: 1, 1: 10}, {0: 2, 1:11} vclockset_psearch(set, {0:2, 1: 9}) might
      return {0: 1, 1: 10}, and not { }.
      This is known and avoided in other places, for example
      recover_remaining_wals(), where vclockset_match() is used instead.
      vclockset_match() starts with the same result as vclockset_psearch() and
      then unwinds the result until the first vclock which is less or equal to
      the search key is found.
      
      Having vclockset_psearch in wal_collect_garbage_f could lead to issues
      even before local space changes became written to 0-th vclock component.
      Once replica subscribes, its' gc consumer is set to the vclock, which
      the replica sent in subscribe request. This vclock might be incomparable
      with xlog vclocks of the master, leading to the same issue of
      potentially deleting a needed xlog during gc.
      
      Closes #7584
      
      NO_DOC=bugfix
      
      (cherry picked from commit c63bfb9a)
      d6fc95f6
  16. Sep 28, 2022
    • Georgiy Lebedev's avatar
      memtx: fix transaction manager MVCC invariant violation · 1fac9eef
      Georgiy Lebedev authored
      We hold the following invariant in MVCC: the story at the top of the
      history chain is present in index.
      
      If a story is subject to be deleted from index and there is an older story
      in the history chain, the older story starts to be at the top of the
      history chain and is not present in index, which violates our invariant:
      explicitly check for this case when evaluating whether a story can be
      garbage collected and add an assertion to check the invariant above is not
      violated.
      
      Rollbacked stories need to be handled in a special way: they are
      present at the end of some history chains and completely unlinked from
      others (which also implies they are not present in the corresponding
      indexes).
      
      `memtx_tx_story_full_unlink` is called in two contexts: space deletion, in
      which we delete all stories, and garbage collection step — the former case
      can break the invariant described above, while the latter must preserve it,
      hence add two different functions for the corresponding contexts.
      
      Closes #7490
      
      NO_CHANGELOG=<internal bugfix not user observable>
      NO_DOC=<bugfix>
      
      (cherry picked from commit c8eccfbb)
      1fac9eef
    • Georgiy Lebedev's avatar
      memtx: rework transaction rollback · 61be2c8f
      Georgiy Lebedev authored
      When we rollback a transaction statement, we relink its read trackers
      to a newer story in the history chain, if present (6c990a7b), but we do not
      handle the case when there is no newer story.
      
      If there is an older story in the history chain, we can relink the
      rollbacked story's reader to it, but if the rollbacked story is the
      only one left, we need to retain it, because it stores the reader list
      needed for conflict resolution — such stories are distinguished by the
      rollbacked flag, and there can be no more than one such story located
      strictly at the end of a given history chain (which means a story can be
      fully unlinked from some indexes and present at the end of others).
      
      There are several nuances we need to account for:
      
      Firstly, such rollbacked stories must be impossible to read from an index:
      this is ensured by `memtx_tx_story_is_visible`.
      
      Secondly, rollbacked transactions need to be treated as prepared with
      stories that have `add_psn == del_psn`, so that they are correctly deleted
      during garbage collection.
      
      After this logical change we have the following partially ordered set over
      tuple stories:
      ———————————————————————————————————————————————————————> serialization time
      |- - - - - - - -|— — — — — -|— — — — — |— — — — — — -|— — — — — — — -
      | No more than  | Committed | Prepared | In-progress | One dirty
      | one rollbacked|           |          |             | story in index
      | story         |           |          |             |
      |- - - - - - - -|— — — — — -| — — — — —|— — — — — — -|— — — — — — — —
      
      Closes #7343
      
      NO_DOC=bugfix
      
      (cherry picked from commit 56cf737c)
      61be2c8f
Loading