Skip to content
Snippets Groups Projects
  1. Nov 07, 2022
  2. Nov 01, 2022
    • Nikolay Shirokovskiy's avatar
      test: turn LuaJIT off in tests with dlsym · 67e79b15
      Nikolay Shirokovskiy authored
      `dlsym` is known to be buggy in FreeBSD. See #7640.
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      67e79b15
    • Nikolay Shirokovskiy's avatar
      box: straighten log configuration code · 571cdc3e
      Nikolay Shirokovskiy authored
      See [1] for some details on why the code for log configuration needs
      some care. In short log config validity checks are spread thru many
      places, on some code paths we use one checks and on other code paths
      other checks. And we make repetitive validity checks many times in
      runtime on single configure call.
      
      We can also reuse code for setting default values, checking options
      type and resetting values to default.
      
      - As a side effect of refactoring one can now reset values to default thru
        `log.cfg()` so now `log.cfg()` is on par with `box.cfg` in this respect.
      
      - This patch also drops conversion `log_level` from string to number.
      
        Before (shorten):
      
          tarantool> box.cfg{log_level='warn'}
          tarantool> box.cfg.log_level
          - info
          tarantool> log.cfg.level
          - 5
      
        Also:
          tarantool> log.cfg{level='info'}
          tarantool> log.cfg.level
          - 5
          tarantool> box.cfg{}
          tarantool> box.cfg.log_level
          - 5
      
        After patch if `log_level`/`level` is given as string than it is saved
        and returned as string too. I guess it should not affect users but looks
        more handy.
      
      - Also fixed issue with inconsistent setting `log_nonblock` thru
        `box.cfg()` and `log.cfg()`. In former case `nil` means setting default
        depending on logger type. In the latter case `nil` meant setting
        `nonblock` to `false`.
      
      - Also patch fixes #7447.
      
      Closes #7447.
      
      [1] PR for this refactoring
      https://github.com/tarantool/tarantool/pull/7454
      
      NO_DOC=refactoring/tiny API improvemnent
      571cdc3e
  3. Oct 21, 2022
    • Georgiy Lebedev's avatar
      build: use relative paths in diagnostics and debugging information · 256da010
      Georgiy Lebedev authored
      Since our diagnostics use the `__FILE__` macro, they provide absolute
      paths, which is kind of redundant and inconsistent: replace them with
      relative ones.
      
      As for debugging information, replacing absolute paths with relative ones
      also requires an extra command to tell the debugger where to find the
      source files, which is not convenient for developers: provide a new
      `DEV_BUILD` option (turned off by default), which replaces absolute paths
      with relative ones in debugging information if turned off.
      
      Strip the prefix map flags from compiler flags exported to tarantool via
      `src/trvia/config.h`.
      
      Closes #7808
      
      NO_DOC=<verbosity>
      NO_TEST=<verbosity>
      256da010
    • Georgiy Lebedev's avatar
      build: refactor setting hardening compiler flags · dd51a2fa
      Georgiy Lebedev authored
      Setting hardening compiler flags is used in three places: default build,
      static build and enterprise build — refactor it into a separate module.
      
      Follow-up e6abe1c9
      
      NO_CHANGELOG=refactoring
      NO_DOC=refactoring
      NO_TEST=refactoring
      dd51a2fa
  4. Sep 15, 2022
    • Ilya Verbin's avatar
      cmake: add extra security compiler options · e6abe1c9
      Ilya Verbin authored
      Introduce cmake option ENABLE_HARDENING, which is TRUE by default for
      non-debug regular and static builds, excluding AArch64 and FreeBSD.
      It passess compiler flags that harden Tarantool (including the bundled
      libraries) against memory corruption attacks. The following flags are
      passed:
      
      * -Wformat - Check calls to printf and scanf, etc., to make sure that
        the arguments supplied have types appropriate to the format string
        specified.
      
      * -Wformat-security -Werror=format-security - Warn about uses of format
        functions that represent possible security problems. And make the
        warning into an error.
      
      * -fstack-protector-strong - Emit extra code to check for buffer
        overflows, such as stack smashing attacks.
      
      * -fPIC -pie - Generate position-independent code (PIC). It allows to
        take advantage of the Address Space Layout Randomization (ASLR).
      
      * -z relro -z now - Resolve all dynamically linked functions at the
        beginning of the execution, and then make the GOT read-only.
      
      Also do not disable hardening for Debian and RPM-based Linux distros.
      
      Closes #5372
      Closes #7536
      
      NO_DOC=build
      NO_TEST=build
      e6abe1c9
  5. Aug 16, 2022
    • Ilya Verbin's avatar
      cmake: detect changes in source files upon static build · a1f554bd
      Ilya Verbin authored
      Currently `make` in `static-build` doesn't rebuild Tarantool when source
      files are changed. Fix this by setting BUILD_ALWAYS option, which forces
      rescan for changes of the external project [1]:
      
      > This option is not normally needed unless developers are expected to
      > modify something the external project's build depends on in a way that
      > is not detectable via the step target dependencies (e.g. SOURCE_DIR is
      > used without a download method and developers might modify the sources
      > in SOURCE_DIR).
      
      It is available since CMake 3.1, so update cmake_minimum_required, as we
      already require it (fa8d70ca).
      
      [1] https://cmake.org/cmake/help/latest/module/ExternalProject.html
      
      Part of #7536
      
      NO_DOC=build
      NO_TEST=build
      NO_CHANGELOG=minor
      a1f554bd
  6. Aug 03, 2022
    • Serge Petrenko's avatar
      build: fix check-dependencies test on OS X · 116993d2
      Serge Petrenko authored
      Commit ed16c1e5 ("build: fix bundled
      libcurl and c-ares build on OS X") fixed building libcurl with ares by
      adding a dependency on libresolv for Mac OS X. It was forgotten to add
      libresolv to check-dependencies exceptions for static build. Do this now.
      
      NO_DOC=CI stuff
      NO_TEST=CI stuff
      NO_CHANGELOG=CI stuff
      116993d2
    • Vladimir Davydov's avatar
      static-build: update zlib to version 1.2.12 · 5da1f6be
      Vladimir Davydov authored
      NO_DOC=build
      NO_TEST=build
      5da1f6be
    • Alexander Turenko's avatar
      static-build: update libiconv to 1.17 (Mac OS) · 3ca56bc4
      Alexander Turenko authored
      The static build for Linux leans on glibc provided iconv functions. On
      Mac OS it links GNU libiconv statically. This patch updates the libiconv
      version from 1.16 (released in 2019) to 1.17 (released in 2022).
      
      It is just regular maintenance update. No behaviour changes are
      expected. The libiconv 1.17 changelog (see the NEWS file in the archive)
      does not mention anything that may have influence on behavior of
      tarantool's iconv built-in module.
      
      NO_DOC=no user-visible changes
      NO_TEST=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      3ca56bc4
  7. Aug 02, 2022
  8. Jul 15, 2022
  9. Apr 14, 2022
  10. Apr 13, 2022
  11. Mar 29, 2022
  12. Mar 24, 2022
  13. Mar 18, 2022
    • Georgiy Lebedev's avatar
      libunwind: add new submodule · 7dc9fe44
      Georgiy Lebedev authored
      Investigation of GNU libunwind problems on the aarch64-linux-gnu
      platform drive us to the conclusion that libunwind-1.2.1 provided by
      major distribution packages is broken. Not to mention that its test
      suite fails with SEGFAULTs.
      
      Last but not least, some distributions, e.g. CentOS 8 (see #4611) do
      not provide a libunwind package.
      
      Hence, bundle libunwind: bundling is enabled by default on all
      platforms, except for macOS — a system package can be used if its
      version is greater or equal than 1.3.0 (minimal version that does not
      seem to be broken on aarch64-linux-gnu).
      
      * Add new submodule: bump it to current master.
      * Refactor libunwind package search logic out of compiler.cmake.
      * Add CMake script for building bundled libunwind.
      * Add CMake script for extracting version of libunwind.
      * Re-enable backtrace for all RHEL distributions by default.
      * Remove libunwind from static build.
      
      Needed for #4002
      Closes #4611
      
      NO_DOC=build system
      NO_TEST=build system
      7dc9fe44
  14. Jan 31, 2022
    • Andrei Sidorov's avatar
      Fix static build for macOS Bug Sur · a7724d8f
      Andrei Sidorov authored
      Fix static build for macOS 11.5 or higher.
      On macOS SDK ver. 11.5 some `*.dylib` files was replaced with `*.tbd`.
      So we replace `libunwind.dylib` on `libunwind.tbd`.
      Because of macOS 10.15 support being dropped
      conditional is not needed.
      
      Closes #6052
      a7724d8f
  15. Dec 13, 2021
    • Vladimir Davydov's avatar
      Make lookup of statically built binary more robust · 88332b35
      Vladimir Davydov authored
      If the tarantool repository is used as a submodule named <foobar> in
      another repository, then the statically built binary will be placed in
      
        <binary_dir>/<foobar>/src/tarantool
      
      not in
      
        <binary_dir>/src/tarantool
      
      where static-build/CMakeLists.txt currently tries to look it up in order
      to run `ctest` and so we can't use static-build/CMakeLists.txt as is.
      
      Let's instead use
      
        <install_dir>/bin/tarantool
      
      This way `ctest` will work for static-build in both open-source and EE
      repository without requiring any modifications.
      88332b35
    • mechanik20051988's avatar
      uri: implement ability to parse URIs passed in different ways · 37c35677
      mechanik20051988 authored
      Previously, URI can be passed as a string, which contains one URI
      or several URIs separated by commas. Now URIs can be passed in
      different ways: as before, as a table which contains URI and it's
      parameters in "param" table, as a table which contains URI strings
      and URI tables. Also there are different ways to specify properties
      for URI: in a string which contains URI, after '?' delimiter, in a
      table which contains URI in "params" table, in "default_params" table
      if it is default parameters for all URIs.
      For this purposes new method `parse_many` was implemented in tarantool
      `uri` library. Also `parse` method was updated to make possible the
      same as new `parse_many` method but only for single URI.
      ```lua
      uri = require('uri')
      -- Single URI, passed as before
      uri.parse_many("/tmp/unix.sock")
      -- Single URI, with query paramters
      uri.parse_many("/tmp/unix.sock?q1=v1&q2=v2")
      -- Several URIs with parameters in one string, separated by commas
      uri.parse_many("/tmp/unix.sock_1?q=v, /tmp/unix.sock_2?q=v")
      -- Single URI passed in table, with additional parameters, passed
      -- in "params" table. This parameters overwrite parameters from
      -- URI string (q1 = "v2" in example below).
      uri.parse_many({"/tmp/unix.sock?q1=v1", params = {q1 = "v2"}})
      -- For parse it's also works now
      uri.parse({"/tmp/unix.sock?q1=v1", params = {q1 = "v2"}})
      -- Several URIs passed in table with default parameters, passed
      -- in "default_params" table, which are used for parameters, which
      -- not specified for URI (q3 parameter with "v3" value corresponds
      -- to all URIs, and used if there is no such parameter in URI).
      uri.parse_many({
          "/tmp/unix.sock_1?q1=v1",
          { uri = "/tmp/unix.sock_2", params = { q2 = "v2" } },
          default_params = { q3 = "v3" }
      })
      ```
      37c35677
    • mechanik20051988's avatar
      uri: implement ability to parse several URIs · 3661bc3a
      mechanik20051988 authored
      Implement ability to parse a string, which contains several URIs
      separated by commas. Each URI can contain different query parameters
      after '?'. All created URIs saved in new implemented struct `uri_set`.
      
      Part of #5928
      3661bc3a
    • mechanik20051988's avatar
      uri: rework struct uri to use a copy of the original splitted string. · c65fd012
      mechanik20051988 authored
      In the future, it is planned to extend the URI structure to allow its
      passing with different options and in different formats (see next commit
      `uri: implement ability to parse URI query paramters` for example). For
      these purposes, it is planned to use functions that modify the source
      string, for example `strtok_r`, so we need to rework the URI structure
      to create copies of the string for each of the URI components.
      
      Part of #5928
      c65fd012
  16. Dec 10, 2021
  17. Nov 18, 2021
  18. Sep 27, 2021
    • Leonid Vasiliev's avatar
      export: wrap exported msgpack symbols · 592db3b0
      Leonid Vasiliev authored
      Exporting symbols of a third party library is not a best practice,
      as we know from [1]. Let's wrap the msgpack symbols that need to
      be exported with the "tnt_" prefix.
      
      While working on the patch, it was decided to export the msgpack
      symbols that are used in "msgpuckffi.lua".
      In test shared libraries where the symbols "mp_***_{decimal,uuid}"
      are used, they are replaced to exported "tnt_mp_***_{decimal,uuid}",
      because in the case of linking with "libcore.a" the "libcore.a"
      needs to be rebuild with the "-fPIC" flag, that seems as overkill
      for tests.
      
      1. https://github.com/tarantool/memcached/issues/59
      
      Closes #5932
      592db3b0
  19. Aug 09, 2021
    • Leonid Vasiliev's avatar
      cmake: hide tarantool symbols back · 5ceabb37
      Leonid Vasiliev authored
      After unhiding all internal symbols([1]) we experience a bunch of
      problems ([2], [3]). The second one (clash of symbols from different version
      of the "small" library) still have no good solution.
      You can find more on the topic [4].
      
      The situation for tarantool executable is the same as for any other library.
      A library should expose only its public API and should not increase probability
      of hard to debug problems due to clash of a user's code with an internal name
      from the library.
      
      Let's hide all symbols by default and create a list of exported symbols.
      (In fact, this patch is a revert of the patch
      03790ac5 ([5]) taking into account the changes
      made to the code)
      
      Explanation of adding some controversial symbols to the export list:
      
      * The following symbols are used in shared libraries used in tests
        ("cfunc*.so", "sql_uuid.so", "gh-6024-funcs-return-bin.so", "function1.so",
        "gh-5938-wrong-string-length.so", "module_api.so")
      
          mp_check
          mp_encode_array
          mp_encode_bin
          mp_encode_bool
          mp_encode_int
          mp_encode_map
          mp_encode_nil
          mp_encode_str
          mp_encode_uint
          mp_decode_array_slowpath
          mp_decode_str
          mp_next_slowpath
          mp_load_u8
          mp_next
          mp_sizeof_array
          mp_sizeof_str
          mp_type_hint
          decimal_from_string
      
      * These symbols are used in "crypto.lua" and, if absent, will lead to the
        failure of the "static_build_cmake_linux" on CI (the crypto prefix was
        used to avoid the clashes of names)
      
          crypto_ERR_error_string
          crypto_ERR_get_error
          crypto_EVP_DigestInit_ex
          crypto_EVP_DigestUpdate
          crypto_EVP_DigestFinal_ex
          crypto_EVP_get_digestbyname
          crypto_HMAC_Init_ex
          crypto_HMAC_Update
          crypto_HMAC_Final
      
      * For correct work of "schema.lua" in the "static_build_cmake_linux"
      
          rl_get_screen_size
      
      * The following symbols are used in "ssl-cert-paths-discover.test.lua"
        (I think these symbols will have to be wrapped in the to avoid clashes
        problems)
      
          X509_get_default_cert_dir_env
          X509_get_default_cert_file_env
          ssl_cert_paths_discover
      
      From "exports.test.lua" have been removed ZSTD symbols checking (see [6]) and
      "tt_uuid_str" (see [7]).
      
      1. https://github.com/tarantool/tarantool/issues/2971
      2. https://github.com/tarantool/tarantool/issues/5001
      3. https://github.com/tarantool/memcached/issues/59
      4. https://lists.tarantool.org/pipermail/tarantool-discussions/2020-September/000095.html
      5. https://github.com/tarantool/tarantool/commit/03790ac5510648d1d9648bb2281857a7992d0593
      6. https://github.com/tarantool/tarantool/issues/4225
      7. https://github.com/tarantool/tarantool/commit/acf8745ed8fef47e6d1f1c31708c7c9d6324d2f3
      
      Part of #5932
      5ceabb37
  20. May 20, 2021
  21. Mar 24, 2021
    • Vladislav Shpilevoy's avatar
      cord_buf: introduce cord_buf API · ade45685
      Vladislav Shpilevoy authored
      There was a global ibuf object called tarantool_lua_ibuf. It was
      used in all the places working with Lua which didn't have yields,
      and where fiber's region could be potentially slower due to not
      being able to guarantee the allocated memory is contiguous.
      
      Yields during the ibuf usage were prohibited because another fiber
      would take the same ibuf and override its previous content which
      was still used by another fiber.
      
      But it wasn't taken into account that there is Lua GC. It can be
      invoked from any Lua function in Lua C code, and almost on any
      line in the Lua scripts. During GC some deleted objects might have
      GC handlers installed as __gc metamethods. From the handler they
      could call Tarantool functions, including the ones using the
      global ibuf.
      
      Therefore ibuf could be overridden not only at yields, but almost
      in any moment. Because with the Lua GC at hand, the multitasking
      is not strictly "cooperative" anymore.
      
      It is necessary to implement ownership for the global buffer. The
      patch prepares the API for this: the buffer is moved to its own
      file, and has methods take(), put(), and drop().
      
      Take() is supposed to make the current fiber own the buffer. Put()
      makes it available again. Drop() does the same but also clears the
      buffer (frees its memory). The ownership itself is a subject for
      the next patches. Here only the API is prepared.
      
      The patch "hits" performance a little. Previously the get of
      buffer.IBUF_SHARED cost around 1 ns. Now cord_ibuf_take() +
      cord_ibuf_put() cost around 5 ns together. The next patches will
      make it worse, up to 15 ns until #5871 is done.
      
      Part of #5632
      ade45685
  22. Mar 15, 2021
  23. Jan 29, 2021
  24. Jan 25, 2021
    • Alexander V. Tikhonov's avatar
      build: bump zstd submodule · c7094cd7
      Alexander V. Tikhonov authored
      Updated third_party/zstd submodule from v1.3.3 to v1.4.8 version.
      
      Found issue building on Fedora 33:
      
        third_party/zstd/lib/decompress/zstd_decompress.c: In function ‘ZSTD_findFrameCompressedSize’:
        third_party/zstd/lib/decompress/zstd_decompress.c:1502:18: error: ‘zfh.headerSize’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
         1502 |         ip += zfh.headerSize;
              |                  ^
      
      Also found that later releases of third_party/zstd submodule already
      fixed it. Decided to bump third_party/zstd submodule from v1.3.3 to
      v1.4.8. Added to zstd cmake build rules new files appeared on bumping.
      
      Found that some checking in static-build test exporting symbols like:
      
        ZSTD_free
        ZSTD_malloc
      
      never were public symbols and should not be tested for presence in the
      Tarantool executable, but also these symbols currently outdated and
      broke the testing. To avoid of it these symbols removed from test.
      
      Needed for #5502
      Closes #5697
      c7094cd7
  25. Sep 15, 2020
    • HustonMmmavr's avatar
      build: refactor static build process · 800e5ed6
      HustonMmmavr authored
      
      Refactored static build process to use static-build/CMakeLists.txt
      instead of Dockerfile.staticbuild (this allows to support static
      build on macOS). Following third-party dependencies for static build
      are installed via cmake `ExternalProject_Add`:
        - OpenSSL
        - Zlib
        - Ncurses
        - Readline
        - Unwind
        - ICU
      
      * Added support static build for macOS
      * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static
        build at file cmake/BuildLibCURL.cmake:
          - disable building shared libcurl libraries (by setting
            `--disable-shared` option)
          - disable hiding libcurl symbols (by setting
            `--disable-symbol-hiding` option)
          - prevent linking libcurl with system libz (by setting
            `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
      * Removed Dockerfile.staticbuild
      * Added new gitlab.ci jobs to test new style static build:
        - static_build_cmake_linux
        - static_build_cmake_osx_15
      * Removed static_docker_build gitlab.ci job
      
      Closes #5095
      
      Co-authored-by: default avatarYaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
      800e5ed6
Loading