Skip to content
Snippets Groups Projects
  1. Jul 12, 2022
    • Alexander Turenko's avatar
      http client: return back CA certificates search · 9d620bd0
      Alexander Turenko authored
      Without CA certificates the HTTP client will unable to verify server's
      certificate, so the only way to perform an HTTPS request would be use
      the `verify_peer = false` option -- disable certificate validation at
      all.
      
      The runtime search of system CA bundle/certificates was unintentionally
      disabled in 2.10.0 (PR #7119). The patch enabled is back.
      
      The main motivation behind the runtime search is difference in paths on
      different systems. Since we ship Tarantool Enterprise Edition as
      executable with ability to run on different Linux distributions, we
      can't choose one particular path at build time. See details in #5746.
      
      The `CURL_CA_BUNDLE_SET` and `CURL_CA_PATH_SET` options were removed,
      because they are not 'real' curl configuration options, but rather
      cached values to don't repeat file/directory search at re-configuration.
      It looks as internal logic of Curl's CMake script.
      
      NO_DOC=Lack of proper HTTPS support is definitely broken behavior, there
             is no sense to document it or the opposite.
      NO_TEST=A simple test would require to send a request to external host.
              It would not work without internet connection or in a sandbox.
              Such test also would be unstable and would fail from time to
              time due to network conditions. I verified the patch manually. I
              have an idea to add more thorough http client testing later.
      
      Fixes #7372
      9d620bd0
  2. Jul 08, 2022
  3. Jun 30, 2022
    • Boris Stepanenko's avatar
      gcov: use __gcov_dump + __gcov_reset instead of __gcov_flush · bd813168
      Boris Stepanenko authored
      __gcov_flush was removed in gcc11.
      
      Since gcc11 __gcov_dump calls __gcov_lock at the start and
      __gcov_unlock before returning. Same is true for __gcov_reset.
      Because of that using __gcov_reset right after __gcov_dump since gcc11
      is the same as using __gcov_flush before gcc11.
      
      Closes #7302
      
      NO_CHANGELOG=internal
      NO_DOC=internal
      NO_TEST=internal
      bd813168
  4. Jun 23, 2022
    • Georgiy Lebedev's avatar
      build: pass necessary compiler flags to libunwind project submodule · bd026399
      Georgiy Lebedev authored
      For the sake of maximizing backtrace collection performance, build
      the libunwind project submodule with "-O2" compiler flag.
      
      Also, build it with the "-g" compiler flag just in case to simplify
      debugging.
      
      Last but not least, pass the same archive-maintaining program used by the
      main CMake project to the libunwind project submodule to make the build
      homogeneous.
      
      Needed for #7207
      
      NO_CHANGELOG=build
      NO_DOC=build
      NO_TEST=build
      bd026399
  5. Jun 20, 2022
  6. Jun 02, 2022
  7. May 19, 2022
  8. Apr 19, 2022
    • Georgiy Lebedev's avatar
      core: re-enable backtrace feature on AARCH64 · ef486a09
      Georgiy Lebedev authored
      After fixing libunwind issues in 7dc9fe44, libcoro fibers issues in
      761053f0 and c8ad49f0, we can now re-enable backtrace feature on
      AARCH64.
      
      For some not investigated reason `unw_backtrace` does not collect any
      ips when "-mbranch-protection" compile feature is enabled: provide a
      workaround for this case.
      
      Closes #6060
      
      NO_DOC=build system
      NO_TEST=build system
      ef486a09
  9. Apr 04, 2022
    • Timur Safin's avatar
      datetime: huge dates support in parse functions · 5511dda7
      Timur Safin authored
      * Default parse
        - new c-dt version used which handles extended years range
          while parse relaxed iso8601 gformat strings;
        - family of functions like dt_from_ymd_checked functions
          added to the new c-dt version, now used by conversion code
          to properly handle validation of a 32-bit boundary values;
        - datetime_parse_full() modified to properly handle huge years values;
        - added tests for extended years range.
      
      * strptime-like parse
        - properly handle longer than 4 years values, negative values,
          and handle zulu suffix, which may be generated by Tarantool
          stringization routines;
      
      Part of #6731
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      5511dda7
  10. Mar 31, 2022
    • Vladimir Davydov's avatar
      cmake: fix lua_source for files stored in superproject · 3bacde16
      Vladimir Davydov authored
      If Tarantool is built as a subproject, and there's a Lua source file
      stored in the superproject directory, its path won't start with
      PROJECT_SOURCE_DIR and hence won't be replaced with PROJECT_BINARY_DIR
      by the lua_source() function. As a result, the lua.c file generated by
      lua_source() will be put in the source directory instead of the binary
      directory. Fix this by using CMAKE dirs instead of PROJECT dirs.
      
      Ironically, CMAKE dirs were replaced with PROJECT dirs everywhere in
      the first place to allow building Tarantool as a subproject, see commit
      d8097325 ("cmake: align folders dependencies"). This patch basically
      reverts that change.
      
      NO_DOC=cmake
      NO_TEST=cmake
      NO_CHANGELOG=cmake
      3bacde16
  11. Mar 29, 2022
  12. Mar 18, 2022
    • Georgiy Lebedev's avatar
      libunwind: add new submodule · 7dc9fe44
      Georgiy Lebedev authored
      Investigation of GNU libunwind problems on the aarch64-linux-gnu
      platform drive us to the conclusion that libunwind-1.2.1 provided by
      major distribution packages is broken. Not to mention that its test
      suite fails with SEGFAULTs.
      
      Last but not least, some distributions, e.g. CentOS 8 (see #4611) do
      not provide a libunwind package.
      
      Hence, bundle libunwind: bundling is enabled by default on all
      platforms, except for macOS — a system package can be used if its
      version is greater or equal than 1.3.0 (minimal version that does not
      seem to be broken on aarch64-linux-gnu).
      
      * Add new submodule: bump it to current master.
      * Refactor libunwind package search logic out of compiler.cmake.
      * Add CMake script for building bundled libunwind.
      * Add CMake script for extracting version of libunwind.
      * Re-enable backtrace for all RHEL distributions by default.
      * Remove libunwind from static build.
      
      Needed for #4002
      Closes #4611
      
      NO_DOC=build system
      NO_TEST=build system
      7dc9fe44
  13. Dec 21, 2021
    • AnaNek's avatar
      build: link bundled libcurl with nghttp2 · fa8d70ca
      AnaNek authored
      Before this patch Tarantool http client did not support HTTP/2.
      The reasons to enable HTTP/2 support are efficiency offered by
      the protocol and potential ability to interact with a GRPC server.
      The CMake version requirement is updated from 3.2 to 3.3, because
      we need generator expressions in cmake for setting multiple paths
      in CMAKE_FIND_ROOT_PATH for nghttp2 support.
      
      Closes #5771
      
      @TarantoolBot document
      Title: Now we require CMake 3.3 to build tarantool
      
      In tarantool/doc#2065 we requested to update
      the CMake minimum version to 3.2. Now it is time for 3.3.
      See details in the linked commit.
      fa8d70ca
  14. Dec 17, 2021
  15. Dec 10, 2021
    • Vladimir Davydov's avatar
      cmake: pass symbol name explicitly to lua_source and bin_source · f5a7bb42
      Vladimir Davydov authored
      Currently, the symbol name is deducted from the filename, but it may
      lead to name clashes if different packages have modules with same names.
      Let's pass the symbol name explicitly to avoid that.
      
      This is needed to embed luarocks in static build.
      f5a7bb42
    • Vladimir Davydov's avatar
      cmake: factor out FindZLIB · 4bd44330
      Vladimir Davydov authored
      Zlib is used in OpenSSL and in libCURL. Besides, we will need it for
      lua-zlib. So better define FindZLIB.
      
      Note, `add_library(ZLIB::ZLIB UNKNOWN IMPORTED)` and setting the
      corresponding properties in FindZLIB.cmake is needed to build libcurl,
      which depends on zlib.
      4bd44330
  16. Dec 09, 2021
  17. Dec 08, 2021
  18. Oct 14, 2021
    • Timur Safin's avatar
      build, lua: built-in module datetime · 43e10ed3
      Timur Safin authored
      Introduce a new builtin Tarantool module `datetime.lua` for timestamp
      and interval types support.
      
      New third_party module - c-dt
      -----------------------------
      
      * Integrated chansen/c-dt parser as 3rd party module to the
        Tarantool cmake build process;
      * We use tarantool/c-dt instead of original chansen/c-dt to
        have an easier cmake build integration, as we have added some
        changes, which provide cmake support, and allow to rename symbols
        if necessary (this symbol renaming is similar to that we see
        with xxhash or icu).
      
      New built-in module `datetime`
      ------------------------------
      
      * created a new Tarantool built-in module `datetime`, which uses
        `struct datetime` data structure for keeping timestamp values;
      * Lua module uses a number of `dt_*` functions from `c-dt` library,
        but they were renamed to `tnt_dt_*` at the moment of exporting
        from executable - to avoid possible name clashes with external
        libraries.
      
      * At the moment we libc `strftime` for formatting of datetime
        values according to flags passed, i.e. `date:format('%FT%T%z')`
        will return something like '1970-01-01T00:00:00+0000', but
        `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970'
      
      * if there is no format provided then we use default
        `tnt_datetime_to_string()` function, which converts datetime
        to their default ISO-8601 output format, i.e.
        `tostring(date)` will return string like "1970-01-01T00:00:00Z"
      
      * There are a number of simplified interfaces
        - totable() for exporting table with attributes names as provided
          by `os.date('*t')`
        - set() method provides unified interface to set values using
          the set of attributes as defined above in totable()
      
      Example,
      
      ```
      local dt = datetime.new {
      	nsec      = 123456789,
      
      	sec       = 19,
      	min       = 29,
      	hour      = 18,
      
      	day       = 20,
      	month     = 8,
      	year      = 2021,
      
      	tzoffset  = 180
      }
      
      local t = dt:totable()
      --[[
      {
      	sec = 19,
      	min = 29,
      	wday = 6,
      	day = 20,
      	nsec = 123456789,
      	isdst = false,
      	yday = 232,
      	tzoffset = 180,
      	month = 8,
      	year = 2021,
      	hour = 18
      }
      --]]
      
      dt:format()   -- 2021-08-21T14:53:34.032Z
      dt:format('%Y-%m-%dT%H:%M:%S')   -- 2021-08-21T14:53:34
      
      dt:set {
      	usec      = 123456,
      
      	sec       = 19,
      	min       = 29,
      	hour      = 18,
      
      	day       = 20,
      	month     = 8,
      	year      = 2021,
      
      	tzoffset  = 180,
      
      }
      dt:set {
      	timestamp = 1629476485.124,
      
      	tzoffset  = 180,
      }
      
      ```
      
      Coverage is
      
      File                 Hits Missed Coverage
      -----------------------------------------
      builtin/datetime.lua 299  23     92.86%
      -----------------------------------------
      Total                299  23     92.86%
      
      Part of #5941
      
      @TarantoolBot document
      Title: Introduced a new `datetime` module for timestamp and interval support
      
      Create `datetime` module for timestamp and interval types support.
      It allows to create date and timestamp values using either object interface,
      or via parsing of string values conforming to iso-8601 standard.
      One may manipulate (modify, subtract or add) timestamp and interval values.
      
      Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool
      for a more detailed description of module API.
      43e10ed3
  19. Sep 28, 2021
    • VitaliyaIoffe's avatar
      github-ci: use openssl@1.1 · 33830978
      VitaliyaIoffe authored
      OSX workflows use brew for install openssl.
      There was a new release of openssl@3.0 and
      homebrew updated the openssl formula.
      
      Close #6468
      33830978
  20. Sep 01, 2021
    • Kirill Yukhin's avatar
      build: build libcurl with PIC enabled · 01964d4c
      Kirill Yukhin authored
      To be able to configure libcurl for ARM64 CPU
      on Fedora distros with linker hardening it is
      necessary to enable `-fPIC`. It fails with wrong
      relocation type for glibc symbols (e.g. socket)
      otherwise.
      
      Closes #6366
      01964d4c
  21. Aug 18, 2021
    • Egor Elchinov's avatar
      build: disable backtraces on arm · 94925b53
      Egor Elchinov authored
      MacOS has its own libunwind which works not good with fibers
      under M1. Backtraces need to be enabled again after #6060 is fixed.
      Other arm backtraces are temporarily disabled too for #6222.
      
      Closes #6272
      94925b53
  22. Jun 22, 2021
    • Oleg Babin's avatar
      build: cleanup BuildZSTD.cmake · 2e854223
      Oleg Babin authored
      Seems CMakeLists.txt contains the same line as inside
      BuildZSTD.cmake. Seems we don't need to duplicate this logic
      twice let's remove it.
      2e854223
  23. Jun 18, 2021
    • Oleg Babin's avatar
      build: add libxxhash to third_party · ce382f96
      Oleg Babin authored
      This patch is the first step for fixing regression introduced in
      f998ea39 (digest: introduce FFI bindings for xxHash32/64).
      We used xxhash library that is shipped with zstd. However it's
      possible that user doesn't use bundled zstd. In such cases we
      couldn't export xxhash symbols and build failed with following
      error:
      
      ```
      [ 59%] Linking CXX executable tarantool
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd80): undefined reference to `XXH32'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd88): undefined reference to `XXH32_copyState'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd90): undefined reference to `XXH32_digest'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd98): undefined reference to `XXH32_reset'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xda0): undefined reference to `XXH32_update'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xda8): undefined reference to `XXH64'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdb0): undefined reference to `XXH64_copyState'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdb8): undefined reference to `XXH64_digest'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdc0): undefined reference to `XXH64_reset'
      /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdc8): undefined reference to `XXH64_update'
      collect2: error: ld returned 1 exit status
      ```
      
      To avoid a problem this patch introduces standalone xxhash library
      that will be bundled anyway. It's worth to mention that our
      approach is still related to zstd. We use Cyan4973/xxHash that is
      used in zstd and passes the same compile flags to it. Single
      difference is usage of XXH_NAMESPACE to avoid symbols clashing
      with zstd.
      
      Need for #6135
      ce382f96
  24. Jun 12, 2021
  25. Apr 30, 2021
  26. Apr 14, 2021
  27. Apr 13, 2021
    • Alexander Turenko's avatar
      build: fix configuring using cmake3 command · 820d2be6
      Alexander Turenko authored
      `cmake` command was hardcoded for configuring libcurl, however only
      `cmake3` may be installed in a system. Now we use the same cmake command
      for configuring libcurl as one that is used for configuring tarantool
      itself.
      
      The problem exists since 2.6.0-196-g2b0760192 ('build: enable cmake in
      curl build').
      
      Fixes #5955
      820d2be6
  28. Mar 17, 2021
    • Sergey Kaplun's avatar
      luajit: bump new version · a0ade206
      Sergey Kaplun authored
      LuaJIT submodule is bumped to introduce the following changes:
      * test: disable LuaJIT CLI tests in lua-Harness suite
      * test: set USERNAME env var for lua-Harness suite
      * test: adjust lua-Harness tests that use dofile
      * test: adjust lua-Harness suite to CMake machinery
      * test: add lua-Harness test suite
      
      Within this changeset lua-Harness suite[1] is added to Tarantool
      testing. Considering Tarantool specific changes in runtime the suite
      itself is adjusted in LuaJIT submodule.
      
      However, Tarantool provides and unconditionally loads TAP module
      conflicting with the one used in the new suite. Hence, the Tarantool
      built-in module is "unloaded" in test/luajit-test-init.lua.
      
      Furthermore, Tarantool provides UTF-8 support via another built-in
      module. Its interfaces differ from the ones implemented in Lua5.3 and
      moonjit. At the same time our LuaJIT fork provides no UTF-8 support, so
      lua-Harness UTF-8 detector is simply confused with non-nil utf8 global
      variable. As a result, utf8 is set to nil in test/luajit-test-init.lua.
      
      There are also some tests launching Lua interpreter, so strict need to
      be disabled for their child tests too. Hence `strict.off()` is added to
      `progname` (i.e. arg[-1] considering the way Tarantool parses its CLI
      arguments) command used in these tests.
      
      [1]: https://framagit.org/fperrad/lua-Harness/tree/a74be27/test_lua
      
      
      
      Closes #5844
      Part of #4473
      
      Reviewed-by: default avatarSergey Ostanevich <sergos@tarantool.org>
      Reviewed-by: default avatarIgor Munkin <imun@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      Unverified
      a0ade206
  29. Mar 10, 2021
    • Igor Munkin's avatar
      luajit: bump new version · 1458a2b2
      Igor Munkin authored
      LuaJIT submodule is bumped to introduce the following changes:
      * test: adjust LuaJIT test suite for Tarantool
      * test: change LuaJIT test suite to match b4e6bf0
      * test: change LuaJIT test suite to match 5a61e1a
      * test: change LuaJIT test suite to match c198167
      * test: change LuaJIT test suite to match de5568e
      * test: add LuaJIT-test-cleanup test suite
      * test: fix Lua command in utils.selfrun
      * test: fix luacheck invocation for non-real paths
      
      Within this changeset LuaJIT-test-cleanup suite[1] is added to Tarantool
      testing. Considering Tarantool specific changes in runtime the suite
      itself is adjusted in LuaJIT submodule. However, there is <strict>
      module enabled by default in Debug build, so the testing environment
      need to be tweaked via test/luajit-test-init.lua script to be run prior
      to every LuaJIT suite test is started.
      
      [1]: https://github.com/LuaJIT/LuaJIT-test-cleanup/tree/014708b/test
      
      
      
      Closes #4064
      Part of #4473
      
      Reviewed-by: default avatarSergey Kaplun <skaplun@tarantool.org>
      Reviewed-by: default avatarSergey Ostanevich <sergos@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      Unverified
      1458a2b2
  30. Mar 05, 2021
  31. Feb 28, 2021
    • Igor Munkin's avatar
      build: adjust LuaJIT build system · 07c83aab
      Igor Munkin authored
      LuaJIT submodule is bumped to introduce the following changes:
      * test: run luacheck static analysis via CMake
      * test: fix warnings found with luacheck in misclib*
      * test: run LuaJIT tests via CMake
      * build: replace GNU Make with CMake
      * build: preserve the original build system
      
      Since LuaJIT build system is ported to CMake in scope of the changeset
      mentioned above, the module building the LuaJIT bundled in Tarantool is
      completely reworked. There is no option to build Tarantool against
      another prebuilt LuaJIT due to a91962c0
      ('Until Bug#962848 is fixed, don't try to compile with external
      LuaJIT'), so all redundant options defining the libluajit to be used in
      Tarantool are dropped with the related auxiliary files.
      
      To run LuaJIT related tests or static analysis for Lua files within
      LuaJIT repository, <LuaJIT-test> and <LuaJIT-luacheck> targets are used
      respectively as a dependency of the corresponding Tarantool targets.
      
      As an additional dependency to run LuaJIT tests, prove[1] utility is
      required, so the necessary binary packages are added to the lists with
      build requirements.
      
      [1]: https://metacpan.org/pod/TAP::Harness#prove
      
      
      
      Closes #4862
      Closes #5470
      Closes #5631
      
      Reviewed-by: default avatarSergey Kaplun <skaplun@tarantool.org>
      Reviewed-by: default avatarTimur Safin <tsafin@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      Unverified
      07c83aab
    • Igor Munkin's avatar
      build: fix lua.c file generation · 8484cabc
      Igor Munkin authored
      
      If Lua source path given to <lua_source> function is relative, the
      output file is generated in the binary directory. At the same time if
      the given path to be compiled to *.lua.c is absolute, the output
      file is generated in source directory instead of the binary one. This
      patch fixes the latter case, providing the valid behaviour for out of
      source build type.
      
      Needed for #4862
      
      Reviewed-by: default avatarSergey Kaplun <skaplun@tarantool.org>
      Reviewed-by: default avatarTimur Safin <tsafin@tarantool.org>
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      Unverified
      8484cabc
  32. Jan 25, 2021
    • Alexander V. Tikhonov's avatar
      build: bump zstd submodule · c7094cd7
      Alexander V. Tikhonov authored
      Updated third_party/zstd submodule from v1.3.3 to v1.4.8 version.
      
      Found issue building on Fedora 33:
      
        third_party/zstd/lib/decompress/zstd_decompress.c: In function ‘ZSTD_findFrameCompressedSize’:
        third_party/zstd/lib/decompress/zstd_decompress.c:1502:18: error: ‘zfh.headerSize’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
         1502 |         ip += zfh.headerSize;
              |                  ^
      
      Also found that later releases of third_party/zstd submodule already
      fixed it. Decided to bump third_party/zstd submodule from v1.3.3 to
      v1.4.8. Added to zstd cmake build rules new files appeared on bumping.
      
      Found that some checking in static-build test exporting symbols like:
      
        ZSTD_free
        ZSTD_malloc
      
      never were public symbols and should not be tested for presence in the
      Tarantool executable, but also these symbols currently outdated and
      broke the testing. To avoid of it these symbols removed from test.
      
      Needed for #5502
      Closes #5697
      c7094cd7
  33. Dec 25, 2020
    • Sergey Bronnikov's avatar
      test: integrate with OSS Fuzz · 7680948f
      Sergey Bronnikov authored
      To run Tarantool fuzzers on OSS Fuzz infrastructure it is needed to pass
      library $LIB_FUZZING_ENGINE to linker and use external CFLAGS and
      CXXFLAGS. Full description how to integrate with OSS Fuzz is in [1] and
      [2].
      
      Patch to OSS Fuzz repository [2] is ready to merge.
      
      We need to pass options with "-fsanitize=fuzzer" two times
      (in cmake/profile.cmake and test/fuzz/CMakeLists.txt) because:
      
      - cmake/profile.cmake is for project source files,
        -fsanitize=fuzzer-no-link option allows to instrument project source
        files for fuzzing, but LibFuzzer will not replace main() in these
        files.
      
      - test/fuzz/CMakeLists.txt uses -fsanitize=fuzzer and not
        -fsanitize=fuzzer-no-link because we want to add automatically
        generated main() for each fuzzer.
      
      1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/
      2. https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/
      3. https://github.com/google/oss-fuzz/pull/4723
      
      Closes #1809
      7680948f
    • Sergey Bronnikov's avatar
      test: add fuzzers and support for fuzzing testing · 2ad7caca
      Sergey Bronnikov authored
      There is a number of bugs related to parsing and encoding/decoding data.
      Examples:
      
      - csv: #2692, #4497, #2692
      - uri: #585
      
      One of the effective method to find such issues is a fuzzing testing.
      Patch introduces a CMake flag to enable building fuzzers (ENABLE_FUZZER)
      and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri
      modules. Note that fuzzers must return 0 exit code only, other exit
      codes are not supported [2].
      
      NOTE: LibFuzzer requires Clang compiler.
      
      1. https://llvm.org/docs/LibFuzzer.html
      2. http://llvm.org/docs/LibFuzzer.html#id22
      
      How-To Use:
      
      $ mkdir build && cd build
      $ cmake -DENABLE_FUZZER=ON \
      	-DENABLE_ASAN=ON \
      	-DCMAKE_BUILD_TYPE=Debug \
      	-DCMAKE_C_COMPILER="/usr/bin/clang" \
      	-DCMAKE_CXX_COMPILER="/usr/bin/clang++" ..
      $ make -j
      $ ./test/fuzz/csv_fuzzer -workers=4 ../test/static/corpus/csv
      
      Part of #1809
      2ad7caca
  34. Oct 22, 2020
    • Sergey Kaplun's avatar
      build: provide missing LuaJIT lmisclib.h header · 08bde945
      Sergey Kaplun authored
      Since LuaJIT provides extended LuaC API introduced in the scope of
      5a61e1ab54b5c66bfebd836db1ac47996611e065 ('misc: add C and Lua API for
      platform metrics') corresponding header should be provided along with
      other Tarantool development files.
      
      Follows up #5187
      08bde945
  35. Oct 16, 2020
    • Alexander V. Tikhonov's avatar
      build: enable cmake in curl build · 2b076019
      Alexander V. Tikhonov authored
      Initially tried to change autoconf tools in Curl build to cmake and
      found the following build issue on:
      
        CentOS 6
        CentOS 7
        Debian 8
        Ubuntu 14.04
      
      Issue found:
      
        CMake Error at CMakeLists.txt:41 (cmake_minimum_required):
          CMake 3.0 or higher is required.  You are running version 2.8.12.2
      
      To fix the issue check is removed of the version from curl sources
      in Tarantool's third party '<root Tarantool sources>/third_party/curl':
      
        12af024bc85606b14ffc415413a7e86e6bbee7eb ('Enable curl build with old cmake')
      
      After this fix completely changed autoconf to cmake in curl build.
      Autoconf part was completely removed and code cleaned up for cmake.
      For curl cmake build all autoconf options were ported to cmake
      configuration call, check the accurate list of the change in [1].
      
      Also the following issues resolved:
      
      1. Found that CURL cmake configuration file:
      
           third_party/curl/lib/CMakeLists.txt
      
         has installation part for built libcurl.a library:
      
           install(TARGETS ${LIB_NAME}
             EXPORT ${TARGETS_EXPORT_NAME}
             ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
             LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
      
         where it changes CMAKE_INSTALL_LIBDIR to appropriate name with
         suffix of the architecture, like:
      
           lib
           lib64
           x86_64
      
         Found that find_library routine from the file:
      
           cmake/FindLibCURL.cmake
      
         returns only 'lib' value and it breaks the building of the depends
         binaries. To avoid of it the CMAKE_INSTALL_LIBDIR option was set to
         cmake call:
      
           -DCMAKE_INSTALL_LIBDIR=lib
      
      2. Found issue with building on CentOS 6:
      
           Linking C executable curl
           build/ares/dest/lib/libcares.a(ares__timeval.c.o): In function `ares__tvnow':
           ares__timeval.c:(.text+0x15): undefined reference to `clock_gettime'
           collect2: error: ld returned 1 exit status
      
         It was fixed with added "-lrt" flag to CMAKE_C_FLAGS and
         CMAKE_CXX_FLAGS build flags, when cmake version is lower
         than 3.0 and RT library had needed function.
      
      3. Found issues with building Tarantool statically on Debian 9 and
         its package build on CentOS 8.
      
         Building statically got the issues with openssl linking, like [2]:
      
           static-build/openssl-prefix/lib/libcrypto.a(threads_pthread.o): In function `CRYPTO_THREAD_lock_new':
           threads_pthread.c:(.text+0x45): undefined reference to `pthread_rwlock_init'
      
         It happened because in openssl radically changed how threads-related
         things were handled before 1.1.0 version. It required the application
         to provide us with the locking functionality in form of callbacks.
         Since 1.1.0, these matters are entirely internal, so libcrypto
         requires the presence of the platform specific thread implementation
         of our choosing, which is pthread on everything.
      
         After '-pthread' added to C compile flags package build on CentOS 8
         failed with the issue [3]:
      
           /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:101:4: error: #error "SIZEOF_CURL_OFF_T not defined"
            #  error "SIZEOF_CURL_OFF_T not defined"
               ^~~~~
           /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c: In function 'curlx_uztoso':
           /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:192:40: error: 'CURL_MASK_SCOFFT' undeclared (first use in this function); did you mean 'CURL_MASK_SINT'?
            return (curl_off_t)(uznum & (size_t) CURL_MASK_SCOFFT);
                                                 ^~~~~~~~~~~~~~~~
                                                 CURL_MASK_SINT
      
         To avoid of the issue decided to use '-pthread' flag only for openssl
         when it had not this flag in openssl compilation.
      
      4. Found issue with static build using CentOS 7, where SSL cmake rule
         failed. Building the image got the issue:
      
            [  1%] Performing configure step for 'bundled-libcurl-project'
            CMake Warning at CMakeLists.txt:50 (message):
              the curl cmake build system is poorly maintained.  Be aware
      
            -- curl version=[7.66.0-DEV]
            -- Found c-ares: /tarantool/build/ares/dest/lib/libcares.a
            Found *nroff option: -- -man
            CMake Error at /usr/share/cmake/Modules/FindOpenSSL.cmake:278 (list):
              list GET given empty list
            Call Stack (most recent call first):
              CMakeLists.txt:347 (find_package)
      
         Root cause of the issue that Dockerfile uses globaly
         installed openSSL with:
      
           cmake ... -DOPENSSL_ROOT_DIR=/usr/local ...
      
         Its cmake build file:
      
           /usr/share/cmake/Modules/FindOpenSSL.cmake
      
         fails on parsing the SSL version:
      
         it has:
                REGEX "^#define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*")
      
         but it should to use:
                REGEX "^#[\t ]*define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*")
      
         Anyway we want to use the same OpenSSL library for libcurl, as is used
         for Tarantool itself. So the path to it set for cURL build:
      
           list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_MODULE_PATH=${PROJECT_SOURCE_DIR}/cmake")
      
      5. In CMake build CMAKE_USE_LIBSSH2 flag is enabled by default, while
         in autoconf --with-libssh2 was disabled by default. We need to switch
         CMAKE_USE_LIBSSH2 flag off with:
      
           list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF")
      
         to avoid of linking issues, like:
      
           ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*...
      
         this issue exists in curl issues [4].
      
         Furthermore the following defaults are also disabled to keep the
         configuration consistent with autoconf one:
      
           list(APPEND LIBCURL_CMAKE_FLAGS "-DPICKY_COMPILER=OFF")
           list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_CURL_EXE=OFF")
      
      Closes #4968
      Closes #5019
      Closes #5396
      
      [1] - https://github.com/tarantool/tarantool/issues/4968#issue-617183031
      [2] - https://gitlab.com/tarantool/tarantool/-/jobs/779176133#L6021
      [3] - https://gitlab.com/tarantool/tarantool/-/jobs/778309145#L3060
      [4] - https://github.com/curl/curl/issues/1146
      2b076019
    • Alexander V. Tikhonov's avatar
      build: generate bootstrap.h in CMAKE_BINARY_DIR · cca763ad
      Alexander V. Tikhonov authored
      Prior to these changes bootstrap.h was generated right in the source
      directory even for out of source build. Firstly such approach doesn't
      respect the idea of building outside the source files. Furthermore
      this leads to build failures when the source directory is located on
      read-only file system.
      
      As a result of the patch bootstrap.h is generated within the build
      tree and include directories are adjusted the corresponding way.
      
      Also changed destination build directory from source to binary path.
      
      Part of #4968
      cca763ad
  36. Sep 15, 2020
    • HustonMmmavr's avatar
      build: refactor static build process · 800e5ed6
      HustonMmmavr authored
      
      Refactored static build process to use static-build/CMakeLists.txt
      instead of Dockerfile.staticbuild (this allows to support static
      build on macOS). Following third-party dependencies for static build
      are installed via cmake `ExternalProject_Add`:
        - OpenSSL
        - Zlib
        - Ncurses
        - Readline
        - Unwind
        - ICU
      
      * Added support static build for macOS
      * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static
        build at file cmake/BuildLibCURL.cmake:
          - disable building shared libcurl libraries (by setting
            `--disable-shared` option)
          - disable hiding libcurl symbols (by setting
            `--disable-symbol-hiding` option)
          - prevent linking libcurl with system libz (by setting
            `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option)
      * Removed Dockerfile.staticbuild
      * Added new gitlab.ci jobs to test new style static build:
        - static_build_cmake_linux
        - static_build_cmake_osx_15
      * Removed static_docker_build gitlab.ci job
      
      Closes #5095
      
      Co-authored-by: default avatarYaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
      800e5ed6
Loading