- Oct 14, 2021
-
-
Timur Safin authored
Introduce a new builtin Tarantool module `datetime.lua` for timestamp and interval types support. New third_party module - c-dt ----------------------------- * Integrated chansen/c-dt parser as 3rd party module to the Tarantool cmake build process; * We use tarantool/c-dt instead of original chansen/c-dt to have an easier cmake build integration, as we have added some changes, which provide cmake support, and allow to rename symbols if necessary (this symbol renaming is similar to that we see with xxhash or icu). New built-in module `datetime` ------------------------------ * created a new Tarantool built-in module `datetime`, which uses `struct datetime` data structure for keeping timestamp values; * Lua module uses a number of `dt_*` functions from `c-dt` library, but they were renamed to `tnt_dt_*` at the moment of exporting from executable - to avoid possible name clashes with external libraries. * At the moment we libc `strftime` for formatting of datetime values according to flags passed, i.e. `date:format('%FT%T%z')` will return something like '1970-01-01T00:00:00+0000', but `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970' * if there is no format provided then we use default `tnt_datetime_to_string()` function, which converts datetime to their default ISO-8601 output format, i.e. `tostring(date)` will return string like "1970-01-01T00:00:00Z" * There are a number of simplified interfaces - totable() for exporting table with attributes names as provided by `os.date('*t')` - set() method provides unified interface to set values using the set of attributes as defined above in totable() Example, ``` local dt = datetime.new { nsec = 123456789, sec = 19, min = 29, hour = 18, day = 20, month = 8, year = 2021, tzoffset = 180 } local t = dt:totable() --[[ { sec = 19, min = 29, wday = 6, day = 20, nsec = 123456789, isdst = false, yday = 232, tzoffset = 180, month = 8, year = 2021, hour = 18 } --]] dt:format() -- 2021-08-21T14:53:34.032Z dt:format('%Y-%m-%dT%H:%M:%S') -- 2021-08-21T14:53:34 dt:set { usec = 123456, sec = 19, min = 29, hour = 18, day = 20, month = 8, year = 2021, tzoffset = 180, } dt:set { timestamp = 1629476485.124, tzoffset = 180, } ``` Coverage is File Hits Missed Coverage ----------------------------------------- builtin/datetime.lua 299 23 92.86% ----------------------------------------- Total 299 23 92.86% Part of #5941 @TarantoolBot document Title: Introduced a new `datetime` module for timestamp and interval support Create `datetime` module for timestamp and interval types support. It allows to create date and timestamp values using either object interface, or via parsing of string values conforming to iso-8601 standard. One may manipulate (modify, subtract or add) timestamp and interval values. Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool for a more detailed description of module API.
-
- Sep 28, 2021
-
-
VitaliyaIoffe authored
OSX workflows use brew for install openssl. There was a new release of openssl@3.0 and homebrew updated the openssl formula. Close #6468
-
- Sep 01, 2021
-
-
Kirill Yukhin authored
To be able to configure libcurl for ARM64 CPU on Fedora distros with linker hardening it is necessary to enable `-fPIC`. It fails with wrong relocation type for glibc symbols (e.g. socket) otherwise. Closes #6366
-
- Aug 18, 2021
-
-
Egor Elchinov authored
MacOS has its own libunwind which works not good with fibers under M1. Backtraces need to be enabled again after #6060 is fixed. Other arm backtraces are temporarily disabled too for #6222. Closes #6272
-
- Jun 22, 2021
-
-
Oleg Babin authored
Seems CMakeLists.txt contains the same line as inside BuildZSTD.cmake. Seems we don't need to duplicate this logic twice let's remove it.
-
- Jun 18, 2021
-
-
Oleg Babin authored
This patch is the first step for fixing regression introduced in f998ea39 (digest: introduce FFI bindings for xxHash32/64). We used xxhash library that is shipped with zstd. However it's possible that user doesn't use bundled zstd. In such cases we couldn't export xxhash symbols and build failed with following error: ``` [ 59%] Linking CXX executable tarantool /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd80): undefined reference to `XXH32' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd88): undefined reference to `XXH32_copyState' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd90): undefined reference to `XXH32_digest' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd98): undefined reference to `XXH32_reset' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xda0): undefined reference to `XXH32_update' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xda8): undefined reference to `XXH64' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdb0): undefined reference to `XXH64_copyState' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdb8): undefined reference to `XXH64_digest' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdc0): undefined reference to `XXH64_reset' /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdc8): undefined reference to `XXH64_update' collect2: error: ld returned 1 exit status ``` To avoid a problem this patch introduces standalone xxhash library that will be bundled anyway. It's worth to mention that our approach is still related to zstd. We use Cyan4973/xxHash that is used in zstd and passes the same compile flags to it. Single difference is usage of XXH_NAMESPACE to avoid symbols clashing with zstd. Need for #6135
-
- Jun 12, 2021
-
-
Sergey Kaplun authored
This patch fixes inaccuracy in Tarantool build configuration introduced by commit 07c83aab ('build: adjust LuaJIT build system'). That patch sets LUAJIT_USE_ASSERT and LUAJIT_USE_APICHECK options for Debug build instead of LUA_USE_ASSERT and LUA_USE_APICHECK respectively. This patch fixes these typos. Follows up #4862 Reviewed-by:
Sergey Ostanevich <sergos@tarantool.org> Reviewed-by:
Igor Munkin <imun@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Apr 30, 2021
-
-
Igor Munkin authored
This patch fixes inaccuracy in Tarantool build configuration introduced by commit 07c83aab ('build: adjust LuaJIT build system'). All those MacOS-related tweaks for __PAGEZERO size and preferred load address for the bundle are necessary only for builds with 32-bit GC area on 64-bit host. The only case fitting these conditions is x86_64 with no LUAJIT_ENABLE_GC64. All other 64-bit builds use 64-bit GC area unconditionally. Part of #5983 Needed for #5629 Follows up #4862 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Nikita Pettik <korablev@tarantool.org> Reviewed-by:
Sergey Ostanevich <sergos@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Apr 14, 2021
-
-
Roman Khabibov authored
Enable smtp and smtps protocols in bundled libcurl. It is needed to use SMTP client with tarantool's libcurl instead of system libcurl. See related issue: https://github.com/tarantool/smtp/issues/24 Part of #4559
-
- Apr 13, 2021
-
-
Alexander Turenko authored
`cmake` command was hardcoded for configuring libcurl, however only `cmake3` may be installed in a system. Now we use the same cmake command for configuring libcurl as one that is used for configuring tarantool itself. The problem exists since 2.6.0-196-g2b0760192 ('build: enable cmake in curl build'). Fixes #5955
-
- Mar 17, 2021
-
-
Sergey Kaplun authored
LuaJIT submodule is bumped to introduce the following changes: * test: disable LuaJIT CLI tests in lua-Harness suite * test: set USERNAME env var for lua-Harness suite * test: adjust lua-Harness tests that use dofile * test: adjust lua-Harness suite to CMake machinery * test: add lua-Harness test suite Within this changeset lua-Harness suite[1] is added to Tarantool testing. Considering Tarantool specific changes in runtime the suite itself is adjusted in LuaJIT submodule. However, Tarantool provides and unconditionally loads TAP module conflicting with the one used in the new suite. Hence, the Tarantool built-in module is "unloaded" in test/luajit-test-init.lua. Furthermore, Tarantool provides UTF-8 support via another built-in module. Its interfaces differ from the ones implemented in Lua5.3 and moonjit. At the same time our LuaJIT fork provides no UTF-8 support, so lua-Harness UTF-8 detector is simply confused with non-nil utf8 global variable. As a result, utf8 is set to nil in test/luajit-test-init.lua. There are also some tests launching Lua interpreter, so strict need to be disabled for their child tests too. Hence `strict.off()` is added to `progname` (i.e. arg[-1] considering the way Tarantool parses its CLI arguments) command used in these tests. [1]: https://framagit.org/fperrad/lua-Harness/tree/a74be27/test_lua Closes #5844 Part of #4473 Reviewed-by:
Sergey Ostanevich <sergos@tarantool.org> Reviewed-by:
Igor Munkin <imun@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Mar 10, 2021
-
-
Igor Munkin authored
LuaJIT submodule is bumped to introduce the following changes: * test: adjust LuaJIT test suite for Tarantool * test: change LuaJIT test suite to match b4e6bf0 * test: change LuaJIT test suite to match 5a61e1a * test: change LuaJIT test suite to match c198167 * test: change LuaJIT test suite to match de5568e * test: add LuaJIT-test-cleanup test suite * test: fix Lua command in utils.selfrun * test: fix luacheck invocation for non-real paths Within this changeset LuaJIT-test-cleanup suite[1] is added to Tarantool testing. Considering Tarantool specific changes in runtime the suite itself is adjusted in LuaJIT submodule. However, there is <strict> module enabled by default in Debug build, so the testing environment need to be tweaked via test/luajit-test-init.lua script to be run prior to every LuaJIT suite test is started. [1]: https://github.com/LuaJIT/LuaJIT-test-cleanup/tree/014708b/test Closes #4064 Part of #4473 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Sergey Ostanevich <sergos@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Mar 05, 2021
-
-
Alexander Turenko authored
Now `make luacheck` gracefully handles different cases[^1]: in-source and out-of-source build (within the source tree or outside), current working directory as a real path or with symlink components. As result of looking into those problems I filed the issue [1] against luacheck. It seems, there are problems around absolute paths with symlinks components. [^1]: We have the similar problems with LuaJIT's luacheck rule. They will be fixed in a separate patch. [1]: https://github.com/mpeterv/luacheck/issues/208 Reviewed-by:
Sergey Bronnikov <sergeyb@tarantool.org> Reviewed-by:
Igor Munkin <imun@tarantool.org>
-
- Feb 28, 2021
-
-
Igor Munkin authored
LuaJIT submodule is bumped to introduce the following changes: * test: run luacheck static analysis via CMake * test: fix warnings found with luacheck in misclib* * test: run LuaJIT tests via CMake * build: replace GNU Make with CMake * build: preserve the original build system Since LuaJIT build system is ported to CMake in scope of the changeset mentioned above, the module building the LuaJIT bundled in Tarantool is completely reworked. There is no option to build Tarantool against another prebuilt LuaJIT due to a91962c0 ('Until Bug#962848 is fixed, don't try to compile with external LuaJIT'), so all redundant options defining the libluajit to be used in Tarantool are dropped with the related auxiliary files. To run LuaJIT related tests or static analysis for Lua files within LuaJIT repository, <LuaJIT-test> and <LuaJIT-luacheck> targets are used respectively as a dependency of the corresponding Tarantool targets. As an additional dependency to run LuaJIT tests, prove[1] utility is required, so the necessary binary packages are added to the lists with build requirements. [1]: https://metacpan.org/pod/TAP::Harness#prove Closes #4862 Closes #5470 Closes #5631 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Timur Safin <tsafin@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
Igor Munkin authored
If Lua source path given to <lua_source> function is relative, the output file is generated in the binary directory. At the same time if the given path to be compiled to *.lua.c is absolute, the output file is generated in source directory instead of the binary one. This patch fixes the latter case, providing the valid behaviour for out of source build type. Needed for #4862 Reviewed-by:
Sergey Kaplun <skaplun@tarantool.org> Reviewed-by:
Timur Safin <tsafin@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Jan 25, 2021
-
-
Alexander V. Tikhonov authored
Updated third_party/zstd submodule from v1.3.3 to v1.4.8 version. Found issue building on Fedora 33: third_party/zstd/lib/decompress/zstd_decompress.c: In function ‘ZSTD_findFrameCompressedSize’: third_party/zstd/lib/decompress/zstd_decompress.c:1502:18: error: ‘zfh.headerSize’ may be used uninitialized in this function [-Werror=maybe-uninitialized] 1502 | ip += zfh.headerSize; | ^ Also found that later releases of third_party/zstd submodule already fixed it. Decided to bump third_party/zstd submodule from v1.3.3 to v1.4.8. Added to zstd cmake build rules new files appeared on bumping. Found that some checking in static-build test exporting symbols like: ZSTD_free ZSTD_malloc never were public symbols and should not be tested for presence in the Tarantool executable, but also these symbols currently outdated and broke the testing. To avoid of it these symbols removed from test. Needed for #5502 Closes #5697
-
- Dec 25, 2020
-
-
Sergey Bronnikov authored
To run Tarantool fuzzers on OSS Fuzz infrastructure it is needed to pass library $LIB_FUZZING_ENGINE to linker and use external CFLAGS and CXXFLAGS. Full description how to integrate with OSS Fuzz is in [1] and [2]. Patch to OSS Fuzz repository [2] is ready to merge. We need to pass options with "-fsanitize=fuzzer" two times (in cmake/profile.cmake and test/fuzz/CMakeLists.txt) because: - cmake/profile.cmake is for project source files, -fsanitize=fuzzer-no-link option allows to instrument project source files for fuzzing, but LibFuzzer will not replace main() in these files. - test/fuzz/CMakeLists.txt uses -fsanitize=fuzzer and not -fsanitize=fuzzer-no-link because we want to add automatically generated main() for each fuzzer. 1. https://google.github.io/oss-fuzz/getting-started/new-project-guide/ 2. https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/ 3. https://github.com/google/oss-fuzz/pull/4723 Closes #1809
-
Sergey Bronnikov authored
There is a number of bugs related to parsing and encoding/decoding data. Examples: - csv: #2692, #4497, #2692 - uri: #585 One of the effective method to find such issues is a fuzzing testing. Patch introduces a CMake flag to enable building fuzzers (ENABLE_FUZZER) and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri modules. Note that fuzzers must return 0 exit code only, other exit codes are not supported [2]. NOTE: LibFuzzer requires Clang compiler. 1. https://llvm.org/docs/LibFuzzer.html 2. http://llvm.org/docs/LibFuzzer.html#id22 How-To Use: $ mkdir build && cd build $ cmake -DENABLE_FUZZER=ON \ -DENABLE_ASAN=ON \ -DCMAKE_BUILD_TYPE=Debug \ -DCMAKE_C_COMPILER="/usr/bin/clang" \ -DCMAKE_CXX_COMPILER="/usr/bin/clang++" .. $ make -j $ ./test/fuzz/csv_fuzzer -workers=4 ../test/static/corpus/csv Part of #1809
-
- Oct 22, 2020
-
-
Sergey Kaplun authored
Since LuaJIT provides extended LuaC API introduced in the scope of 5a61e1ab54b5c66bfebd836db1ac47996611e065 ('misc: add C and Lua API for platform metrics') corresponding header should be provided along with other Tarantool development files. Follows up #5187
-
- Oct 16, 2020
-
-
Alexander V. Tikhonov authored
Initially tried to change autoconf tools in Curl build to cmake and found the following build issue on: CentOS 6 CentOS 7 Debian 8 Ubuntu 14.04 Issue found: CMake Error at CMakeLists.txt:41 (cmake_minimum_required): CMake 3.0 or higher is required. You are running version 2.8.12.2 To fix the issue check is removed of the version from curl sources in Tarantool's third party '<root Tarantool sources>/third_party/curl': 12af024bc85606b14ffc415413a7e86e6bbee7eb ('Enable curl build with old cmake') After this fix completely changed autoconf to cmake in curl build. Autoconf part was completely removed and code cleaned up for cmake. For curl cmake build all autoconf options were ported to cmake configuration call, check the accurate list of the change in [1]. Also the following issues resolved: 1. Found that CURL cmake configuration file: third_party/curl/lib/CMakeLists.txt has installation part for built libcurl.a library: install(TARGETS ${LIB_NAME} EXPORT ${TARGETS_EXPORT_NAME} ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR} LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR} where it changes CMAKE_INSTALL_LIBDIR to appropriate name with suffix of the architecture, like: lib lib64 x86_64 Found that find_library routine from the file: cmake/FindLibCURL.cmake returns only 'lib' value and it breaks the building of the depends binaries. To avoid of it the CMAKE_INSTALL_LIBDIR option was set to cmake call: -DCMAKE_INSTALL_LIBDIR=lib 2. Found issue with building on CentOS 6: Linking C executable curl build/ares/dest/lib/libcares.a(ares__timeval.c.o): In function `ares__tvnow': ares__timeval.c:(.text+0x15): undefined reference to `clock_gettime' collect2: error: ld returned 1 exit status It was fixed with added "-lrt" flag to CMAKE_C_FLAGS and CMAKE_CXX_FLAGS build flags, when cmake version is lower than 3.0 and RT library had needed function. 3. Found issues with building Tarantool statically on Debian 9 and its package build on CentOS 8. Building statically got the issues with openssl linking, like [2]: static-build/openssl-prefix/lib/libcrypto.a(threads_pthread.o): In function `CRYPTO_THREAD_lock_new': threads_pthread.c:(.text+0x45): undefined reference to `pthread_rwlock_init' It happened because in openssl radically changed how threads-related things were handled before 1.1.0 version. It required the application to provide us with the locking functionality in form of callbacks. Since 1.1.0, these matters are entirely internal, so libcrypto requires the presence of the platform specific thread implementation of our choosing, which is pthread on everything. After '-pthread' added to C compile flags package build on CentOS 8 failed with the issue [3]: /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:101:4: error: #error "SIZEOF_CURL_OFF_T not defined" # error "SIZEOF_CURL_OFF_T not defined" ^~~~~ /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c: In function 'curlx_uztoso': /build/usr/src/debug/tarantool-2.6.0.141/third_party/curl/lib/warnless.c:192:40: error: 'CURL_MASK_SCOFFT' undeclared (first use in this function); did you mean 'CURL_MASK_SINT'? return (curl_off_t)(uznum & (size_t) CURL_MASK_SCOFFT); ^~~~~~~~~~~~~~~~ CURL_MASK_SINT To avoid of the issue decided to use '-pthread' flag only for openssl when it had not this flag in openssl compilation. 4. Found issue with static build using CentOS 7, where SSL cmake rule failed. Building the image got the issue: [ 1%] Performing configure step for 'bundled-libcurl-project' CMake Warning at CMakeLists.txt:50 (message): the curl cmake build system is poorly maintained. Be aware -- curl version=[7.66.0-DEV] -- Found c-ares: /tarantool/build/ares/dest/lib/libcares.a Found *nroff option: -- -man CMake Error at /usr/share/cmake/Modules/FindOpenSSL.cmake:278 (list): list GET given empty list Call Stack (most recent call first): CMakeLists.txt:347 (find_package) Root cause of the issue that Dockerfile uses globaly installed openSSL with: cmake ... -DOPENSSL_ROOT_DIR=/usr/local ... Its cmake build file: /usr/share/cmake/Modules/FindOpenSSL.cmake fails on parsing the SSL version: it has: REGEX "^#define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*") but it should to use: REGEX "^#[\t ]*define[\t ]+OPENSSL_VERSION_NUMBER[\t ]+0x([0-9a-fA-F])+.*") Anyway we want to use the same OpenSSL library for libcurl, as is used for Tarantool itself. So the path to it set for cURL build: list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_MODULE_PATH=${PROJECT_SOURCE_DIR}/cmake") 5. In CMake build CMAKE_USE_LIBSSH2 flag is enabled by default, while in autoconf --with-libssh2 was disabled by default. We need to switch CMAKE_USE_LIBSSH2 flag off with: list(APPEND LIBCURL_CMAKE_FLAGS "-DCMAKE_USE_LIBSSH2=OFF") to avoid of linking issues, like: ld: libssh2.c:(.text+0x4d8): undefined reference to `libssh2_*... this issue exists in curl issues [4]. Furthermore the following defaults are also disabled to keep the configuration consistent with autoconf one: list(APPEND LIBCURL_CMAKE_FLAGS "-DPICKY_COMPILER=OFF") list(APPEND LIBCURL_CMAKE_FLAGS "-DBUILD_CURL_EXE=OFF") Closes #4968 Closes #5019 Closes #5396 [1] - https://github.com/tarantool/tarantool/issues/4968#issue-617183031 [2] - https://gitlab.com/tarantool/tarantool/-/jobs/779176133#L6021 [3] - https://gitlab.com/tarantool/tarantool/-/jobs/778309145#L3060 [4] - https://github.com/curl/curl/issues/1146
-
Alexander V. Tikhonov authored
Prior to these changes bootstrap.h was generated right in the source directory even for out of source build. Firstly such approach doesn't respect the idea of building outside the source files. Furthermore this leads to build failures when the source directory is located on read-only file system. As a result of the patch bootstrap.h is generated within the build tree and include directories are adjusted the corresponding way. Also changed destination build directory from source to binary path. Part of #4968
-
- Sep 15, 2020
-
-
HustonMmmavr authored
Refactored static build process to use static-build/CMakeLists.txt instead of Dockerfile.staticbuild (this allows to support static build on macOS). Following third-party dependencies for static build are installed via cmake `ExternalProject_Add`: - OpenSSL - Zlib - Ncurses - Readline - Unwind - ICU * Added support static build for macOS * Fixed `CONFIGURE_COMMAND` while building bundled libcurl for static build at file cmake/BuildLibCURL.cmake: - disable building shared libcurl libraries (by setting `--disable-shared` option) - disable hiding libcurl symbols (by setting `--disable-symbol-hiding` option) - prevent linking libcurl with system libz (by setting `--with-zlib=${FOUND_ZLIB_ROOT_DIR}` option) * Removed Dockerfile.staticbuild * Added new gitlab.ci jobs to test new style static build: - static_build_cmake_linux - static_build_cmake_osx_15 * Removed static_docker_build gitlab.ci job Closes #5095 Co-authored-by:
Yaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
-
- Jul 08, 2020
-
-
Olga Arkhangelskaia authored
Tarantool has LTO support, however while building luajit this opt. was omitted. Patch adds necessary flag to turn it on. Closes #3743
-
- Jun 16, 2020
-
-
Vladislav Shpilevoy authored
Clang undefined behaviour sanitizer was turned on using -fsanitize=undefined flag, which is supposed to turn on all the sanitizations, except a few ones. Not needed sanitations were turned off explicitly, using -fno-sanitize=<type> flags. However appeared it does not work with some flags. For example, nullability sanitations can't be turned off when -fsanitize=undefined is used. Nullability sanitations lead to lots of false-positive fails such as typeof(*obj) where obj is NULL, or memcpy() with NULL destination but 0 size. The patch splits -fsanitize=undefined into separate flags and never turns on nullability checks. Part of #4609
-
- Jun 10, 2020
-
-
HustonMmmavr authored
Fixed static build with '-DBUILD_STATIC=ON' option: * Added cmake option CMAKE_DL_LIBS to icu library for test/unit tests binaries builds at file: cmake/FindICU.cmake due to fail: [ 84%] Linking CXX executable vy_point_lookup.test /usr/local/lib/libicuuc.a(putil.ao): In function `uprv_dl_open_62': putil.cpp:(.text+0x1a22): undefined reference to `dlopen' /usr/local/lib/libicuuc.a(putil.ao): In function `uprv_dlsym_func_62': putil.cpp:(.text+0x1a7d): undefined reference to `dlsym' /usr/local/lib/libicuuc.a(putil.ao): In function `uprv_dl_close_62': putil.cpp:(.text+0x1a61): undefined reference to `dlclose' collect2: error: ld returned 1 exit status * Added cmake option CMAKE_DL_LIBS to gomp library for test/unit tests binaries builds at file: cmake/BuildMisc.cmake due to fail: [ 91%] Linking CXX executable bps_tree.test /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function `gomp_target_init': (.text+0x8b): undefined reference to `dlopen' (.text+0xa2): undefined reference to `dlsym' (.text+0xd9): undefined reference to `dlclose' (.text+0x29b): undefined reference to `dlsym' (.text+0x2a8): undefined reference to `dlerror' (.text+0x2d0): undefined reference to `dlsym' (.text+0x2e9): undefined reference to `dlsym' (.text+0x300): undefined reference to `dlsym' (.text+0x317): undefined reference to `dlsym' (.text+0x330): undefined reference to `dlsym' /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d): more undefined references to `dlsym' follow /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function `gomp_target_init': (.text+0x9cc): undefined reference to `dlerror' collect2: error: ld returned 1 exit status Close #5024
-
- Jun 09, 2020
-
-
Vladislav Shpilevoy authored
Option ENABLE_UB_SANITIZER enables clang undefined behaviour sanitizer. So far the only UB to detect was alignment violation. This was the biggest problem found by the sanitizer. Now when it is fixed, most of the other types of UB are also turned on to fix them as well. There is a few of exceptions - pointer type overflow, vptr check, and all types of integer overflow and truncation. Pointer type overflow detection is disabled because it is abused in the source code a lot, by stailq data structure. Vptr sanitation is a runtime check ensuring that a pointer at a non-POD type really points at an object of this type, using RTTI. The check false-positively fails in alter.cc when AlterSpaceOp class objects are stored in an rlist, and the list is iterated using rlist_foreach_entry(). In the cycle there is a condition: &item->member != head In the end the 'item' points not at an AlterSpaceOp, but at the rlist head - offsetof(typeof(item), member), at an rlist structure. Despite 'item' is never dereferenced, clang anyway generates vptr check here, which of course fails. Note, '&item->member' does not dereference item. It is item + offsetof(typeof(item), member). Just another address a few bytes after item. Integer overflow and truncation are disabled because SQL uses int64_t variables as a container of values of range [INT64_MIN, UINT64_MAX]. This works because there is a flag 'is_neg' near each such value which tells how to interpret it - as negative int64_t, or as positive uint64_t. As a result, some operations lead to a false-positive overflow. For example, consider expr_code_int() function. It essentially can do this: int64_t value; ((uint64_t *)&value) = 9223372036854775808; value = -value; 9223372036854775808 is -INT64_MIN. It can't be stored in int64_t. But the thing is that (uint64_t)9223372036854775808 is stored exactly like (int64_t)INT64_MIN, in binary form. So the expression "value = -value" looks perfectly valid: "value = -9223372036854775808", But in fact it is interpreted as "value = -(-9223372036854775808)". These integer overflow/truncation problems are going to be fixed in a separate commit due to big amount of changes needed for that. Part of #4609
-
- Jun 08, 2020
-
-
Vladislav Shpilevoy authored
Clang has a built-in sanitizer for undefined behaviour. Such as wrong memory alignment, array boundaries violation, 0 division, bool values with non standard content, etc. The sanitizer emits runtime checks which lead to either crash, or a trap, or a warning print, depending on what is chosen. The patch makes it possible to turn the sanitizer on and catch UBs. The only supported UB so far is alignment check. Other types can be added gradually, along with fixing bugs which they find. The UB sanitizer is activated for ASAN builds in CI. Part of #4609
-
Vladislav Shpilevoy authored
Warning about invalid offsetof() (used on non-POD types) was set for g++, but wasn't for clang++. Warning about invalid alignof() (when expression is passed to it instead of a type) wasn't ignored, but is going to be very useful in upcoming unaligned memory access patches. That allows to write something like: struct some_long_type *object = region_aligned_alloc( region, size, alignof(*object)); This will work even if type of 'object' will change in future, and so it is safer. And shorter. Part of #4609
-
- May 20, 2020
-
-
Sergey Bronnikov authored
OpenBSD includes DL library in a base system Part of #4967
-
Sergey Bronnikov authored
Part of #4967
-
Sergey Bronnikov authored
Part of #4967
-
- Apr 02, 2020
-
-
Alexander V. Tikhonov authored
Fixed static build with '-DBUILD_STATIC=ON' option: - installed liblzma-dev library for libunwind static, due to found that static libunwind library uses undefined lzma functions: nm -a /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a | grep lzma U lzma_index_buffer_decode U lzma_index_end U lzma_index_size U lzma_index_uncompressed_size U lzma_stream_buffer_decode U lzma_stream_footer_decode while dynamic libunwind correctly sees liblzma installed: ldd /usr/lib/x86_64-linux-gnu/libunwind-x86_64.so | grep lzma liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f8fd1c23000) so to fix it the static library of lzma was needed. - added lzma library to unwind library for Tarantool build at file: cmake/compiler.cmake due to fail: /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o): In function `xz_uncompressed_size': ./src/elfxx.c:194: undefined reference to `lzma_stream_footer_decode' ./src/elfxx.c:201: undefined reference to `lzma_index_buffer_decode' ./src/elfxx.c:205: undefined reference to `lzma_index_size' ./src/elfxx.c:210: undefined reference to `lzma_index_end' ./src/elfxx.c:207: undefined reference to `lzma_index_uncompressed_size' ./src/elfxx.c:210: undefined reference to `lzma_index_end' /usr/lib/x86_64-linux-gnu/libunwind-x86_64.a(elf64.o): In function `_Uelf64_extract_minidebuginfo': ./src/elfxx.c:278: undefined reference to `lzma_stream_buffer_decode' collect2: error: ld returned 1 exit status test/unit/CMakeFiles/luaL_iterator.test.dir/build.make:134: recipe for target 'test/unit/luaL_iterator.test' failed make[2]: *** [test/unit/luaL_iterator.test] Error 1 - added dl library to gomp library for test/unit tests binaries builds at file: cmake/BuildMisc.cmake due to fail: /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o):(.text+0x34d): more undefined references to `dlsym' follow /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.a(target.o): In function `gomp_target_init': (.text+0x9cc): undefined reference to `dlerror' collect2: error: ld returned 1 exit status - added dl library to icu library for test/unit tests binaries builds at file: cmake/FindICU.cmake due to fail: /usr/x86_64-linux-gnu/libicuuc.a(putil.ao): In function `uprv_dl_open_60': (.text+0x1ce2): undefined reference to `dlopen' /usr/x86_64-linux-gnu/libicuuc.a(putil.ao): In function `uprv_dlsym_func_60': (.text+0x1d3d): undefined reference to `dlsym' /usr/x86_64-linux-gnu/libicuuc.a(putil.ao): In function `uprv_dl_close_60': (.text+0x1d21): undefined reference to `dlclose' collect2: error: ld returned 1 exit status Added static build to gitlab-ci in release check criteria named as static_build job. Previously named static_build job renamed to static_docker_build, due to it checks the build at Dockerfile. Also moved static build make targets from .gitlab.mk to .travis.mk to store it in common place with the other test/build make targets. Moved environement from .gitlab-ci.yml file into make targets to make this targets true building in static w/o additional setup. Close #4551
-
Alexander V. Tikhonov authored
The change enables memory leaks detection to existing ASAN testing routine and introduces suppression files with the corresponding exception list: * address sanitizer for compile-time: asan/asan.supp * memory leak sanitizer for run-time: asan/lsan.supp Furthermore, added engine and replication suites for ASAN testing routine. Additionally to the tests blacklisted within #4359, 'box/on_shutdown.test.lua' is also disabled since it fails the introduced leak check. All blacklisted tests have to be enabled within #4360. Close #2058
-
- Mar 23, 2020
-
-
Timur Safin authored
SO_LINGER makes no much sense for unix-sockets, and Microsoft WSL is returning EINVAL if setsockopts called for SO_LINGER over unix sockets: [004] 2020-03-11 18:42:29.592 [29182] main/102/app sio.c:169 !> SystemError setsockopt(SO_LINGER), called on fd 16, aka [004] 2020-03-11 18:42:29.592 [29182] main/102/app F> can't initialize storage: setsockopt(SO_LINGER), called on fd 16, [004] 2020-03-11 18:42:29.592 [29182] main/102/app F> can't initialize storage: setsockopt(SO_LINGER), called on fd 16, And it's sort of correct here, but the problem is Linux is simply silently ignoring it, which passes tests. After much debates we decided to work-around this case via CMAKE define. NB! In a future (April/May 2020), when WSL2 with full Linux kernel would be released we should disable this check. Acked-by:
Cyrill Gorcunov <gorcunov@gmail.com>
-
- Mar 17, 2020
-
-
Igor Munkin authored
Since this build flag has been removed as a result of reverting the tarantool/luajit@d4e985a, its definition in the corresponding Tarantool cmake file is irrelevant. Furthermore, considering the breakage faced in #4770 the following tests are introduced: * the check whether space __pairs metamethod is set to space.pairs to create a Lua Fun iterator that handles __pairs manually underneath. * the check whether pairs builtin behaviour doesn't change when __pairs is set e.g. on space object. Follow-up #4560 Closes #4770 Reviewed-by:
Vladislav Shpilevoy <v.shpilevoy@tarantool.org> Signed-off-by:
Igor Munkin <imun@tarantool.org>
-
- Mar 05, 2020
-
-
Serge Petrenko authored
libcurl has a built-in threaded resolver used for asynchronous DNS requests, however, when DNS server is slow to respond, the request still hangs tarantool until it is finished. The reason is that curl calls thread_join on the resolving thread internally upon timeout, making the calling thread hang until resolution has ended. Use c-ares as an asynchronous resolver instead to eliminate the problem. Closes #4591
-
- Dec 05, 2019
-
-
Olga Arkhangelskaia authored
Turns on LUAJIT_ENABLE_PAIRSMM flag for tarantool build. Now __pairs/__ipairs metamethods are available. Closes #4650
-
- Nov 08, 2019
-
-
Alexander Turenko authored
After ea5929db ('build: fix OpenSSL linking problems on FreeBSD') we set CFLAGS explicitly (possibly to an empty value) when invoking a configure script for curl. When this parameter is set the script does not use a value of environment variable CFLAGS. Before this commit LDFLAGS environment variable can affect build of curl submodule. This can lead to a problem when a user or a tool set CFLAGS and LDFLAGS both and some linker flag assumes that some compilation flag is present. Here we set empty LDFLAGS explicitly to avoid using of the environment variable. A distributive build tool such as rpmbuild or emerge usually sets CFLAGS and LDFLAGS. The problem with incompatible compiler / linker options has been reveal under rpmbuild on CentOS 8 with hardened build enabled (which is so when backtraces are disabled). It is not clear whether we should follow environment variables or values determined by CMake for CFLAGS, CPPFLAGS and LDFLAGS when building a submodule (such as luajit and curl). Let's decide about this later. Part of #4543. Reviewed-by:
Alexander V. Tikhonov <avtikhon@tarantool.org> Reviewed-by:
Igor Munkin <imun@tarantool.org>
-
- Oct 28, 2019
-
-
Alexander Turenko authored
This allows to overcome problems when CMake chooses one toolchain to build tarantool, but a library (libluajit.a or libcurl.a) is built using another (incompatible) toolchain. Fixes #4587.
-
Alexander Turenko authored
FreeBSD has OpenSSL as part of the base system: libraries are located in /usr/lib, headers are in /usr/include. However a user may install the library into /usr/local/{lib,include} from ports / pkg. In this case tarantool did choose /usr/local version, while libcurl will pick up a base system library. This is fixed by passing --with-ssl option with an argument (/usr/local or /usr if custom -DOPENSSL_ROOT_DIR=<...> is not passed). Now the behaviour is the following. If -DOPENSSL_ROOT_DIR=<...> is passed, then try to use OpenSSL from it. Otherwise find the library in /usr/local and then in /usr. This is right as for tarantool's crypto module as well as for libcurl submodule. There is a flaw here: a user is unable to choose a base system library if a ports / pkg version of OpenSSL is installed. The reason here is that tarantool's crypto module depends on other libraries and -I/usr/local/include may be added to build options. I have no good solution for that, so `cmake . -DOPENSSL_ROOT_DIR=/usr` will give a warning on FreeBSD and `gmake` likely will fail if libraries are of different versions (see cmake/os.cmake comments for more information). See also a [discussion][1] in FreeBSD community about all those /usr and /usr/local problems. There were two other problems that may fail tarantool build on FreeBSD: they are fixed in this commit and described below. First, libcurl's configure script chooses GCC by default if it exists (say, installed from ports / pkg). It is unexpected behaviour when tarantool sources itself are built with clang. Now it is fixed by passing a compiler explicitly to the libcurl's configure script: the library will use base system clang by default or one that a user pass to tarantool's cmake. Side note: GCC has /usr/local/include in its default headers search paths; libcurl's configure script chooses GCC as a compiler and OpenSSL from a base system by default (when CC and --with-ssl=<...> are not set) that leads to OpenSSL header / library mismatch. It is the primary reason of the build fail that was fixed in 1f2338bd ('build: FreeBSD packages installation'). It is not much relevant anymore, because we don't try to link with a base system OpenSSL if /usr/local one exists (however if it is asked explicitly with -DOPENSSL_ROOT_DIR=<...> we'll do, but will give a warning). Anyway, it is important to know such details if we'll change build scripts in a future. Second, backtraces are not supported on FreeBSD, but were enabled if libunwind headers is found. This leads to an error on cmake stage, because of inability to find a right library (this is a bug). Now we disable backtraces on FreeBSD by default even if libunwind is found. See When CC is passed to libcurl's configure script, the new problem opens on Mac OS. CMake chooses XCode toolchain by default (at least on a particular system where I tried it), which requires -isysroot=<SDK_PATH> option to be passed to a preprocessor and a compiler in order to find system headers. See [2] for more information. [1]: https://wiki.freebsd.org/WarnerLosh/UsrLocal [2]: https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes#3035623 Follows up #4490.
-