Introduce a new builtin Tarantool module `datetime.lua` for timestamp
and interval types support.

New third_party module - c-dt
-----------------------------

* Integrated chansen/c-dt parser as 3rd party module to the
  Tarantool cmake build process;
* We use tarantool/c-dt instead of original chansen/c-dt to
  have an easier cmake build integration, as we have added some
  changes, which provide cmake support, and allow to rename symbols
  if necessary (this symbol renaming is similar to that we see
  with xxhash or icu).

New built-in module `datetime`
------------------------------

* created a new Tarantool built-in module `datetime`, which uses
  `struct datetime` data structure for keeping timestamp values;
* Lua module uses a number of `dt_*` functions from `c-dt` library,
  but they were renamed to `tnt_dt_*` at the moment of exporting
  from executable - to avoid possible name clashes with external
  libraries.

* At the moment we libc `strftime` for formatting of datetime
  values according to flags passed, i.e. `date:format('%FT%T%z')`
  will return something like '1970-01-01T00:00:00+0000', but
  `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970'

* if there is no format provided then we use default
  `tnt_datetime_to_string()` function, which converts datetime
  to their default ISO-8601 output format, i.e.
  `tostring(date)` will return string like "1970-01-01T00:00:00Z"

* There are a number of simplified interfaces
  - totable() for exporting table with attributes names as provided
    by `os.date('*t')`
  - set() method provides unified interface to set values using
    the set of attributes as defined above in totable()

Example,

```
local dt = datetime.new {
	nsec      = 123456789,

	sec       = 19,
	min       = 29,
	hour      = 18,

	day       = 20,
	month     = 8,
	year      = 2021,

	tzoffset  = 180
}

local t = dt:totable()
--[[
{
	sec = 19,
	min = 29,
	wday = 6,
	day = 20,
	nsec = 123456789,
	isdst = false,
	yday = 232,
	tzoffset = 180,
	month = 8,
	year = 2021,
	hour = 18
}
--]]

dt:format()   -- 2021-08-21T14:53:34.032Z
dt:format('%Y-%m-%dT%H:%M:%S')   -- 2021-08-21T14:53:34

dt:set {
	usec      = 123456,

	sec       = 19,
	min       = 29,
	hour      = 18,

	day       = 20,
	month     = 8,
	year      = 2021,

	tzoffset  = 180,

}
dt:set {
	timestamp = 1629476485.124,

	tzoffset  = 180,
}

```

Coverage is

File                 Hits Missed Coverage
-----------------------------------------
builtin/datetime.lua 299  23     92.86%
-----------------------------------------
Total                299  23     92.86%

Part of #5941

@TarantoolBot document
Title: Introduced a new `datetime` module for timestamp and interval support

Create `datetime` module for timestamp and interval types support.
It allows to create date and timestamp values using either object interface,
or via parsing of string values conforming to iso-8601 standard.
One may manipulate (modify, subtract or add) timestamp and interval values.

Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool
for a more detailed description of module API.

Part of #5385

This patch is the first step for fixing regression introduced in
f998ea39 (digest: introduce FFI bindings for xxHash32/64).
We used xxhash library that is shipped with zstd. However it's
possible that user doesn't use bundled zstd. In such cases we
couldn't export xxhash symbols and build failed with following
error:

```
[ 59%] Linking CXX executable tarantool
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd80): undefined reference to `XXH32'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd88): undefined reference to `XXH32_copyState'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd90): undefined reference to `XXH32_digest'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xd98): undefined reference to `XXH32_reset'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xda0): undefined reference to `XXH32_update'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xda8): undefined reference to `XXH64'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdb0): undefined reference to `XXH64_copyState'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdb8): undefined reference to `XXH64_digest'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdc0): undefined reference to `XXH64_reset'
/usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/../../../../x86_64-pc-linux-gnu/bin/ld: CMakeFiles/tarantool.dir/exports.c.o:(.data.rel+0xdc8): undefined reference to `XXH64_update'
collect2: error: ld returned 1 exit status
```

To avoid a problem this patch introduces standalone xxhash library
that will be bundled anyway. It's worth to mention that our
approach is still related to zstd. We use Cyan4973/xxHash that is
used in zstd and passes the same compile flags to it. Single
difference is usage of XXH_NAMESPACE to avoid symbols clashing
with zstd.

Need for #6135

`VERSION` files in small subproject and in tarantool are treated as C++
standard library on a filesystem with case-insensitive names. So we have
to delete the root of tarantool project from `include_directories` in
tarantool CMake. Also we have to change `include_directories` in tarantool
CMake from the root of `small` project to `include` subfolder in `small`
project.
Closes #6076

Ship libcurl headers to system path "${PREFIX}/include/tarantool"
in the case of libcurl included as bundled library or static
build. It is needed to use SMTP client with tarantool's libcurl
instead of system libcurl.

See related issue: https://github.com/tarantool/smtp/issues/24

Closes #4559

Now `make luacheck` gracefully handles different cases[^1]: in-source
and out-of-source build (within the source tree or outside), current
working directory as a real path or with symlink components.

As result of looking into those problems I filed the issue [1] against
luacheck. It seems, there are problems around absolute paths with
symlinks components.

[^1]: We have the similar problems with LuaJIT's luacheck rule. They
      will be fixed in a separate patch.

[1]: https://github.com/mpeterv/luacheck/issues/208



Reviewed-by: Sergey Bronnikov <sergeyb@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>

LuaJIT submodule is bumped to introduce the following changes:
* test: run luacheck static analysis via CMake
* test: fix warnings found with luacheck in misclib*
* test: run LuaJIT tests via CMake
* build: replace GNU Make with CMake
* build: preserve the original build system

Since LuaJIT build system is ported to CMake in scope of the changeset
mentioned above, the module building the LuaJIT bundled in Tarantool is
completely reworked. There is no option to build Tarantool against
another prebuilt LuaJIT due to a91962c0
('Until Bug#962848 is fixed, don't try to compile with external
LuaJIT'), so all redundant options defining the libluajit to be used in
Tarantool are dropped with the related auxiliary files.

To run LuaJIT related tests or static analysis for Lua files within
LuaJIT repository, <LuaJIT-test> and <LuaJIT-luacheck> targets are used
respectively as a dependency of the corresponding Tarantool targets.

As an additional dependency to run LuaJIT tests, prove[1] utility is
required, so the necessary binary packages are added to the lists with
build requirements.

[1]: https://metacpan.org/pod/TAP::Harness#prove



Closes #4862
Closes #5470
Closes #5631

Reviewed-by: Sergey Kaplun <skaplun@tarantool.org>
Reviewed-by: Timur Safin <tsafin@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

Export libcurl's symbols only when they are provided by tarantool
itself: when the library is linked statically into the tarantool's
executable. There is no much sense to export the symbols when we link
against the library dynamically.

Regarding motivation of the change. Since 2.6.0-36-g29ec62891 ('Ensure
all curl symbols are exported') the curl_multi_poll() function is
exported from the tarantool executable. It leads to a failure in
Homebrew's build, because there we link (dynamically) with a system
libcurl. On Mac OS 10.15 it is libcurl 7.64.1, while the function
appears since libcurl 7.66.0. So a linker reports the undefined symbol:
`curl_multi_poll`.

Now the symbols are not exported at dynamic linking with libcurl, so the
linker is happy.

This commit relaxes bounds for dynamic linking, but an attempt to link
with libcurl older than 7.66.0 statically still leads to a linking
failure. The box-tap/gh-5223-curl-exports.test.lua test still fails when
tarantool is linked (dynamically) against an old libcurl.

It looks as the good compromise. When libcurl functionality is provided
by tarantool itself, *all* functions listed in the test are present
(otherwise a linker will complain). But tarantool does not enforce a
newer libcurl version, when it just *uses* this functionality and don't
provide it for modules and stored procedured. It is not tarantool's
responsibility in the case.

We possibly should skip the box-tap/gh-5223-curl-exports.test.lua test
when tarantool is built against libcurl dynamically or revisit the
described approach. I'll leave it as possible follow up activity.

Fixes #5542

There is a number of bugs related to parsing and encoding/decoding data.
Examples:

- csv: #2692, #4497, #2692
- uri: #585

One of the effective method to find such issues is a fuzzing testing.
Patch introduces a CMake flag to enable building fuzzers (ENABLE_FUZZER)
and add fuzzers based on LibFuzzer [1] to csv, http_parser and uri
modules. Note that fuzzers must return 0 exit code only, other exit
codes are not supported [2].

NOTE: LibFuzzer requires Clang compiler.

1. https://llvm.org/docs/LibFuzzer.html
2. http://llvm.org/docs/LibFuzzer.html#id22

How-To Use:

$ mkdir build && cd build
$ cmake -DENABLE_FUZZER=ON \
	-DENABLE_ASAN=ON \
	-DCMAKE_BUILD_TYPE=Debug \
	-DCMAKE_C_COMPILER="/usr/bin/clang" \
	-DCMAKE_CXX_COMPILER="/usr/bin/clang++" ..
$ make -j
$ ./test/fuzz/csv_fuzzer -workers=4 ../test/static/corpus/csv

Part of #1809

Very convenient to have this string extension.
We will use it in crash handling.

Acked-by: Serge Petrenko <sergepetrenko@tarantool.org>
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>

To run Jepsen tests in different configurations we need to parametrize run
script by options, so lein options and number of nodes passed with environment
variables. By default script runs testing with Tarantool built from latest
commit.

Added these configurations:

- single instance
- single instance with enabled TXM
- cluster with enabled Raft
- cluster with enabled Raft and TXM

Closes #5437

For running Jepsen tests we need to checkout external repository with tests
source code on a build stage. This behaviour brokes a Tarantool build under
Gentoo. Option WITH_JEPSEN enables targets only when they needed.

Closes #5325

Added targets 'make jepsen-single' and 'make jepsen-cluster'
to run Jepsen tests on a single Tarantool instance and
cluster of Tarantool instances.

Part of #5277

Part of #4681

Reviewed-by: Igor Munkin <imun@tarantool.org>

Clang has a built-in sanitizer for undefined behaviour. Such as
wrong memory alignment, array boundaries violation, 0 division,
bool values with non standard content, etc.

The sanitizer emits runtime checks which lead to either crash, or
a trap, or a warning print, depending on what is chosen.

The patch makes it possible to turn the sanitizer on and catch
UBs. The only supported UB so far is alignment check. Other types
can be added gradually, along with fixing bugs which they find.

The UB sanitizer is activated for ASAN builds in CI.

Part of #4609

libcurl has a built-in threaded resolver used for asynchronous DNS
requests, however, when DNS server is slow to respond, the request still
hangs tarantool until it is finished. The reason is that curl calls
thread_join on the resolving thread internally upon timeout, making the
calling thread hang until resolution has ended.
Use c-ares as an asynchronous resolver instead to eliminate the problem.

Closes #4591

After the commit 77fa45bd
('lua: add fiber.top() listing fiber cpu consumption')
the unit tests builds failed like:

/opt/rh/devtoolset-6/root/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/ld:
  ../../src/lib/core/libcore.a(fiber.c.o): undefined reference to symbol
  'clock_gettime@@GLIBC_2.2.5'
//lib64/librt.so.1: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
test/unit/CMakeFiles/cbus.test.dir/build.make:108: recipe for target
  'test/unit/cbus.test' failed
make[2]: *** [test/unit/cbus.test] Error 1

Found that fiber.cc is using now clock_gettime(), which requires -lrt
with glibc. To fix it added librt dependency for core library for glibc.
Due to glibc requires for -lrt for clock_gettime() only for some
versions, check 'man clock_gettime.2':
  'Link with -lrt (only for glibc versions before 2.17).'
the check whether is able to use clock_gettime() w/o librt library is
added.

Close #4639

(cherry picked from commit 99b0ef771135eab66ef3758371b1b5431e21cbff)

To make it possible to develop and test related features on
systems without systemd.

WITH_SYSTEMD cmake flag is used to generate systemd related files:
unit, generator script, etc. To keep this behavior and make it possible
to use NOTIFY_SOCKET without other systemd-related stuff,
I added WITH_NOTIFY_SOCKET cmake flag.

It also required some changes to support other OS:

SOCK_CLOEXEC (not available on macOS) flag for socket()
is replaced with `fcntl(fd, F_SETFD, FD_CLOEXEC)` which has the same effect.

MSG_NOSIGNAL flag for sendmsg is also not available on macOS.
However it has SO_NOSIGPIPE flag for setsockopt which disables SIGPIPE.
So it requires different solution for different OS. Inspired by
https://nwat.xyz/blog/2014/01/16/porting-msg_more-and-msg_nosigpipe-to-osx/

Have to reduce send-buffer size to 4MB because larger values
are not supported on macOS by default. This value should be enough
for all systems because notification messages are usually less than 1KB.

Fixes #4436

System-wide dynamic libraries usually (always?) have NEEDED and RUNPATH
tags in a dynamic section (as `readelf -d /usr/lib/lib<...>.so` shows),
so when we link, say, with libssl.so, which depends on libz.so, a linker
does not complain against unresolved symbols that can be found in Z
library (if it is installed within a system).

Things are different when we linking with a static library. Say, when we
linking with libssl.a, which contains an unresolved symbol from Z
library, a linker reports an error. It is not possible to store an
information where to find unresolved symbols (NEEDED / RUNPATH) in a
static library (AFAIK).

We depend on three libraries that are depend on Z library: libcurl,
libssl and libcrypto (two latter are part of OpenSSL). When one of those
libraries is linked statically we should link with libz.so or libz.a
(depending on BUILD_STATIC flag). The patch doing exactly this.

The patch changes OPENSSL_LIBRARIES variable to fix the issue with
static linking of OpenSSL libraries. It also changes CURL_LIBRARIES in
the same way, however this does not alter any visible behaviour, because
OPENSSL_LIBRARIES is added to CURL_LIBRARIES. The latter change was made
to unify the way to choose libraries to link with: it is pure
refactoring part.

Fixes #4437.

Hold libcurl-7.65.3. This version is not affected by the following
issues:

* #4180 ('httpc: redirects are broken with libcurl-7.30 and older');
* #4389 ('libcurl memory leak');
* #4397 ('HTTPS seem to be unstable').

After this patch libcurl will be statically linked when
ENABLE_BUNDLED_LIBCURL option is set. This option is set by default.

Closes #4318

@TarantoolBot document
Title: Tarantool dependency list was changed

* Added build dependencies: autoconf, automake, libtool, zlib-devel
  (zlib1g-dev on Debian).
* Added runtime dependencies: zlib (zlib1g on Debian).
* Removed build dependencies: libcurl-devel (libcurl4-openssl-dev on
  Debian).
* Removed runtime dependencies: curl.

The reason is that now we use compiled-in libcurl: so we don't depend on
a system libcurl, but inherit its dependencies.

Add fixed-point decimal type to tarantool core.
Adapt decNumber floating-point decimal library for the purpose, write a
small wrapper and add unit tests.

A new decimal type is an alias for decNumber numbers from the decNumber
library.
Arithmetic operations (+, -, *, /) and some mathematic functions
(ln, log10, exp, pow, sqrt) are available together with methods to
pack and unpack decimal to and from its packed representation (useful
for serialization).

We introduce a single context for all the arithmetic operations
on decimals, which enforces both number precision and scale to be
in range [0, 38]. NaNs and Infinities are restricted.

Part of #692

We want to detect a situation where task in fiber is too eager for
stack memory and relax rss usage in such case. For this sake upon
stack creation we put 8 marks near 64K bound (such params allows us
to fill ~1/4 of a page, which seem reasonable but we might change
this params with time).

Once stack get recycled we investigate the marks and if they were
overwritten we drop all pages behind to relax memory usage (if OS
supports madvise syscall).

Another important moment is that we're marking the whole stack
as not present thus if fiber never stepped over 64K limit the
marks will be in tact and it means the fibers are light ones
there won't be much #pf in future.

Later we plan to implement an intelligent fiber scheduling
considering how many memory fibers consume in average.

@locker:
 - fix watermark page alignment for grow-up stack
 - improve MADV_DONTNEED check
 - clean up code and elaborate comments
 - add test case to unit/fiber
 - fix unit/guard test

Follow-up #3418

When building "tags" target we scan the whole working
directory which is redundant. In particular .git,.pc,patches
directories should not be scanned for sure.

For the same reason why 'uri' was moved to src/lib - SWIM needs
core and uuid, and SWIM will live in src/lib.

This commit follows 'uri' relocation as a separate one because
'uri' relocation required some changes in the files, moved by
this commit.

Needed for #3234

posix_fallocate(), which is used for preallocating disk space for WAL
files, increases the file size and fills the allocated space with zeros.
The problem is a WAL file may be read by a relay thread at the same time
it is written to. We try to handle the zeroed space in xlog_cursor (see
xlog_cursor_next_tx()), however this turns out to be not enough, because
transactions are written not atomically so it may occur that a writer
writes half a transaction when a reader reads it. Without fallocate, the
reader would stop at EOF until the rest of the transaction is written,
but with fallocate it reads zeroes instead and thinks that the xlog file
is corrupted while actually it is not.

Fix this issue by using fallocate() with FALLOC_FL_KEEP_SIZE flag
instead of posix_fallocate(). With the flag fallocate() won't increase
the file size, it will only allocate disk space beyond EOF.

Closes #3883

Added -DENABLE_LTO=ON/OFF cmake option, OFF by default.

LTO speeds up cpu-intensive workloads by up to 20% (see [1] and [2]).

Requirements to enable LTO:

- cmake >= 3.9;
- Linux: ld.bfd / ld.gold from binutils >= 2.31 (or later 2.30)
  (gold >= 1.15);
- Mac OS: xcode >= 8 (earlier versions are not tested).

The requirement of the recent ld version is due to bug with exporting
symbols from dynamic list when LTO is enabled, see [3].

Note: -Wlto-type-mismatch on GCC (enabled by default with -flto) gives
namy warnings. Filed [4] to investigate it.

Note: LuaJIT will be compiled w/o LTO despite the option set, filed [5].

[1]: https://github.com/tarantool/tarantool/wiki/performance-research
[2]: https://gist.github.com/Khatskevich/31a2da6ab46ce903120e7a03d65966db
[3]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84901
[4]: https://github.com/tarantool/tarantool/issues/3742
[5]: https://github.com/tarantool/tarantool/issues/3743

Closes #3117

f

This function introduces a new xlog method xlog_fallocate() that makes
sure that the requested amount of disk space is available at the current
write position. It does that with posix_fallocate(). The new method is
called before writing anything to WAL, see wal_fallocate(). In order not
to invoke the system call too often, wal_fallocate() allocates disk
space in big chunks (1 MB).

The reason why I'm doing this is that I want to have a single and
clearly defined point in the code to handle ENOSPC errors, where
I could delete old WALs and retry (this is what #3397 is about).

Needed for #3397

sync_file_range is declared only if _GNU_SOURCE macro is defined.
Also, in order to be used in a source file, HAVE_SYNC_FILE_RANGE
must be present in config.h.cmake.

Fixes commit caae99e5 ("Refactor xlog writer").

A possibility to build tarantool with included library dependencies.
Use the flag -DBUILD_STATIC=ON to build statically against curl, readline,
ncurses, icu and z.
Use the flag -DOPENSSL_USE_STATIC_LIBS=ON to build with static
openssl

Changes:
  * Add FindOpenSSL.cmake because some distributions do not support the use of
  openssl static libraries.
  * Find libssl before curl because of build dependency.
  * Catch all bundled libraries API and export then it in case of static
  build.
  * Rename crc32 internal functions to avoid a name clash with linked libraries.

Notes:
  * Bundled libyaml is not properly exported, use the system one.
  * Dockerfile to build static with docker is included

Fixes #3445

Fixing build under FreeBSD:
Undefined symbol "iconv_open"
Add compile time build check with FindICONV.cmake
and a wrapper to import relevant symbol names with include file.

Closes #3441

The option is '-DENABLE_BUNDLED_ZSTD' and defaults to ON.

There are two goals of making this conditional, both related to building
Tarantool on Gentoo Linux from an ebuild:

* Avoid bundled ZStd building issue w/o pay to investivage it.
* Allow user to choose between system and bundled library.

Fix misleading "C atomics not supported" when git submodules
are missing.

Closes #2088

getopt_long() is available on all supported platforms.
Get rid of legacy gopt and use getopt_long().

Incompatible changes:

 * `tarantool --version --no-such-option` printed
    "unrecognized option '--no-such-option'", now it displays version.
    `tarantool --no-such-option --version` still prints an error message.

Needed for #1265

Add an option to enable LuaJIT's GC64 mode and enable this option
by default on OS X.

Closes #2643

Remove libbfd from dependencies and use libunwind for stack traces.

Closes #2103

Inpspired by tarantool/curl module by Vasiliy Soshnikov.
Reviewed and refactored by Roman Tsisyk.

Closes #2083