libcurl has a built-in threaded resolver used for asynchronous DNS
requests, however, when DNS server is slow to respond, the request still
hangs tarantool until it is finished. The reason is that curl calls
thread_join on the resolving thread internally upon timeout, making the
calling thread hang until resolution has ended.
Use c-ares as an asynchronous resolver instead to eliminate the problem.

Closes #4591

(cherry picked from commit 23837076)

To make it possible to develop and test related features on
systems without systemd.

WITH_SYSTEMD cmake flag is used to generate systemd related files:
unit, generator script, etc. To keep this behavior and make it possible
to use NOTIFY_SOCKET without other systemd-related stuff,
I added WITH_NOTIFY_SOCKET cmake flag.

It also required some changes to support other OS:

SOCK_CLOEXEC (not available on macOS) flag for socket()
is replaced with `fcntl(fd, F_SETFD, FD_CLOEXEC)` which has the same effect.

MSG_NOSIGNAL flag for sendmsg is also not available on macOS.
However it has SO_NOSIGPIPE flag for setsockopt which disables SIGPIPE.
So it requires different solution for different OS. Inspired by
https://nwat.xyz/blog/2014/01/16/porting-msg_more-and-msg_nosigpipe-to-osx/

Have to reduce send-buffer size to 4MB because larger values
are not supported on macOS by default. This value should be enough
for all systems because notification messages are usually less than 1KB.

Fixes #4436

(cherry picked from commit 1e509dde)

System-wide dynamic libraries usually (always?) have NEEDED and RUNPATH
tags in a dynamic section (as `readelf -d /usr/lib/lib<...>.so` shows),
so when we link, say, with libssl.so, which depends on libz.so, a linker
does not complain against unresolved symbols that can be found in Z
library (if it is installed within a system).

Things are different when we linking with a static library. Say, when we
linking with libssl.a, which contains an unresolved symbol from Z
library, a linker reports an error. It is not possible to store an
information where to find unresolved symbols (NEEDED / RUNPATH) in a
static library (AFAIK).

We depend on three libraries that are depend on Z library: libcurl,
libssl and libcrypto (two latter are part of OpenSSL). When one of those
libraries is linked statically we should link with libz.so or libz.a
(depending on BUILD_STATIC flag). The patch doing exactly this.

The patch changes OPENSSL_LIBRARIES variable to fix the issue with
static linking of OpenSSL libraries. It also changes CURL_LIBRARIES in
the same way, however this does not alter any visible behaviour, because
OPENSSL_LIBRARIES is added to CURL_LIBRARIES. The latter change was made
to unify the way to choose libraries to link with: it is pure
refactoring part.

Fixes #4437.

(cherry picked from commit 2cdfaf3b)

Hold libcurl-7.65.3. This version is not affected by the following
issues:

* #4180 ('httpc: redirects are broken with libcurl-7.30 and older');
* #4389 ('libcurl memory leak');
* #4397 ('HTTPS seem to be unstable').

After this patch libcurl will be statically linked when
ENABLE_BUNDLED_LIBCURL option is set. This option is set by default.

Closes #4318

@TarantoolBot document
Title: Tarantool dependency list was changed

* Added build dependencies: autoconf, automake, libtool, zlib-devel
  (zlib1g-dev on Debian).
* Added runtime dependencies: zlib (zlib1g on Debian).
* Removed build dependencies: libcurl-devel (libcurl4-openssl-dev on
  Debian).
* Removed runtime dependencies: curl.

The reason is that now we use compiled-in libcurl: so we don't depend on
a system libcurl, but inherit its dependencies.

(cherry picked from commit 7e51aebb)

Add fixed-point decimal type to tarantool core.
Adapt decNumber floating-point decimal library for the purpose, write a
small wrapper and add unit tests.

A new decimal type is an alias for decNumber numbers from the decNumber
library.
Arithmetic operations (+, -, *, /) and some mathematic functions
(ln, log10, exp, pow, sqrt) are available together with methods to
pack and unpack decimal to and from its packed representation (useful
for serialization).

We introduce a single context for all the arithmetic operations
on decimals, which enforces both number precision and scale to be
in range [0, 38]. NaNs and Infinities are restricted.

Part of #692

We want to detect a situation where task in fiber is too eager for
stack memory and relax rss usage in such case. For this sake upon
stack creation we put 8 marks near 64K bound (such params allows us
to fill ~1/4 of a page, which seem reasonable but we might change
this params with time).

Once stack get recycled we investigate the marks and if they were
overwritten we drop all pages behind to relax memory usage (if OS
supports madvise syscall).

Another important moment is that we're marking the whole stack
as not present thus if fiber never stepped over 64K limit the
marks will be in tact and it means the fibers are light ones
there won't be much #pf in future.

Later we plan to implement an intelligent fiber scheduling
considering how many memory fibers consume in average.

@locker:
 - fix watermark page alignment for grow-up stack
 - improve MADV_DONTNEED check
 - clean up code and elaborate comments
 - add test case to unit/fiber
 - fix unit/guard test

Follow-up #3418

When building "tags" target we scan the whole working
directory which is redundant. In particular .git,.pc,patches
directories should not be scanned for sure.

For the same reason why 'uri' was moved to src/lib - SWIM needs
core and uuid, and SWIM will live in src/lib.

This commit follows 'uri' relocation as a separate one because
'uri' relocation required some changes in the files, moved by
this commit.

Needed for #3234

posix_fallocate(), which is used for preallocating disk space for WAL
files, increases the file size and fills the allocated space with zeros.
The problem is a WAL file may be read by a relay thread at the same time
it is written to. We try to handle the zeroed space in xlog_cursor (see
xlog_cursor_next_tx()), however this turns out to be not enough, because
transactions are written not atomically so it may occur that a writer
writes half a transaction when a reader reads it. Without fallocate, the
reader would stop at EOF until the rest of the transaction is written,
but with fallocate it reads zeroes instead and thinks that the xlog file
is corrupted while actually it is not.

Fix this issue by using fallocate() with FALLOC_FL_KEEP_SIZE flag
instead of posix_fallocate(). With the flag fallocate() won't increase
the file size, it will only allocate disk space beyond EOF.

Closes #3883

Added -DENABLE_LTO=ON/OFF cmake option, OFF by default.

LTO speeds up cpu-intensive workloads by up to 20% (see [1] and [2]).

Requirements to enable LTO:

- cmake >= 3.9;
- Linux: ld.bfd / ld.gold from binutils >= 2.31 (or later 2.30)
  (gold >= 1.15);
- Mac OS: xcode >= 8 (earlier versions are not tested).

The requirement of the recent ld version is due to bug with exporting
symbols from dynamic list when LTO is enabled, see [3].

Note: -Wlto-type-mismatch on GCC (enabled by default with -flto) gives
namy warnings. Filed [4] to investigate it.

Note: LuaJIT will be compiled w/o LTO despite the option set, filed [5].

[1]: https://github.com/tarantool/tarantool/wiki/performance-research
[2]: https://gist.github.com/Khatskevich/31a2da6ab46ce903120e7a03d65966db
[3]: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84901
[4]: https://github.com/tarantool/tarantool/issues/3742
[5]: https://github.com/tarantool/tarantool/issues/3743

Closes #3117

f

This function introduces a new xlog method xlog_fallocate() that makes
sure that the requested amount of disk space is available at the current
write position. It does that with posix_fallocate(). The new method is
called before writing anything to WAL, see wal_fallocate(). In order not
to invoke the system call too often, wal_fallocate() allocates disk
space in big chunks (1 MB).

The reason why I'm doing this is that I want to have a single and
clearly defined point in the code to handle ENOSPC errors, where
I could delete old WALs and retry (this is what #3397 is about).

Needed for #3397

sync_file_range is declared only if _GNU_SOURCE macro is defined.
Also, in order to be used in a source file, HAVE_SYNC_FILE_RANGE
must be present in config.h.cmake.

Fixes commit caae99e5 ("Refactor xlog writer").

A possibility to build tarantool with included library dependencies.
Use the flag -DBUILD_STATIC=ON to build statically against curl, readline,
ncurses, icu and z.
Use the flag -DOPENSSL_USE_STATIC_LIBS=ON to build with static
openssl

Changes:
  * Add FindOpenSSL.cmake because some distributions do not support the use of
  openssl static libraries.
  * Find libssl before curl because of build dependency.
  * Catch all bundled libraries API and export then it in case of static
  build.
  * Rename crc32 internal functions to avoid a name clash with linked libraries.

Notes:
  * Bundled libyaml is not properly exported, use the system one.
  * Dockerfile to build static with docker is included

Fixes #3445

Fixing build under FreeBSD:
Undefined symbol "iconv_open"
Add compile time build check with FindICONV.cmake
and a wrapper to import relevant symbol names with include file.

Closes #3441

The option is '-DENABLE_BUNDLED_ZSTD' and defaults to ON.

There are two goals of making this conditional, both related to building
Tarantool on Gentoo Linux from an ebuild:

* Avoid bundled ZStd building issue w/o pay to investivage it.
* Allow user to choose between system and bundled library.

Fix misleading "C atomics not supported" when git submodules
are missing.

Closes #2088

getopt_long() is available on all supported platforms.
Get rid of legacy gopt and use getopt_long().

Incompatible changes:

 * `tarantool --version --no-such-option` printed
    "unrecognized option '--no-such-option'", now it displays version.
    `tarantool --no-such-option --version` still prints an error message.

Needed for #1265

Add an option to enable LuaJIT's GC64 mode and enable this option
by default on OS X.

Closes #2643

Remove libbfd from dependencies and use libunwind for stack traces.

Closes #2103

Inpspired by tarantool/curl module by Vasiliy Soshnikov.
Reviewed and refactored by Roman Tsisyk.

Closes #2083

This patch removes following source files regeneration:
  1. keywordhash.h
  2. opcodes.[hc]
  3. parse.[hc]
  4. sqlite3.h
adding them to the source tree.

To re-generate them (p.p. 1-3) one must pass
-DSQL_MAINTAINER_MODE=ON option to cmake. Files will be
copied to source tree automatically iif actual difference will
be detected.

As far we don't need such platform variety - remove sqlite3.h
re-generation at all.
Remove TCL script responsible for generation and template header
as well.

Compilation of auxilary SQL tools was also moved under this option:
  1. Lemon parser
  2. mkkeywordhash
since they redundant in non-maintainer mode.

Finally it removes dependency on TCL from all Tarantool package
variants.

Remove sqliteConfig.h, we don't use it.

Set working directory to root of source tree while doing
`git modules update`.

* do not include autotools build files

sql: [#2387] [#2267] Cleanup unused SQLite fines.

sql: Checkin SQLite test coverage

sql: Remove TCL-based tests

Remove sqlite-tcl testsuite along with all TCL-relared libs.
Clean up sqlite's CMakeLists and remove redundant TCL-related
sources.

	* src/lib/sqlite/CMakeLists.txt: Remove dependency on TCL library.
	* src/lib/sqlite/src/CMakeLists.txt: Remove testfixture target.
	* src/lib/sqlite/src/test.*: Remove.
	* src/lib/sqlite/src/sqlite3.rc: Ditto.
	* src/lib/sqlite/src/tclsqlite.c: Ditto.
	* src/lib/sqlite/ext: Ditto.
	* test/sqlite-tcl: Ditto.

Add -o option in lemon (output file name)

This is necessary for out-of-source CMake builds.

Use dummy commit date and UUID in sqlite3.h

Last commit date and UUID are included in generated sqlite3.h. We don't
distribute standalone sqlite, and Tarantool itself is already
version-stamped.

sqlite: Add VERSION
Implement CMake build rules for sqlite.

Sophia was an experimental storage engine in 1.6.x.
Please use 'vinyl' engine in Tarantool 1.7 instead.

Closes #2040

We don't need these functions anymore.

* Check for missing cpuid.h header
* Fix sed magic in extra/mkexports
* Fix open_memstream implementation
* Fix CLOCK_PROCESS_CPUTIME_ID / CLOCK_THREAD_CPUTIME_ID
* Disable stupid -Wchar-subscripts warnings
* Add a workaround for "undefined __gcc_personality_v0"

Tested on NetBSD 7 and on x86_64-rumprun-netbsd.
Now you can use Tarantool on toasters.

See #1874

This patch is not perfect, but it is better than nothing.

+ Remove third_party/valgrind and third_party/pmatomic which
  also exist in src/lib/small/third_party.

* Write xlog via fd instead of fiob with FILE * interface

* Remove O_DIRECT mode, use fdatasync(2)/sync_file_range(2) with
  posix_fadvise(2) to free the page cache. Disk cache polution
  on snapshot was tested with `vmtouch` utility and fadvise()
  provides similar result to O_DIRECT.

* Remove fiob.c implementation (unused)

Make sure llvm-symbolizer is available. If not, reports will contain raw
addresses instead of line info.

Since leak suppressions aren't ready yet, we suggest setting
ASAN_OPTIONS=detect_leaks=0 environment variable when running tests.

Regression by d36ba279

Needed for #1689

Fixes #810

Fixes #1617