This directory belongs to the EE repository.

Also, delete test/enterprise-tap because we don't have any TAP tests in
the EE repository.

Follow-up commit d8faca69 ("lua: implement stubs for etcd client").

NO_DOC=gitignore
NO_TEST=gitignore
NO_CHANGELOG=gitignore

Patch made following changes:
- add a CMake function lua_multi_source that unpacks symbol names and
  Lua source paths in passed list and compiles Lua source into *.lua.c
  sources using lua_source() function. Function is needed for embedding
  Lua modules consisting from a number of files to Tarantool EE.
- add using lua_multi_source instead of lua_source for etcd client module.
- remove function prototypes and declarations related to extra features.
  All this code is moved to Tarantool EE source code.

Part of #8620
Follows up https://github.com/tarantool/tarantool-ee/issues/443

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

Fefactor lua name resolution, simplify and comment.
Add an ability to specify path with brackets, for example in
'box.space[512]:get' or 'box.space["test"]:get'.
Only literals (strings and numbers) are supported.

Closes #8604

@TarantoolBot document
Title: square brackets in procedure resolution for Lua calls

Square brackets are now supported in Lua call procedure resolution. This is
applicable to `net.box` connection objects `call` method as well as
`box.schema.func.call`.

Examples of function calls with square brackets can be found in the test to
this patch.

Default token for actions provided by GitHub lacks
read and write permissions for private projects. This
commit changes the token used for the stale PRs workflow
to the dedicated token with needed permissions.

NO_DOC=Workflow fix
NO_TEST=Workflow fix
NO_CHANGELOG=Workflow fix

Sometimes we need negative timestamps to work with dates before
1970. But seems such cases were even covered in tests. So there
wasn't any handling of negative timestamps with fraction part.
Such datetime objects had incorrect string representation (e.g.
"1963-11-22T12:30:02.-999"). This patch fixes it.

Closes #8570

NO_DOC=bugfix

Seems that problem code part was ported from Lua as is. But there
is some difference between modulo operator in C and in Lua. Lua
always returns positive value but in C result could be negavive.
This difference led to the case when after subtraction nsec part
of datetime object become negative that yielded weird result on
attempt to get string representation (e.g."2008-02-03T03:36:43.-100Z").
This patch fixes it.

Part of #8570

NO_DOC=bugfix
NO_CHANGELOG=see next commit

Currently, to export IPROTO constants to Lua, we define a translation
table in C manually. As a result, some constants are forgotten and some
are exported with mistakes.

Fortunately, we have a mechanism to generate enums and its stringified
names in compile-time. Let's rewrite iproto constants using such mechanism
and generate translation table automatically.

Closes #8443
Closes #8574
Closes #8577

NO_DOC=bugfix

We are going to generate iproto_key enum automatically to generate iproto
constants for Lua as well. This mechanism generates *_MAX constant for enum
using its name, which is in lower case. So let's convert IPROTO_KEY_MAX to
form which is appropriate for enum generator.

Part of #8443

NO_TEST=rename constant
NO_CHANGELOG=internal
NO_DOC=internal

This commit adds another possible bootstrap_strategy to accompany "auto"
and "config": "supervised".

Such a strategy may be useful to pin the desired bootstrap leader on an
active cluster (so that the user may join the replicas from the desired
node without changing their box.cfg) or to manually set the bootstrap
leader among the nodes that managed to start without issues.

More details are in the docbot request.

Closes #8509

@TarantoolBot document
Title: new bootstrap strategy - "supervised"

The `bootstrap_strategy` configuration option may now be set to
"supervised".

This strategy works as follows:
When bootstrapping a new replicaset, the nodes do not choose a bootstrap
leader automatically and instead wait for it to be appointed by the
user. The configuration will fail if no bootstrap leader is appointed
during a `replication_connect_timeout`.

In order to apppoint a bootstrap leader, the user has to issue
`box.ctl.make_bootstrap_leader()` call on the desired node. This must be
done during the initial `box.cfg()` call.

Possible ways to achieve this are:

1. In interactive mode:
```lua
fiber.create(box.cfg, desired_config)
box.ctl.make_bootstrap_leader()
```
2. Via an init script:
```lua
-- Init script:
console.listen(admin_port)
box.cfg(desired_config)
-- User console:
tarantoolctl enter admin port
> box.ctl.make_bootstrap_leader()
```

When joining a new replica with `bootstrap_strategy` = "supervised" to
an existing replica set, the replica will not choose the bootstrap
leader automatically, but will instead join to the node on which
`box.ctl.make_bootstrap_leader()` was issued last. In case such a node
isn't found after a `replication_connect_timeout`, the configuration
fails.

Both triggers are used to wait for specific applier state (applier is
connected, remote node ballot is known, etc.) and are often used one
after another. Keeping these two trigger lists only adds complexity of
creating 2 triggers instead of 1 in some places. Let's unify the
triggers.

Now the triggers might be fired multiple times, so make all the existing
trigger functions reenterable.

In-scope-of #8509

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

The idea is to simplify applier_on_connect_f as much as possible, so
that it only wakes the waiter up, but doesn't participate in any
decisions regarding the replicaset connection state.

In-scope-of #8509

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Now that the event isn't used to pass the success status, we may pass
applier there for consistency with on_state triggers.

Besides, this allows to simplify applier_ballot_data struct, which
doesn't need to store applier anymore.

In-scope-of #8509

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Ballot watcher used to pass a boolean as an event to ballot_update
triggers, with "false" meaning there was some error while waiting for
ballot and one has to check diag for the error.

This was intended as a guard against stale diag contents (diag might
contain some leftovers from previous errors).

Let's make ballot watcher fiber clear the diagnostics when it's not
planning on using them. Now ballot update triggers may rely on
`diag_is_empty()` test instead of the "success" parameter.

In-scope-of #8509

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

It was used for getting _schema tuple[1] field. Worked because
BOX_CLUSTER_FIELD_UUID is also 1. This commit introduces
BOX_SCHEMA_FIELD_VALUE.

It is going to be used in the next commits more extensively for
new _schema tuples.

Part of #5029

NO_DOC=refactoring
NO_CHANGELOG=refactoring
NO_TEST=already covered

This patch fixes a case when timestamp is passed to datetime.set
function at the same time with nsec, usec or msec.
It works fine for datetime.new but some logic was missed for set
function. Here we fix that and introduce a test.

Closes #8583

NO_DOC=bugfix

With MVCC off (box.cfg.memtx_use_mvcc_engine = false), a memtx space
read view may include a dirty (not committed to WAL) record. To prevent
such records from being written to a snapshot, we sync WAL after
creating a read view for a snapshot. The problem is that it doesn't work
for long (yielding) DDL operations, such as building a new index,
because such operations yield before waiting on WAL. As a result,
a dirty DDL record may make it to a snapshot even though it may fail
eventually. To fix that, let's keep track of all yielding DDL statements
and exclude them from a read view using the memtx snapshot cleaner.

Closes #8530

NO_DOC=bug fix

We don't use this functionality in net.box (sync number is always 0 for
all watch/event packets), but other clients may actually use it.

Closes #8393

@TarantoolBot document
Title: Document that `IPROTO_EVENT` has sync number

Initially the sync number sent by a client in an `IPROTO_WATCH` request
was ignored and `IPROTO_EVENT` packet didn't have a sync number. There
were complaints about it from users so we consider this to be a bug.
Now the server sends the same sync number in an `IPROTO_EVENT` packet
as the one sent by the client in the last corresponding `IPROTO_WATCH`
request.

Closes #8636

NO_TEST=<dropped functionality>

@TarantoolBot document
Title: Remove mentions about `IPROTO_CALL16` from `net.box`

Support for `IPROTO_CALL16` was dropped: remove mentions about `call_16`
option for `net.box` connections.

Add necessary wait for replication to appear on the replica, enforce
correct txn isolation to avoid memtx/vinyl discrepancy.
Remove the test from the fragile list.

Closes tarantool/tarantool-qa#292

NO_DOC=test fix
NO_CHANGELOG=test fix

The setup-python action defaults to searching for
a dependency file (requirements.txt or pyproject.toml
for pip, Pipfile.lock for pipenv or poetry.lock for
poetry) in the repository, and uses its hash as a
part of the cache key. We don't have one, so the
caching option is disabled.

NO_DOC=Workflow fix
NO_TEST=Workflow fix
NO_CHANGELOG=Workflow fix

There was a typo in the commit 55295f5f ("box: populate
index_object.parts with key_def module methods"):
the function luaT_push_key_def() was erroneously used instead
of luaT_push_key_def_nodup().

Follow-up #7356

NO_DOC=bug fix
NO_TEST=memory leak
NO_CHANGELOG=unreleased

According to libFuzzer documentation [1] backslash should be escaped.

1. https://llvm.org/docs/LibFuzzer.html#dictionaries

```
$ swim_proto_meta_fuzzer -dict=swim_proto_meta_fuzzer.dict
ParseDictionaryFile: error in line 1
                "\001\000\000\004"
$ swim_proto_member_fuzzer -dict=swim_proto_member_fuzzer.dict
ParseDictionaryFile: error in line 1
                "\022\000\000\000\000\000\000\000"
```

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

This patch removes the 'size' argument from macros, as it was only used
to set an error on failure, which is not possible for x* versions. In
addition, both macros now cast the value to the specified type, as is
done in the original macros.

Closes #8522

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

This patch adds a Python script, that uses GitHub's GraphQL
API to provide automatic last update date for PR items on
any specified GitHub ProjectsV2 board.

Since there is no versatile enough  way to automate a
GitHub ProjectV2 by its own means, that is done via
standard mechanism of on schedule workflow, which runs
the script every hour.

NO_DOC=workflow
NO_TEST=workflow
NO_CHANGELOG=workflow

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

Synced with description in the README updated in #5679.

Show manual page: pod2man doc/man/tarantool.pod | mandoc -a

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

Follows up #7456
Follows up #5541

NO_CHANGELOG=doc
NO_DOC=doc
NO_TEST=doc

All box.cfg options values used to be plain values or arrays. Now some
options contain key-value or nested tables.

This patch allows all such options to be set through environment
variables too.

Closes #8494
Closes #8051

@TarantoolBot document
Title: Set box.cfg table value via env vars

This patch implements a way to set tables as box.cfg options value in
two different ways:
* as plain key-value table:
```
bash> export TT_LOG_MODULES=aaa=info,bbb=error && ./tarantool
Tarantool 2.10.0-beta1-1977-g2970bd57a
type 'help' for interactive help
tarantool> box.cfg{}
...
2023-04-11 07:22:10.951 [219020] main/103/interactive/box.load_cfg I> set \
'log_modules' configuration option to {"aaa":"info","bbb":"error"}
---
...

tarantool> box.cfg.log_modules.aaa
---
- info
...

tarantool> box.cfg.log_modules.bbb
---
- error
...
```

* as a table in json encoding (important: don't forget to put env var
  value into single quotes):
NO_WRAP
```
bash> export TT_METRICS='{"labels":{"alias":"mystorage"},"include":"all","exclude":["vinyl"]}' && ./tarantool
Tarantool 2.10.0-beta1-1977-g2970bd57a
type 'help' for interactive help
tarantool> box.cfg{}
...
2023-04-11 07:26:03.635 [219288] main/103/interactive/box.load_cfg I> set \
'metrics' configuration option to {"exclude":["vinyl"],"include":"all",\
"labels":{"alias":"mystorage"}}
---
...

tarantool> box.cfg.metrics.include
---
- all
...
```
NO_WRAP

Response body decoding of DML and call/eval requests is very ad-hoc and
hard to extend: introduce a new `response_body_decode` helper that decodes
the response body similarly to `xrow_decode_dml` — this will allow to
separate decoding from processing.

Needed for #8147

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

An autoconf-generated configure script doesn't enable compiler
optimization flags if CFLAGS / CXXFLAGS options are set explicitly.
We started setting CFLAGS / CXXFLAGS in commit e6abe1c9
("cmake: add extra security compiler options"). As a result, users
started experiencing performance degradation issues, like the one
described in tarantool/tarantool-ee#440.

Let's set -O2 in CFLAGS / CXXFLAGS explicitly to fix that.

Closes #8606
Needed for tarantool/tarantool-ee#440

NO_DOC=build
NO_TEST=build

To enable etcd client set the following CMake variables:

 - ENABLE_ETCD_CLIENT: ON
 - ETCD_CLIENT_DIR: path to a directory with etcd client Lua source
   files

If ENABLE_ETCD_CLIENT is unset, then a stub implementation is built,
which spits an error on attempt require etcd-client.

Needed for https://github.com/tarantool/tarantool-ee/issues/443

MSG="stubs for enterprise version, no user-visible changes"

NO_CHANGELOG=MSG
NO_DOC=MSG
NO_TEST=MSG

The test flaked a lot on FreeBSD and Mac OS due to some proxy problems
which were fixed in scope of tarantool/luatest@5b704ac.

Another possible failure looked like this:

NO_WRAP
[006] not ok 5  linearizable-read.test_leader_change
[006] #   Can't modify data on a read-only instance - it is an orphan
[006] #   stack traceback:
[006] #         ...tarantool/test/replication-luatest/linearizable_test.lua:196: in function 'linearizable-read.test_leader_change'
[006] #         ...
[006] #         [C]: in function 'xpcall'
NO_WRAP

Fix it by waiting for the server to follow its upstreams.

Closes tarantool/tarantool-qa#277

NO_DOC=test fix
NO_CHANGELOG=test fix

In case box.cfg.replication isn't update by box.cfg, we don't try to
reestablish replication connections, but we still need to reload URIs
(recreate IO stream contexts created from URIs) because a URI parameter
may store a path to an SSL certificate file, which could change. This
was done in commit 655290aa ("replication: always reload URIs on
reconfiguration").

There's a bug in the commit: it works as expected for appliers that
already connected in the past and received UUID but it ignores anonymous
appliers, i.e. appliers that never connected, as a result the following
code fails:

  -- Configure replication using a wrong SSL certificate.
  -- This leaves anonymous appliers reconnecting in the background.
  box.cfg{replication = {...}}
  -- Update the SSL certificate file and reload replication URIs.
  -- At this point, the anonymous applier should finally connect.
  box.cfg{replication = box.cfg.replication}

Needed for tarantool/tarantool-ee#432

NO_TEST=ee
NO_DOC=bug fix
NO_CHANGELOG=unreleased

The netbox_transport_send_and_recv() function takes a part of the
response size as a parameter, and it is possible that the resulting size
could become larger than SIZE_MAX, resulting in an integer overflow and
a segmentation fault.

Closes #tarantool/security#121

NO_DOC=bugfix

Currently on applier death `is_candidate` is updated after trying
to start election. So, raft assumes it has healthy quorum and
bumps term even when there's not enough healthy nodes to do that.

Trigger on updating above-mentioned flag is run in
`replicaset_on_health_change`. So, let's move it before executing
`raft_notify_is_leader_seen`, which tries to start election.

Closes #8433

NO_DOC=bugfix

This patch reworks the parsing of statements associated with
transactions. There are six of them:
START TRANSACTION;
COMMIT;
ROLLBACK;
SAVEPOINT savepoint_name;
RELEASE [SAVEPOINT] savepoint_name;
ROLLBACK TO savepoint_name;

Prior to this patch, VDBE for these statements were generated while the
statement was in the process of being parsed. After this patch, VDBE for
all these statements will be generated only after the parsing has
successfully completed.

Part of #5485

NO_DOC=Will be added later
NO_TEST=No user-visible changes are made
NO_CHANGELOG=Will be added later

Even if the new value of box.cfg.listen is the same as the old one, we
still need to recreate evio service entries' IO stream contexts from the
URIs on reconfiguration, because a URI parameter may store a path to a
file (for example, an SSL certificate), which could change.

Note that we have to rework the way how IO stream contexts are handled
on evio_service_attach. Before this patch, we copied struct iostream_ctx
directly from the source service to the attached service. It was fine,
because (a) we didn't destroy iostream_ctx on detach (we only cleared
it) and (b) we detached IPROTO threads' services from tx_binary before
stopping tx_binary. Now, to reload URIs, we first call the new function
evio_service_reload_uris on tx_binary to recreate iostream_ctx, and only
then reattach IPROTO threads' services, which opens a window for an
IPROTO thread's service to access a destroyed iostream_ctx context while
accepting a new connection. To avoid that, we now copy iostream_ctx to
attached services with the new function iostream_ctx_copy. This function
copies the underlying SSL_CTX with a stub function ssl_iostream_ctx_dup,
which will be implemented in the EE repository using SSL_CTX_up_ref
(i.e. no actual copying will happen - we'll just increment the internal
SSL_CTX reference counter).

Needed for tarantool/tarantool-ee#432

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

Even if the new value of box.cfg.replication is the same as the old one,
we still need to recreate appliers' IO stream contexts from the URIs on
reconfiguration, because a URI parameter may store a path to a file (for
example, an SSL certificate), which could change.

Needed for tarantool/tarantool-ee#432

NO_DOC=ee
NO_TEST=ee
NO_CHANGELOG=ee

If a configuration option value passed to `box.cfg` is the same as the
old one, the option handler isn't called. However, sometimes we might
need to do some extra work in the handler in this case. For example,
even if listen/replication URI is the same, we still need to recreate
the derived IO stream context, because a URI parameter may contain a
path to a file (for example, an SSL certificate), which could change.

So we remove the corresponding check from load_cfg.lua and instead move
it to option handlers. Most of the existing dynamic option handlers
already do nothing if the old and the new values are the same. There are
only three exceptions:

 1. box.cfg.memtx_memory. The problem is that the value is rounded up
    to a multiple of the quota unit size by the handler so even if we
    pass exactly the same value, it may mistakenly assume that the new
    value is less and fail. To avoid that, we make the handler round up
    the new value before checking it.

 2. box.cfg.replication. Even though commit 5994892c ("replication:
    fix replica disconnect upon reconfiguration") made the handler avoid
    disconnect, it still tries to connect to the new URIs and sync,
    which may pose a problem to our users. For example, resetting
    box.cfg.replication to the same value on the leader after right
    after a new cluster bootstrap (before granting access to replicas)
    would lead to a replication sync, which would obviously fail on
    timeout. To avoid that, we make the handler compare the new and old
    URIs internally and do nothing if they are the same.

 3. box.cfg.listen. Handled similarly to box.cfg.replication: to avoid
    futile rebinding of IPROTO ports, we store the last configured
    listen URIs in a C variable and do nothing on box.cfg if the
    configuration is the same.

Needed for tarantool/tarantool-ee#432

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

 - Call listen() once per socket in the TX thread instead of doing it in
   each IPROTO thread because calling listen() more than once has no
   effect.
 - Merge evio_service_bind and evio_service_listen into one method
   named evio_service_start to match evio_service_stop.
 - Rename IPROTO_CFG_LISTEN message to IPROTO_CFG_START to match
   IPROTO_CFG_STOP and drop the binary argument because it always
   attaches to tx_binary.
 - Drop the dead evio_service_is_active check from IPROTO_CFG_START
   because the service must be inactive at this point and we have an
   assertion for that in evio_service_attach.
 - Move ERRINJ_IPROTO_CFG_LISTEN error injection to iproto_listen and
   drop the error code of iproto_send_start_msg because it should never
   fail by design.

Needed for tarantool/tarantool-ee#432

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring