`datetime` module has a function `:totable()` that converts
the information from a datetime object into the table format.
The field `timestamp` can be accessed in a datetime object:

```
tarantool> dt = datetime.new({timestamp=100})
---
...

tarantool> dt.timestamp
---
- 100
...
```

However, the timestamp is missed in a table produced by
`:totable()` method. The patch fixes that.

Fixes #10374

@TarantoolBot document
Title: Support of timestamp field in :totable()

The field `timestamp` is available in a table produced by
`:totable`:

```
tarantool> datetime.new({timestamp=100}):totable()
---
- timestamp: 100
  tz:
  sec: 40
  min: 1
  yday: 1
  day: 1
  nsec: 0
  isdst: false
  wday: 5
  tzoffset: 0
  month: 1
  year: 1970
  hour: 0
...

tarantool>
```

(cherry picked from commit f66020b670296642547cbfe0a34ab6c11d69989d)

There is no requirement for using fractional numbers
in datetime constructors. This issue will be addressed in [1].
The patch removes testcase with fractional numbers in datetime
constructor.

1. https://github.com/tarantool/tarantool/issues/10391

See also #10391
Needed for #10374

NO_CHANGELOG=testing
NO_DOC=testing

(cherry picked from commit 989c63c75f5f8c7b38dc67f8e5c32ce8bef1da08)

The commit 3c403661 ("datetime, lua: date parsing functions")
introduced a function `datetime.parse()`. The function accepts
a datetime format: 'iso8601', 'rfc3339' and a custom datetime
format. However, 'iso8601' as well as 'rfc3339' is a set of
datetime formats (see [1] and [2]), and `datetime.parse()`
supports a subset of these formats. There is a documentation
issue [3], where we should describe what formats are supported
and what formats are not.

The patch adds a tests for most of supported formats.
The tests were created using a page about date and time formats
in ISO 8601 and RFC 3339 standards [2] and these tests are not
exhaustive. More thorough testing can be done with property-based
testing and randomization. It will be a subject of the following
commits.

1. https://datatracker.ietf.org/doc/html/rfc3339
2. https://ijmacd.github.io/rfc3339-iso8601/
3. https://github.com/tarantool/doc/issues/3155



Follows up #6731
Needed for tarantool/doc#3155

Co-authored-by: Sergey Kaplun <skaplun@tarantool.org>

NO_CHANGELOG=testing
NO_DOC=tarantool/doc#3155

(cherry picked from commit dcb196366621db4ab39ef97029ee126d836b77fb)

`datetime` module has a function `:totable()` that converts
the information from a datetime object into the table format.
The commit 43e10ed3 ("build, lua: built-in module datetime")
added `tzoffset` field to the datetime object and to table
produced by `:totable()`. The commit 9ee45289
("datetime: datetime.TZ array") added fields `tz` and `tzindex` to
the datetime object, but not to the table produced by
`:totable()`. The patch fixes that. Note, `tzindex` is not added,
because it is an internal field.

```
tarantool> datetime.parse('2004-12-01T00:00 Europe/Moscow'):totable()
---
- tz: Europe/Moscow
  sec: 0
  min: 0
  yday: 336
  day: 1
  nsec: 0
  isdst: false
  wday: 4
  tzoffset: 180
  month: 12
  year: 2004
  hour: 0
...
```

Fixes #10331
Follows up #6751

@TarantoolBot document
Title: Support of tz field in :totable()

In addition to the `tzoffset` in a table produced by `:totable`
we added `tz` field.

```
tarantool> datetime.parse('2004-12-01T00:00 Europe/Moscow'):totable()
---
- tz: Europe/Moscow
  sec: 0
  min: 0
  yday: 336
  day: 1
  nsec: 0
  isdst: false
  wday: 4
  tzoffset: 180
  month: 12
  year: 2004
  hour: 0
...
```

(cherry picked from commit 90552e55e0921405c43ea086ae418a72c9f000e4)

This patch fixes a bug found by the ASAN instrumentation of LuaJIT
allocator [1]. The problem is using a Lua serializer object that has
been cleaned up by GC.

The crash occurs when executing `tarantool> \set output lua`.

Failing tests:
 - ./test/app-luatest/gh_7031_configure_eos_in_lua_console_test.lua
 - ./test/app-tap/console.test.lua
 - ./test/box/push.test.lua
 - ./app-tap/console_lua.test.lua
 - ./app-luatest/varbinary_test.lua

The `serializer_lua` static member is removed from
`src/box/lua/serialize_lua.c` along with `serializer_lua_init`,
because it is not used by any function other than `serializer_lua_init`,
which is not needed now too.

[1]: Issue #10231

Closes #10177 (this issue is a duplicate of #7404)

NO_TEST=rely on existing tests
NO_CHANGELOG=codehealth
NO_DOC=codehealth

(cherry picked from commit 830f27bbaf94dadcacc66a7f543e0e5feec3988d)

To ensure better encapsulation, maintainability, and portability of
the code, it is necessary to replace direct access to the fields of
global structures with calls using the Lua C API.

Closes #10284

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit f7bb3fc7a1222107753dc420675ecda043b0a5d2)

Since commit 8f4be322 ("txm: disallow yields after DDL operation in
TX"), any DDL operation aborts **all** active transactions, even those
that wouldn't be affected by it anyway, see `memtx_engine_prepare()`,
`memtx_tx_abort_all_for_ddl()`. Actually, there's no need to do that in
Vinyl because it properly handles concurrent DDL operations, see commit
d3e12369 ("vinyl: abort affected transactions when space is removed
from cache"). Let's skip Vinyl transactions from consideration by
marking the Vinyl engine with a special flag.

Closes #10375

NO_DOC=bug fix

(cherry picked from commit f5f061d051dc6268949bfcb141d211142282578d)

CMake 3.29.0 produces a warning on configuration stage:

NO_WRAP
 | CMake Warning (dev) in cmake/ProtobufMutator.cmake:
 |  A logical block opening on the line
 |    /home/sergeyb/sources/MRG/tarantool/cmake/ProtobufMutator.cmake:38 (if)
 |   closes on the line
 |    /home/sergeyb/sources/MRG/tarantool/cmake/ProtobufMutator.cmake:40 (endif)
 |  with mis-matching arguments.
NO_WRAP

The patch fixes the warning.

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

(cherry picked from commit 1e9e70f4c55c74c94922cb0beef44ae2b82255a9)

Protocol Buffers library has a name `protobufd` when CMAKE_BUILD_TYPE is
equal to "Debug". In other cases the name is `protobuf`. The patch fixes
the library name.

Follows up commit b11072a6 ("cmake: add dependencies for LuaJIT and
SQL fuzzers").

Follows up #4823

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

(cherry picked from commit c0d8cb54)

An attempt to build Tarantool statically by Ninja (with the
enabled CMake option BUILD_STATIC_WITH_BUNDLED_LIBS), error below
is produced:

 | ninja: error: build.ninja:1405: bad $-escape
 | (literal $ must be written as $$)

The commit fixes that error. Follows up commit c92a1699
("cmake: support build using Ninja").

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

(cherry picked from commit d48c40e69495b8cc3475ea0b184768054d87b780)

CMake 3.24+ produces a warning about the missed option
DOWNLOAD_EXTRACT_TIMESTAMP in `ExternalProject_Add()` [1]:

 | The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy
 | CMP0135 is not set.

Documentation about policy CMP0135 [2] said:

CMake 3.23 and below set the timestamps of the extracted contents
to the same as the timestamps in the archive. When the URL
changes, the new archive is downloaded and extracted, but the
timestamps of the extracted contents might not be newer than the
previous contents. Anything that depends on the extracted contents
might not be rebuilt, even though the contents may change.

CMake 3.24 and above prefer to set the timestamps of all
extracted contents to the time of the extraction. This ensures
that anything that depends on the extracted contents will be
rebuilt whenever the URL changes.

1. https://cmake.org/cmake/help/latest/module/ExternalProject.html#url
2. https://cmake.org/cmake/help/latest/policy/CMP0135.html

NO_CHANGELOG=build
NO_DOC=build
NO_TEST=build

(cherry picked from commit 4c0e08d85cd39916818bc4b915ac3acdacce8fe4)

datetime object can be created using `datetime.new()` and
`datetime.parse()`. `datetime.new()` sets values of unspecified
fields to appropriate values of fields in a Unix time
(00:00:00 UTC on 1 January 1971, Thursday), see commit
43e10ed3 ("build, lua: built-in module datetime"):

```
tarantool> datetime.new()
---
- 1970-01-01T00:00:00Z
...

tarantool> datetime.new():totable()
---
- sec: 0
  min: 0
  yday: 1
  day: 1
  nsec: 0
  isdst: false
  wday: 5
  tzoffset: 0
  month: 1
  year: 1970
  hour: 0
```

The function `datetime.parse` converts an input string with the
date and time information into a datetime object. When fields are
not specified in a input string their values becomes undefined:

```
tarantool> dt.parse('01:01:01', {format ='%H:%M:%S'}):totable()
---
- sec: -59
  min: -58
  yday: 366
  day: 31
  nsec: 0
  isdst: false
  wday: 1
  tzoffset: 0
  month: 12
  year: 0
  hour: -22
...

tarantool>
```

The commit fixes aforementioned behaviour of `datetime.parse()`
by setting values for fields not specified by user to values of
appropriate fields of Unix time.

NO_DOC=bugfix

Fixes #8588

(cherry picked from commit 9ac56a1222e916951010091fbdbfc497f4a4971d)

The function `datetime_gmtoff()` has been introduced in commit
aec6fbac ("datetime: implement date.isdst") for a function
`datetime_increment_by()`. In commit 6ca07285
("datetime: fix interval arithmetic for DST") the function
`datetime_increment_by()` has been updated and call of the
function `datetime_gmtoff()` was removed. `datetime_gmtoff` is
dead code now. The patch removes `datetime_gmtoff()`.

NO_CHANGELOG=codehealth
NO_DOC=codehealth
NO_TEST=codehealth

(cherry picked from commit 625afe418151de1015483ed24acaa57e1f85efb3)

The patch fixes a number of typos in datetime source code.

NO_CHANGELOG=codehealth
NO_DOC=codehealth
NO_TEST=codehealth

(cherry picked from commit c9c5b9f139ccbf5372d1568827fbb50bec7239bb)

Since tarantool/luajit@a16313f large exponent double strings are not
considered convertible to number. It broke encoding lua objects to YAML
because single quotes weren't considered necessary for decoding.

This commit adds wrapping of every string containing infinite double values
into a single quotes.

Closes #10164

NO_DOC=bug fix

(cherry picked from commit 7c3f42590240525d2e543305b6c289ddb30054a2)

This patch adds encapsulation for IPv6 addresses in brackets when
calling uri.format (as per RFC 2732).

Closes #9556

NO_DOC=bugfix

(cherry picked from commit a49ec23b85edb684744eb525427465dfa4f660e1)

Before this patch `uri.parse(<uri-string-with-ipv6-addres>)`
did not work correctly. In particular, it did not parse an
IPv6 address if it contained `A-F`. It is a regression caused
by commit 1376aad9 ("Refactor src/uri.rl to support RFC3986
and add Lua bindings").

This patch fixes a bug where characters `A-F` are not
supported in IPv6.

Part of #9556

NO_DOC=bugfix

(cherry picked from commit 2eefc56a3e08a60f0b71e33621be851794131546)

The motivation is to exclude the file from diffs on GitHub's web
interface [1] and to exclude it from checkpatch checks [2].

[1]: https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github
[2]: https://github.com/tarantool/checkpatch/pull/75

NO_DOC=the patch is about the development process, it doesn't change
       anything in the shipped product
NO_CHANGELOG=see NO_DOC
NO_TEST=see NO_DOC

(cherry picked from commit fbea9862b087e161a48dc0ac181a1da06cc0ce9d)

When checkpoint fails, we abort it in all engines even if it wasn't
started successfully. If it fails right from the start so that checkpoint
in memtx wasn't started, assertion in `memtx_engine_abort_checkpoint`
fails - memtx doesn't expect that checkpoint will be aborted if it
failed to start. Let's do the same thing as vinyl does - no-op if
there is no checkpoint in progress.

Closes #10265

NO_CHANGELOG=reproducible only with error injection
NO_DOC=bugfix

(cherry picked from commit 6b484622259c01a2468b1f248dd6f1bcdc227021)

We've had a number of issues when Tarantool was permanently broken
(unable to recover after restart) because of a bad vylog record.
The `force_recovery` mode didn't help so the user would have no other
choice but to rebootstrap. A funny thing is those bugs were usually
caused by a race between the garbage collector and dump/compaction when
a vylog record was written for a dropped index. The worst thing that
could happen if we ignored such a bad record is an unused run file not
deleted from disk. Apparently, this is better than a permanent recovery
failure so let's support the `force_recovery` mode in vylog.

The tricky part here is handling checkpoint after restart. The problem
is that to create a vylog checkpoint, we load the previous vylog file
so we have to ignore errors if it was loaded in the `force_recovery`
mode.

Closes #10292

NO_DOC=bug fix

(cherry picked from commit c68e8a8e029d849d68c6018ed00b5a79cc769222)

The vinyl metadata log processor allocates its internal objects either
from malloc or region, neither of which should fail for small
allocations. Let's switch to xalloc to simplify the code. A good thing
about this change is that now we can ignore all errors raised by
vy_log_record_decode() and vy_recovery_process_record() if the
force_recovery flag is set (see the next commit).

Needed for #10292

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit b6f015e98b7c30861dc33032ba7eca47de0cc198)

Temporary allocations from a region don't fail so let's use xalloc to
simplify the code.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 28d51f8076b390c50d5b18f8e767c1eb540e5dcc)

Let's log the new value when an error injection is set in orer to ease
debugging in tests.

NO_DOC=logging
NO_TEST=logging
NO_CHANGELOG=logging

(cherry picked from commit 019bacbe)

There is no much sense in testing it, but it is sensitive to source code
changes, especially `ERRINJ_*_COUNTDOWN` injections, e.g. see commit
697123d0 ("box: use maximal space id instead of _schema.max_id").

Needed for tarantool/tarantool-ee#712

NO_DOC=test
NO_CHANGELOG=test

(cherry picked from commit dc0fd81c)

An index can be dropped while a memory dump is in progress. If the vinyl
garbage collector happens to delete the index from the vylog by the time
the memory dump completes, the dump will log an entry for a deleted
index, resulting in an error next time we try to recover the vylog,
like:

```
ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Run 2 committed after deletion
```

or

```
ER_INVALID_VYLOG_FILE: Invalid VYLOG file: Deleted range 9 has run slices
```

We already fixed a similar issue with compaction in commit 29e2931c
("vinyl: fix race between compaction and gc of dropped LSM"). Let's fix
this one in exactly the same way: discard the new run without logging it
to the vylog on a memory dump completion if the index was dropped while
the dump was in progress.

Closes #10277

NO_DOC=bug fix

(cherry picked from commit ae6a02eb)

The tuple formats table may be accessed with `tuple_format_by_id()` from
any thread, not just tx. For example, it's accessed by a vinyl writer
thread when it deletes a tuple. If a thread happens to access the table
while it's being reallocated by tx, see `tuple_format_register()`,
the accessing thread may crash with a use-after-free or NULL pointer
dereference bug, like the one below:

```
 # 1  0x64bd45c09e22 in crash_signal_cb+162
 # 2  0x76ce74e45320 in __sigaction+80
 # 3  0x64bd45ab070c in vy_run_writer_append_stmt+700
 # 4  0x64bd45ada32a in vy_task_write_run+234
 # 5  0x64bd45ad84fe in vy_task_f+46
 # 6  0x64bd45a4aba0 in fiber_cxx_invoke(int (*)(__va_list_tag*), __va_list_tag*)+16
 # 7  0x64bd45c13e66 in fiber_loop+70
 # 8  0x64bd45e83b9c in coro_init+76
```

To avoid that, let's make the tuple formats table statically allocated.
This shouldn't increase actual memory usage because system memory is
allocated lazily, on page fault. The max number of tuple formats isn't
that big (64K) to care about the increase in virtual memory usage.

Closes #10278

NO_DOC=bug fix
NO_TEST=mt race

(cherry picked from commit a2da1de7)

Can use the regular applier_apply_tx(), they do the same. The
latter is just more protective, but doesn't matter much in this
case if the code does a few latch locks.

The patch also drops an old test about double-received row panic
during final join. The logic is that absolutely the same situation
could happen during subscribe, but it was always filtered out by
checking replicaset.applier.vclock and skipping duplicate rows.

There doesn't seem to be a reason why final join must be any
different. It is, after all, same subscribe logic but the received
rows go into replica's initial snapshot instead of xlogs. Now it
even uses the same txn processing function applier_apply_tx().

The patch also moves `replication_skip_conflict` option setting
after bootstrap is finished. In theory, final join could deliver
a conflicting row and it must not be ignored. The problem is that
it can't be reproduced anyhow without illegal error injection
(which would corrupt something in an unrealistic way). But lets
anyway move it below bootstrap for clarity.

Follow-up #10113

NO_DOC=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit da158b9b)

No code besides box.cc can now update instance's vclock
explicitly. That is a protection against hacks like #9916.

Closes #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 19b2cc20)

The goal is to make sure that no files except box.cc can change
instance_vclock_storage directly. That leads to all sorts of hacks
which in turn lead to bugs - #9916 is a good example.

Now applier on final join only sends rows into the journal. The
journal then is handled by box.cc where vclock is properly
updated.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit fe338ed4)

The function writes a single xrow into the journal in a blocking
way. It isn't so simple, so makes sense to keep as a function,
especially given that it will be used more in the next commit.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 7d10096c)

Recovery journal uses word "recovery" to say that it works with
xlogs. For snapshot recovery there is bootstrap_journal. Lets use
it during local snapshot recovery.

The reasoning is that while right now there is no difference, in
next commits the recovery_journal will do more.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 2620eb9e)

Storing vclock of the instance in replicaset.vclock wasn't right.
It wasn't vclock of the whole replicaset. It was local to this
instance. There is no such thing as "replicaset vclock".

The patch moves it to box.h/cc.

Part of #10113

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit f1e8e4e1)

Applier during the registration waiting (for registering a new ID
or a name) could keep doing the master txns received before the
registration was started. They could still be inside WAL doing a
disk write, when the replica sends a register request.

Before this commit, it could cause an assertion failure in debug
and a double LSN error in release.

The reason was that during the registration waiting the applier
treated all incoming txns as "final join" txns. I.e. it wasn't
checking if those txns were already received, but not committed
yet.

During normal subscribe process the appliers (potentially
multiple) protect themselves from that by keeping track of the
vclocks which are already applied and also being applied right now
(replicaset.applier.vclock).

Such protection ensures that receiving same row from 2 appliers
wouldn't result into its double write. It also protects from the
case when a txn was received, goes to WAL, but then the applier
reconnects, resubscribes, and gets the same txn again - it
shouldn't be applied.

The patch makes so that the registration waiting after recovery
works like subscribe. Registration during recovery would mean
bootstrap via join. And outside of recovery it means the instance
is already running.

Closes #9916

NO_DOC=bugfix

(cherry picked from commit 51751f87)

The function `vy_space_build_index`, which builds a new index on DDL,
calls `vy_scheduler_dump` on completion. If there's a checkpoint in
progress, the latter will wait on `vy_scheduler::dump_cond` until
`vy_scheduler::checkpoint_in_progress` is cleared. The problem is
`vy_scheduler_end_checkpoint` doesn't broadcast `dump_cond` when it
clears the flag. Usually, everything works fine because the condition
variable is broadcast on any dump completion, and vinyl checkpoint
implies a dump, but under certain conditions this may lead to a fiber
hang. Let's broadcast `dump_cond` in `vy_scheduler_end_checkpoint`
to be on the safe side.

While we are at it, let's also inject a dump delay to the original
test to make it more robust.

Closes #10267
Follow-up #10234

NO_DOC=bug fix

(cherry picked from commit fc3196dc)

After receiving async transaction from an old term applier_apply_tx
exits without unlocking the latch. If the same applier tries to
subscribe for replication, it fails with assertion, as the latch is
already locked.

Let's fix the function, which raises error so that it just sets
diag and returns -1.

Closes #10073

NO_DOC=bugfix
NO_CHANGELOG=no crash on release version

(cherry picked from commit 5ce010c5)

Now `sio_bind` function prints address into error message directly
instead of relying on `fd` used in `bind` that failed to execute.

`sio_bind` used `sio_socketname_to_buffer` for error message
effectively attempting printing address bound to `fd` while there
actually was an error in binding that address to that socket in the
first place.

Fixes #5925

NO_DOC=bugfix
NO_CHANGELOG=minor

(cherry picked from commit a5214bfc)

This test checks, that when PROMOTE from the previous term is
encountered we immediately notice split-brain situation and break
replication without corrupting data.

Closes #9943

NO_DOC=test
NO_CHANGELOG=test

(cherry picked from commit 06b87e27)

For symmetry with the update of the synchronous replication quorum on
insertion into the `_cluster` space, let's reuse the
`on_replace_cluster_update_quorum` on_commit trigger.

Follows-up #10087

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>

(cherry picked from commit 9b63ced3)

Currently, we update the synchronous replication quorum from the
`on_replace` trigger of the `_cluster` space when registering a new
replica. However, during the join process, the replica cannot ack its own
insertion into the `_cluster` space. In the scope of #9723, we are going to
enable synchronous replication for most of the system spaces, including the
`_cluster` space. There are several problems with this:

1. Joining a replica to a 1-member cluster without manual changing of
quorum won't work: it is impossible to commit the insertion into the
`_cluster` space with only 1 node, since the quorum will equal to 2 right
after the insertion.

2. Joining a replica to a 3-member cluster may fail: the quorum will become
equal to 3 right after the insertion, the newly joined replica cannot ACK
its own insertion into the `_cluster` space — if one out of original 3
nodes fails, then reconfiguration will fail.

Generally speaking, it will be impossible to join a new replica to the
cluster, if a quorum, which includes the newly added replica (which cannot
ACK), cannot be gathered.

To solve these problems, let's update the quorum in the `on_commit`
trigger. This way we’ll be able to insert a node regardless of the current
configuration. This somewhat contradicts with the Raft specification, which
requires application of all configuration changes in the `on_replace`
trigger (i.e., as soon as they are persisted in the WAL, without quorum
confirmation), but still forbids several reconfigurations at the same time.

Closes #10087

NO_DOC=<no special documentation page devoted to cluster reconfiguration>

(cherry picked from commit 29d1c0fa)

There may be more than one fiber waiting on `vy_scheduler::dump_cond`:

```
box.snapshot
  vinyl_engine_wait_checkpoint
    vy_scheduler_wait_checkpoint

space.create_index
  vinyl_space_build_index
    vy_scheduler_dump
```

To avoid hang, we should use `fiber_cond_broadcast`.

Closes #10233

NO_DOC=bug fix

(cherry picked from commit 30547157)