First beta version of 2 release serie
2.10.0-beta1 is the [beta][release_policy] version of the 2 release series.
This release introduces 28 new features and resolves 56 bugs since
the 2.8 version. There can be bugs in less common areas. If you find any,
feel free to [report an issue][issues] on GitHub.
Notable changes are:
* Preliminary support of ARM64 architecture (GNU/Linux).
* Preliminary support of Apple M1 architecture (macOS).
* Consistent SQL type system.
* Lightning fast net.box module.
* Interative transactions in binary protocol.
* Small tuples are now stored more efficiently.
[release_policy]: https://www.tarantool.io/en/doc/latest/release/policy/
[issues]: https://github.com/tarantool/tarantool/issues
Tarantool 2.x is backward compatible with Tarantool 1.10.x in the binary data
layout, client-server protocol, and replication protocol.
[Upgrade][upgrade] using the `box.schema.upgrade()` procedure to unlock
all the new features of the 2.x series.
[upgrade]: https://www.tarantool.io/en/doc/latest/book/admin/upgrades/
* Field type UUID is now part of field type SCALAR (gh-6042).
* Field type UUID is now available in SQL, and new UUID can be generated using new SQL built-in function uuid() (gh-5886).
* **[Breaking change]** timeout() method of net.box connection, which was
marked deprecated more than four years ago (in 1.7.4) was dropped, because
it negatively affected performance of hot net.box methods, like call() and
select(), in case those are called without specifying a timeout (gh-6242).
* **[Breaking change]**: timeout() method of net.box connection was dropped.
* Improved net.box performance by up to 70% by rewriting hot code paths in C
(gh-6241).
* Introduce compact tuples that allow to save 4 bytes per tuple in case of small userdata (gh-5385)
* Streams and interactive transactions over streams are implemented
in iproto. Stream is associated with it's ID, which is unique within
one connection. All requests with same not zero stream ID belongs to
the same stream. All requests in stream processed synchronously. The
execution of the next request will not start until the previous one is
completed. If request has zero stream ID it does not belong to stream
and is processed in the old way.
In `net.box`, stream is an object above connection that has the same
methods, but allows to execute requests sequentially. ID is generated
on the client side automatically. If user writes his own connector and
wants to use streams, he must transmit stream_id over iproto protocol.
The main purpose of streams is transactions via iproto. Each stream
can start its own transaction, so they allows multiplexing several
transactions over one connection. There are multiple ways to begin,
commit and rollback transaction: using appropriate stream methods, using
`call` or `eval` methods or using `execute` method with sql transaction
syntax. User can mix these methods, for example, start transaction using
`stream:begin()`, and commit transaction using `stream:call('box.commit')`
or stream:execute('COMMIT').
If any request fails during the transaction, it will not affect the other
requests in the transaction. If disconnect occurs when there is some active
transaction in stream, this transaction will be rollbacked, if it does not
have time to commit before this moment.
* Add new 'memtx_allocator' option to box.cfg{} which allows to
select the appropriate allocator for memtx tuples if necessary.
Possible values are "system" for malloc allocator and "small"
for default small allocator.
Implement system allocator, based on malloc: slab allocator, which
is used for tuples allocation, has a certain disadvantage - it tends
to unresolvable fragmentation on certain workloads (size migration).
In this case user should be able to choose other allocator. System
allocator based on malloc function, but restricted by the same qouta
as slab allocator. System allocator does not alloc all memory at start,
istead, it allocate memory as needed, checking that quota is not exceeded
(gh-5419).
* Introduced `box.info.replication[n].downstream.lag` field to monitor
state of replication. This member represents a lag between the main
node writes a certain transaction to it's own WAL and a moment it
receives an ack for this transaction from a replica (gh-5447).
* Introduce on_election triggers. The triggers may be registered via
`box.ctl.on_election()` interface and are run asynchronously each time
`box.info.election` changes (gh-5819).
* Introduced support for `LJ_DUALNUM` mode in luajit-gdb.py (gh-6224).
* Introduced preliminary support of GNU/Linux ARM64 and MacOS M1. In scope of
this activity the following issues have been resolved:
- Introduced support for full 64-bit range of lightuserdata values (gh-2712)
- Fixed memory remapping issue when the page leaves 47-bit segments
- Fixed M1 architecture detection (gh-6065)
- Fixed variadic arguments handling in FFI on M1 (gh-6066)
- Fixed `table.move` misbehaviour when table reallocation occurs (gh-6084)
- Fixed Lua stack inconsistency when xpcall is called with invalid second
argument on ARM64 (gh-6093)
- Fixed `BC_USETS` bytecode semantics for closed upvalues and gray strings
- Fixed side exit jump target patching considering the range values of the
particular instruction (gh-6098)
- Fixed current Lua coroutine restoring on exceptional path on ARM64 (gh-6189)
* Introduce method `table.equals`. It compares 2 tables by value and respects
`__eq` metamethod.
* Introduce new hash types in digest module - `xxhash32` and `xxhash64`
(gh-2003).
* Introduce `fiber_object:info()` to get `info` from fiber. Works as `require(fiber).info()` but only for one fiber.
* Introduce `fiber_object:csw()` to get `csw` from fiber (gh-5799).
* Changed `fiber.info()` to hide backtraces of idle fibers (gh-4235).
* Implemented support of symbolic log levels representation
in `log` module (gh-5882). Now it is possible to specify
levels the same way as in `box.cfg{}` call. For example
instead of
``` Lua
require('log').cfg{level = 6}
```
One can use
``` Lua
require('log').cfg{level = 'verbose'}
```
* Descriptions of type mismatch error and inconsistent type error become more
informative (gh-6176).
* Removed explicit cast from BOOLEAN to numeric types and vice versa (gh-4770).
* Removed explicit cast from VARBINARY to numeric types and vice versa (gh-4772,
gh-5852).
* Fixed a bug due to which a string that is not NULL terminated could not be
cast to BOOLEAN, even if the conversion should be successful according to the
rules.
* Now a numeric value can be cast to another numeric type only if the cast is
precise. In addition, a UUID value cannot be implicitly cast to
STRING/VARBINARY, and a STRING/VARBINARY value cannot be implicitly cast to
a UUID (gh-4470).
* Now any number can be compared to any other number, and values of any scalar
type can be compared to any other value of the same type. A value of a
non-numeric scalar type cannot be compared with a value of any other scalar
type (gh-4230).
* SQL built-in functions were removed from \_func system space (gh-6106).
* Function are now looked up first in SQL built-in functions and then in
user-defined functions.
* Fixed incorrect error message in case of misuse of the function used to set
the default value.
* The typeof() function with NULL as an argument now returns "NULL" (gh-5956).
* The SCALAR and NUMBER types have been reworked in SQL. Now SCALAR values
cannot be implicitly cast to any other scalar type, and NUMBER values cannot be
implicitly cast to any other numeric type. This means that arithmetic and
bitwise operations and concatenation are no longer allowed for SCALAR and NUMBER
values. In addition, any SCALAR value can now be compared with values of any
other scalar type using the SCALAR rules (gh-6221).
* Field type DECIMAL is now available in SQL. Decimal can be implcitly cast to
and from INTEGER and DOUBLE, it can participate in arithmetic operations and
comparison between DECIMAL and all other numeric types are defined (gh-4415).
* The argument types of SQL built-in functions are now checked in most cases
during parsing. In addition, the number of arguments is now always checked
during parsing (gh-6105).
* Previously csw (Context SWitch) of new fiber can be greater than 0, now it is always 0 (gh-5799).
* Set FORCE_CONFIG=false for luarocks config to allow loading project-side .rocks/config-5.1.lua
* Fedora 34 builds are now supported. (gh-6074)
* Fedora 28 and 29 builds are no longer supported.
* **[Breaking change]** `fiber.wakeup()` in Lua and `fiber_wakeup()` in C became
NOP on the currently running fiber. Previously they allowed to "ignore" the
next yield or sleep leading to unexpected spurious wakeups. Could lead to a
crash (in debug build) or undefined behaviour (in release build) if called
right before `fiber.create()` in Lua or `fiber_start()` in C (gh-6043).
There was a single usecase for that - reschedule in the same event loop
iteration which is not the same as `fiber.sleep(0)` in Lua and
`fiber_sleep(0)` in C. Could be done in C like that:
```C
fiber_wakeup(fiber_self());
fiber_yield();
```
and in Lua like that:
```Lua
fiber.self():wakeup()
fiber.yield()
```
Now to get the same effect in C use `fiber_reschedule()`. In Lua it is now
simply impossible to reschedule the current fiber in the same event loop
iteration directly. But still can reschedule self through a second fiber like
this (**never use it, please**):
```Lua
local self = fiber.self()
fiber.new(function() self:wakeup() end)
fiber.sleep(0)
```
* Fixed memory leak on each `box.on_commit()` and `box.on_rollback()` (gh-6025).
* Fixed lack of testing for non noinable fibers in `fiber_join()` call.
This could lead to unpredictable results. Note the issue affects C
level only, in Lua interface `fiber:join()` the protection is
turned on already.
* Now tarantool yields when scanning `.xlog` files for the latest applied vclock
and when finding the right place in `.xlog`s to start recovering. This means
that the instance is responsive right after `box.cfg` call even when an empty
`.xlog` was not created on previous exit.
Also this prevents relay from timing out when a freshly subscribed replica
needs rows from the end of a relatively long (hundreds of MBs) `.xlog`
(gh-5979).
* The counter in `x.yM rows processed` log messages does not reset on each new
recovered `xlog` anymore.
* Fixed wrong type specification when printing fiber state
change which lead to negative fiber's ID logging (gh-5846).
For example
```
main/-244760339/cartridge.failover.task I> Instance state changed
```
instead of proper
```
main/4050206957/cartridge.failover.task I> Instance state changed
```
* Fiber IDs are switched to monotonically increasing unsigned 8 byte
integers so that there won't be IDs wrapping anymore. This allows
to detect fiber's precedence by their IDs if needed (gh-5846).
* Fixed a crash in JSON update on tuple/space when it had more than one
operation, they accessed fields in reversed order, and these fields didn't
exist. Example: `box.tuple.new({1}):update({{'=', 4, 4}, {'=', 3, 3}})`
(gh-6069).
* Fixed invalid results produced by `json` module's `encode` function when it
was used from Lua's garbage collector. For instance, in functions used as
`ffi.gc()` (gh-6050).
* Added check for user input of the number of iproto threads - value
must be > 0 and less then or equal to 1000 (gh-6005).
* Fixed error, related to the fact, that if user changed listen address,
all iproto threads closed same socket multiple times.
Fixed error, related to the fact, that tarantool not deleting the unix
socket path, when it's finishing work.
* Fixed a crash in MVCC during simultaneous update of a key in different transactions (gh-6131)
* Fixed a bug when memtx mvcc crashed during reading uncommitted DDL (gh-5515)
* Fixed a bug when memtx mvcc crashed if an index was created in transaction (gh-6137)
* Fixed segmentation fault with mvcc when entire space was updated concurrently (gh-5892)
* Fixed a bug with failed assertion after stress update of the same key. (gh-6193)
* Fixed a crash if you call box.snapshot during an incomplete transaction (gh-6229)
* Fixed console client connection breakage if request times out (gh-6249).
* Added missing broadcast to net.box.future:discard() so that now fibers
waiting for a request result are woken up when the request is discarded
(gh-6250).
* `box.info.uuid`, `box.info.cluster.uuid`, and `tostring(decimal)` with any
decimal number in Lua sometimes could return garbage if `__gc` handlers are
used in user's code (gh-6259).
* Fix an error message that happened in very specific case
during mvcc operation (gh-6247)
* Fixed a repeatable read violation after delete (gh-6206)
* Fixed a bug when hash select{} was not tracked by mvcc engine (gh-6040)
* Fixed a crash in mvcc after drop of a space with several indexes (gh-6274)
* Fixed a bug when GC at some state could leave tuples in secondary indexes (gh-6234)
* Disallow yields after DDL operations in MVCC mode. It fixes crash which takes
place in case several transactions refer to system spaces (gh-5998).
* Fixed bug in MVCC connected which happens on rollback after
DDL operation (gh-5998).
* Fix a bug when rollback resulted in unserializable behaviour (gh-6325)
* Fixed possible keys divergence during secondary index build which might
lead to missing tuples in it (gh-6045).
* Fixed a race between Vinyl garbage collection and compaction resulting in
broken vylog and recovery (gh-5436).
* Fixed use after free in relay thread when using elections (gh-6031).
* Fixed a possible crash when a synchronous transaction was followed by an
asynchronous transaction right when its confirmation was being written
(gh-6057).
* Fixed an error when a replica, at attempt to subscribe to a foreign cluster
(with different replicaset UUID), didn't notice it is not possible, and
instead was stuck in an infinite retry loop printing an error about "too
early subscribe" (gh-6094).
* Fixed an error when a replica, at attempt to join a cluster with exclusively
read-only replicas available, instead of failing or retrying just decided to
boot its own replicaset. Now it fails with an error about the other nodes
being read-only so they can't register it (gh-5613).
* When an error happened during appliance of a transaction received from a
remote instance via replication, it was always reported as "Failed to write
to disk" regardless of what really happened. Now the correct error is shown.
For example, "Out of memory", or "Transaction has been aborted by conflict",
and so on (gh-6027).
* Fixed replication stopping occasionally with `ER_INVALID_MSGPACK` when replica
is under high load (gh-4040).
* Fixed a cluster sometimes being unable to bootstrap if it contains nodes with
`election_mode` `manual` or `voter` (gh-6018).
* Fixed a possible crash when `box.ctl.promote()` was called in a cluster with
>= 3 instances, happened in debug build. In release build it could lead to
undefined behaviour. It was likely to happen if a new node was added shortly
before the promotion (gh-5430).
* Fixed a rare error appearing when MVCC (`box.cfg.memtx_use_mvcc_engine`) was
enabled and more than one replica was joined to a cluster. The join could fail
with the error `"ER_TUPLE_FOUND: Duplicate key exists in unique index
'primary' in space '_cluster'"`. The same could happen at bootstrap of a
cluster having >= 3 nodes (gh-5601).
* Fixed a rare crash with the leader election enabled (any mode except `off`),
which could happen if a leader resigned from its role at the same time as some
other node was writing something related to the elections to WAL. The crash
was in debug build and in the release build it would lead to undefined
behaviour (gh-6129).
* Fixed an error when a new replica in a Raft cluster could try to join from a
follower instead of a leader and failed with an error `ER_READONLY` (gh-6127).
* Fixed optimization for single-char strings in `IR_BUFPUT` assembly routine.
* Fixed slots alignment in `lj-stack` command output when `LJ_GC64` is enabled
(gh-5876).
* Fixed dummy frame unwinding in `lj-stack` command.
* Fixed detection of inconsistent renames even in the presence of sunk values
(gh-4252, gh-5049, gh-5118).
* Fixed the order VM registers are allocated by LuaJIT frontend in case of
`BC_ISGE` and `BC_ISGT` (gh-6227).
* Fixed a bug when multibyte characters broke `space:fselect()` output.
* When error is raised during encoding call results, auxiliary lightuserdata
value is not removed from the main Lua coroutine stack. Prior to the fix it
leads to undefined behaviour during the next usage of this Lua coroutine
(gh-4617).
* Fixed Lua C API misuse, when the error is raised during call results encoding
on unprotected coroutine and expected to be catched on the different one, that
is protected (gh-6248).
* Fixed possibility crash in case when trigger removes itself.
Fixed possibility crash in case when someone destroy trigger,
when it's yield (gh-6266).
* User-defined functions can now return VARBINARY to SQL as result (gh-6024).
* Fixed assert on cast of DOUBLE value that greater than -1.0 and less than 0.0
to INTEGER and UNSIGNED (gh-6255).
* Removed spontaneous conversion from INTEGER to DOUBLE in a field of type
NUMBER (gh-5335).
* All arithmetic operations can now only accept numeric values (gh-5756).
* Now function quote() will return the argument in case the argument is DOUBLE.
The same for all other numeric types. For types other than numeric, STRING
will be returned (gh-6239).
* The TRIM() function now does not lose collation when executed with the
keywords BOTH, LEADING, or TRAILING (gh-6299).
* Fixed MVCC interaction with ephemeral spaces: TX manager now ignores such
spaces (gh-6095).
* Fixed a loss of tuple after a conflict exception (gh-6132)
* Fixed a segfault in update/delete of the same tuple (gh-6021)