It prevents password sniffing and avoids storing passwords on the
server in plain text but provides no protection if an attacker
manages to steal the password hash from the server.

Usage example:
```lua
-- Enable the md5 authentication method for all new users.
box.cfg({auth_type = 'md5'})

-- Reset existing user passwords to use the md5 authentication method.
box.schema.user.passwd('alice', 'topsecret')

-- Authenticate using the md5 authentication method via net.box.
conn = require('net.box').connect(uri, {
    user = 'alice',
    password = 'topsecret',
    -- Specifying the authentication method isn't strictly necessary:
    -- by default the client will use the method set in the remote
	-- server config (box.cfg.auth_type)
    auth_type = 'md5',
})
```

part of picodata/picodata/sbroad!377

@TarantoolBot document
Title: md5 authentication method

See the commit message.

User name is usually used as a salt for user password in order to
avoid password repeating.
For instance, postgres md5 authentication stores passwords as
md5("password", "user"), so that the same passwords are represented by
different hashes.

part of picodata/picodata/sbroad!377

@TarantoolBot document
Title: Document updated `box.schema.user.password` declaration.

Since auth methods can use user name for hashing, user name is
added to argument list of `box.schema.user.password`.

NO_TEST=there are no methods that use user name

Calling a `TRASH` macro after calling the `free`
function dereferences the pointer to the already
freed memory.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

Gleb Kashkin authored 2 years ago and Дмитрий Кольцов committed 1 year ago

There used to be a rare error when failed to connect via tarantoolctl to
listening cartridge console. It was caused by unclear
console.local_print() contract. Starting from gh-7031 fix, the function
assumed string-only arguments, while in some cases cdata error was
passed.

Now console.local_print() prints all non-string arguments as is, without
modifying potential local_eos.

Closes #8374

NO_DOC=bugfix
NO_TEST=very hard to test

Denis Smirnov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

Picodata supports cluster-wide SQL and needs some predictable
method to calculate tuple hashes for the bucket ids. Method
should be available for Lua, C and Rust users. It was decided
to expose a murmur3 hash calculation method of the key_def module.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

godzie44 authored 1 year ago and Дмитрий Кольцов committed 1 year ago

Introduced a new type of cbus pipe - lcpipe. The current pipe in the
cbus - cpipe, has a number of limitations, first of all - the cpipe
cannot be used from the 3rd party threads, cpipe only works as a channel
between two cords. That why lcpipe is needed. Its main responsibility -
create channel between any thread and tarantool cord.

Internally lcpipe is a cpipe, but:
- on flush triggers removed, cause triggers use thread-local mem-pool,
this is not possible on a third party thread
- producer event loop removed, cause there is no libev event loop in
third party thread

Also, lcpipe interface is exported to the outside world.

NO_DOC=core feature

NO_DOC=disable feedback
NO_TEST=disable feedback

Vladimir Davydov authored 2 years ago and Дмитрий Кольцов committed 1 year ago

The index directory is created on demand since commit c00ba8e7
("xlog: make log directory if needed") and removed when it becomes
empty. There's no need to create it when an index is created anymore.

Follow-up #8441

NO_DOC=bugfix

Denis Smirnov authored 2 years ago and Дмитрий Кольцов committed 1 year ago

When vinyl space is dropped, its files are left on the file system
until GC removes them. At the moment GC removes only run files,
but not the root directory. These empty directories are never
removed and occupy 4KB on ext-family file systems each. In a case
of many dropped vinyl spaces it can become a serious disk space
and inode leak. Current commit makes gc always remove root directory
if there are no runs in it.

Closes #8441

NO_DOC=bugfix

NO_DOC=core feature
NO_TEST=no Lua API
NO_CHANGELOG=bugfix

Due to inconsistency of Tarantool type casting while using strict
data types as "double" or "unsigned" it is needed
to use "number" data type in a whole bunch of cases.
However "number" may contain "decimal" that will be serialized into
string by JSON builtin module.

This commit adds "encode_decimal_as_number" parameter to json.cfg{}.
That forces to encode `decimal` as JSON number to force type
consistency in JSON output.
Use with catious - most of JSON parsers assume that number is restricted
to float64.

NO_DOC=we do not host doc

Denis Smirnov authored 2 years ago and Дмитрий Кольцов committed 1 year ago

Actually there is no reason to throw an error and make a user
manually recreate prepared statement when it expires. A much more
user friendly way is to recreate it under hood when statement's
schema version differs from the box one.

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

Denis Smirnov authored 2 years ago and Дмитрий Кольцов committed 1 year ago

Problem description.

When we prepare a statement with parameters in the result columns
(for example box.prepare('select ?')) Tarantool has no information
about the type of the output column and set it to default boolean.
Then, on the execution phase, the type would be recalculated during
the parameter binding.

Tarantool expects that there is no way for parameter to appear in the
result tuple other than exactly be mentioned in the final projection.
But it is incorrect - we can easily propagate parameter from the inner
part of the join. For example

box.prepare([[select COLUMN_1 from t1 join (values (?)) as t2 on true]])

In this case column COLUMN_1 in the final projection is not a
parameter, but a "reference" to it and its type depends on the
parameter from the inner part of the join. But as Tarantool
recalculates only binded parameters in the result projection,
it doesn't change the default boolean metadata type of the COLUMN_1
and the query fails on comparison with the actual type of the tuple.

Solution.
As we don't want to patch Vdbe to make COLUMN_1 refer inner parameter,
it was decided to make a simple workaround: change the default
column type from BOOLEAN to ANY for parameters. It fixes the
comparison with the actual tuple type (we do not fail), but in some
cases get ANY column in the results where we would like to have
explicitly defined type. Also NULL parameters would also have ANY
type, though Tarantool prefers to have BOOLEAN in this case.

Closes https://github.com/tarantool/tarantool/issues/7283

NO_DOC=bug fix

godzie44 authored 2 years ago and Дмитрий Кольцов committed 1 year ago

sql: add sql_execute_prepared_ext function, same as sql_execute_prepared but without `region` parameter
closes #2

NO_DOC=minor
NO_TEST=minor

godzie44 authored 2 years ago and Дмитрий Кольцов committed 1 year ago

- add box_tuple_data_offset function (return offset of the messagePack encoded data from the beginning of the tuple)
- add more export functions

closes #1

NO_DOC=build
NO_TEST=build

In some cases unsafe extension decoding was done without bound and type
checks: add necessary checks.

Closes tarantool/security#73

NO_DOC=bugfix

(cherry picked from commit 1de6a071)

This patch addresses coverity complain 1535241.

Follow-up #8047

NO_TEST=nit
NO_CHANGELOG=nit
NO_DOC=nit

(cherry picked from commit 089cbfa9)

If the 'after' key is less than the search key in case of ge/gt or
greater than the search key in case of le/lt, the iterator either
crashes (vinyl) or returns invalid result (memtx). This happens because
the engine implementation doesn't expect an invalid 'after' key.
Let's fix this by raising an error at the top level in case the 'after'
key doesn't meet the search criteria.

Closes #8403
Closes #8404

NO_DOC=bug fix
NO_CHANGELOG=unreleased

(cherry picked from commit c561202d)

Currently, if the position isn't compatible with the index, we raise
an error like "Invalid key part count ...". From this error it's
difficult to figure out whether it's for the given iterator position of
for the search key. Let's always raise ER_ITERATOR_POSITION in this
case. Later on we'll use stacked diag to add extra error info.

Needed for #8403
Needed for #8404

NO_DOC=bug fix
NO_CHANGELOG=unreleased

(cherry picked from commit 81d43c17)

We need to compare a tuple position with a search key in select() and
pairs() to make sure the tuple position meets the search criteria. The
problem is that we strip the MessagePack header from the position while
key_compare() takes keys with headers. Let's make key_compare take keys
without headers like the rest of comparator functions. Since in Vinyl we
often need to compare keys with headers, we also add vy_key_compare()
helper function.

Needed for #8403
Needed for #8404

NO_DOC=refactoring
NO_TEST=refactoring
NO_CHANGELOG=refactoring

(cherry picked from commit 41b8a012)

To improve customer experience it was decided to disable JIT engine on
Tarantool startup for macOS builds. Either way, JIT will be aboard as a
result of the changes and more adventurous users will be able to enable
it via <jit.on> in their code.

Furthermore, for convenient maintenance of JIT default behaviour CMake
configuration option "LUAJIT_JIT_STATUS" is introduced.

Closes #8252

NO_DOC=no behaviour changes

(cherry picked from commit ae0db476)

If the tuple pointed to by 'after' isn't in the space, eq pagination
would skip it because of the broken 'equals' flag update in the memtx
iterator.

Closes #8373

NO_DOC=bug fix
NO_CHANGELOG=unreleased

(cherry picked from commit bf275ed5)

Refactor logging of dangerous `select` call out of `check_select_safety`
and add it to `box.internal` so that it can be reused for read views.

Change all occurrences of 'dangerous' to 'long' to be consistent with the
log message.

Needed for tarantool/tarantool-ee#211

NO_CHANGELOG=<refactoring>
NO_DOC=<refactoring>
NO_TEST=<refactoring>

(cherry picked from commit 4221a983)

The `log_level' configuration parameter can be set as a number or a string.
When it is a string, cfg_geti() returns 0. Use log_default->level instead,
which is initialized earlier during box_init_say().

Closes #8287

NO_DOC=bugfix
NO_CHANGELOG=minor bug

(cherry picked from commit 41ead021)

This patch fixes an assertion or segmentation error if a FOREIGN KEY or
CHECK constraint is declared before the first column.

Closes #8392

NO_DOC=bugfix of the bug added in the current release
NO_CHANGELOG=bugfix of the bug added in the current release

We use coio_connect() to connect the replica to a remote peer. It
implies no timeout, and does a non-blocking connect() to the peer and
then waits for the socket to become writable indefinitely.

When the remote peer changes its IP address, connect() might try
connecting to the old address for as long as ~ 2 minutes (given the
default tcp_syn_retries value of 6).

This blocks replica from trying to reconnect to the updated address and
is pretty inconvenient.

Let's use coio_connect_timeout() instead and use
replication_disconnect_timeout() as a timeout, like everywhere else in
master-replica communication.

Closes #7294

NO_DOC=bugfix

(cherry picked from commit 0486a489)

Force recovery needs to follow the following logic: any unsuccessful system
space request must make recovery fail (including failure to decode an xrow,
when we are not sure we have finished processing system space request). If
the request is a non-insert one (e.g., raft or synchro) or addresses a user
space, it means we have finished processing system space requests, and from
this moment force recovery can be enabled — change the behaviour
accordingly.

We assume the request order in the snapshot is the following:
1. system space requests;
2. user space requests;
3. non-insert requests (e.g., raft or synchro).

Refactor the force recovery logic: add a enumeration to track snapshot
recovery state and add a new diagnostic for the case when the snapshot
contains has no system spaces.

Closes #7974

NO_DOC=bugfix

(cherry picked from commit b1095c1c)

In some cases we don't have the whole space struct, but we want to
determine whether the provided space identifier corresponds to a system
space — add a `space_id_is_system` helper and refactor `space_is_system` to
reuse it.

Needed for #7974

NO_CHANGELOG=refactoring
NO_DOC=refactoring
NO_TEST=refactoring

(cherry picked from commit 928e5733)

Patch fixes a bug when body in response couldn't be decoded:

NO_WRAP
```
tarantool> httpc = require('http.client').new()
tarantool> response = httpc:get('https://jsonplaceholder.typicode.com/todos/1'

)
tarantool> response:decode()

---
- error: 'builtin/http.client.lua:301: attempt to index field ''decoders'' (a nil
    value)'
...
```
NO_WRAP

Now response object contains table with decoders defined by user in his
http client instance. We hide this table on response serialization by
adding underscore because decoders there is not a part of API.

Reported-by: Alexander Turenko <alexander.turenko@tarantool.org>

Fixes #8363

NO_DOC=bugfix

(cherry picked from commit 83168b25)

Error labes got mixed up. Fix the order and add a test.

Fixes commit 3f026339 ("vinyl: implement iterator pagination").

Closes #8372

NO_DOC=bug fix
NO_CHANGELOG=fix for unreleased feature

(cherry picked from commit 074e4eb8)

When calculating `rows_index` in `netbox_decode_execute` etc we use
implementation defined behavior of compiler. `i < mapsize` and this is
fine according to standard as unsigned arithmetic is well defined on
overflows. But then we cast the result to int and this is implementation
defined as the result can not be represented by int.

Closes https://github.com/tarantool/security/issues/98

NO_DOC=minor
NO_TEST=minor
NO_CHANGELOG=minor

(cherry picked from commit baba38e6)

The new compat option 'fiber_slice_default' is added to control
the default value of the max fiber slice. The old default is no limit
(both warning and error slice equals TIMEOUT_INFINITY). The new default
is {warn = 0.5, err = 1.0}.

Follow-up #6085

NO_DOC=tarantool/doc#3057
NO_CHANGELOG=unreleased

This commit adds the new field 'max_slice' to fiber.info() that reports
the max slice applied to the given fiber. The value is a table with two
fields: 'err' for error slice and 'warn' for warning slice. Values
greater than or equal to TIMEOUT_INFINITY aren't reported.

Follow-up #6085

NO_DOC=tarantool/doc#3057
NO_CHANGELOG=unreleased

Error messages raised when an invalid slice is specified are confusing:

  tarantool> fiber = require('fiber')
  ---
  ...

  tarantool> fiber.set_max_slice('foo')
  ---
  - error: slice must be a table or a number
  ...

  tarantool> fiber.set_max_slice({})
  ---
  - error: 'bad argument #3 to ''?'' (number expected, got nil)'
  ...

Let's change the error message to "slice must be a number or a table
{warn = <number>, err = <number>}".

Follow-up #6085

NO_DOC=undocumented
NO_CHANGELOG=unreleased

It doesn't belong to Tarantool sources. The tools directory looks like
the right place for it.

NO_DOC=code cleanup
NO_TEST=code cleanup
NO_CHANGELOG=code cleanup

This patch allows to downgrade tuple foreign keys to SQL foreign keys
and some tuple check constraints to SQL check constraints. The only
tuple check constraints that can be downgraded are those using functions
with the SQL_EXPR language.

Closes #7718

NO_DOC=already introduced
NO_CHANGELOG=already introduced

Added `-d` option for activation of debugger shell:
- it calls debugger shell in `luadebug.lua` instead of
  a standard interactive shell from `console.lua`;
- that option complements original way for starting a
  debugging shell via `require 'luadebug'()`, but is a
  little bit easier.

NB! At the moment when we enter debugging mode instead of
a standard Tarantool console, we change banner to:
```
Tarantool debugger 2.11.0-entrypoint-852-g9e6ed28ae
type 'help' for interactive help
```

Part of #7456

@TarantoolBot document
Title: Command-line option `-d` for console debugger.

Please see third_party/lua/README-luadebug.md for a full description
of different ways to activate debugging shell.

2.10.5 version is going to be released at the same time as 2.11.0 thus
let's add it to the box.schema.downgrade_versions().

NO_DOC=internal
NO_CHANGELOG=internal
NO_TEST=internal

Currently it's possible to set the log level for a particular Lua module,
or to change the default log level, however there is no way to change it
only for Tarantool system messages. This patch introduces a fake module
name "tarantool" for this purpose.

Closes #8320

NO_CHANGELOG=Unreleased feature
NO_DOC=Will update tarantool/doc#3264

Add a streaming data input/output object for http.client. The
input/output object can be created using the same methods and the
same options as a normal request, but with a new option
{chunked = true}.

Closes #7845

@TarantoolBot document
Title: Stream input/output interface for http.client

An uncompleted io object has only headers and cookies fields. A
completed io object has the same fields as a normal request, but
without the `body` field.

The io object interface looks like the socket object interface
and should have the same description:

```
io_object:read(chunk[, timeout])
io_object:read(delimiter[, timeout])
io_object:read({chunk = chunk, delimiter = delimiter}[, timeout])
io_object:write(data[, timeout])
```

The difference is in the method `finish`. Unlike socket:close()
it has an optional parameter `timeout`:

```
io_object:finish([timeout])
```

Be careful, the call may yield a fiber. The idea is to wait
until a HTTP connection is finished by the server-side or
force finish the connection from client-time after a timeout
value.

The default timeout value is 10 seconds for all methods.

Usage example:

```lua
local io = httpc:post(url, nil, {chunked = true})

local write_chan = fiber.channel()
fiber.new(function()
    fiber.name("write to " .. url)
    while true do
        local data = write_chan:get()
        if data == nil then
            break
        end
        io:write(data, 1)
    end
end)

local recvdata
while recvdata = io:read('\r\n', 1) do
    local decoded = json.decode(recvdata)
    if condition(decoded) then
        write_chan:put(data)
    end
    if condition(decoded) then
        io:finish(1)
    end
end
write_chan:close()
```

See also:
* https://www.tarantool.io/en/doc/latest/reference/reference_lua/socket/#lua-function.socket_object.read
* https://github.com/tarantool/tarantool/issues/7845#issuecomment-1298538412
* https://github.com/tarantool/tarantool/issues/7845#issuecomment-1298821779