NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Picodata SQL now manages the Tarantool statement cache using a
dedicated SQL fiber that handles preparation and unpreparation of
statements based on an LRU eviction policy. Prepared statements
can be executed across different sessions by SQL clients.

Previously, when a client executed a prepared statement, it
increased the reference count in the statement cache and linked
the statement to the client's session. While this approach seemed
fine, it caused issues during eviction, as references to these
statements remained in client sessions, preventing proper eviction.

This commit addresses the issue by ensuring that if a statement is
added to the current session during execution, it is removed and
the session state is restored once execution is complete.

NO_DOC=internal
NO_CHANGELOG=internal

Temporary spaces, used for cluster-wide SQL data materialization,
were causing unnecessary netbox schema version bumps, leading to
schema downloading via netbox, excessive Lua garbage and GC blocks.
Since these tables are for internal SQL use, we don't need to inform
netbox clients about schema changes. We now maintain separate schema
versions: one for netbox clients and one for the internal prepared
statement cache.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Dmitry Rodionov authored 1 year ago and Dmitry Ivanov committed 7 months ago

* smoke test that verifies that cluster can be successfully bootstrapped
* test diverging limbo owner (litmus test)
  - doesn't work on fork master, works with promote chains
* test aba leader
  - two cases where elected leader fails to deliver promote to others
* attempt to model 2+2 quorum lowering case
* two cases with two outstanding promotes
* use new injections for particular xrow types

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Co-authored-by: Dmitry Ivanov <ivadmi5@gmail.com>

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Further reading:
* https://git.picodata.io/picodata/tarantool/-/merge_requests/175
* https://github.com/tarantool/tarantool/pull/10334

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

- If you prepare and execute statement with params
in the projection and then unprepare the
statement, byte counter may show the wrong value
or even overflow.
- The problem is that when we compile sql
statement, we set parameter type to 'any'.
But when we execute the statement we set parameter
type to actual type. Then we use this type in
calculation of estimated of sql cache entry size.
This leads to different estimated sizes of cache
entry during prepare and during unprepare after
statement was executed
- Fix this by resetting type to 'any' after
executing the statement

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

BREAKING CHANGE!:
1. add session id argument to sql_prepare_ext
2. introduce sql_unprepare_ext function.
This function removes prepared stmt using
given session id.

In picodata SQL, we may prepare stmt in one
session and unprepare it in some
other session, which does not know in
what session the statement was prepared. Now
sql_prepare_ext returns not only statement id,
but also a session id. This way statement can
be unprepared from other session using
sql_unprepare_ext.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

Replace fiber.sleep with luatest.helpers.retrying
to make tests less flaky

NO_DOC=internal
NO_CHANGELOG=internal

godzie44 authored 1 year ago and Дмитрий Кольцов committed 7 months ago

WAL extensions allows to add auxiliary information to each
write-ahead log record. WAL extensions configured by
`box.cfg.wal_ext` option. Currently, there is only one builtin
extension: `new_old`. `new_old` extension add information about
new and old tuples for ddl operations.

NO_DOC=internal
NO_CHANGELOG=internal

BREAKING CHANGE!:
1. remove sql_prepare from the export list;
2. introduce sql_prepare_ext.

The sql_prepare symbol previously included the tarantool port as
an output parameter. However, this structure was inconvenient for
libraries using the C API, as they primarily required just the
statement ID. To address this issue, the sql_prepare symbol was
replaced with the sql_prepare_ext symbol.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch

Previously, users with multiple connections to tarantool instance
couldn't share prepared statements across sessions. They had to
manually call prepare in each session before execution.

This commit automates this process for the exported version of
SQL prepared statement execution (sql_execute_prepared_ext symbol).
Original Lua execution keeps the old behavior for backward
compatibility.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

BREAKING CHANGE!:
1. sql_bind_list_decode - removed
2. sql_execute_prepared_ext - new arguments
3. sql_prepare_and_execute_ext - exported

There were several reasons to refactor the API.
1. sql_bind_list_decode (decodes message pack parameters into
   internal C bind structure) is very difficult to use without
   memory leaks (as it allocates results on fiber()->gc).
2. sql_execute_prepared_ext missed vdbe step limit in parameters
   and used the default value.
3. Sometimes SQL queries don't fit into prepared statement cache
   and the user still wants to execute them via a slow pass with
   full compilations from the query text. That was the reason to
   export sql_prepare_and_execute_ext symbol.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Also remove CI pipeline for fuzz_until, as running it in CI is not planned

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

- add cli flag whether to wait for 2x coverage
- add timestamps to log lines
- remove dictionary passing as it is not needed when
  corpus already exist

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

- add corpus for xrow_decode_error and xrow_header_decode
- update corpus for xrow_decode_call and xrow_decode_dml
  for increased coverage

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=bug fix
NO_TEST=bug fix
NO_CHANGELOG=bug fix

Add new transport for IPROTO connections: ssl.
SSL transport may be configured with (at client and server side):
- certificate (mandatory at server side)
- private key (mandatory at server side)
- password for PK
- certificate authorities (for peer certificate verification)
- cipher list

SSL transport also can be used in replication and net.box mechanisms.

@TarantoolBot document
Title: add IPROTO traffic encryption.

New ssl transport allows
creating a secure connection between two IPROTO peers.
TLS protocol using and openssl v1.1 or later required.

To configure traffic encryption, you need to set the
special URI parameters for a particular connection.
The parameters can be set for the following box.cfg
options and nex.box method:
- box.cfg.listen – on the server side.
- box.cfg.replication–on the client side.
- net_box_object.connect()–on the client side.

Дмитрий Кибирев authored 1 year ago and Yaroslav Dynnikov committed 1 year ago

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Egor Ivkov authored 1 year ago and Егор Ивков committed 1 year ago

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

godzie44 authored 1 year ago and Дмитрий Кольцов committed 1 year ago

Currently, DDL transaction may abort other in-progress
transactions (this behavior added in #8f4be322 - avoid sharing
(ergo phantom reads) metadata object for different
transactions in MVCC mode).
This behavior makes no sense if the MVCC is
disabled (cause these conflicts never happen).

Closes #47

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

(cherry picked from commit 05b62fe0)

Current commit is a part of the SQL procedures track for Picodata.
It introduces several features:
- the reserved id range in the _func space, that can be only consumed
  implicitly;
- an additional "id" option for function creation with Lua (to set
  the function identifier implicitly);
- helper function to generate function ids either in reserved or default
  ranges

NO_DOC=internal
NO_CHANGELOG=internal

Dmitry Ivanov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

This reverts commit c8594fbd.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Dmitry Ivanov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

This reverts commit 9bde48a8.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Dmitry Ivanov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

This reverts commit 4643a26a.

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal

Dmitry Rodionov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

This patch makes schema_object_type to be exported.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

Dmitry Rodionov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch

Georgy Moshkin authored 1 year ago and Дмитрий Кольцов committed 1 year ago

NO_CHANGELOG=bug fix for unreleased feature
NO_DOC=bug fix for unreleased feature

Feodor Alexandrov authored 1 year ago and Дмитрий Кольцов committed 1 year ago

Picodata is built statically, and name crypto clashes between
`src/lib/crypto` and libcrypto from `openssl`. To resolve the
ambiguity this patch renames tarantool library to `tcrypto`.

NO_DOC=picodata internal patch
NO_CHANGELOG=picodata internal patch
NO_TEST=picodata internal patch