Maksim Kaitmazian authored 5 months ago and Dmitry Ivanov committed 4 months ago

Disabling the tracing feature has shown a significant performance improvement
of 72.50%, as demonstrated in https://git.picodata.io/picodata/picodata/picodata/-/issues/1026#note_107222.

This commit disables "tracing" feature for sbroad and eliminates
the opentelemetry wrappers and statistics tables.

Arseniy Volynets authored 6 months ago and Denis Smirnov committed 5 months ago

Now SQL explain shows the user what buckets the query would be
executed on. Some results are exact, others are not (we return
`buckets: unknown` or `buckets: <= [..]` according to the query).

Denis Smirnov authored 5 months ago and Dmitry Rodionov committed 5 months ago

There was a bug when SQL statistics LRU removed some old query
from the _sql_query table. The query ref_counter field in unsigned,
but the code decremented it below zero in the space on eviction.
That produced an error (fixed).

Changed sbroad vshard related traits and it's capabilities
about working with several vshard groups

1. A better VDBE caching was implemented for local SQL.
2. Local SQL is called via C API rather then Lua.
3. A new prepare-unprepare fiber added to manage SQL statement
   cache.

Arseniy Volynets authored 8 months ago and Arseniy Volynets committed 7 months ago

- update sbroad submodule to commit with
new normalization rules
- update integration tests

- update sbroad submodule to commit with helper
patches for dml on global tables
- add support for sql dml insert/update/delete
on global tables. Insert on conflict is not
supported yet
- Current implementation uses a single batch of
dml commands via CAS. No additional predicates
are used in CAS, so no isolation level is guaranteed
even when query only refers to global tables

- rename it to proc_sql_dispatch
- modify args so that it's more suitable for public RPC API

Previously when running `pico.sql [[ create table foo ]]` we would get
this output:
```
---
- null
- "sbroad: rule parsing error:  --> 1:9\n  |\n1 |  create table foo \n  |  ^---\n
  \ |\n  = expected Unique"
...
```

But now the output will be:
```
---
- null
- |+
  sbroad: rule parsing error:  --> 1:9
    |
  1 |  create table foo
    |         ^---
    |
    = expected Unique

...
```

Maksim Kaitmazian authored 1 year ago and Dmitry Ivanov committed 10 months ago

The backend module implements the logic for handling postgres messages.
Note that this commit doesn't bring anything new: it just moves picodata's
handlers into pgproto.

Arseniy Volynets authored 11 months ago and Denis Smirnov committed 10 months ago

- update sbroad submodule to commit
with support of UNION operator
- add integration tests for global
tables (sharded tables were tested
in cartridge tests)
- Usage:

`select a from t union select b from t2`

Fedor Telnov authored 11 months ago and Denis Smirnov committed 10 months ago

It syncs API with new sbroad's one.
For more context refer to sbroad!424

Arseniy Volynets authored 11 months ago and Denis Smirnov committed 11 months ago

- update sbroad submodule with support of
datetime type and new `to_date` builtin
function.
- initialize sbroad builtin functions in
init common. Currently there is only one
builtin function `to_date`
- add integration tests for datetime types
- update tarantool module to commit with
fixes for datetime deserialization

We have recently replaced String with SmolStr in sbroad.
As a result, we need to make corresponding changes in
picodata as it uses sbroad-core for cluster SQL.

BREAKING CHANGE: "name" index in the _pico_index has changed:
previously it guarantees uniqueness for a pair of (space_id,
index_name). Now index name must be unique among the cluster.
So, all the global indexes for _pico tables were renamed.

This commit introduces:
 * statement and portal storages;
 * statements with parameter oids;
 * portal encoding;
 * interactive portals;

Portal encoding defines in which format (binary or text) columns
are returned.

Interactive portals allow to limit the number of rows returned from
execute method. Other rows can be extracted by calling execute again
on the same portal.

- add new ddl opcode for renaming procedure
- update sbroad submodule to get sql support
- syntax:

alter procedure "foo" rename to "bar" option(timeout=3)
alter procedure "foo"(int, int) rename to "bar"

- Update sbroad submodule to commit,
with refactored tracing api: tracer
can be provided by library user (us).
- Use tracer that creates traces under
admin, because traces are created randomly
for 1% of the queries and the user may not
have the privileges to create/modify tracing
tables
- Move tracing related logic to picodata repo.
In particular, now all tables storing tracing
- Add sql tests that were removed from sbroad
during refactoring
- added extra parameter 'query_id' to pico.sql
lua api. 'query_id' is used to identify
each query in tracing tables, if not specified
hash function on query text is used.
- changed signature of pico.sql to:
pico.sql(pattern[, params, options)
options={'traceable'=boolean,'query_id'=string}

Current commit introduces
- _pico_routine replicated table
- procedure creation with SQL API