godzie44 authored 10 months ago and Georgy Moshkin committed 10 months ago

chore(plugins): `update_plugin_tiers` API call is split into `service_append_tier` and `service_remove_tier` calls

godzie44 authored 11 months ago and Georgy Moshkin committed 10 months ago

- add description field to `_pico_plugin` and `_pico_service`
- plugin loading: filter files by its extensions
- plugin loading: check possible service collisions
- plugin loading: now plugin searching at {plugin_dir}/{plugin_name} path
- now all errors between plugin and picodata wrapping with `BoxError` (this protects plugin caller from UAF)

This commit fixes that non-admin users can't read queries from the query cache.
An attempt to read from the cache used to result in the following error:
sbroad error: failed to get space_def: box error: AccessDenied: Read access to space '_pico_table' is denied for user 'alex3'

Previously, an error would occur when a user ran the same query twice.
During the first execution, the query was prepared and cached. However,
on the second execution, the client attempted to retrieve the query from the
cache and encountered an access denied error.

- we didn't check that index name is unique
in indexes meta table. That led to panics
when index with same name was created for
two tables.
- previous commit set wrong sbroad submodule.
Set sbroad submodule to correct commit

Arseniy Volynets authored 11 months ago and Denis Smirnov committed 10 months ago

- update sbroad submodule to commit
with support of UNION operator
- add integration tests for global
tables (sharded tables were tested
in cartridge tests)
- Usage:

`select a from t union select b from t2`

godzie44 authored 11 months ago and Yaroslav Dynnikov committed 10 months ago

Plugin topology is a mapping plugin service to tiers.

Introduce new `pico.update_plugin_tiers` API.
This function may be used for control
where plugin services must be running and where not.

Plugin integration tests have also been refactored.

godzie44 authored 11 months ago and Yaroslav Dynnikov committed 11 months ago

Now for installation and enable plugin, there are two steps instead of one
(`pico.load_plugin` previously).
1. Install plugin (`pico.install_plugin`):
validate plugin, create system tables.
2. Enable plugin (`pico.enable_plugin`):
loads plugin into instances memory, call `on_start` callback.

Same for a plugin removing process (`pico.remove_plugin` previously):
1. Disable plugin (`pico.disable_plugin`): call `on_stop` callbacks, drop
plugin from instances.
2. Remove plugin (`pico.remove_plugin`):
remove plugin meta from system tables.

Currently only 2 cases are supported:
- config_file
- commandline_or_environment

Before this change we would silently replace an instance with current
grade not Online if a new instance with the same name wanted to join.
This was a mistake as it would allow a race condition when joining
several instances at once.

- update sbroad submodule to commit with support of to_char
- add some integration tests, note: there are also
integration tests in cartridge
- This function is implemented as calling `format` on
given datetime object
- Usage:

to_char(datetime, format)

select to_char(datetime_column, 'Last updated: %Y-%m-%d') from t

Second argument is optional and could be specified as `null`:

select to_char(datetime_column, null) from t

- format is specified with the same rules as in strftime function
https://man.freebsd.org/cgi/man.cgi?query=strftime&sektion=3

Arseniy Volynets authored 11 months ago and Denis Smirnov committed 11 months ago

- update sbroad submodule with support of
datetime type and new `to_date` builtin
function.
- initialize sbroad builtin functions in
init common. Currently there is only one
builtin function `to_date`
- add integration tests for datetime types
- update tarantool module to commit with
fixes for datetime deserialization

BREAKING CHANGE: "name" index in the _pico_index has changed:
previously it guarantees uniqueness for a pair of (space_id,
index_name). Now index name must be unique among the cluster.
So, all the global indexes for _pico tables were renamed.

BREAKING CHANGE: _pico_index structure has changed.

The reason for refactoring is the request for the cluster wide
indexes with all possible options. The old structure didn't allow
us to support them all.

godzie44 authored 11 months ago and Yaroslav Dynnikov committed 11 months ago

Introduce a plugin system for picodata cluster.
Plugin system contains:
- `picoplugin` crate with plugin interfaces.
This crate exposes plugin interfaces for user implementation and API
that used by picodata to interact with plugin as with a code from dynamic library.
- Support for plugin work cycle in picodata instance work cycle.
Picodata instance interacts with running plugin by `PluginManager`
component.
Typically, plugins react to events that occur in instance or whole cluster.
- New lua API with plugin CRUD actions.
    - pico.create_plugin - load a new plugin into picodata cluster
    - pico.remove_plugin - remove plugin from picodata cluster
    - pico.update_plugin_config - update plugin service configuration