Instead of linking against static vendored libcurl unconditionally, use
pkg_config/pkgconf to discover system libcurl and its version. At the
moment of writing, 8.4.0 is sufficient and thus can be used in
dynamic build mode.

This patch boils down to the following improvements:

* Various linkage-related shenanigans have been documented and revamped.
* All logic to build tarantool-sys is now encapsulated in tarantool-build.
* Most of the magic `println!`s are now encapsulated in build-rs-helpers.
* Unsound function `export_symbols` has been dropped in favor of
  explicit linker flags (`--require-defined` for linux & `-u` for mac).
* `TarantoolBuildRoot` is now in charge of building and linking
  tarantool libs. It defines the build tree via path helper functions.
* Picodata's `build.rs` should be a little easier to read.

Needed since introduction of plugin version compatibility check in
86a08ef9. For release candidate versions
we need to properly bump all versions. We could've bumped picodata
version instead but for simplicity just revert to currently unreleased
24.6.0. Ideally we should describe release candidates in our versioning
policy.

The patch prepares picodata-plugin and picodata-plugin-proc-macro
for publishing to crates.io.

This includes Makefile wrapper over cargo commands and manual CI job

For now minimum of required fields have been filled. We should come back
to fill proper readmes, categories, home page etc.

picoplugin -> picodata-plugin
picoplugin_gen -> picodata-plugin-proc-macro

This commit embeds in plugins a new variable, containing picoplugin
version used to build it. If this version differs from the picodata version
when the plugin gets loaded, the loading fails with an error reporting
the expected and the current versions. This check can be disabled via env
PICODATA_UNSAFE_DISABLE_PLUGIN_COMPATIBILITY_CHECK.

Maksim Kaitmazian authored 5 months ago and Maksim Kaitmazian committed 5 months ago

This commit resolves the performance issues observed in pgbench
benchmark*. Because of the Nagle's algorithm, every message was read with
50ms delay, while the query execution took only 1ms. By disabling the
algorithm via TCP_NODELAY option, we achieve 50 times performance
improvement for a single client in simple query mode.

* https://git.picodata.io/picodata/picodata/picodata/-/issues/1002

Needed because $CI_COMMIT_BRANCH is not available for tags

Up to this patch `instance_uuid` and `replicaset_uuid` were generated
using name-based UUID v3. So all instances named `i1` in all clusters
had the same UUID.

With this patch UUIDs become random v4. Particular UUID values are
generated in two locations:

1. `start_boot` (src/lib.rs)
2. `handle_join_request_and_wait` -> `build_instance` (src/rpc/join.rs)

Function `Instance::new()` is removed in favor of explicit struct
construction.

Dmitry Rodionov authored 8 months ago and Denis Smirnov committed 5 months ago

Previous version was using yaml-rust which is marked as unmaintained.
This gets reported by dependency scanner we use for certification.
Though worth noting that serde-yaml itself is now archived. There is a
promising replacement library but no need to hurry with that just yet.
But anyway, we don't need this library in the production build, only
for tests. So, moved it to development dependencies.

Co-authored-by: Denis Smirnov <sd@picodata.io>

Co-authored-by: Georgy Moshkin <gmoshkin@picodata.io>

1. A better VDBE caching was implemented for local SQL.
2. Local SQL is called via C API rather then Lua.
3. A new prepare-unprepare fiber added to manage SQL statement
   cache.

Arseniy Volynets authored 9 months ago and Arseniy Volynets committed 6 months ago

- For cacheable requests, we now send only cache required data needed to
execute the query if it is in the storage cache. If query is not in the
storage cache, storage returns empty result with flag indicating there
was a cache miss. Then coordinator sends the second round of requests to
replicasets where cache miss happened. It sends the full message there.
- Parameters for required message are now computed without compilation
to SyntaxPlan - Fixed parameter order in SyntaxPlan

Previously we cached only the leaf sub-trees of the IR plan under
the first level of motions. The problem was in temporary tables as
we used random names for them and dropped after usage (and invalidated
corresponding prepared statements as well). Now we use idempotent
names for temporary tables and truncate them after usage. It allows
us to keep prepared statements in the cache. One more thing: as we
use the same temporary table for multiple queries, we need to use
latch locks for the prepared statement cache. One day we'll switch
to more granular locks (mostly for vinyl spaces), but let's use
current approach for the start.

We also removed can_be_cached flag from the code. Only DQL uses VDBE
and needs caching, and all these queries can be cached. The only
exception are the queries with Local and LocalSegment motion policies
but only DML can have them. So, no need for this flag and let's make the
code cleaner.

Previously macros won't work when using solely picoplugin as SDK.
It is fixed in linked tarantool-module MR, this MR introduces tests for that.

Linked [MR](https://git.picodata.io/picodata/picodata/tarantool-module/-/merge_requests/542) in tarantool-module.

See https://git.picodata.io/picodata/picodata/tarantool-module/-/merge_requests/537

godzie44 authored 8 months ago and Maksim Kaitmazian committed 7 months ago

Closes #584

godzie44 authored 8 months ago and Maksim Kaitmazian committed 7 months ago

Closes #741

Dmitry Rodionov authored 7 months ago and Dmitry Ivanov committed 7 months ago

> cargo tree -i either
either v1.13.0
├── itertools v0.10.5
│   └── sbroad-core v0.1.0 (picodata/sbroad/sbroad-core)
│       └── picodata v24.5.0 (picodata/picodata)

feature(plugin migration): use md5 from migration files meta for to confirm consistency of migrations between versions

Closes #652

The problem appeared in form of a failed debian package build. aws-lc-rs
build script was failing with:

thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.19.0/builder/cc_builder.rs:244:13:
Your compiler (cc) is not supported due to a memcmp related bug
reported in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=95189.We
strongly recommend against using this compiler.

Since we use system openssl we dont need to rely on these backends.

Error message is probably correct, so upgrading the compiler is still a
good idea.