- Nov 22, 2023
-
-
- Sep 28, 2023
-
-
- Jul 13, 2023
-
-
Denis Smirnov authored
-
- Jul 04, 2023
-
-
Denis Smirnov authored
-
- Jun 15, 2023
-
-
Denis Smirnov authored
Current commit redesigns distributed INSERT command. Previously we dispatched insert SQL command to the storages. If INSERT could be done locally (without building a virtual table on the router) we used SQL "bucket_id(<string>)" function to recalculate buckets on the starage via SQL. This approach had the main disadvantage - it worked only with a "bucket_id" SQL function that had a single argument as a parameter. An attempt to support multiple parameters of different types (tuple columns as we plan to implement for Picodata engine in future) faced serious technical problems. Also, the old implementation had performance issues: - we created temporary spaces on the storage even for a single VALUE insertion - we always dispatched VALUES to the storage to build a virtual table - the resulting SQL was too verbose (as we produced a subquery under INSERT node) It was desided to get rid of the old approach and migrate to non-SQL insertion. It means that a new type of motion was introduled - local segment. It allows us to build a virtual table on the storage in Rust memory, then using space API transform and insert collected tuples directly into the target space within a single transaction. Also we can materailize VALUES (if they contain only constants) on the router and get rid of the redundant network transmission over the network.
-
- May 02, 2023
-
-
Denis Smirnov authored
Implement the picodata engine for sbroad to integrate distributed SQL into the picodata binary. In fact, picodata engine simple exports public rust functions from api.rs. On the picodata side they (and some lua functions) are imported and included into the bunary on the build phase. Current commit is POC and have some problems: - DDL is still not implemented in the picodata (_pico_space is mocked); - we include tarantool module symbols twice (first time in sbroad, second time in picodata binary); But anyway, it works and all these problems should be solved in the next commits.
-
- Feb 08, 2023
-
-
Denis Smirnov authored
-
- Jan 19, 2023
-
-
Denis Smirnov authored
We stop using VALUES to store temporary tuple on the storages and switch to the tarantool spaces instead. This is done to avoid the problems with the auto generated column names in VALUES, parser stack and parameters limitations. Tarantool forbids to use multiple space engines in a single transaction. So for vinyl tables we have to use vinyl spaces as a tepmorary storage. For memtx tables we can use temporary memtx spaces. One more important change is that we can't insert values of different numeric types in a number column (as we don't cast them as the local SQL does).
-
- Jan 16, 2023
-
-
Denis Smirnov authored
Reduce the amount of the heap allocations (use recursion instead of the heap stack).
-
- Dec 29, 2022
-
-
Denis Smirnov authored
BREAKING CHANGE: api functions have changed signatures. This commit changes the way how the router dispatches commands to the storages. Previously, the router compiled the SQL statements with parameters from the plan subtrees and sent them to the storages. Now the router sends the raw IR subtrees to the storages. 1. The subtrees are constructed from the original plan nodes for performance reasons: the node's memory chunk is extracted from the original plan tree (replaced with invalid parameter node) and reused in the sub-plan. 2. The router-storage message consists of the two parts: required and optional. The required part is the hash of the sub-plan (excluding constants - analogue of the SQL pattern in the previous version) and parameters. The optional part is the IR itself and the syntax node tree (precompiled on the router to skip redundant work on the multiple storages). Storage uses a lazy deserialization of the message: - first it deserialized the part with the hash and parameters (to check the plan cache) - if the cache lookup failed, it deserializes the IR and the syntax node tree and updates the cache. 3. The SHA256 hash was replaced with BLAKE3 for performance reasons.
-
- Nov 22, 2022
-
-
Denis Smirnov authored
-
- Sep 21, 2022
-
-
Igor Kuznetsov authored
-
- Sep 19, 2022
-
-
Igor Kuznetsov authored
-
- Sep 15, 2022
-
-
Igor Kuznetsov authored
-
- Sep 09, 2022
-
-
Denis Smirnov authored
Implement opentelemetry instrumentation in sbroad. How to test: docker run --name jaeger -d --rm -p6831:6831/udp -p6832:6832/udp \ -p16686:16686 -p14268:14268 jaegertracing/all-in-one:latest Then run a stress test with sbroad. The results would be available at http://localhost:16686/
-
- Sep 08, 2022
-
-
Igor Kuznetsov authored
-
- Aug 09, 2022
-
-
Denis Smirnov authored
Thanks to the updates in tarantool module we don't use tarantool symbols to work with decimal. As a bonus we can remove our mocking framework with dynamic linking of the decNumber library to cargo test binary.
-
- Aug 02, 2022
-
-
Denis Smirnov authored
-
Denis Smirnov authored
Previously, we created a static archives of the msgpuck and decNumber libraries and made a statically linked them into the test executable. After tarrantool module migrated to dlsym, we can no longer use static linking. As a result we build shared libraries for msgpuck and decNumber to dynamic link them into the unit test binary.
-
- Jul 05, 2022
-
-
Дмитрий Кольцов authored
picodata features enables Picodata Tarantool fork functionality Schema enables functionality needed to operate spaces metadata
-
Дмитрий Кольцов authored
We want to get reproducible builds. Cargo.lock does exactly this. On the start of the development we were following the rule that "libraries does not include cargo.lock in git, binary does". However we misinterpreted the role of Sbroad. It is built as a "cdylib" so we should treat it as an executable and include cargo.lock
-