Skip to content
Snippets Groups Projects
  1. Nov 22, 2023
  2. Sep 28, 2023
  3. Jul 13, 2023
  4. Jul 04, 2023
  5. Jun 15, 2023
    • Denis Smirnov's avatar
      feat: non-SQL insert · c64f009b
      Denis Smirnov authored
      Current commit redesigns distributed INSERT command. Previously we
      dispatched insert SQL command to the storages. If INSERT could be
      done locally (without building a virtual table on the router) we
      used SQL "bucket_id(<string>)" function to recalculate buckets
      on the starage via SQL.
      
      This approach had the main disadvantage - it worked only with a
      "bucket_id" SQL function that had a single argument as a parameter.
      An attempt to support multiple parameters of different types (tuple
      columns as we plan to implement for Picodata engine in future) faced
      serious technical problems.
      Also, the old implementation had performance issues:
      - we created temporary spaces on the storage even for a single VALUE
        insertion
      - we always dispatched VALUES to the storage to build a virtual table
      - the resulting SQL was too verbose (as we produced a subquery under
        INSERT node)
      
      It was desided to get rid of the old approach and migrate to non-SQL
      insertion. It means that a new type of motion was introduled - local
      segment. It allows us to build a virtual table on the storage in
      Rust memory, then using space API transform and insert collected tuples
      directly into the target space within a single transaction. Also we
      can materailize VALUES (if they contain only constants) on the router
      and get rid of the redundant network transmission over the network.
      c64f009b
  6. May 02, 2023
    • Denis Smirnov's avatar
      feat(POC): picodata engine · 93685c03
      Denis Smirnov authored
      Implement the picodata engine for sbroad to integrate distributed
      SQL into the picodata binary. In fact, picodata engine simple
      exports public rust functions from api.rs. On the picodata side
      they (and some lua functions) are imported and included into the
      bunary on the build phase.
      
      Current commit is POC and have some problems:
      - DDL is still not implemented in the picodata (_pico_space is mocked);
      - we include tarantool module symbols twice (first time in sbroad,
        second time in picodata binary);
      
      But anyway, it works and all these problems should be solved in the
      next commits.
      93685c03
  7. Feb 08, 2023
  8. Jan 19, 2023
    • Denis Smirnov's avatar
      feat: use spaces as virtual tables · 21f90b86
      Denis Smirnov authored
      We stop using VALUES to store temporary tuple on the storages and
      switch to the tarantool spaces instead. This is done to avoid the
      problems with the auto generated column names in VALUES, parser
      stack and parameters limitations.
      
      Tarantool forbids to use multiple space engines in a single transaction.
      So for vinyl tables we have to use vinyl spaces as a tepmorary storage.
      For memtx tables we can use temporary memtx spaces.
      
      One more important change is that we can't insert values of
      different numeric types in a number column (as we don't cast them
      as the local SQL does).
      21f90b86
  9. Jan 16, 2023
  10. Dec 29, 2022
    • Denis Smirnov's avatar
      feat!: dispatch IR instead of the SQL with parameters · 214b55a5
      Denis Smirnov authored
      BREAKING CHANGE: api functions have changed signatures.
      
      This commit changes the way how the router dispatches commands to
      the storages. Previously, the router compiled the SQL statements
      with parameters from the plan subtrees and sent them to the storages.
      
      Now the router sends the raw IR subtrees to the storages.
      
      1. The subtrees are constructed from the original plan nodes
         for performance reasons: the node's memory chunk is extracted
         from the original plan tree (replaced with invalid parameter node)
         and reused in the sub-plan.
      2. The router-storage message consists of the two parts: required and
         optional. The required part is the hash of the sub-plan (excluding
         constants - analogue of the SQL pattern in the previous version)
         and parameters. The optional part is the IR itself and the syntax
         node tree (precompiled on the router to skip redundant work on the
         multiple storages). Storage uses a lazy deserialization of the message:
         - first it deserialized the part with the hash and parameters (to
           check the plan cache)
         - if the cache lookup failed, it deserializes the IR and the syntax
           node tree and updates the cache.
      3. The SHA256 hash was replaced with BLAKE3 for performance reasons.
      214b55a5
  11. Nov 22, 2022
  12. Sep 21, 2022
  13. Sep 19, 2022
  14. Sep 15, 2022
  15. Sep 09, 2022
    • Denis Smirnov's avatar
      feat: opentelemetry · fad79ec7
      Denis Smirnov authored
      Implement opentelemetry instrumentation in sbroad.
      How to test:
      
      docker run --name jaeger -d --rm -p6831:6831/udp -p6832:6832/udp \
      -p16686:16686 -p14268:14268 jaegertracing/all-in-one:latest
      
      Then run a stress test with sbroad. The results would be available
      at http://localhost:16686/
      fad79ec7
  16. Sep 08, 2022
  17. Aug 09, 2022
    • Denis Smirnov's avatar
      feat: get rid of the decimal symbols in sbroad · f23bcd2c
      Denis Smirnov authored
      Thanks to the updates in tarantool module we don't use tarantool
      symbols to work with decimal. As a bonus we can remove our mocking
      framework with dynamic linking of the decNumber library to cargo
      test binary.
      f23bcd2c
  18. Aug 02, 2022
  19. Jul 05, 2022
Loading