Skip to content

Migrate to non-SQL INSERT

This task is inspired by the problems with a custom bucket_id SQL function for Picodata and some performance issues (one, two). The main idea is to modify the plan and the storage runtime.

  1. Plan.
    • Always generate a Motion node under Insert.
    • If insertion can be done locally we should use a new local segment motion policy. It produces a Rust virtual table (instead of tarantool space as we do for selects) on the storage (with buckets in the index as we know the distribution key from insertion table).
  2. Storage runtime.
    • Validates that Insert node has a single Motion child.
    • Insert into the target space tuples from the virtual table. Do not forget to extend them with the bucket_id field (in the case of the sharded spaces when the VT index is non-empty). All insertions must be done within a single transaction.
  3. Router runtime.
    • If we have a chain of node INSERT - LOCAL SEGMENT MOTION - VALUES and the VALUES node has a replicated distribution (i.e. contains only constants), then we can build a virtual table on the router without VALUES dispatch to the storages.
Edited by Denis Smirnov