Migrate to non-SQL INSERT
This task is inspired by the problems with a custom bucket_id SQL function for Picodata and some performance issues (one, two). The main idea is to modify the plan and the storage runtime.
- Plan.
- Always generate a Motion node under Insert.
- If insertion can be done locally we should use a new local segment motion policy. It produces a Rust virtual table (instead of tarantool space as we do for selects) on the storage (with buckets in the index as we know the distribution key from insertion table).
- Storage runtime.
- Validates that Insert node has a single Motion child.
- Insert into the target space tuples from the virtual table. Do not forget to extend them with the
bucket_idfield (in the case of the sharded spaces when the VT index is non-empty). All insertions must be done within a single transaction.
- Router runtime.
- If we have a chain of node
INSERT - LOCAL SEGMENT MOTION - VALUESand the VALUES node has a replicated distribution (i.e. contains only constants), then we can build a virtual table on the router without VALUES dispatch to the storages.
- If we have a chain of node
Edited by Denis Smirnov