feature: once eval lua implementation
feature: picolib.once_synced_tx_checked_lua_eval (aka cluster.once) feature: picolib.once_synced_tx_checked_lua_eval (aka cluster.once)
Introduce new functions in Lua picolib public API:
-
raft_sync(timeout_millis)-- checks within the timeout if the instance's Raft state is up-to-date. -
once_synced_tx_checked_lua_eval(timeout_millis, lua_code)-- executes the following actions within the timeout:-
Checks within the timeout if the instance's Raft state is not outdated too much (same as
raft_sync). -
"Validates" the lua code using
box.begin(); <lua_code> ; box.rollback (). This is supposed to catch lua syntax errors and errors related to box API like creating an already existing space. Assumes the side effects of thelua_codecan be rolled back. -
Proposes a Raft entry with the
lua_codeto the Raft leader. -
Waits until the proposed Raft entry is committed in the cluster and the Raft commit handler for this entry executes the
lua_code. There is also a persistent compare-and-swap safety mechanism that ensures that this function call does not interfere with it's concurrent calls in the Raft cluster. The mechanism usesonce_indexwhich is stored in raft_state space in Tarantool.
-
#19 (closed) https://docs.google.com/document/d/1aKmsTsKgpjRshwjjIJtkt17w-A_hJU-64TJ6nPxk79w/edit
Example (Click to expand)
tarantool> picolib.once_synced_tx_checked_lua_eval(1000, 'jkljk')
2022-01-25 18:06:13.156 [47631] main/103/interactive I> propose binary data (24 bytes).......................................
2022-01-25 18:06:13.156 [47631] main/103/interactive I> ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2022-01-25 18:06:13.165 [47631] main/118/lua I> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
2022-01-25 18:06:13.165 [47631] main/118/lua I> --- Ready { number: 2, ss: None, hs: None, read_states: [], entries: [term: 7 index: 28 data: "\222\244sync\330\002\r\253\340V\335\316OC\240\350\356\256\333\031\301\017"], snapshot: , is_persisted_msg: false, light: LightReady { commit_index: None, committed_entries: [], messages: [] }, must_sync: true }
2022-01-25 18:06:13.165 [47631] main/118/lua I> --- uncommitted_entry: term: 7 index: 28 data: "\222\244sync\330\002\r\253\340V\335\316OC\240\350\356\256\333\031\301\017"
2022-01-25 18:06:13.165 [47631] main/118/lua I> ADVANCE -----------------------------------------
2022-01-25 18:06:13.165 [47631] main/118/lua V> +++ term(idx=28) -> 7
2022-01-25 18:06:13.165 [47631] main/118/lua V> persisted index 28, raft_id: 1
2022-01-25 18:06:13.166 [47631] main/118/lua V> +++ term(idx=28) -> 7
2022-01-25 18:06:13.166 [47631] main/118/lua V> committing index 28, index: 28, raft_id: 1
2022-01-25 18:06:13.166 [47631] main/118/lua I> --- LightReady { commit_index: Some(28), committed_entries: [term: 7 index: 28 data: "\222\244sync\330\002\r\253\340V\335\316OC\240\350\356\256\333\031\301\017"], messages: [] }
2022-01-25 18:06:13.166 [47631] main/118/lua I> --- committed_entry: term: 7 index: 28 data: "\222\244sync\330\002\r\253\340V\335\316OC\240\350\356\256\333\031\301\017"
2022-01-25 18:06:13.166 [47631] main/118/lua I> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
---
- null
- 'Lua error: Syntax error: [string "chunk"]:1: ''='' expected near ''<eof>'''
...
tarantool> picolib.once_synced_tx_checked_lua_eval(1000, 'box.schema.space.create("s222")')
2022-01-25 18:06:19.300 [47631] main/103/interactive I> propose binary data (24 bytes).......................................
2022-01-25 18:06:19.300 [47631] main/103/interactive I> ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2022-01-25 18:06:19.358 [47631] main/118/lua I> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
2022-01-25 18:06:19.358 [47631] main/118/lua I> --- Ready { number: 3, ss: None, hs: None, read_states: [], entries: [term: 7 index: 29 data: "\222\244sync\330\002\022+xPY\256@c\205\346\226\357\262sZ\266"], snapshot: , is_persisted_msg: false, light: LightReady { commit_index: None, committed_entries: [], messages: [] }, must_sync: true }
2022-01-25 18:06:19.358 [47631] main/118/lua I> --- uncommitted_entry: term: 7 index: 29 data: "\222\244sync\330\002\022+xPY\256@c\205\346\226\357\262sZ\266"
2022-01-25 18:06:19.358 [47631] main/118/lua I> ADVANCE -----------------------------------------
2022-01-25 18:06:19.358 [47631] main/118/lua V> +++ term(idx=29) -> 7
2022-01-25 18:06:19.358 [47631] main/118/lua V> persisted index 29, raft_id: 1
2022-01-25 18:06:19.358 [47631] main/118/lua V> +++ term(idx=29) -> 7
2022-01-25 18:06:19.358 [47631] main/118/lua V> committing index 29, index: 29, raft_id: 1
2022-01-25 18:06:19.358 [47631] main/118/lua I> --- LightReady { commit_index: Some(29), committed_entries: [term: 7 index: 29 data: "\222\244sync\330\002\022+xPY\256@c\205\346\226\357\262sZ\266"], messages: [] }
2022-01-25 18:06:19.359 [47631] main/118/lua I> --- committed_entry: term: 7 index: 29 data: "\222\244sync\330\002\022+xPY\256@c\205\346\226\357\262sZ\266"
2022-01-25 18:06:19.359 [47631] main/118/lua I> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
---
- null
- 'Lua error: Execution error: Space ''s222'' already exists'
...
tarantool> picolib.once_synced_tx_checked_lua_eval(1000, 'box.schema.space.create("s2222")')
2022-01-25 18:06:23.141 [47631] main/103/interactive I> propose binary data (24 bytes).......................................
2022-01-25 18:06:23.141 [47631] main/103/interactive I> ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2022-01-25 18:06:23.235 [47631] main/118/lua I> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
2022-01-25 18:06:23.235 [47631] main/118/lua I> --- Ready { number: 4, ss: None, hs: None, read_states: [], entries: [term: 7 index: 30 data: "\222\244sync\330\002\321\352w\244#\315JM\222\006`,\217\321P\036"], snapshot: , is_persisted_msg: false, light: LightReady { commit_index: None, committed_entries: [], messages: [] }, must_sync: true }
2022-01-25 18:06:23.235 [47631] main/118/lua I> --- uncommitted_entry: term: 7 index: 30 data: "\222\244sync\330\002\321\352w\244#\315JM\222\006`,\217\321P\036"
2022-01-25 18:06:23.235 [47631] main/118/lua I> ADVANCE -----------------------------------------
2022-01-25 18:06:23.236 [47631] main/118/lua V> +++ term(idx=30) -> 7
2022-01-25 18:06:23.236 [47631] main/118/lua V> persisted index 30, raft_id: 1
2022-01-25 18:06:23.236 [47631] main/118/lua V> +++ term(idx=30) -> 7
2022-01-25 18:06:23.236 [47631] main/118/lua V> committing index 30, index: 30, raft_id: 1
2022-01-25 18:06:23.236 [47631] main/118/lua I> --- LightReady { commit_index: Some(30), committed_entries: [term: 7 index: 30 data: "\222\244sync\330\002\321\352w\244#\315JM\222\006`,\217\321P\036"], messages: [] }
2022-01-25 18:06:23.236 [47631] main/118/lua I> --- committed_entry: term: 7 index: 30 data: "\222\244sync\330\002\321\352w\244#\315JM\222\006`,\217\321P\036"
2022-01-25 18:06:23.236 [47631] main/118/lua I> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2022-01-25 18:06:23.237 [47631] main/103/interactive I> propose binary data (68 bytes).......................................
2022-01-25 18:06:23.237 [47631] main/103/interactive I> ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
2022-01-25 18:06:23.341 [47631] main/118/lua I> vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
2022-01-25 18:06:23.341 [47631] main/118/lua I> --- Ready { number: 5, ss: None, hs: None, read_states: [], entries: [term: 7 index: 31 data: "\224\255once_eval_lua\330\002\t\007\371^4wD\346\257X\247\273\"\357\222\227\005\331 box.schema.space.create(\"s2222\")"], snapshot: , is_persisted_msg: false, light: LightReady { commit_index: None, committed_entries: [], messages: [] }, must_sync: true }
2022-01-25 18:06:23.341 [47631] main/118/lua I> --- uncommitted_entry: term: 7 index: 31 data: "\224\255once_eval_lua\330\002\t\007\371^4wD\346\257X\247\273\"\357\222\227\005\331 box.schema.space.create(\"s2222\")"
2022-01-25 18:06:23.341 [47631] main/118/lua I> ADVANCE -----------------------------------------
2022-01-25 18:06:23.341 [47631] main/118/lua V> +++ term(idx=31) -> 7
2022-01-25 18:06:23.342 [47631] main/118/lua V> persisted index 31, raft_id: 1
2022-01-25 18:06:23.342 [47631] main/118/lua V> +++ term(idx=31) -> 7
2022-01-25 18:06:23.342 [47631] main/118/lua V> committing index 31, index: 31, raft_id: 1
2022-01-25 18:06:23.342 [47631] main/118/lua I> --- LightReady { commit_index: Some(31), committed_entries: [term: 7 index: 31 data: "\224\255once_eval_lua\330\002\t\007\371^4wD\346\257X\247\273\"\357\222\227\005\331 box.schema.space.create(\"s2222\")"], messages: [] }
2022-01-25 18:06:23.342 [47631] main/118/lua I> --- committed_entry: term: 7 index: 31 data: "\224\255once_eval_lua\330\002\t\007\371^4wD\346\257X\247\273\"\357\222\227\005\331 box.schema.space.create(\"s2222\")"
2022-01-25 18:06:23.343 [47631] main/118/lua I> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
---
- true
...
tarantool> inspect()
---
- raft_log:
- +-------------+-----+-----+-----------------------------------------------------------------------------------------------+
- | entry_type |index|term | msg |
- +-------------+-----+-----+-----------------------------------------------------------------------------------------------+
- |"EntryNormal"| 1 | 1 | ["empty"] |
- |"EntryNormal"| 2 | 1 | ["sync","72108ecc-4363-4c6c-9ac7-04e06aaf0734"] |
- |"EntryNormal"| 3 | 1 | ["once_eval_lua",1,"box.schema.space.create(\"s1\")"] |
- |"EntryNormal"| 4 | 1 | ["sync","3e9db43a-f851-45de-ba79-94091a00431f"] |
- |"EntryNormal"| 5 | 1 | ["once_eval_lua",2,"box.schema.space.create(\"s1\")"] |
- |"EntryNormal"| 6 | 2 | ["empty"] |
- |"EntryNormal"| 7 | 2 | ["sync","0615e264-1ea6-42ea-8407-015d8397a50d"] |
- |"EntryNormal"| 8 | 2 | ["sync","1a6b32d6-e754-43f8-8f4b-77ac257982a8"] |
- |"EntryNormal"| 9 | 2 | ["once_eval_lua",2,"box.schema.space.create(\"s2\")"] |
- |"EntryNormal"| 10 | 2 | ["sync","fdb7b48a-c0c2-476e-876f-6676884bbaa1"] |
- |"EntryNormal"| 11 | 3 | ["empty"] |
- |"EntryNormal"| 12 | 3 | ["sync","eeec2cb1-6660-4d15-9d67-9b1192dac909"] |
- |"EntryNormal"| 13 | 3 | ["sync","5ef73af0-c91b-4949-9a1f-c3770712169d"] |
- |"EntryNormal"| 14 | 4 | ["empty"] |
- |"EntryNormal"| 15 | 4 | ["sync","b2d18efb-22c5-4803-add7-4ff4ee646bec"] |
- |"EntryNormal"| 16 | 4 | ["sync","bd499066-fa80-428c-afd7-d1faabc6d3cd"] |
- |"EntryNormal"| 17 | 5 | ["empty"] |
- |"EntryNormal"| 18 | 5 | ["sync","b3cfdbdd-3bfc-40df-aa39-ec85fe5aea49"] |
- |"EntryNormal"| 19 | 5 | ["sync","62201993-b4e4-48b2-9b32-c68e03f19f9d"] |
- |"EntryNormal"| 20 | 5 | ["sync","46a9652d-cdb4-4d6e-b803-3d21521f308f"] |
- |"EntryNormal"| 21 | 5 | ["sync","da48aae3-437d-4e9c-8646-2321b1eb98f6"] |
- |"EntryNormal"| 22 | 5 | ["once_eval_lua","31b129cc-1171-4568-8923-bc13a830530a",3,"box.schema.space.create(\"s22\")"] |
- |"EntryNormal"| 23 | 6 | ["empty"] |
- |"EntryNormal"| 24 | 6 | ["sync","e7672342-4eb5-4556-be76-26225ec26e93"] |
- |"EntryNormal"| 25 | 6 | ["sync","18b892c5-45c8-4f9e-87bc-3d00425f9194"] |
- |"EntryNormal"| 26 | 6 |["once_eval_lua","258256be-86b8-4d9c-8519-6abc9d499e80",4,"box.schema.space.create(\"s222\")"] |
- |"EntryNormal"| 27 | 7 | ["empty"] |
- |"EntryNormal"| 28 | 7 | ["sync","0dabe056-ddce-4f43-a0e8-eeaedb19c10f"] |
- |"EntryNormal"| 29 | 7 | ["sync","122b7850-59ae-4063-85e6-96efb2735ab6"] |
- |"EntryNormal"| 30 | 7 | ["sync","d1ea77a4-23cd-4a4d-9206-602c8fd1501e"] |
- |"EntryNormal"| 31 | 7 |["once_eval_lua","0907f95e-3477-44e6-af58-a7bb22ef9297",5,"box.schema.space.create(\"s2222\")"]|
- +-------------+-----+-----+-----------------------------------------------------------------------------------------------+
- raft_state:
- +------------+-----+
- | key |value|
- +------------+-----+
- | "applied" | 31 |
- | "commit" | 31 |
- |"once_index"| 5 |
- | "term" | 7 |
- | "vote" | 1 |
- +------------+-----+
...Edited by Alexey Protsenko