- May 12, 2022
-
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
Commit 1a3b5233 missed a bug. Iteration over instances could be aborted by an exception during teardown. It resulted in garbage process remaining alive after pytest termination.
-
Yaroslav Dynnikov authored
1. Lower log level of connection errors in `netork.rs`. 2. Give raft fibers a name.
-
Yaroslav Dynnikov authored
There were some problems with join requests synchronization. Raft forbids proposing a configuration change if there's another one uncommitted (see [1]). In that case, it replaces an `EntryConfChange` with an `EntryNormal`. It could happen at any time even without bugs in code due to the network partitioning, and its the repsonsibility of the picodata product to handle it properly. Earlier, there was no way to wait when raft leaves the joint state. It used to slow down cluster assembling and made it race-prone. The waiting for the cluster readiness is also important in tests. Some operations (the most important amongst them is leader switching) are impossible until instance finishes promotion to a voter. For instance, raft rejects `MsgTimeoutNow` unless the node is promotable (see [2]). It makes some testing scenarios flaky. This patch introduces new synchronization primitive - `JointStateLatch`. The latch is held on the leader and is locked upon `raw_node.propose_conf_change()`. It's unlocked only when the second (implicit) conf change that represents leaving joint state is committed. The latch also tracks the index of the corresponding `EntryConfChange`. Even if raft ignores it for any reason, the latch is still unlocked as soon as the committed index exceeds the one of the latch. [1] https://github.com/tikv/raft-rs/blob/v0.6.0/src/raft.rs#L2014-L2026 [2] https://github.com/tikv/raft-rs/blob/v0.6.0/src/raft.rs#L2314 Close https://git.picodata.io/picodata/picodata/picodata/-/issues/47 Close https://git.picodata.io/picodata/picodata/picodata/-/issues/53
-
Yaroslav Dynnikov authored
Waiting for a valid `leader_id` on a node isn't enough. It may already have one, but still be a Learner. Instead, the fixture should wait until the node is promoted to voter.
-
Yaroslav Dynnikov authored
The assertion `status == "Leader"` was in the first place, and `raft_timeout_now` call was unreachable.
-
- May 11, 2022
-
-
Yaroslav Dynnikov authored
1. Print logs to the stderr so that they interleave with tarantool logs. 2. Fix `cluster.__repr__()`.
-
Yaroslav Dynnikov authored
1. Review `Pipfile`: - Remove unused `filelock`; - Install `mypy` - static type checker for Python. 2. Add new command `pipenv run lint`. 3. Enable `mypy` in CI. Fix reported errors in `test_basics.py`. 4. Renew readme.
-
Yaroslav Dynnikov authored
In `conftest.py`: - *Add* function `xdist_worker_number`. It converts `str(worker_id)` into `int`. It serves as a substitute for `session_data_mutex` for parallel test runs. - *Change* `normalize_net_box_result`. Replace a function with a decorator. Also, handle typical kinds of picodata responses. Extensively test it in `test_basics.py::test_eval/call_normalization`. - *Add* dataclass `RaftStatus`. It shouldn't be used outside `conftest.py`. It only makes assertions more brief in logs and understndable in code. - *Add* all raft stuff into the `Instance` class. This implies `raft_propose_eval`, `assert_raft_status` all `promote_or_fail` moved from `util.py`. - *Change* fixture `compile`. No need in extra logics since commit 59c31cb8. - *Preserve* fixture `binary_path`. - *Remove* fixtures `session_data_mutex` and `run_id`. Superseded with `xdist_worker_number`. - *Remove* fixtures `run_cluster` and `run_instance`. Superseded with `cluster.deploy(...)`. - *Remove* function `wait_tcp_port`. It's never enough to check raw socket. Superseded with `instance.wait_ready()`. - Give the instances clean names `i1, i2, ...`, and simple addresses `127.7.0.1:3301`. For the parallel test run use different IPs `127.7.N.1` etc. In `test_basics.py`: - *Add* `test_xdist_worker_number`. - *Add* `test_call_normalization` and `test_eval_normalization`. - *Add* `test_process_management`. It's brand new, never implemented in luatest before. - *Rename* `test_single_instance_raft_eval` to `test_propose_eval` and extend it with additional assertion from `single_test.lua`. - *Remove* `test_instance`. A part of its logics is moved to `test_call/eval_normalization`. The other part is rewritten and extended in `test_process_management`. - *Remove* `test_cluster`. It was completely useless because of inappropriate synchronization and no valuable assertions. - *Remove* `test_propose_eval`. It wasn't that useful, but failed because of inappropriate synchronization. In `test_couple.py`: - *Preserve* `test_follower_proposal` and `test_failover`. Just slightly refactor according to the new `conftest.py` API. In `util.py` (completely removed): - *Remove* decorator `retry`. Needless. - *Remove* decorator `retry_on_network_errors`. Inappropriate predicate didn't catch Lua errors. - *Remove* everything related to raft. Move it into `conftest.py`. Part of https://git.picodata.io/picodata/picodata/picodata/-/issues/59
-
- May 08, 2022
-
-
Yaroslav Dynnikov authored
-
- May 06, 2022
-
-
- Apr 29, 2022
-
-
Yaroslav Dynnikov authored
-
- Apr 28, 2022
-
-
Yaroslav Dynnikov authored
We've recently noticed many files in `vendor/` dir have windows-like EOL `crlf`. When cloning the repo, git produces a huge diff due to the EOL normalization built-in feature. This patch disbles it for the certain path.
-
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-
Sergey V authored
-
- Apr 27, 2022
-
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-
Georgy Moshkin authored
-
- Apr 26, 2022
-
-
Georgy Moshkin authored
-
- Apr 24, 2022
-
-
Yaroslav Dynnikov authored
We don't want a child process to live without the supervisor. Usually, supervisor waits for child forever and retransmits termination signals. But if the parent is killed with a SIGKILL there's no way to pass anything. This patch supplies a child process with a `supervisor_fuse` fiber. It tries to read from a pipe (that supervisor never writes to), and if the writing end is closed, it means the supervisor has terminated. In this case, child process terminates too. Part of https://git.picodata.io/picodata/picodata/picodata/-/issues/56
-
Yaroslav Dynnikov authored
This patch reduces the variable passing trace length. Instead of returning it from function to function, call `exit()` in place. The trace of exit code before the patch: ``` exit() <- main <- main_run <- fork(child) <- tarantool_main ``` And now: ``` exit() <- tarantool_main ``` Type safety is ensured at comilation time by using `!` type for `fn main()`. See https://doc.rust-lang.org/reference/types/never.html
-
Yaroslav Dynnikov authored
-
-
Yaroslav Dynnikov authored
```console $ cargo clippy Compiling picodata v0.1.0 (/home/rosik/w/picodata) warning: 'tarantool-sys/patches-applied' exists, so patching step is skipped warning: single-character string constant used as pattern --> src/args.rs:194:47 | 194 | let (host, port) = match text.rsplit_once(":") { | ^^^ help: try using a `char` instead: `':'` | = note: `#[warn(clippy::single_char_pattern)]` on by default = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_char_pattern warning: called `cloned().next()` on an `Iterator`. It may be more efficient to call `next().cloned()` instead --> src/discovery.rs:154:27 | 154 | let res = peers.difference(&self.visited).cloned().next(); | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `peers.difference(&self.visited).next().cloned()` | = note: `#[warn(clippy::iter_overeager_cloned)]` on by default = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#iter_overeager_cloned warning: you seem to be trying to use `match` for destructuring a single pattern. Consider using `if let` --> src/ipc.rs:27:9 | 27 | / match rmp_serde::encode::write(&mut self.fd, msg) { 28 | | Err(e) => tlog!(Error, "ipc error: {e}"), 29 | | Ok(()) => (), 30 | | } | |_________^ help: try this: `if let Err(e) = rmp_serde::encode::write(&mut self.fd, msg) { tlog!(Error, "ipc error: {e}") }` | = note: `#[warn(clippy::single_match)]` on by default = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#single_match warning: `picodata` (bin "picodata") generated 3 warnings Finished dev [unoptimized + debuginfo] target(s) in 1.41s ```
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
- Apr 21, 2022
-
-
Yaroslav Dynnikov authored
1. Remove it from functions args where it's not necessary. 2. In other places rename it into `supervisor_ipc`.
-
Yaroslav Dynnikov authored
-
- Apr 20, 2022
-
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
Since dynamic discovery isn't implemented yet, instance after `raft_join` has discovery uninitialized. It's not a reason to panic.
-
Yaroslav Dynnikov authored
Use common `Notify` instead of a cond. It makes the code more clear and reliable, because each request is tracked individually and can't be affected by different yields in the main_loop.
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-
Yaroslav Dynnikov authored
-