Describe `conf_change_loop` and graceful shutdown logic.

Bring the algorithm description in accordance with actual
implementation. Also provide some comments to the code.

Everywhere in code use `RaftId`, `RaftTerm`, `RaftIndex` instead of
obscure `u64`.

Georgy Moshkin authored 2 years ago and Georgy Moshkin committed 2 years ago

It prints the raft log in a concise and readable way

Yaroslav Dynnikov authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

When it was added, it used to be a dirty hack for running tests.
Nowadays it's needless.

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

In some states `raft-rs` ignores the ReadIndex request.
Check it preliminary, don't wait for the timeout.

See for details:
- <https://github.com/tikv/raft-rs/blob/v0.6.0/src/raft.rs#L2058>
- <https://github.com/tikv/raft-rs/blob/v0.6.0/src/raft.rs#L2323>

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

32MiB is the minimum allowed memtx_memory size.

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

No need to DDoS a poor raft leader, especially when it's already under
high load.

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

The failure of `raw_node.propose_conf_change` isn't easy to handle.
For now we only know that it shouldn't short-circuit the
`raft_conf_change_loop`. Fuse it with a `fiber.sleep()`.

Current pause value of 1 second is chosen arbitrarily with no actual
reason.

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

Join requests should be made without timeout restrictions. Otherways
it's impossible to tell retried request from instance_id collision.
Retried requests never succeed and return the "already joined" error.