Skip to content
Snippets Groups Projects
  1. May 16, 2022
    • Yaroslav Dynnikov's avatar
      test: fix flaky args::tests · 18ccf158
      Yaroslav Dynnikov authored
      By default cargo runs tests in parallel in multiple threads.  Both
      `test_log_level` and `test_parse` access environment variables which are
      shared across threads. Consequently, their concurrent modification
      results in the test failure.
      
      This patch unites these two tests making it linear.
      18ccf158
  2. May 13, 2022
  3. May 12, 2022
    • Yaroslav Dynnikov's avatar
      chore: refine logging · c01f07df
      Yaroslav Dynnikov authored
      1. Lower log level of connection errors in `netork.rs`.
      2. Give raft fibers a name.
      c01f07df
    • Yaroslav Dynnikov's avatar
      feature: concurrent join requests handling · 9b079eae
      Yaroslav Dynnikov authored
      There were some problems with join requests synchronization. Raft
      forbids proposing a configuration change if there's another one
      uncommitted (see [1]). In that case, it replaces an `EntryConfChange`
      with an `EntryNormal`. It could happen at any time even without bugs in
      code due to the network partitioning, and its the repsonsibility of
      the picodata product to handle it properly.
      
      Earlier, there was no way to wait when raft leaves the joint state. It
      used to slow down cluster assembling and made it race-prone. The waiting
      for the cluster readiness is also important in tests. Some operations
      (the most important amongst them is leader switching) are impossible
      until instance finishes promotion to a voter. For instance, raft rejects
      `MsgTimeoutNow` unless the node is promotable (see [2]). It makes some
      testing scenarios flaky.
      
      This patch introduces new synchronization primitive - `JointStateLatch`.
      The latch is held on the leader and is locked upon
      `raw_node.propose_conf_change()`. It's unlocked only when the second
      (implicit) conf change that represents leaving joint state is committed.
      The latch also tracks the index of the corresponding `EntryConfChange`.
      Even if raft ignores it for any reason, the latch is still unlocked as
      soon as the committed index exceeds the one of the latch.
      
      [1] https://github.com/tikv/raft-rs/blob/v0.6.0/src/raft.rs#L2014-L2026
      [2] https://github.com/tikv/raft-rs/blob/v0.6.0/src/raft.rs#L2314
      
      Close https://git.picodata.io/picodata/picodata/picodata/-/issues/47
      Close https://git.picodata.io/picodata/picodata/picodata/-/issues/53
      9b079eae
    • Yaroslav Dynnikov's avatar
      fix: pytest wait_ready implementation · 7b94717a
      Yaroslav Dynnikov authored
      Waiting for a valid `leader_id` on a node isn't enough. It may already
      have one, but still be a Learner. Instead, the fixture should wait until
      the node is promoted to voter.
      7b94717a
  4. May 08, 2022
  5. Apr 27, 2022
  6. Apr 26, 2022
  7. Apr 24, 2022
  8. Apr 21, 2022
  9. Apr 20, 2022
  10. Apr 18, 2022
    • Yaroslav Dynnikov's avatar
      Remove unnecessary comments · e4855d7a
      Yaroslav Dynnikov authored
      e4855d7a
    • Yaroslav Dynnikov's avatar
      Fix macos build · af4a2db7
      Yaroslav Dynnikov authored
      Remove prctl dependency that is not available on mac.
      af4a2db7
    • Yaroslav Dynnikov's avatar
      Fix clippy warnings · 7bb8aa15
      Yaroslav Dynnikov authored
      7bb8aa15
    • Yaroslav Dynnikov's avatar
      Fix tests · d5c28170
      Yaroslav Dynnikov authored
      d5c28170
    • Yaroslav Dynnikov's avatar
      Befriend two features · 88ea8ae4
      Yaroslav Dynnikov authored
      This pach establishes a friendship between discovery and raft_join.
      88ea8ae4
    • Yaroslav Dynnikov's avatar
      Fix merge conflicts · 151dd191
      Yaroslav Dynnikov authored
      151dd191
    • Yaroslav Dynnikov's avatar
      Squash the huge patch · 10b6eb1c
      Yaroslav Dynnikov authored
      feat: raft peer discovery PoC
      
      chore: prevent C functions from being optimized out
      
      feat: improve peer discovery
      
      fix: fix tests after making instance_id arg mandatory
      
      Smart supevision with fork
      
      Make it work
      
      Under development
      
      IPC messages to supervisor
      
      One more little step: entrypoint enum
      
      Arrange IPC from child to supervisor
      
      Remove tarantool_main macro
      
      Persist snapshot
      
      Fix some fresh bugs
      
      Implement postjoin
      
      Discovery under refactoring
      
      Enhance discovery
      
      Working on discovery
      
      Fix all discovery bugs known so far
      
      Draft join algorithm
      
      Fail applying snapshot
      
      Joining a learner works
      
      Cleanup snapshot generation
      
      Reorganize traft code and call join automatically
      
      Change peer.commit_index type from option to u64
      
      Implement autopromotion to voter
      
      Implement read_index
      
      Take read_index before self-romotion
      
      Cleanup excess logs
      
      Cleanup logs and code
      
      Deep refactoring in progress
      
      Finish refactoring db schema
      
      Embed entries applying inside traft node
      
      Refactor raft node communication
      
      Replace fiber channel with a mailbox, which is a `Vec<_>` + fiber cond.
      It allows to batch raft commands in a more predictable way and makes
      the code less error-prone.
      
      Remove commented code
      
      Simplify raft nodes interaction over net_box
      
      Eliminate `traft::Message` struct because its internals aren't used.
      Instead, serialize `raft::Message` using protobuf.
      
      Batch ConnectionPool requests
      
      1. Send messages in batches.
      2. Allow changing connection uri.
      3. Close unused connections after `inactivity_timeout`.
      
      Enhance the raft node
      
      1. Collect results from raft node
      2. Fix initial bootstrap which used to fail due to fiber race.
      3. Wrap raft storage operations in a transaction.
      
      Bump tarantool module
      
      Add documentation draft
      
      Cleanup warnings
      
      Try fixing tests
      
      Fix test_storage_log
      
      Fix test_traft_pool
      
      Fix luatest single and couple
      
      Start fixing threesome test
      
      Implement concurrent join requests handling
      10b6eb1c
Loading