Georgy Moshkin authored 2 years ago and Georgy Moshkin committed 2 years ago

replication_factor now goes into cluster_state

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

This patch introduces a new type of tests - manual. They aren't run by
default. You can use it for development experiments:

```bash
pytest test/manual -s -k instant
pytest test/manual -s -k chunked
```

test_concurrency makes no sense anymore. There's no batching since the
join_loop was refactored. The same is true for test_uuids.

Georgy Moshkin authored 2 years ago and Georgy Moshkin committed 2 years ago

deactivation means that instance
- is demoted to learner (if it wasn't one already)
- is marked "inactive" so that it's ignored when determining
    the number of voters required for the cluster

Yaroslav Dynnikov authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

Bootstrapping the replication was implemented in 2dac77c5. But it was
configured on the new instance only. The old instance (that joined
earlier) couldn't update `box.cfg({replication})` until now.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/52

If proc_discover is invoked after raft node was initialized but before
raft leader was elected, it would return an error before this commit.
Because of that it was impossible to restart the whole cluster at once.

This commit change proc_discover such that in case leader_id is not
ready, the normal discovery algorithm takes place.

Closes #93

Restarting both instances doesn't work yet, to be fixed later.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/90

Since commit d87dd4ca `leader_id` became an `Option`, so the `None`
value isn't rendered in the `picolib.raft_status` response:

```python
status={'is_ready': False, 'raft_state': 'Follower', 'id': 1}
```

It makes pytest complain about missing argument:

```
cluster2 = Cluster("127.0.0.1:3300", n=2)

    def test_restart_leader(cluster2: Cluster):
        i1, i2 = cluster2.instances
        i1.assert_raft_status('Leader')
        i2.assert_raft_status('Follower')

        i1.restart()
>       i1.wait_ready()

test/int/test_joining.py:209:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../.local/share/virtualenvs/picodata-6sv6l6y-/lib/python3.10/site-packages/funcy/decorators.py:45:
in wrapper
    return deco(call, *dargs, **dkwargs)
../../.local/share/virtualenvs/picodata-6sv6l6y-/lib/python3.10/site-packages/funcy/flow.py:127:
in retry
    return call()
../../.local/share/virtualenvs/picodata-6sv6l6y-/lib/python3.10/site-packages/funcy/decorators.py:66:
in __call__
    return self._func(*self._args, **self._kwargs)
test/int/conftest.py:305: in wait_ready
    status = self._raft_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Instance(i1, listen=127.0.0.1:3301)

    def _raft_status(self) -> RaftStatus:
        status = self.call("picolib.raft_status")
        assert isinstance(status, dict)
        eprint(f"{status=}")
>       return RaftStatus(**status)
E       TypeError: RaftStatus.__init__() missing 1 required positional
argument: 'leader_id'

test/int/conftest.py:280: TypeError
```

This patch fixes the failure message:

```
self = Instance(i1, listen=127.0.0.1:3301)

    @funcy.retry(tries=20, timeout=0.1)
    def wait_ready(self):
        status = self._raft_status()
>       assert status.is_ready
E       AssertionError: assert False
E        +  where False = RaftStatus(id=1, raft_state='Follower',
is_ready=False, leader_id=None).is_ready

test/int/conftest.py:306: AssertionError
```

* Make `--cluster-id` CLI mandatory.
* Handle cluster_id mismatch in raft_join.
  When an instance attempts to join the cluster and the instances's
  `--instance-id` parameter mismatches the cluster_id of the cluster
  an error is raised inside the raft_join handler.