Georgy Moshkin authored 2 years ago and Georgy Moshkin committed 2 years ago

Some users may change their "target-dir" cargo setting for example to
make it shared between all projects (may speed up some builds).
It that case our integration tests should be looking for the executable
in the right place.

Yaroslav Dynnikov authored 2 years ago and Georgy Moshkin committed 2 years ago

This patch introduces a new type of tests - manual. They aren't run by
default. You can use it for development experiments:

```bash
pytest test/manual -s -k instant
pytest test/manual -s -k chunked
```

Restarting both instances doesn't work yet, to be fixed later.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/90

Since commit d87dd4ca `leader_id` became an `Option`, so the `None`
value isn't rendered in the `picolib.raft_status` response:

```python
status={'is_ready': False, 'raft_state': 'Follower', 'id': 1}
```

It makes pytest complain about missing argument:

```
cluster2 = Cluster("127.0.0.1:3300", n=2)

    def test_restart_leader(cluster2: Cluster):
        i1, i2 = cluster2.instances
        i1.assert_raft_status('Leader')
        i2.assert_raft_status('Follower')

        i1.restart()
>       i1.wait_ready()

test/int/test_joining.py:209:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../.local/share/virtualenvs/picodata-6sv6l6y-/lib/python3.10/site-packages/funcy/decorators.py:45:
in wrapper
    return deco(call, *dargs, **dkwargs)
../../.local/share/virtualenvs/picodata-6sv6l6y-/lib/python3.10/site-packages/funcy/flow.py:127:
in retry
    return call()
../../.local/share/virtualenvs/picodata-6sv6l6y-/lib/python3.10/site-packages/funcy/decorators.py:66:
in __call__
    return self._func(*self._args, **self._kwargs)
test/int/conftest.py:305: in wait_ready
    status = self._raft_status()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = Instance(i1, listen=127.0.0.1:3301)

    def _raft_status(self) -> RaftStatus:
        status = self.call("picolib.raft_status")
        assert isinstance(status, dict)
        eprint(f"{status=}")
>       return RaftStatus(**status)
E       TypeError: RaftStatus.__init__() missing 1 required positional
argument: 'leader_id'

test/int/conftest.py:280: TypeError
```

This patch fixes the failure message:

```
self = Instance(i1, listen=127.0.0.1:3301)

    @funcy.retry(tries=20, timeout=0.1)
    def wait_ready(self):
        status = self._raft_status()
>       assert status.is_ready
E       AssertionError: assert False
E        +  where False = RaftStatus(id=1, raft_state='Follower',
is_ready=False, leader_id=None).is_ready

test/int/conftest.py:306: AssertionError
```

* Make `--cluster-id` CLI mandatory.
* Handle cluster_id mismatch in raft_join.
  When an instance attempts to join the cluster and the instances's
  `--instance-id` parameter mismatches the cluster_id of the cluster
  an error is raised inside the raft_join handler.

The intention is to eliminate ambiguities in the `Instance` API. Make
it more like `subprocess` module (as regards `kill` and `terminate`
functions).

Pytest supports running tests in parallel using the `xdist` plugin. In
order to support it in Picodata, one should avoid ports collision. It
assigns each worker a dedicated IP address `127.7.n.1`, where
`n = xdist_worker_number`. Unfortunately, it doesn't work on MacOS,
because Mac doesn't provide any loopback aliases except `127.0.0.1`
by default.

This patch provides another address generation logics. The `subnet`
parameter is superseeded with a `base_port`, that is `3300 + n * 100`.
In this way, every pytest (xdist) worker gets a dedicated port range
`[3301, 3399]`, `[3401, 3499]` and so on.

Close https://git.picodata.io/picodata/picodata/picodata/-/issues/65

Before this patch, pytest used to launch all instances in a clean
environment. It prevented running with `PICODATA_LOG_LEVEL=verbose`.

Yaroslav Dynnikov authored 2 years ago and Yaroslav Dynnikov committed 2 years ago

This patch covers one more case when discovery request is handled
by an instance that has the discovery module unitialized.

Commit 1a3b5233 missed a bug. Iteration over instances could be aborted
by an exception during teardown. It resulted in garbage process
remaining alive after pytest termination.

Waiting for a valid `leader_id` on a node isn't enough. It may already
have one, but still be a Learner. Instead, the fixture should wait until
the node is promoted to voter.

The assertion `status == "Leader"` was in the first place, and
`raft_timeout_now` call was unreachable.

1. Print logs to the stderr so that they interleave with tarantool logs.
2. Fix `cluster.__repr__()`.

In `conftest.py`:

  - *Add* function `xdist_worker_number`. It converts `str(worker_id)` into `int`.
    It serves as a substitute for `session_data_mutex` for parallel test runs.

  - *Change* `normalize_net_box_result`. Replace a function with a decorator. Also,
    handle typical kinds of picodata responses. Extensively test it in
    `test_basics.py::test_eval/call_normalization`.

  - *Add* dataclass `RaftStatus`. It shouldn't be used outside `conftest.py`. It only makes
    assertions more brief in logs and understndable in code.

  - *Add* all raft stuff into the `Instance` class. This implies `raft_propose_eval`,
    `assert_raft_status` all `promote_or_fail` moved from `util.py`.

  - *Change* fixture `compile`. No need in extra logics since commit 59c31cb8.

  - *Preserve* fixture `binary_path`.

  - *Remove* fixtures `session_data_mutex` and `run_id`. Superseded with `xdist_worker_number`.

  - *Remove* fixtures `run_cluster` and `run_instance`. Superseded with `cluster.deploy(...)`.

  - *Remove* function `wait_tcp_port`. It's never enough to check raw socket. Superseded with
    `instance.wait_ready()`.

  - Give the instances clean names `i1, i2, ...`, and simple addresses `127.7.0.1:3301`.
    For the parallel test run use different IPs `127.7.N.1` etc.

In `test_basics.py`:

  - *Add* `test_xdist_worker_number`.

  - *Add* `test_call_normalization` and `test_eval_normalization`.

  - *Add* `test_process_management`. It's brand new, never implemented in luatest before.

  - *Rename* `test_single_instance_raft_eval` to `test_propose_eval` and extend it with additional
    assertion from `single_test.lua`.

  - *Remove* `test_instance`. A part of its logics is moved to `test_call/eval_normalization`. The
    other part is rewritten and extended in `test_process_management`.

  - *Remove* `test_cluster`. It was completely useless because of inappropriate synchronization and
    no valuable assertions.

  - *Remove* `test_propose_eval`. It wasn't that useful, but failed because of inappropriate
    synchronization.

In `test_couple.py`:

  - *Preserve* `test_follower_proposal` and `test_failover`. Just slightly refactor according to the
    new `conftest.py` API.

In `util.py` (completely removed):

  - *Remove* decorator `retry`. Needless.

  - *Remove* decorator `retry_on_network_errors`. Inappropriate predicate didn't catch Lua errors.

  - *Remove* everything related to raft. Move it into `conftest.py`.

Part of https://git.picodata.io/picodata/picodata/picodata/-/issues/59