test: fix flakiness in app-luatest/http_client_test.lua
The problem could be easily reproduced with following command line: ./test/test-run.py $(yes app-luatest/http_client_test.lua | head -n 1000). Before this commit we did a socket binding to know a free network port, then close a socket and started httpd.py on that network port. However it was not reliable and even with socket options SO_REUSE_PORT start of httpd.py has failed. With proposed patch schema is changed: we start httpd.py and pass only a socket family (AF_INET for TCP connection and AF_UNIX for connection via Unix socket) and then reading output from a process. Successfully started httpd.py prints a path to a Unix socket or a pair of IP address and network port split with ":". With proposed patch test has passed 1000 times without any problems. Tests previously marked as "fragile" are passed too: ./test/test-run.py --builddir=$(pwd)/build box-tap/net.box.test.lua \ box-tap/cfg.test.lua box-tap/session.storage.test.lua \ box-tap/session.test.lua app-tap/tarantoolctl.test.lua \ app-tap/debug.test.lua app-tap/inspector.test.lua \ app-tap/logger.test.lua app-tap/transitive1.test.lua \ app-tap/csv.test.lua app-luatest/http_client_test.lua P.S. The problem with "fragile" tests is that rerunning hides other problems. [1] is about "Address already in use" and [2] is about hangs in test. I made a pull request with changes in http client module and triggered CI run. Job has been passed, but in log [3] I see three test restarts due to fails in http_client test related to my changes. 1. https://github.com/tarantool/tarantool-qa/issues/186 2. https://github.com/tarantool/tarantool-qa/issues/31 3. https://github.com/tarantool/tarantool/runs/7726358823?check_suite_focus=true Closes https://github.com/tarantool/tarantool-qa/issues/186 Closes https://github.com/tarantool/tarantool-qa/issues/31 NO_CHANGELOG=testing NO_DOC=testing NO_TEST=testing
Showing
- test/app-luatest/http_client_test.lua 27 additions, 41 deletionstest/app-luatest/http_client_test.lua
- test/app-luatest/httpd.py 17 additions, 12 deletionstest/app-luatest/httpd.py
- test/app-luatest/suite.ini 0 additions, 8 deletionstest/app-luatest/suite.ini
- test/app-tap/suite.ini 1 addition, 16 deletionstest/app-tap/suite.ini
- test/box-tap/suite.ini 1 addition, 10 deletionstest/box-tap/suite.ini
Loading
Please register or sign in to comment