Skip to content
Snippets Groups Projects
Commit e2803cb1 authored by Yaroslav Lobankov's avatar Yaroslav Lobankov Committed by Yaroslav Lobankov
Browse files

ci: add performance test reporting for sysbench

Since analyzing the performance test results via the workflow artifacts
is quite inconvenient (we need to download the artifact, unpack it, look
through results), we need a more human approach to see the test report.
Moreover, it would be cool to see the performance difference between the
commits.

So this patch adds reporting the performance test results to MyTeam chat
and mailing list. Furthermore, the comparison of test results is added
to the report. Seeing the test report in the chat or mail is very simple
and convenient, plus forever results history as a bonus.

The test report looks something like this:

    # Curr:
    #   branch:  add-feature-y
    #   build:   1.2.3-42-g6c71c3901
    #   summary: Add feature Y
    #   machine: x86_64
    #   distrib: ce
    #   gc64:    false
    # Prev:
    #   branch:  add-feature-x
    #   build:   1.2.3-41-g5271240ea
    #   summary: Add feature X
    #   machine: x86_64
    #   distrib: ce
    #   gc64:    false
    +----------------------+----------+----------+----------+
    |Sysbench              | Curr(rps)| Prev(rps)|     Ratio|
    +----------------------+----------+----------+----------+
    |oltp_delete           | 11408.000| 11532.000|     0.989|
    |oltp_insert           | 31455.000| 31322.000|     1.004|
    |oltp_point_select     | 65031.000| 66486.000|     0.978|
    |oltp_read_only        |  1732.000|  1735.000|     0.998|
    |oltp_read_write       |  1211.000|  1196.000|     1.013|
    |oltp_update_index     | 15580.000| 15570.000|     1.001|
    |oltp_update_non_index | 16129.000| 16093.000|     1.002|
    |oltp_write_only       |  4482.000|  4434.000|     1.011|
    |select_random_points  | 18525.000| 18517.000|     1.000|
    |select_random_ranges  |  4364.000|  4331.000|     1.008|
    +----------------------+----------+----------+----------+
    |gmean                 |  9223.593|  9220.182|     1.000|
    +----------------------+----------+----------+----------+

NO_DOC=ci
NO_TEST=ci
NO_CHANGELOG=ci
parent 7036b55a
No related branches found
No related tags found
No related merge requests found
......@@ -53,6 +53,10 @@ jobs:
- name: Add tarantool path to $PATH
run: echo "${PWD}/src:${PWD}/extra/dist" >> ${GITHUB_PATH}
- name: Get tarantool version
id: tarantool-version
run: echo "::set-output name=value::$(tarantool -e 'print(_TARANTOOL)')"
- name: Build and install sysbench
run: |
./autogen.sh
......@@ -71,13 +75,57 @@ jobs:
run: python3 -m pip install --user --requirement requirements.txt
working-directory: ./bench-run/publishing
- name: Publish metrics
- name: Publish metrics to database
env:
INFLUXDB_ORG: tarantool
INFLUXDB_BUCKET: perf
INFLUXDB_URL: ${{ secrets.INFLUXDB_URL }}
INFLUXDB_TOKEN: ${{ secrets.INFLUXDB_TOKEN }}
run: ./bench-run/publishing/influxdb.py -m sysbench -f Sysbench_result.txt
run: |
./bench-run/publishing/influxdb.py \
--measurement sysbench \
--file Sysbench_result.txt \
--save sysbench_db_record.txt \
--prev
- name: Install requirements for metrics reporter
run: python3 -m pip install --user --requirement requirements.txt
working-directory: ./bench-run/reporting
- name: Generate metrics report
run: |
./bench-run/reporting/table.py \
--input sysbench_db_record.txt \
--output sysbench_result_table.txt
- name: Send metrics report to MyTeam chat
env:
MYTEAM_URL: ${{ secrets.MYTEAM_URL }}
MYTEAM_TOKEN: ${{ secrets.MYTEAM_TOKEN }}
MYTEAM_CHAT_ID: ${{ secrets.MYTEAM_CHAT_ID }}
run: |
./bench-run/reporting/myteam.py \
--message-file sysbench_result_table.txt \
--format MarkdownV2
- name: Prepare email body
run: |
{ echo '<pre>'; cat sysbench_result_table.txt; echo '<pre/>'; } \
> sysbench_result_table.html
- name: Send metrics report to mailing list
uses: dawidd6/action-send-mail@v3
with:
server_address: ${{ secrets.PERF_EMAIL_SERVER }}
server_port: 25
subject: >
Tarantool Perf Testing. Branch: ${{ github.ref_name }}.
Build: ${{ steps.tarantool-version.outputs.value }}.
Test: sysbench
to: ${{ secrets.PERF_EMAIL_RECIPIENTS }}
from: ${{ secrets.PERF_EMAIL_SENDER }}
html_body: file://sysbench_result_table.html
ignore_cert: false
- name: call action to send Telegram message on failure
env:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment