README.md

Cartridge integration test application

The application was created for integration testing of the sbroad library. It may also serve as a usage example of sbroad.

Navigation

• Build the application
• Run test application
• Architecture
  • sbroad.execute
  • sbroad.calculate_bucket_id
  • sbroad.trace
• General observations
• Local load testing

Build the application

cd .. && make build_integration

Run test application

cartridge start

Architecture

The application includes two roles:

• api
• storage

Notice that sbroad function are placed into global _G, so you can call sbroad functions at any place of app. Let's have a look at sbroad functions:

sbroad.execute(sql_query,{params})

execute the parametrized sql query. Currently query supports only SELECT, INSERT and EXPLAIN statements. For details see SQL feature taxonomy.

Parameters:

• sql_query (string) -- sql query to execute on cluster
• params (array) -- params for sql query. The number and order of the arguments correspond to the ? signs inside sql_query.

Return:

• result of query (table|nil)
• error

Example:

sbroad.execute([[insert into "testing_space" ("id", "name") values (?, ?), (?, ?)]], {1, "123", 2, "123"})
---
- {row_count = 2}
- null
...
sbroad.execute([[insert into "testing_space" ("id", "name") values (3, '123'), (4, '123')]])
---
- {row_count = 2}
- null
...

sbroad.execute([[select "name" from "testing_space" where "id" = 2]])
---
- {{{metadata = {name = 'id', type = 'integer'}, {name = 'name', type = 'string'}}, {rows = {2, "123"}}}
- null
...
sbroad.execute([[select "name" from "testing_space" where "id" = ?]], {3})
---
- {{{metadata = {name = 'id', type = 'integer'}, {name = 'name', type = 'string'}}, {rows = {3, "123"}}}
- null
...

sbroad.execute([[EXPLAIN SELECT * FROM (
    SELECT "f1", "f2" FROM "space_1" WHERE "f3" <= 0
    UNION ALL
    SELECT "f1", "f2" FROM "space_2" WHERE "f3" > 0
    ) as "t1"
    WHERE "f1" = 1 ]]
)
---
- "projection (\"t1\".\"f1\" -> \"f1\", \"t1\".\"f2\" -> \"f2\")",
  "    selection ROW(\"t1\".\"f1\") = ROW(1)",
  "        scan \"t1\"",
  "            union all",
  "                projection (\"space_1\".\"f1\" -> \"f1\", \"space_1\".\"f2\" -> \"f2\")",
  "                    selection ROW(\"space_1\".\"f3\") <= ROW(0)",
  "                        scan \"space_1\"",
  "                projection (\"space_2\".\"f1\" -> \"f1\", \"space_2\".\"f2\" -> \"f2\")",
  "                    selection ROW(\"space_2\".\"f3\") > ROW(0)",
  "                        scan \"space_2\"",
...

You can find more examples inside integration tests. To run it use

cd .. && test_integration

sbroad.calculate_bucket_id(value[,space_name])

calculate bucket id.

Parameters:

• value (string|tuple|table) -- value on the basis of which to calculate
• space_name (string) -- name of space.

Restrictions:

• String type value does not need space_name
• Tuple type value (array or box.tuple) needs space_name. The number and order of the elements must be corresponding to space format or less by one (skip bucket_id field).
• Table type value needs space_name. Table must contain all space sharding_key elements.

Return:

• result (number|nil) -- bucket_id
• error

Example:

sbroad.calculate_bucket_id("1123")
---
- 360
...
sbroad.calculate_bucket_id({ id = 1, name = "123" }, "testing_space"})
---
- 360
...
sbroad.calculate_bucket_id"({ box.tuple.new{ 1, "123" }, "testing_space" })
---
- 360
...
sbroad.calculate_bucket_id"({ 1, "123", 1 }, "testing_space" })
---
- 360
...

sbroad.trace(sql_query, {params}, carrier, query_id)

execute the parametrized sql query in the same way as sbroad.execute, but also start trace of query.

Parameters:

• sql_query (string) -- sql query to execute on cluster
• params (array) -- params for sql query. The number and order of the arguments correspond to the ? signs inside sql_query
• carrier (table|box.NULL) -- tracer params that can be represented as W3C format or as Jaegger format
• query_id (string) -- query id at format "idN"

Return:

• result of query (table|nil)
• error

Example: Start jaeger:

docker run --name jaeger -d --rm -p6831:6831/udp -p6832:6832/udp -p16686:16686 -p14268:14268 jaegertracing/all-in-one:latest

The result can be found at http://localhost:16686/ (choose sbroad service at left panel).

sbroad.execute([[select "name" from "testing_space" where "id" = ?]], {3}, box.NULL, "id1")
---
- {{{metadata = {name = 'id', type = 'integer'}, {name = 'name', type = 'string'}}, {rows = {3, "123"}}}
- null
...

General observations

Sbroad library uses internal lua functions in the cartridge executor and preloads them with load_lua_extra_function call in the init cartridge function.

As the sbroad library caches the cluster cartridge schema internally, any sbroad function that is called checks the internal cluster schema, and if that is empty it loads the schema from the main app. If the app schema was updated then the internal cache needs to be cleared. To clear the cache we need to add the invalidate_cached_schema call to the apply_config cartridge function.

Local load testing

More information in stress-test folder.