Skip to content
Snippets Groups Projects
  1. Dec 09, 2024
    • godzie44's avatar
      Part of #37 · eb76f8b7
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Add `current_cord_name` function to get a name of the current cord,
      add `cord_is_main_dont_create` function.
      Add exports for `cord_is_main`, `cord_is_main_dont_create` and
      `current_cord_name` functions.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      eb76f8b7
    • godzie44's avatar
      Part of #37 · 7ca3ccad
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Add `log_default_logger` to get a default logger.
      Add exports for `log_set_format`, `log_set_level` and `log_default_logger` functions.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      7ca3ccad
    • Dmitry Rodionov's avatar
      feat: export box_access_check_ddl · fe036d54
      Dmitry Rodionov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      fe036d54
    • Dmitry Ivanov's avatar
      feat: Add new methods to `say` for FFI use cases · 7701d529
      Dmitry Ivanov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      This patch helps us create and write to non-default
      loggers provided by `say.h`. We'll use this mainly
      for audit log in picodata.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      7701d529
    • Georgy Moshkin's avatar
      fiber: basic api exports · 10465891
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Closes #9237
      
      Add exports for fiber_set_name_n, fiber_name, fiber_id, fiber_csw &
      fiber_find.
      
      Also make fiber_set_joinable, fiber_set_ctx & fiber_get_ctx interpret
      NULL as the current fiber.
      
      @TarantoolBot document
      Title: add basic fiber api to ffi exports.
      
      5 basic functions can now be used via ffi api, which were previously
      only accessible via lua api: fiber_set_name_n, fiber_name, fiber_id,
      fiber_csw & fiber_find.
      
      fiber_set_joinable now interprets NULL as current fiber.
      10465891
    • Dmitry Rodionov's avatar
      feat: export box_access_check_space · 603220a5
      Dmitry Rodionov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      
      Co-authored-by: default avatarYaroslav Dynnikov <yaroslav.dynnikov@gmail.com>
      603220a5
    • Georgy Moshkin's avatar
      box: introduce box_read_view_* ffi API functions · bd338c4c
      Georgy Moshkin authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduce API for opening a read view over the specified spaces and
      indexes and creating iterators over the index read views.
      
      NO_DOC=picodata patch
      bd338c4c
    • Denis Smirnov's avatar
      feat: extend C box API with a new auth method · 1f949ad8
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      1. Current commit introduces 'box_auth_data_prepare()' to prepare
         a data string for any supported authentication methods.
      2. The user name argument is refactored in the auth methods: the
         null-terminated string is replaced with an address range approach.
         Now Rust users don't need to re-allocate username with CString.
      3. Password length type was set to uint32_t (previously it was size_t,
         int, uint32_t for different functions). Tarantool uses murmur3a,
         so all the hashed strings should be up to 32 bit long.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      1f949ad8
    • Denis Smirnov's avatar
      feat: extend C box API with new user methods · 9c2ab33c
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduce new methods:
      
      1. box_user_id_by_name - get the user identifier by name;
      2. box_effective_user_id - get current effective user
         identifier;
      3. box_session_user_id - get current session user identifier;
      4. box_session_su - change current session user;
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      NO_TEST=picodata internal patch
      9c2ab33c
    • Maksim Kaitmazian's avatar
      build: add exports for a simple query flow · 5fe445b2
      Maksim Kaitmazian authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      PG protocol run queries by calling dispatch_query, which stores the
      response in port_c contained in box_function_ctx structure.
      
      NO_DOC=exports
      NO_CHANGELOG=exports
      NO_TEST=exports
      5fe445b2
    • Maksim Kaitmazian's avatar
      feat: implement user_auth_method_name · 2680f19a
      Maksim Kaitmazian authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      This function is used to determine the authentication method when
      authenticating a postgres client.
      
      part of picodata/picodata/sbroad!292
      
      NO_DOC=exports
      NO_CHANGELOG=exports
      NO_TEST=exports
      2680f19a
    • Maksim Kaitmazian's avatar
      build: add exports for PG authentication · 780b5011
      Maksim Kaitmazian authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      These exports allow us to perform user authentication and
      network communication.
      
      part of picodata/picodata/sbroad!292
      
      NO_DOC=exports
      NO_CHANGELOG=exports
      NO_TEST=exports
      780b5011
    • Denis Smirnov's avatar
      feat: expose tuple hash calculation method · e1ce65e8
      Denis Smirnov authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Picodata supports cluster-wide SQL and needs some predictable
      method to calculate tuple hashes for the bucket ids. Method
      should be available for Lua, C and Rust users. It was decided
      to expose a murmur3 hash calculation method of the key_def module.
      
      NO_DOC=picodata internal patch
      NO_CHANGELOG=picodata internal patch
      e1ce65e8
    • godzie44's avatar
      cbus: introduce lcpipe - light cpipe · d653d088
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      Introduced a new type of cbus pipe - lcpipe. The current pipe in the
      cbus - cpipe, has a number of limitations, first of all - the cpipe
      cannot be used from the 3rd party threads, cpipe only works as a channel
      between two cords. That why lcpipe is needed. Its main responsibility -
      create channel between any thread and tarantool cord.
      
      Internally lcpipe is a cpipe, but:
      - on flush triggers removed, cause triggers use thread-local mem-pool,
      this is not possible on a third party thread
      - producer event loop removed, cause there is no libev event loop in
      third party thread
      
      Also, lcpipe interface is exported to the outside world.
      
      NO_DOC=core feature
      d653d088
    • godzie44's avatar
      sql: add sql_execute_prepared_ext function, same as sql_execute_prepared but... · 4db44efa
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      sql: add sql_execute_prepared_ext function, same as sql_execute_prepared but without `region` parameter
      closes #2
      
      NO_DOC=minor
      NO_TEST=minor
      4db44efa
    • godzie44's avatar
      compatibility with tarantool-module: · 4305fee1
      godzie44 authored and Dmitry Ivanov's avatar Dmitry Ivanov committed
      - add box_tuple_data_offset function (return offset of the messagePack encoded data from the beginning of the tuple)
      - add more export functions
      
      closes #1
      
      NO_DOC=build
      NO_TEST=build
      4305fee1
  2. Aug 16, 2024
    • Aleksandr Lyapunov's avatar
      box: export more symbols · 03f8270b
      Aleksandr Lyapunov authored
      box_info_lsn
      box_is_ro
      box_ro_reason
      box_wait_ro
      
      Closes #10378
      
      NO_CHANGELOG=minor change
      
      @TarantoolBot document
      Title: document 4 more symbols in public C API
      
      box_info_lsn
      box_is_ro
      box_ro_reason
      box_wait_ro
      
      Their meaning is identical to lua methods:
      
      box.info.lsn
      box.info.ro
      box.info.ro_reason
      box.wait_ro
      
      (cherry picked from commit f1c76b976077e8078c42990546d9dbbdea315425)
      03f8270b
  3. Sep 12, 2023
    • Vladimir Davydov's avatar
      box: fix schema downgrade replication · b24eade3
      Vladimir Davydov authored
      Some downgrade operations are performed with disabled system space
      triggers because they were prohibited recently (creation of SQL built-in
      functions) or never allowed (dropping a system space). This works fine
      on the instance running downgrade but apparently fails on replicas.
      
      To fix this issue, let's disable the checks the operations that prevent
      downgrade in the following scenarios:
       - in the fiber that is currently running a schema upgrade or downgrade;
       - in the applier fiber so that it can replicate changes done by upgrade
         or downgrade on the master;
       - during recovery so that DDL records written to the WAL can be
         replayed.
      
      We already have all the necessary infrastructure in-place - we use it
      for allowing DDL operations with an old schema for upgrade.
      
      Closes #9049
      
      NO_DOC=bug fix
      
      (cherry picked from commit 71de4b2c)
      
      NOTE: We don't have the commit that disables DDL operations with an old
      schema in 2.11 so we have to backport bits of it from 3.0, see commit
      97c2c9a4 ("box: disable DDL with old schema").
      b24eade3
  4. Mar 07, 2023
  5. Feb 02, 2023
  6. Jan 30, 2023
    • Andrey Saranchin's avatar
      box: make ffi select work on M1 · 1b4a81fa
      Andrey Saranchin authored
      Recently, ffi select was broken on M1 in commit ec1a71ff
      ("box: introduce pagination to memtx_tree and tuple position methods").
      It turned out that ffi on M1 poorly supports a big quantity of arguments.
      Fortunately, there is a workaround - we can pass only 64-bit integer
      arguments beyond the 8th argument. Let's do it.
      
      Closes #7946
      
      NO_TEST=reflected in existing tests
      NO_CHANGELOG=bugfix for unreleased feature
      NO_DOC=bugfix
      1b4a81fa
  7. Jan 13, 2023
    • Aleksandr Lyapunov's avatar
      box: introduce options in box.atomic · 9d1cbda5
      Aleksandr Lyapunov authored
      If the first argument of box.atomic is a non-callable table then
      consider it as options table for box.begin{}.
      
      For test and debug purposes introduce internal getter of current
      transaction isolation level as box.internal.txn_isolation().
      
      Closes #7202
      
      @TarantoolBot document
      Title: Options in box.atomic
      
      Now it's allowed to pass transaction options in the first argument
      of box.atomic(..) call. The options must be a table, exactly as
      in box.begin(..). If options are passed as the first arguments,
      the second and the rest arguments are expected to be a functions
      and its arguments, like in usual box.atomic.
      9d1cbda5
  8. Jan 11, 2023
    • Georgiy Lebedev's avatar
      box: overriding IPROTO request handlers · 998964c8
      Georgiy Lebedev authored
      Add `IPROTO_UNKNOWN` command code for overriding the unknown request
      handler.
      
      Change request type variable types related to IPROTO to `uint32_t`.
      
      Add request handler hash table to transaction thread and request handler
      set to IPROTO threads for storing overridden request handlers: TX thread
      notifies IPROTO threads about overridden request handlers using IPROTO
      configuration message.
      
      If a given request handler is overridden, the IPROTO thread does not
      preprocess it and sends the package immediately over a dedicated route. If
      later it is necessary to fallback to the system handler, the message
      decoding and dispatching is done in the TX thread.
      
      Add new `box.iproto.override` method to Lua and `box_iproto_override` to C
      API, which allow setting IPROTO request handler callbacks.
      
      Closes #7901
      
      @TarantoolBot document
      Title: Document overriding IPROTO request handlers feature
      For the API description and usage examples, see:
      * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#0f84694523214c0e9bf2f3d75cccace4);
      * tarantool/tarantool#7901.
      998964c8
  9. Dec 27, 2022
    • Sergey Bronnikov's avatar
      uri: speedup encode and decode functions · dcd46244
      Sergey Bronnikov authored
      
      Patch replaces encoding and decoding functions written in Lua with
      functions implemented in C.
      
      Performance of Lua implementation (before the patch):
      
      ```
      uri.escape   152.37  runs/sec
      uri.unescape 263.44  runs/sec
      ```
      
      Performance of C implementation (after the patch):
      
      ```
      uri.escape   4983.03  runs/sec
      uri.unescape 4197.19  runs/sec
      ```
      
      Follows up #3682
      
      NO_CHANGELOG=see previous commit
      NO_DOC=see previous commit
      
      Co-authored-by: default avatarAlexander Turenko <alexander.turenko@tarantool.org>
      dcd46244
    • Georgiy Lebedev's avatar
      box: sending arbitrary IPROTO packets · b9892247
      Georgiy Lebedev authored
      Add translation table for `box.iproto.key` constants encoding to simplify
      packet assembly.
      
      Add new `box.iproto.send` method to Lua and `box_iproto_send` function to C
      API, which allow sending arbitrary IPROTO packets, using active IPROTO
      sessions. Packets are sent asynchronously using Kharon.
      
      Add `xregion_join` to the `xalloc` API.
      
      Change gh-7894 test: instead of simply comparing `box.iproto` table to the
      reference table, iterate over `box.iproto` and check that corresponding
      non-{function, thread, userdata} type values exist in the reference table.
      
      Closes #7897
      
      @TarantoolBot document
      Title: Document sending arbitrary IPROTO packets feature
      For the API description and usage examples, see:
      * [design document](https://www.notion.so/tarantool/box-iproto-override-44935a6ac7e04fb5a2c81ca713ed1bce#a2cc04da89d34fad8f8564c150cd9977);
      * tarantool/tarantool#7897.
      b9892247
  10. Dec 14, 2022
  11. Dec 12, 2022
    • Vladimir Davydov's avatar
      lua/digest: use OpenSSL version of SHA1 · c029e63f
      Vladimir Davydov authored
      Since commit f6ea7180 ("Try to load several variants of libssl.")
      the digest module uses an internal version of SHA1. Back then, we didn't
      link the OpenSSL library. Instead, we tried to load it dynamically.
      Since on some distributions the library could be missing, it was decided
      to implement an internal version of SHA1, see #405.
      
      However, since commit 59a55740 ("Link against libssl and libcrypto.
      Issue #1382") we link the OpenSSL library unconditionally so there's no
      need in having an internal implementation of SHA1. Let's drop it and
      switch the digest module to the version of SHA1 implemented by the
      crypto module using OpenSSL.
      
      Part of #7987
      
      NO_DOC=code cleanup
      NO_TEST=code cleanup
      NO_CHANGELOG=code cleanup
      c029e63f
  12. Dec 09, 2022
    • Ilya Verbin's avatar
      log: add log.new() function that creates a new logger · 24323448
      Ilya Verbin authored
      
      It allows to create a new instance of a log module, with a custom name:
      local my_log = require('log').new('my_module')
      
      The name is added to the log message after fiber name:
      YYYY-MM-DD hh:mm:ss.ms [PID]: CORD/FID/FIBERNAME/MODULENAME LEVEL> MSG
      
      Part of #3211
      
      NO_DOC=See next commit
      NO_CHANGELOG=See next commit
      
      Co-authored-by: default avatarAnastasMIPT <beliaev.ab@tarantool.org>
      24323448
    • Vladimir Davydov's avatar
      box: make auth subsystem pluggable · b5754d3f
      Vladimir Davydov authored
      This commit introduces an abstraction for the authentication code so
      that one can easily add new methods. To add a new method, one just needs
      to define a set of authentication callbacks in a struct auth_method and
      register it with auth_method_register.
      
      The IPROTO_AUTH and _user.auth formats were initially designed with
      extensibility in mind: both take the authentication method name
      (currently, only 'chap-sha1' is supported) so no changes to the schema
      are required.
      
      Note that although 'chap-sha1' is now implemented in its own file
      src/box/auth_chap_sha1.c, we don't merge src/scramble.c into it.
      This will be done later, in the scope of #7987.
      
      Since we call authentication plug-ins "methods" (not "mechanisms"),
      let's rename BOX_USER_FIELD_AUTH_MECH_LIST to BOX_USER_FIELD_AUTH while
      we are at it. Anyway, the corresponding field of the _user system space
      is called 'auth' (not 'auth_mech_list').
      
      Closes #7986
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      b5754d3f
  13. Dec 07, 2022
  14. Nov 28, 2022
    • Andrey Saranchin's avatar
      box: introduce after in pairs · 5eee5d7e
      Andrey Saranchin authored
      The patch introduces pagination to pairs: one can create iterator which
      will start after specific tuple or position, described by option after.
      Tuple (both cdata and array) and position (obtained by index:tuple_pos
      or fetch_pos option of select) can be passed as option after.
      
      NO_CHANGELOG=next commit
      NO_DOC=next commit
      5eee5d7e
  15. Nov 01, 2022
    • Nikolay Shirokovskiy's avatar
      box: straighten log configuration code · 571cdc3e
      Nikolay Shirokovskiy authored
      See [1] for some details on why the code for log configuration needs
      some care. In short log config validity checks are spread thru many
      places, on some code paths we use one checks and on other code paths
      other checks. And we make repetitive validity checks many times in
      runtime on single configure call.
      
      We can also reuse code for setting default values, checking options
      type and resetting values to default.
      
      - As a side effect of refactoring one can now reset values to default thru
        `log.cfg()` so now `log.cfg()` is on par with `box.cfg` in this respect.
      
      - This patch also drops conversion `log_level` from string to number.
      
        Before (shorten):
      
          tarantool> box.cfg{log_level='warn'}
          tarantool> box.cfg.log_level
          - info
          tarantool> log.cfg.level
          - 5
      
        Also:
          tarantool> log.cfg{level='info'}
          tarantool> log.cfg.level
          - 5
          tarantool> box.cfg{}
          tarantool> box.cfg.log_level
          - 5
      
        After patch if `log_level`/`level` is given as string than it is saved
        and returned as string too. I guess it should not affect users but looks
        more handy.
      
      - Also fixed issue with inconsistent setting `log_nonblock` thru
        `box.cfg()` and `log.cfg()`. In former case `nil` means setting default
        depending on logger type. In the latter case `nil` meant setting
        `nonblock` to `false`.
      
      - Also patch fixes #7447.
      
      Closes #7447.
      
      [1] PR for this refactoring
      https://github.com/tarantool/tarantool/pull/7454
      
      NO_DOC=refactoring/tiny API improvemnent
      571cdc3e
  16. Oct 18, 2022
    • Ilya Verbin's avatar
      box: use dd_version_id instead of _schema.version in get_version · 3e6393d5
      Ilya Verbin authored
      By default a user might not have privileges to access the _schema space,
      that will cause an error during schema_needs_upgrade(), which calls
      get_version(). Fix this by using C variable dd_version_id, which is
      updated in the _schema.version replace trigger.
      
      There's a special case for upgrade() during bootstrap() - triggers are
      disabled during bootstrap, that's why dd_version_id is not being updated.
      Handle this by passing _initial_version=1.7.5 to the upgrade function.
      
      Part of #7149
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      3e6393d5
  17. Sep 29, 2022
    • Andrey Saranchin's avatar
      box: introduce pagination to memtx_tree and tuple position methods · ec1a71ff
      Andrey Saranchin authored
      The patch introduces C and Lua methods for extracting position
      of tuple in index. Multikey and functional indexes are not supported.
      
      Also this patch extends method index_create_iterator_after and adds
      implementation of iterator_position for memtx tree. Options
      after and fetch_pos are added to Lua select as well (both FFI and LuaC).
      All the types of memtx tree indexes are supported (multikey, functional).
      
      NO_CHANGELOG=see later commits
      NO_DOC=see later commits
      
      Closes #7633
      Closes #7636
      ec1a71ff
  18. Sep 13, 2022
    • Georgy Moshkin's avatar
      fiber: add fiber_set_ctx & fiber_get_ctx functions · 6a60768c
      Georgy Moshkin authored
      Before this change there was no way to create a fiber that accepts
      parameters without yielding from the current fiber using the c api. You
      could pass the function arguments when calling fiber_start, but that
      forces you to yield, which is not acceptable in some scenarios (e.g.
      within a transaction).
      
      This commit introduces 2 new functions to the api: fiber_set_ctx for
      setting an pointer to a context of the given fiber and fiber_get_ctx for
      accessing that context.
      
      Closes https://github.com/tarantool/tarantool/issues/7669
      
      @TarantoolBot document
      Title: fiber: add fiber_set_ctx & fiber_get_ctx functions
      
      Add 2 api functions: `fiber_set_ctx` & `fiber_get_ctx` which can be used
      for passing data to a fiber. Previously this could be done via the
      `fiber_start` function, except that this would force the current fiber
      to yield, which is not acceptable in some scenarios (e.g. during a
      transaction). Now you can create a fiber with `fiber_new`, set it's
      contents with `fiber_set_ctx`, make it ready for execution with
      `fiber_wakeup` and keep executing the current fiber.
      6a60768c
  19. Sep 12, 2022
    • Vladimir Davydov's avatar
      Use MT-Safe strerror_r instead of strerror · 44f46dc8
      Vladimir Davydov authored
      strerror() is MT-Unsafe, because it uses a static buffer under the hood.
      We should use strerror_r() instead, which takes a user-provided buffer.
      The problem is there are two implementations of strerror_r(): XSI and
      GNU. The first one returns an error code and always writes the message
      to the beginning of the buffer while the second one returns a pointer to
      a location within the buffer where the message starts. Let's introduce a
      macro HAVE_STRERROR_R_GNU set if the GNU version is available and define
      tt_strerror() which writes the message to the static buffer, like
      tt_cstr() or tt_sprintf().
      
      Note, we have to export tt_strerror(), because it is used by Lua via
      FFI. We also need to make it available in the module API header, because
      the say_syserror() macro uses strerror() directly. In order to avoid
      adding tt_strerror() to the module API, we introduce an internal helper
      function _say_strerror(), which calls tt_strerror().
      
      NO_DOC=bug fix
      NO_TEST=code is covered by existing tests
      44f46dc8
  20. Aug 05, 2022
    • Alexander Turenko's avatar
      lua/decimal: add Lua value accessors to module API · c75fbce1
      Alexander Turenko authored
      The Rust module (see the issue) needs a getter and a setter for decimal
      values on the Lua stack. Let's make them part of the module API.
      
      Part of #7228
      
      @TarantoolBot document
      Title: Lua/C functions for decimals in the module API
      
      The following functions are added into the module API:
      
      ```c
      /**
       * Allocate a new decimal on the Lua stack and return
       * a pointer to it.
       */
      API_EXPORT box_decimal_t *
      luaT_newdecimal(struct lua_State *L);
      
      /**
       * Allocate a new decimal on the Lua stack with copy of given
       * decimal and return a pointer to it.
       */
      API_EXPORT box_decimal_t *
      luaT_pushdecimal(struct lua_State *L, const box_decimal_t *dec);
      
      /**
       * Check whether a value on the Lua stack is a decimal.
       *
       * Returns a pointer to the decimal on a successful check,
       * NULL otherwise.
       */
      API_EXPORT box_decimal_t *
      luaT_isdecimal(struct lua_State *L, int index);
      ```
      c75fbce1
  21. Aug 04, 2022
    • Alexander Turenko's avatar
      decimal: add the library into the module API · 5c1bc3da
      Alexander Turenko authored
      The main decision made in this patch is how large the public
      `box_decimal_t` type should be. Let's look on some calculations.
      
      We're interested in the following values.
      
      * How much decimal digits is stored?
      * Size of an internal decimal type (`sizeof(decimal_t)`).
      * Size of a buffer to store a string representation of any valid
        `decimat_t` value.
      * Largest signed integer type fully represented in decimal_t (number of
        bits).
      * Largest unsigned integer type fully represented in decimal_t (number
        of bits).
      
      Now `decimal_t` is defined to store 38 decimal digits. It means the
      following values:
      
      | digits | sizeof | string | int???_t | uint???_t |
      | ------ | ------ | ------ | -------- | --------- |
      | 38     | 36     | 52     | 126      | 127       |
      
      In fact, decNumber (the library we currently use under the hood) allows
      to vary the 'decimal digits per unit' parameter, which is 3 by default,
      so we can choose density of the representation. For example, for given
      38 digits the sizeof is 36 by default, but it may vary from 28 to 47
      bytes:
      
      | digits | sizeof     | string | int???_t | uint???_t |
      | ------ | ---------- | ------ | -------- | --------- |
      | 38     | 36 (28-47) | 52     | 126      | 127       |
      
      If we'll want to store `int128_t` and `uint128_t` ranges, we'll need 39
      digits:
      
      | digits | sizeof     | string | int???_t | uint???_t |
      | ------ | ---------- | ------ | -------- | --------- |
      | 39     | 36 (29-48) | 53     | 130      | 129       |
      
      If we'll want to store `int256_t` and `uint256_t` ranges:
      
      | digits | sizeof     | string | int???_t | uint???_t |
      | ------ | ---------- | ------ | -------- | --------- |
      | 78     | 62 (48-87) | 92     | 260      | 259       |
      
      If we'll want to store `int512_t` and `uint512_t` ranges:
      
      | digits | sizeof       | string | int???_t | uint???_t |
      | ------ | ------------ | ------ | -------- | --------- |
      | 155    | 114 (84-164) | 169    | 515      | 514       |
      
      The decision here is what we consdider as possible and what as unlikely.
      The patch freeze the maximum amount of bytes in `decimal_t` as 64. So
      we'll able to store 256 bit integers and will NOT able to store 512 bit
      integers in a future (without the ABI breakage at least).
      
      The script, which helps to calculate those tables, is at end of the
      commit message.
      
      Next, how else `box_decimal_*()` library is different from the internal
      `decimal_*()`?
      
      * Added a structure that may hold any decimal value from any current or
        future tarantool version.
      * Added `box_decimal_copy()`.
      * Left `strtodec()` out of scope -- we can add it later.
      * Left `decimal_str()` out of scope -- it looks dangerous without at
        least a good explanation when data in the static buffer are
        invalidated. There is `box_decimal_to_string()` that writes to an
        explicitly provided buffer.
      * Added `box_decimal_mp_*()` for encoding to/decoding from msgpack.
        Unlike `mp_decimal.h` functions, here we always have `box_decimal_t`
        as the first parameter.
      * Left `decimal_pack()` out of scope, because a user unlikely wants to
        serialize a decimal value piece-by-piece.
      * Exposed `decimal_unpack()` as `box_decimal_mp_decode_data()` to keep a
        consistent terminogoly around msgpack encoding/decoding.
      * More detailed API description, grouping by functionality.
      
      The script, which helps to calculate sizes around `decimal_t`:
      
      ```lua
      -- See notes in decNumber.h.
      
      -- DECOPUN: DECimal Digits Per UNit
      local function unit_size(DECOPUN)
          assert(DECOPUN > 0 and DECOPUN < 10)
          if DECOPUN <= 2 then
              return 1
          elseif DECOPUN <= 4 then
              return 2
          end
          return 4
      end
      
      function sizeof_decimal_t(digits, DECOPUN)
          -- int32_t digits;
          -- int32_t exponent;
          -- uint8_t bits;
          -- <..padding..>
          -- <..units..>
          local us = unit_size(DECOPUN)
          local padding = us - 1
          local unit_count = math.ceil(digits / DECOPUN)
          return 4 + 4 + 1 + padding + us * unit_count
      end
      
      function string_buffer(digits)
          -- -9.{9...}E+999999999# (# is '\0')
          -- ^ ^      ^^^^^^^^^^^^
          return digits + 14
      end
      
      function binary_signed(digits)
          local x = 1
          while math.log10(2 ^ (x - 1)) < digits do
              x = x + 1
          end
          return x - 1
      end
      
      function binary_unsigned(digits)
          local x = 1
          while math.log10(2 ^ x) < digits do
              x = x + 1
          end
          return x - 1
      end
      
      function digits_for_binary_signed(x)
          return math.ceil(math.log10(2 ^ (x - 1)))
      end
      
      function digits_for_binary_unsigned(x)
          return math.ceil(math.log10(2 ^ x))
      end
      
      function summary(digits)
          print('digits', digits)
          local sizeof_min = math.huge
          local sizeof_max = 0
          local DECOPUN_sizeof_min
          local DECOPUN_sizeof_max
          for DECOPUN = 1, 9 do
              local sizeof = sizeof_decimal_t(digits, DECOPUN)
              print('sizeof', sizeof, 'DECOPUN', DECOPUN)
              if sizeof < sizeof_min then
                  sizeof_min = sizeof
                  DECOPUN_sizeof_min = DECOPUN
              end
              if sizeof > sizeof_max then
                  sizeof_max = sizeof
                  DECOPUN_sizeof_max = DECOPUN
              end
          end
          print('sizeof min', sizeof_min, 'DECOPUN', DECOPUN_sizeof_min)
          print('sizeof max', sizeof_max, 'DECOPUN', DECOPUN_sizeof_max)
          print('string', string_buffer(digits))
          print('int???_t', binary_signed(digits))
          print('uint???_t', binary_unsigned(digits))
      end
      ```
      
      Part of #7228
      
      @TarantoolBot document
      Title: Module API for decimals
      
      See the declarations in `src/box/decimal.h` in tarantool sources.
      5c1bc3da
  22. Aug 01, 2022
    • Alexander Turenko's avatar
      fiber_channel: add accessor to internal functions · 395c30e8
      Alexander Turenko authored
      The Rust module [1] leans on several internal symbols. They were open in
      Tarantool 2.8 (see #2971 and #5932), but never were in the public API.
      Tarantool 2.10.0 hides the symbols and we need a way to get them back to
      use in the module.
      
      We have the following options:
      
      1. Design and expose a module API for fiber channels.
      2. Export the symbols with a prefix like `tnt_internal_` (to don't spoil
         the global namespace).
      3. Provide a `dlsym()` alike function to get an address of an internal
         symbol for users who knows what they're doing.
      
      I think that the third way offers the best compromise between amount of
      effort, quality of the result and opportunities to extend. In this
      commit I hardcoded the list of functions to make the change as safe as
      possible. Later I'll return here to autogenerate the list.
      
      Exported the following function from the tarantool executable:
      
      ```c
      void *
      tnt_internal_symbol(const char *name);
      ```
      
      I don't add it into the module API headers, because the function is to
      perform a dark magic and we don't suggest it for users.
      
      While I'm here, added `static` to a couple of fiber channel functions,
      which are only used within the compilation unit.
      
      [1]: https://github.com/picodata/tarantool-module
      
      Part of #7228
      Related to #6372
      
      NO_DOC=don't advertize the dangerous API
      NO_CHANGELOG=don't advertize the dangerous API
      395c30e8
  23. Jul 26, 2022
    • Alexander Turenko's avatar
      tuple: add JSON path field accessor to module API · bcca0b2b
      Alexander Turenko authored
      Added a function (see the API in the documentation request below), which
      reflects the `tuple[json_path]` Lua API (see #1285).
      
      Part of #7228
      
      @TarantoolBot document
      Title: tuple: access a field using JSON path via module API
      
      The following function is added into the module API:
      
      ```c
      /**
       * Return a raw tuple field in the MsgPack format pointed by
       * a JSON path.
       *
       * The JSON path includes the outmost field. For example, "c" in
       * ["a", ["b", "c"], "d"] can be accessed using "[2][2]" path (if
       * index_base is 1, as in Lua). If index_base is set to 0, the
       * same field will be pointed by the "[1][1]" path.
       *
       * The first JSON path token may be a field name if the tuple
       * has associated format with named fields. A field of a nested
       * map can be accessed in the same way: "foo.bar" or ".foo.bar".
       *
       * The return value is valid until the tuple is destroyed, see
       * box_tuple_ref().
       *
       * Return NULL if the field does not exist or if the JSON path is
       * malformed or invalid. Multikey JSON path token [*] is treated
       * as invalid in this context.
       *
       * \param tuple a tuple
       * \param path a JSON path
       * \param path_len a length of @a path
       * \param index_base 0 if array element indexes in @a path are
       *        zero-based (like in C) or 1 if they're one-based (like
       *        in Lua)
       * \retval a pointer to a field data if the field exists or NULL
       */
      API_EXPORT const char *
      box_tuple_field_by_path(box_tuple_t *tuple, const char *path,
      			uint32_t path_len, int index_base);
      ```
      bcca0b2b
Loading