Skip to content
Snippets Groups Projects
  1. Oct 14, 2021
    • Timur Safin's avatar
      build, lua: built-in module datetime · 43e10ed3
      Timur Safin authored
      Introduce a new builtin Tarantool module `datetime.lua` for timestamp
      and interval types support.
      
      New third_party module - c-dt
      -----------------------------
      
      * Integrated chansen/c-dt parser as 3rd party module to the
        Tarantool cmake build process;
      * We use tarantool/c-dt instead of original chansen/c-dt to
        have an easier cmake build integration, as we have added some
        changes, which provide cmake support, and allow to rename symbols
        if necessary (this symbol renaming is similar to that we see
        with xxhash or icu).
      
      New built-in module `datetime`
      ------------------------------
      
      * created a new Tarantool built-in module `datetime`, which uses
        `struct datetime` data structure for keeping timestamp values;
      * Lua module uses a number of `dt_*` functions from `c-dt` library,
        but they were renamed to `tnt_dt_*` at the moment of exporting
        from executable - to avoid possible name clashes with external
        libraries.
      
      * At the moment we libc `strftime` for formatting of datetime
        values according to flags passed, i.e. `date:format('%FT%T%z')`
        will return something like '1970-01-01T00:00:00+0000', but
        `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970'
      
      * if there is no format provided then we use default
        `tnt_datetime_to_string()` function, which converts datetime
        to their default ISO-8601 output format, i.e.
        `tostring(date)` will return string like "1970-01-01T00:00:00Z"
      
      * There are a number of simplified interfaces
        - totable() for exporting table with attributes names as provided
          by `os.date('*t')`
        - set() method provides unified interface to set values using
          the set of attributes as defined above in totable()
      
      Example,
      
      ```
      local dt = datetime.new {
      	nsec      = 123456789,
      
      	sec       = 19,
      	min       = 29,
      	hour      = 18,
      
      	day       = 20,
      	month     = 8,
      	year      = 2021,
      
      	tzoffset  = 180
      }
      
      local t = dt:totable()
      --[[
      {
      	sec = 19,
      	min = 29,
      	wday = 6,
      	day = 20,
      	nsec = 123456789,
      	isdst = false,
      	yday = 232,
      	tzoffset = 180,
      	month = 8,
      	year = 2021,
      	hour = 18
      }
      --]]
      
      dt:format()   -- 2021-08-21T14:53:34.032Z
      dt:format('%Y-%m-%dT%H:%M:%S')   -- 2021-08-21T14:53:34
      
      dt:set {
      	usec      = 123456,
      
      	sec       = 19,
      	min       = 29,
      	hour      = 18,
      
      	day       = 20,
      	month     = 8,
      	year      = 2021,
      
      	tzoffset  = 180,
      
      }
      dt:set {
      	timestamp = 1629476485.124,
      
      	tzoffset  = 180,
      }
      
      ```
      
      Coverage is
      
      File                 Hits Missed Coverage
      -----------------------------------------
      builtin/datetime.lua 299  23     92.86%
      -----------------------------------------
      Total                299  23     92.86%
      
      Part of #5941
      
      @TarantoolBot document
      Title: Introduced a new `datetime` module for timestamp and interval support
      
      Create `datetime` module for timestamp and interval types support.
      It allows to create date and timestamp values using either object interface,
      or via parsing of string values conforming to iso-8601 standard.
      One may manipulate (modify, subtract or add) timestamp and interval values.
      
      Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool
      for a more detailed description of module API.
      43e10ed3
  2. Sep 27, 2021
    • Leonid Vasiliev's avatar
      export: wrap exported msgpack symbols · 592db3b0
      Leonid Vasiliev authored
      Exporting symbols of a third party library is not a best practice,
      as we know from [1]. Let's wrap the msgpack symbols that need to
      be exported with the "tnt_" prefix.
      
      While working on the patch, it was decided to export the msgpack
      symbols that are used in "msgpuckffi.lua".
      In test shared libraries where the symbols "mp_***_{decimal,uuid}"
      are used, they are replaced to exported "tnt_mp_***_{decimal,uuid}",
      because in the case of linking with "libcore.a" the "libcore.a"
      needs to be rebuild with the "-fPIC" flag, that seems as overkill
      for tests.
      
      1. https://github.com/tarantool/memcached/issues/59
      
      Closes #5932
      592db3b0
  3. Sep 17, 2021
    • Vladimir Davydov's avatar
      lua/msgpack: enable encoding errors as msgpack extension · fa1652df
      Vladimir Davydov authored
      This patch adds a new msgpack.cfg: encode_error_as_ext. Setting it makes
      msgpack and msgpackffi modules encode errors as the MP_ERROR msgpack
      extension. If the flag is unset, msgpack.encode behavior depends on
      encode_load_metatables, encode_use_tostring, and encode_invalid_as_nil
      options, see luaL_convertfield(), while msgpackffi.encode() will always
      encode errors as strings. The latter needs to be fixed, but it's out of
      the scope of this work and tracked separately, see #4499.
      
      The new option is enabled by default.
      
      Interaction with box.session.settings.error_marshaling_enabled: errors
      are encoded as the MP_ERROR msgpack extension when returned via IPROTO
      iff both error_marshaling_enabled and encode_error_as_ext are set.
      
      Closes #6433
      
      @TarantoolBot document
      Title: Document msgpack.cfg.encode_error_as_ext
      
      The new option determines how error objects (see box.error.new) are
      encoded in the msgpack format:
       - If it's set, errors are encoded as the MP_ERROR msgpack extension.
         This is the default behavior.
       - If it's unset, the encoded format depends on other msgpack
         configuration options (encode_load_metatables, encode_use_tostring,
         encode_invalid_as_nil). With the otherwise default configuration,
         they are encoded as strings (see error.message).
      
      Functions affected by the default configuration (msgpack.cfg):
       - msgpack and msgpackffi modules
       - Storing errors in tuples and spaces (box.tuple.new)
       - Returning errors from IPROTO CALL/EVAL
      fa1652df
    • Vladimir Davydov's avatar
      lua/msgpack: teach msgpackffi decode MP_ERROR · ab01c2fc
      Vladimir Davydov authored
      There's no way to encode an error yet so the test just hard-codes
      msgpack data. It will be fixed in the future, once we allow to encode
      errors with msgpack/msgpackffi.
      
      Needed for #6433
      ab01c2fc
  4. Aug 20, 2021
    • Alexander Turenko's avatar
      github-ci: support the new version format · 9707a7bd
      Alexander Turenko authored
      The new release policy suggests to use X.Y.Z-alphaN, X.Y.Z-betaN,
      X.Y.Z-rcN tags for pre-releases. See the policy document here:
      https://github.com/tarantool/tarantool/discussions/6182
      
      This commit enforces the new versioning for tarantool builds that're
      made from CI.
      
      Tarballs
      --------
      
      When we'll add the 2.10.0-beta1 tag, the tarball will be named so:
      
      > tarantool-2.10.0-beta1.tar.gz
      
      When we'll add one commit upward the tag, the tarball will be named so:
      
      > tarantool-2.10.0-beta1.1.dev.tar.gz
      
      (In fact, we don't deploy tarballs per push anymore, so it is just for
      the same of completeness.)
      
      For 2.10.0 release it'll be named as:
      
      > tarantool-2.10.0.tar.gz
      
      How to try: `make -f .gitlab.mk source`.
      
      Packages
      --------
      
      When we'll add the 2.10.0-beta1 tag, the packages for Ubuntu Focal will
      be named so:
      
      > tarantool_2.10.0~beta1-1_amd64.deb
      > tarantool-common_2.10.0~beta1-1_all.deb
      > tarantool-dev_2.10.0~beta1-1_amd64.deb
      
      The sources packaged together with the package will be named so:
      
      > tarantool_2.10.0~beta1-1.debian.tar.xz
      > tarantool_2.10.0~beta1-1.dsc
      > tarantool_2.10.0~beta1.orig.tar.xz
      > tarantool-2.10.0~beta1.tar.xz
      
      When we'll add one commit upward the tag, those files will look so:
      
      > tarantool_2.10.0~beta1.1.dev-1_amd64.deb
      > tarantool-common_2.10.0~beta1.1.dev-1_all.deb
      > tarantool-dev_2.10.0~beta1.1.dev-1_amd64.deb
      
      > tarantool_2.10.0~beta1.1.dev-1.debian.tar.xz
      > tarantool_2.10.0~beta1.1.dev-1.dsc
      > tarantool_2.10.0~beta1.1.dev.orig.tar.xz
      > tarantool-2.10.0~beta1.1.dev.tar.xz
      
      (In fact, we don't deploy packages per push anymore, so it is just for
      the same of completeness.)
      
      For 2.10.0 release it'll be named so:
      
      > tarantool_2.10.0.gcb3bdbf2a-1_amd64.deb
      > tarantool-common_2.10.0.gcb3bdbf2a-1_all.deb
      > tarantool-dev_2.10.0.gcb3bdbf2a-1_amd64.deb
      
      > tarantool_2.10.0.gcb3bdbf2a-1.debian.tar.xz
      > tarantool_2.10.0.gcb3bdbf2a-1.dsc
      > tarantool_2.10.0.gcb3bdbf2a.orig.tar.xz
      > tarantool-2.10.0.tar.xz
      
      How to try: `OS=ubuntu DIST=focal make -f .gitlab.mk package`.
      
      Fixes #6184
      9707a7bd
  5. Aug 19, 2021
    • Mergen Imeev's avatar
      sql: introduce field type decimal · ada186d6
      Mergen Imeev authored
      This patch introduces a decimal field type. However, implicit and
      explicit casts and arithmetic operations for this type will be presented
      in next few patches. Literals also will be introduced later.
      
      Part of #4415
      ada186d6
  6. Aug 18, 2021
    • Mons Anderson's avatar
      luarocks: Set FORCE_CONFIG=false in hardcoded config · 5bbe88cd
      Mons Anderson authored
      FORCE_CONFIG=true is preventing local .rocks/config-5.1.lua from loading.
      Local config is often used to populate rocks_servers with own rocks servers and with original luarocks
      
      This patch changes FORCE_CONFIG to true in generated luarocks.core.hardcoded to allow using
      project-side config-5.1.lua for luarocks configuration to avoid dirty and ugly hacks
      5bbe88cd
  7. Aug 12, 2021
  8. Aug 09, 2021
  9. Jun 07, 2021
    • Mergen Imeev's avatar
      sql: introduce UUID field type · 64f9018c
      Mergen Imeev authored
      This patch introduces UUID to SQL. UUID is now available as a new field
      type.
      
      Part of #5886
      
      @TarantoolBot document
      Title: Field type UUID is now available in SQL
      
      The UUID field type is now available in SQL. This means that we can
      create spaces and indexes with UUID, use it in SELECT, UPDATE and
      DELETE. UUID can be accepted and returned by built-in functions and
      user-defined functions.
      
      According to the comparison rules, there will be no implicit casting in
      the comparison. This rule also applies to UUID values: if a value is not
      part of a SCALAR field, it cannot be compared to a value of any other
      type. If the value is in a SCALAR field, it can be compared to any other
      scalar value according to the comparison rules for a SCALAR field.
      
      In case a UUID value is used in an operation that is not a comparison,
      it can be implicitly converted to STRING or VARBINARY.
      
      If a STRING or VARBINARY value is used in an operation that is not a
      comparison, it can be implicitly converted to a UUID.
      
      UUID value can always be explicitly converted to STRING or VARBINARY.
      
      A STRING or VARBINARY value can be explicitly converted to a UUID if it
      conforms to the UUID standard.
      64f9018c
  10. Jun 02, 2021
    • Sergey Bronnikov's avatar
      extra: enable debug options for apt(8) · f40afb85
      Sergey Bronnikov authored
      Sometimes jobs on CI with Jepsen tests failed on installation
      dependencies:
      
      ```
      sudo -S -u root bash -c "cd /; env DEBIAN_FRONTEND=noninteractive apt-get install -y --force-yes apt-transport-https libzip4 ntpdate faketime"
      
      STDIN:
      null
      
      STDOUT:
      Reading package lists...
      Building dependency tree...
      Reading state information...
      
      STDERR:
      W: --force-yes is deprecated, use one of the options starting with --allow instead.
      E: Unable to locate package libzip4
      E: Unable to locate package ntpdate
      E: Unable to locate package faketime
      ```
      
      Problem looks as a flaky, I couldn't reproduce it locally.  I suspect
      the root cause is an infrastructure problem and to get more details
      about it I have enabled debug options in apt-get and added
      `set -o errexit` as it is recommended in documentation [1]
      (see Note section).
      
      1. https://www.terraform.io/docs/language/resources/provisioners/remote-exec.html#argument-reference
      
      Part of: https://github.com/tarantool/jepsen.tarantool/issues/87
      f40afb85
  11. Apr 15, 2021
    • Roman Khabibov's avatar
      box: set box.cfg options via environment variables · 1b330121
      Roman Khabibov authored
      
      Add ability to set box.cfg options via environment variables. These
      variables should have name `TT_<OPTION>`. When Tarantool instance is
      started under tarantoolctl utility, environment variables have higher
      priority than tarantoolctl configuration file.
      
      Closes #5602
      
      Co-authored-by: default avatarLeonid Vasiliev <lvasiliev@tarantool.org>
      Co-authored-by: default avatarAlexander Turenko <alexander.turenko@tarantool.org>
      
      @TarantoolBot document
      Title: Set box.cfg options via environment variables
      
      Now, it is possible to set box.cfg options via environment variables.
      The name of variable should correspond the following pattern:
      `TT_<NAME>`, where `<NAME>` is uppercase box.cfg option name. For
      example: `TT_LISTEN`, `TT_READAHEAD`.
      
      Array values are separated by comma. Example:
      
      ```sh
      export TT_REPLICATION=localhost:3301,localhost:3302
      ```
      
      An empty variable is the same as unset one.
      1b330121
  12. Nov 18, 2020
    • Roman Khabibov's avatar
      sql: support column addition · 6cff9025
      Roman Khabibov authored
      Enable to add column to existing space with
      <ALTER TABLE ADD [COLUMN]> statement. Column definition can be
      supplemented with the four types of constraints, <DEFAULT>,
      <COLLATE> clauses and <[NOT] NULL>, AUTOINCREMENT.
      
      Closes #2349, #3075
      
      @TarantoolBot document
      Title: Add columns to existing tables in SQL
      Now, it is possible to add columns to existing empty spaces using
      <ALTER TABLE table_name ADD [COLUMN] column_name column_type ...>
      statement. The column definition is the same as in <CREATE TABLE>
      statement.
      
      * Space emptiness is Tarantool's restriction. Possibilty to add
      column to non empty space will be implemented later.
      
      For example:
      
      ```
      tarantool> box.execute("CREATE TABLE test (a INTEGER PRIMARY KEY)")
      ---
      - row_count: 1
      ...
      
      tarantool> box.execute([[ALTER TABLE test ADD COLUMN b TEXT
               >                                           CHECK (LENGTH(b) > 1)
               >                                           NOT NULL
               >                                           DEFAULT ('aa')
               >                                           COLLATE "unicode_ci"
               >             ]])
      ---
      - row_count: 1
      ...
      ```
      6cff9025
    • Roman Khabibov's avatar
      sql: rename TK_COLUMN to TK_COLUMN_REF · e0850326
      Roman Khabibov authored
      Rename TK_COLUMN used for tokens treated as a column name to
      TK_COLUMN_REF. It is needed to allow the typing of COLUMN keyword
      in <ALTER TABLE ADD COLUMN> statement.
      
      Needed for #3075
      e0850326
  13. Oct 14, 2020
  14. Sep 18, 2020
    • Sergey Bronnikov's avatar
      extra: add Terraform config files · 0b59bc93
      Sergey Bronnikov authored
      For testing Tarantool with Jepsen we use virtual machines as they provides
      better resource isolation in comparison to containers. Jepsen tests may need a
      single instance or a set of instances for testing cluster.  To setup virtual
      machines we use Terraform [1]. Patch adds a set of configuration files for
      Terraform that can create required number of virtual machines in MCS and output
      IP addresses to stdout.
      
      Terraform needs some parameters before run. They are:
      
      - id, identificator of a test stand that should be specific for this run, id
      also is a part of virtual machine name
      - keypair_name, name of keypair used in a cloud, public SSH key of that key pair
      will be placed to virtual machine
      - instance_count, number of virtual machines in a test stand
      - ssh_key, SSH private key, used to access to a virtual machine
      - user_name
      - password
      - tenant_id
      - user_domain_id
      
      These parameters can be passed via enviroment variables with TF_VAR_ prefix
      (like TF_VAR_id) or via command-line parameters.
      
      To demonstrate full lifecycle of a test stand with Terraform one needs to
      perform these commands:
      
      terraform init extra/tf
      terraform apply extra/tf
      terraform output instance_names
      terraform output instance_ips
      terraform destroy extra/tf
      
      1. https://www.terraform.io/
      
      Part of #5277
      0b59bc93
  15. Jul 15, 2020
  16. Jun 10, 2020
  17. May 18, 2020
    • Vladislav Shpilevoy's avatar
      cmake: remove dynamic-list linker option · 03790ac5
      Vladislav Shpilevoy authored
      dynamic-list (exported_symbols_list on Mac) was used to forbid
      export of all symbols of the tarantool executable except a given
      list. Motivation of that was to avoid hacking the linker with
      false usage of symbols needed to be exported. As a consequence,
      symbols not listed in these options became invisible.
      
      Before these options, when a symbol was defined, but not used in
      the final executable, the linker could throw it away, even though
      many symbols were used by Lua FFI, or should be visible for user's
      dynamic modules. Where the linker, obviously, can't see if they
      are needed.
      
      To make the linker believe the symbols are actually needed there
      was a hack with getting pointers at these functions and doing
      something with them.
      
      For example, assume we have 'test()' function in 'box' static
      library:
      
          int
          test(void);
      
      It is not used anywhere in the final executable. So to trick the
      linker there is a function 'export_syms()' declared, which takes a
      pointer at 'test()' and seemingly does something with it (or
      actually does - it does not matter):
      
          void
          export_syms()
          {
              void *syms[] = {test};
              if (time(NULL) == 0) {
                  syms[0]();
                  syms[1]();
                  ...
              }
          }
      
      Some users want to use not documented but visible symbols, so the
      patch removes the dynamic-list option, and returns the linker
      hack back. But with 0 dependencies in the export file.
      
      Closes #2971
      03790ac5
  18. Apr 28, 2020
    • Vladislav Shpilevoy's avatar
      box: introduce box_return_mp() public C function · dd36c610
      Vladislav Shpilevoy authored
      Closes #4641
      
      @TarantoolBot document
      Title: box_return_mp() public C function
      
      Stored C functions could return a result only via
      `box_return_tuple()` function. That made users create a tuple
      every time they wanted to return something from a C function.
      
      Now public C API offers another way to return - `box_return_mp()`.
      It allows to return arbitrary MessagePack, not wrapped into a
      tuple object. This is simpler to use for small results like a
      number, boolean, or a short string. Besides, `box_return_mp()` is
      much faster than `box_return_tuple()`, especially for small
      MessagePack.
      
      Note, that it is faster only if an alternative is to create a
      tuple by yourself. If an already existing tuple was obtained from
      an iterator, and you want to return it, then of course it is
      faster to return via `box_return_tuple()`, than via extraction of
      tuple data, and calling `box_return_mp()`.
      
      Here is the function declaration from module.h:
      ```C
      /**
       * Return MessagePack from a stored C procedure. The MessagePack
       * is copied, so it is safe to free/reuse the passed arguments
       * after the call.
       * MessagePack is not validated, for the sake of speed. It is
       * expected to be a single encoded object. An attempt to encode
       * and return multiple objects without wrapping them into an
       * MP_ARRAY or MP_MAP is undefined behaviour.
       *
       * \param ctx An opaque structure passed to the stored C procedure
       *        by Tarantool.
       * \param mp Begin of MessagePack.
       * \param mp_end End of MessagePack.
       * \retval -1 Error.
       * \retval 0 Success.
       */
      API_EXPORT int
      box_return_mp(box_function_ctx_t *ctx, const char *mp, const char *mp_end);
      ```
      dd36c610
  19. Apr 20, 2020
    • Vladislav Shpilevoy's avatar
      error: make iproto errors reuse mp_error module · 712af455
      Vladislav Shpilevoy authored
      After error objects marshaling was implemented in #4398, there
      were essentially 2 versions of the marshaling - when an error is
      sent inside response body, and when it is thrown and is encoded
      in iproto fields IPROTO_ERROR and IPROTO_ERROR_STACK. That is not
      really useful to have 2 implementation of the same feature. This
      commit drops the old iproto error encoding (its IPROTO_ERROR_STACK
      part), and makes it reuse the common error encoder.
      
      Note, the encoder skips MP_EXT header. This is because
      
      * The header is not needed - error is encoded as a value of
        IPROTO_ERROR_STACK key, so it is known this is an error. MP_EXT
        is needed only when type is unknown on decoding side in advance;
      
      * Old clients may not expect MP_EXT in iproto fields. That is the
        case of netbox connector, at least.
      
      Follow up #4398
      
      @TarantoolBot document
      Title: Stacked diagnostics binary protocol
      Stacked diagnostics is described in details in
      https://github.com/tarantool/doc/issues/1224. This commit
      changes nothing except binary protocol. The old protocol should
      not be documented anywhere.
      
      `IPROTO_ERROR_STACK` is still 0x52, but format of its value is
      different now. It looks exactly like `MP_ERROR` object, without
      `MP_EXT` header.
      
      ```
      IPROTO_ERROR_STACK: <MP_MAP> {
          MP_ERROR_STACK: <MP_ARRAY> [
              <MP_MAP> {
                  ... <all the other fields of MP_ERROR> ...
              },
              ...
          ]
      }
      ```
      
      It is easy to see, that key `IPROTO_ERROR_STACK` is called
      'stack', and `MP_ERROR_STACK` is also 'stack'. So it may be good
      to rename the former key in the documentation. For example, the
      old `IPROTO_ERROR` can be renamed to `IPROTO_ERROR_24` and
      `IPROTO_ERROR_STACK` can be renamed to just `IPROTO_ERROR`.
      712af455
    • Vladislav Shpilevoy's avatar
      error: export error_unref() function · ed217292
      Vladislav Shpilevoy authored
      C struct error objects can be created directly only in C.
      C-side increments their reference counter when pushes to the Lua
      stack.
      
      It is not going to be so convenient soon. error_unpack() function
      will be used in netbox to decode error object via Lua FFI.
      
      Such error object will have 0 refs and no Lua GC callback
      established. Because it won't be pushed on Lua stack natually,
      from Lua C. To make such errors alive their reference counter
      will be incremented and error_unref() will be set as GC callback.
      
      Follow up for #4398
      ed217292
    • Leonid Vasiliev's avatar
      error: add custom error type · b728e7af
      Leonid Vasiliev authored
      
      Co-authored-by: default avatarVladislav <Shpilevoy&lt;v.shpilevoy@tarantool.org>
      
      Part of #4398
      
      @TarantoolBot document
      Title: Custom error types for Lua errors
      
      Errors can be created in 2 ways: `box.error.new()` and `box.error()`.
      
      Both used to take either `code, reason, <reason string args>` or
      `{code = code, reason = reason, ...}` arguments.
      
      Now in the first option instead of code a user can specify a
      string as its own error type. In the second option a user can
      specify both code and type. For example:
      
      ```Lua
      box.error('MyErrorType', 'Message')
      box.error({type = 'MyErrorType', code = 1024, reason = 'Message'})
      ```
      Or no-throw version:
      ```Lua
      box.error.new('MyErrorType', 'Message')
      box.error.new({type = 'MyErrorType', code = 1024, reason = 'Message'})
      ```
      When a custom type is specified, it is shown in `err.type`
      attribute. When it is not specified, `err.type` shows one of
      built-in errors such as 'ClientError', 'OurOfMemory', etc.
      
      Name length limit on the custom type is 63 bytes. All what is
      longer is truncated.
      
      Original error type can be checked using `err.base_type` member,
      although normally it should not be used. For user-defined types
      base type is 'CustomError'.
      
      For example:
      ```
      tarantool> e = box.error.new({type = 'MyErrorType', code = 1024, reason = 'Message'})
      ---
      ...
      
      tarantool> e:unpack()
      ---
      - code: 1024
        trace:
        - file: '[string "e = box.error.new({type = ''MyErrorType'', code..."]'
          line: 1
        type: MyErrorType
        custom_type: MyErrorType
        message: Message
        base_type: CustomError
      ...
      ```
      b728e7af
  20. Apr 14, 2020
    • Vladislav Shpilevoy's avatar
      box: export box_session_push to the public C API · 5d795527
      Vladislav Shpilevoy authored
      API is different from box.session.push() - sync argument was
      removed. It will disappear from Lua API as well, because it just
      is not needed here. Session is omitted as well. Indeed, a user
      can't push to a foreign session, and the current session can be
      obtained inside box_session_push(). And anyway session is not in
      the public C API.
      
      Internally dump into iproto is done using obuf_dup(), just like
      tuple_to_obuf() does. obuf_alloc() would be a bad call here,
      because it wouldn't be able to split the pushed data into several
      obuf chunks, and would cause obuf fragmentation.
      
      Dump into plain text behaves just like a Lua push - it produces a
      YAML formatted text or Lua text depending on output format. But to
      turn MessagePack into YAML or Lua text an intermediate Lua
      representation is used, because there are no a MessagePack -> YAML
      and MessagePack -> Lua text translators yet.
      
      Closes #4734
      
      @TarantoolBot document
      Title: box_session_push() C API
      
      There is a new function in the public C API:
      ```C
          int
          box_session_push(const char *data, const char *data_end);
      ```
      
      It takes raw MessagePack, and behaves just like Lua
      `box.session.push()`.
      5d795527
    • Leonid's avatar
      rocks: forward options to luarocks · ce8c6ef1
      Leonid authored
      The policy for check of luarocks flags has been changed
      (moved from tarantoolctl to luarocks).
      Chdir has been moved to luarocks.
      
      Closes #4629
      
      @TarantoolBot document
      Title: Update tarantoolctl rocks
      tarantoolctl rocks commands has been added:
      build
      config
      download
      init
      lint
      new_version
      purge
      which
      write_rockspec
      
      https://github.com/luarocks/luarocks/wiki/luarocks
      ce8c6ef1
  21. Apr 13, 2020
    • Chris Sosnin's avatar
      session: store output format in struct session · 9c0e40f9
      Chris Sosnin authored
      box.session.storage is a general-purpose table, which can be
      used by user. Therefore, we shouldn't store any internal details
      in it.
      
      Needed for #4686
      9c0e40f9
    • Chris Sosnin's avatar
      sql: provide a user friendly frontend for accessing session settings · 6706f659
      Chris Sosnin authored
      Currently if a user wants to change session setting with SQL, one has
      to execute UPDATE query like:
      [[UPDATE "_session_settings" SET "value" = true WHERE "name" = 'name']]
      However, direct access to system spaces isn't considered to be a good practice.
      To avoid that and a bit simplify user's life, we introduce SQL shortcut command
      SET SESSION.
      
      Closes #4711
      
      @TarantoolBot document
      Title: API for accessing _session_settings space.
      There are two ways of updating values of session settings:
      via Lua and SQL.
      
      Lua:
      box.session.settings is a table, which is always accessible
      to user. The syntax is the following:
      `box.session.settings.<setting_name> = <new_value>`.
      
      Example of usage:
      ```
      tarantool> box.session.settings.sql_default_engine
      ---
      - memtx
      ...
      
      tarantool> box.session.settings.sql_default_engine = 'vinyl'
      ---
      ...
      
      ```
      
      The table itself represents the (unordered) result of select
      from _session_settings space.
      
      SQL:
      Instead of typing long UPDATE query one can use the SET SESSION command:
      `box.execute([[SET SESSION "<setting_name>" = <new_value>]])`.
      Note, that this query is case sensitive so the name must be quoted.
      Also, SET SESSION doesn't provide any implicit casts, so <new_value> must
      be of the type corresponding to the setting being updated.
      
      Example:
      ```
      tarantool> box.execute([[set session "sql_default_engine" = 'memtx']])
      ---
      - row_count: 1
      ...
      
      tarantool> box.execute([[set session "sql_defer_foreign_keys" = true]])
      ---
      - row_count: 1
      ...
      
      ```
      6706f659
    • Serge Petrenko's avatar
      box: add MsgPack encoding/decoding for UUID · d68fc292
      Serge Petrenko authored
      A special format for encoding UUIDs to MsgPack is introduced.
      It is supported by both lua and C encoders/decoders, and it is now
      possible to insert UUIDs into spaces, but only into unindexed fields
      without format for now.
      
      Prerequisite #4268
      
      @TarantoolBot document
      Title: Internals: msgpack format for UUID
      
      UUID values share the MessagePack type with decimals:
      both use MP_EXT. A new subtype is introduced for UUIDs,
      MP_UUID = `0x02`
      UUID is encoded as follows:
      ```
          +--------+---------+-----------+
          | MP_EXT | MP_UUID | UuidValue |
          +--------+---------+-----------+
      ```
      Since UUID is 16 bytes in size, the header, MP_EXT, is always the same:
      `0xd8`. MP_UUID = `0x02` follows. The header is followed by the 16
      bytes of the UuidValue.
      
      UuidValue consists of 11 fields, which are encoded as big endian
      unsigned integers in the following order: `time_low` (4 bytes), `time_mid`
      (2 bytes), `time_hi_and_version` (2 bytes), `clock_seq_hi_and_reserved` (1
      byte), `clock_seq_low` (1 byte), `node[0], ..., node[5]` (1 byte each).
      
      The total size of such a representation is 18 bytes, whereas storing
      uuids as strings requires from 34 (when '-'s are ommitted) to 38 bytes
      per UUID, giving a 2x space usage improvement.
      d68fc292
  22. Apr 07, 2020
    • Nikita Pettik's avatar
      box: introduce stacked diagnostic area · 3b887d04
      Nikita Pettik authored
      In terms of implementation, now struct error objects can be organized
      into double-linked lists. To achieve this pointers to the next and
      previous elements (cause and effect correspondingly) have been added to
      struct error. It is worth mentioning that already existing rlist and
      stailq list implementations are not suitable: rlist is cycled list, as a
      result it is impossible to start iteration over the list from random
      list entry and finish it at the logical end of the list; stailq is
      single-linked list leaving no possibility to remove elements from the
      middle of the list.
      
      As a part of C interface, box_error_add() has been introduced. In
      contrast to box_error_set() it does not replace last raised error, but
      instead it adds error to the list of diagnostic errors having already
      been set. If error is to be deleted (its reference counter hits 0 value)
      it is unlinked from the list it belongs to and destroyed. Meanwhile,
      error destruction leads to decrement of reference counter of its
      previous error and so on.
      
      To organize errors into lists in Lua, table representing error object in
      Lua now has .prev field (corresponding to 'previous' error) and method
      :set_prev(e). The latter accepts error object (i.e. created via
      box.error.new() or box.error.last()) and nil value. Both field .prev and
      :set_prev() method are implemented as ffi functions. Also note that
      cycles are not allowed while organizing errors into lists:
      e1 -> e2 -> e3; e3:set_prev(e1) -- would lead to error.
      
      Part of #1148
      3b887d04
  23. Apr 02, 2020
  24. Mar 18, 2020
    • Oleg Babin's avatar
      box: allow to retrieve the last generated value of sequence · 64c69fe0
      Oleg Babin authored
      
      This patch introduces "current" function for sequences.
      It returns the last retrieved value of specified sequence or
      throws an error if no value has been generated yet.
      
      This patch partially reverts 3ff1f1e3
      (box: remove sequence_get) here similar function "get" was removed
      to avoid possible misleading with "currval" function of PosgreSQL
      that returns the last obtained value of the sequence in the scope
      of current session. In contrast "current" returns the last globally
      retrieved value of the sequence.
      
      Closes #4752
      
      Reviewed-by: default avatarVladislav Shpilevoy <v.shpilevoy@tarantool.org>
      Reviewed-by: default avatarNikita Pettik <korablev@tarantool.org>
      
      @TarantoolBot document
      Title: sequence:current()
      
      This patch introduces "current" function for sequences.
      It returns the last retrieved value of specified sequence or
      throws an error if no value has been generated yet ("next"
      has not been called yet or right after "reset" is called).
      
      Lua:
      
      Example:
      
      ```lua
      sq = box.schema.sequence.create('test')
      ---
      ...
      sq:current()
      ---
      - error: Sequence 'test' is not started
      ...
      sq:next()
      ---
      - 1
      ...
      sq:current()
      ---
      - 1
      ...
      sq:set(42)
      ---
      ...
      sq:current()
      ---
      - 42
      ...
      sq:reset()
      ---
      ...
      sq:current()  -- error
      ---
      - error: Sequence 'test' is not started
      ...
      ```
      
      C API:
      
      ```C
      int
      box_sequence_current(uint32_t seq_id, int64_t *result);
      ```
      
      Where:
        * seq_id - sequence identifier;
        * result - pointer to a variable where the current sequence
        value will be stored on success.
      
      Returns 0 on success and -1 otherwise. In case of an error user
      could get it via `box_error_last()`.
      64c69fe0
  25. Mar 05, 2020
  26. Dec 28, 2019
    • Nikita Pettik's avatar
      sql: display line and position in syntax errors · ea958f41
      Nikita Pettik authored
      When it comes for huge queries, it may turn out to be useful to see
      exact position of occurred error. Hence, let's now display line and
      position within line near which syntax error takes place. Note that it
      can be done only during parsing process (since AST can be analysed only
      after its construction is completed), so most of semantic errors still
      don't contain position. A few errors have been reworked to match new
      formatting patterns.
      
      First iteration of this patch is implemented by @romanhabibov
      
      Closes #2611
      ea958f41
  27. Dec 27, 2019
    • Mergen Imeev's avatar
      sql: introduce DOUBLE type · 64745b10
      Mergen Imeev authored
      This patch introduces type DOUBLE in SQL.
      
      Closes #3812
      Needed for #4233
      
      @TarantoolBot document
      Title: Tarantool DOUBLE field type and DOUBLE type in SQL
      The DOUBLE field type was added to Tarantool mainly for adding the
      DOUBLE type to SQL. Values of this type are stored as MP_DOUBLE in
      msgpack. The size of the encoded value is always 9 bytes.
      
      In Lua, only non-integer numbers and CDATA of type DOUBLE can be
      inserted in this field. You cannot insert integers of type Lua
      NUMBER or CDATA of type int64 or uint64 in this field. The same
      rules apply to key in get(), select(), update() and upsert()
      methods. It was done this way to avoid unwanted implicit casts
      that could affect performance.
      
      It is important to note that you can use the ffi.cast() function
      to cast numbers to CDATA of type DOUBLE. An example of this can be
      seen below.
      
      Another very important point is that CDATA of type DOUBLE in lua
      can be used in arithmetic, but arithmetic for them does not work
      correctly. This comes from LuaJIT and most likely will not be
      fixed.
      
      Example of usage in Lua:
      s = box.schema.space.create('s', {format = {{'d', 'double'}}})
      _ = s:create_index('ii')
      s:insert({1.1})
      ffi = require('ffi')
      s:insert({ffi.cast('double', 1)})
      s:insert({ffi.cast('double', tonumber('123'))})
      s:select(1.1)
      s:select({ffi.cast('double', 1)})
      
      In SQL, DOUBLE type behavior is different due to implicit casting.
      In a column of type DOUBLE, the number of any supported type can
      be inserted. However, it is possible that the number that will be
      inserted will be different from that which is inserted due to the
      rules for casting to DOUBLE. In addition, after this patch, all
      floating point literals will be recognized as DOUBLE. Prior to
      that, they were considered as NUMBER.
      
      Example of usage in SQL:
      box.execute('CREATE TABLE t (d DOUBLE PRIMARY KEY);')
      box.execute('INSERT INTO t VALUES (10), (-2.0), (3.3);')
      box.execute('SELECT * FROM t;')
      box.execute('SELECT d / 100 FROM t;')
      box.execute('SELECT * from t WHERE d < 15;')
      box.execute('SELECT * from t WHERE d = 3.3;')
      64745b10
  28. Oct 16, 2019
    • Kirill Shcherbatov's avatar
      sql: add an ability to disable CK constraints · c4781f93
      Kirill Shcherbatov authored
      Closes #4244
      
      @TarantoolBot document
      Title: an ability to disable CK constraints
      
      Now it is possible to disable and enable ck constraints.
      All ck constraints are enabled by default when Tarantool is
      configured. Ck constraints checks are not performed during
      standard recovery, but performed during force_recovery -
      all conflicting tuples are skipped in case of ck_constraint
      conflict.
      
      To change CK constraint "is_enabled" state, call
      -- in LUA
      ck_obj:enable(new_state in {true, false})
      -- in SQL
      ALTER TABLE {TABLE_NAME} {EN, DIS}ABLE CHECK CONSTRAINT {CK_NAME};
      
      Example:
      box.space.T6.ck_constraint.ck_unnamed_T6_1:enable(false)
      box.space.T6.ck_constraint.ck_unnamed_T6_1
      - space_id: 512
        is_enabled: false
        name: ck_unnamed_T6_1
        expr: a < 10
      box.space.T6:insert({11})
      -- passed
      box.execute("ALTER TABLE t6 ENABLE CHECK CONSTRAINT \"ck_unnamed_T6_1\"")
      box.space.T6:insert({12})
      - error: 'Check constraint failed ''ck_unnamed_T6_1'': a < 10'
      c4781f93
  29. Sep 16, 2019
    • Nikita Pettik's avatar
      sql: remove ENGINE from list of reserved keywords · 07618dad
      Nikita Pettik authored
      ENGINE became reserved keyword in 1013a744. There's no any actual
      reason why ENGINE should be reserved keyword. What is more, we are going
      to use this word as a name of some fields for tables forming
      informational schema. Hence, until it is too late (it is not documented
      yet), let's remove ENGINE from the list of reserved keywords and allow
      identifiers be that word.
      07618dad
  30. Aug 22, 2019
    • Max Melentiev's avatar
      tarantoolctl: allow to start instances with delayed box.cfg{} · 17df9edf
      Max Melentiev authored
      `tarantoolctl start` patches box.cfg two times:
      1) before the init script to set default values and enforce some others,
      2) after the init script to prevent changing a pid_file in runtime.
      
      The second patching fails if an init file does not call
      box.cfg{} before it's finished. This can take a place in apps with
      managed instances which receive configuration from external server.
      
      This patch moves the second patching into the box.cfg
      wrapper created during the first patching. So the second patching
      is performed only after box.cfg{} was invoked, so it does not fail anymore.
      
      However there is relatively minor flaw for applications that
      invoke box.cfg{} after init script is finished:
      `tarantoolctl start` goes to background only when box.cfg{} is called.
      Though this is not the case for daemon management systems like systemd,
      as they handle backgrounding on their side
      
      Fixes #4435
      
      @TarantoolBot document
      Title: tarantoolctl allows to start instances without a box.cfg{}
      
      tarantoolctl now works for instances without box.cfg{} or
      with delayed box.cfg{} call. This can be managed instances which receive
      configuration from external server.
      
      For such instances `tarantoolctl start` goes to background when
      box.cfg{} is called, so it will wait until options for box.cfg are received.
      However this is not the case for daemon management systems like systemd,
      as they handle backgrounding on their side.
      17df9edf
Loading