Skip to content
Snippets Groups Projects
  1. Jun 28, 2019
    • Vladislav Shpilevoy's avatar
      swim: fix a leak when a trigger is installed · eb403598
      Vladislav Shpilevoy authored
      SWIM wraps user triggers to prepare arguments. The wrapper
      function kept a reference to SWIM object, and prevented its
      automatic deletion at GC.
      
      The patch makes this reference weak.
      
      Follow up #4250
      eb403598
    • Mergen Imeev's avatar
      sql: allow to use vectors as left value of IN operator · 7418c373
      Mergen Imeev authored
      In SQL, it is allowed to use vector expressions, that is, an
      operation that uses vectors as operands. For instance, vector
      comparison:
      SELECT (1,2,3) < (1,2,4);
      
      Accidentally, routines handling IN operator contained a bug: in
      cases where we used a vector as the left value in the IN operator,
      we received an assertion in debug build or a segmentation fault in
      release. This was due to some legacy code in which it was assumed
      that the left value of the IN operator can have only one column in
      case it is a vector. Let's fix this by allowing vectors of the
      other sizes as the left value of the IN operator and providing
      check which verifies that both sides of IN operator have the same
      dimension.
      
      Closes #4204
      7418c373
    • Georgy Kirichenko's avatar
      test: fix flaky test · 3f5806ab
      Georgy Kirichenko authored
      This test fails sporadically
      3f5806ab
  2. Jun 27, 2019
  3. Jun 25, 2019
    • Georgy Kirichenko's avatar
      applier: apply transaction in parallel · 8c84932a
      Georgy Kirichenko authored
      Applier use asynchronous transaction to batch journal writes. All
      appliers share the replicaset.applier.tx_vclock which means the vclock
      applied but not necessarily written to a journal. Appliers use a trigger
      to coordinate in case of failure - when a transaction is going to
      be rolled back.
      
      Closes: #1254
      8c84932a
    • Georgy Kirichenko's avatar
      txn: introduce asynchronous txn commit · 18ea440a
      Georgy Kirichenko authored
      This commit implements asynchronous transaction processing using
      txn_write. The method prepares a transaction and sends it to an journal
      without an yield until the transaction was finished. The transaction
      status could be controlled via on_commit/on_rollback triggers.
      In order to support asynchronous transaction journal_write method turned
      to an asynchronous one and now a transaction engine controls journal status
      using journal entry finalization callback.
      
      Prerequisites: #1254
      18ea440a
    • Georgy Kirichenko's avatar
      wal: introduce a journal entry finalization callback · a16ff869
      Georgy Kirichenko authored
      Finalize a transaction thorough a journal entry callback. So transaction
      processing doesn't rely on fiber schedule.
      Also allow to steal locked latch ownership for fiber which isn't owner
      of the latch. This is required to process transaction triggers
      asynchronously.
      
      Prerequisites: #1254
      a16ff869
    • Georgy Kirichenko's avatar
      txn: get rid of fiber_gc from txn_rollback · 68404cfd
      Georgy Kirichenko authored
      Refactoring: don't touch a fiber gc storage on a transaction rollback
      explicitly. This relaxes dependencies between fiber and transaction
      life cycles.
      
      Prerequisites: #1254
      68404cfd
    • Georgy Kirichenko's avatar
      txn: get rid of autocommit from a txn structure · e070cc4d
      Georgy Kirichenko authored
      Move transaction auto start and auto commit behavior to the box level.
      From now a transaction won't start and commit automatically without
      txn_begin/txn_commit invocations. This is a part of a bigger transaction
      refactoring in order to implement detachable transactions and a parallel
      applier.
      
      Prerequisites: #1254
      e070cc4d
    • Georgy Kirichenko's avatar
      txn: unref statement at txn_free · 399e2b65
      Georgy Kirichenko authored
      Refactoring: put txn statement unref code into transaction free function.
      399e2b65
    • Stanislav Zudin's avatar
      ddl: No replication for temp and local spaces · 564ba89a
      Stanislav Zudin authored
      Do not spread the space:truncate() to replicas if the
      affected space is local or temporary.
      
      Closes #4263
      564ba89a
    • Konstantin Osipov's avatar
      Revert "box: introduce Lua persistent functions" · a758bced
      Konstantin Osipov authored
      This reverts commit 4e3470ce.
      
      The RFC did not pass review yet.
      a758bced
  4. Jun 24, 2019
    • Vladislav Shpilevoy's avatar
      test: fix unit/crypto test flakiness · ad1ec950
      Vladislav Shpilevoy authored
      One of subtests was checking if crypto_decode returns an error
      when fails to decode. But due to randomness of the test sometimes
      it happened, that initial vector of encrypted data somehow didn't
      lead to an error. Decryption was not correct, but only in terms
      of result, not in terms of decryption algorithm. -1 was not
      returned, and diag was not set.
      
      This patch checks all the cases.
      
      Closes #4306
      ad1ec950
    • Kirill Shcherbatov's avatar
      box: introduce Lua persistent functions · 4e3470ce
      Kirill Shcherbatov authored
      Closes #4182
      Needed for #1260
      
      @TarantoolBot document
      Title: Persistent Lua functions
      
      Now Tarantool supports 'persistent' Lua functions.
      Such functions are stored in snapshot and are available after
      restart.
      To create a persistent Lua function, specify a function body
      in box.schema.func.create call:
      e.g. body = "function(a, b) return a + b end"
      
      A Lua persistent function may be 'sandboxed'. The 'sandboxed'
      function is executed in isolated environment:
        a. only limited set of Lua functions and modules are available:
          -assert -error -pairs -ipairs -next -pcall -xpcall -type
          -print -select -string -tonumber -tostring -unpack -math -utf8;
        b. global variables are forbidden
      
      Finally, the new 'is_deterministic' flag allows to mark a
      registered function as deterministic, i.e. the function that
      can produce only one result for a given list of parameters.
      
      The new box.schema.func.create interface is:
      box.schema.func.create('funcname', <setuid = true|FALSE>,
      	<if_not_exists = true|FALSE>, <language = LUA|c>,
      	<body = string ('')>, <is_deterministic = true|FALSE>,
      	<is_sandboxed = true|FALSE>, <comment = string ('')>)
      
      Example:
      lua_code = [[function(a, b) return a + b end]]
      box.schema.func.create('sum', {body = lua_code,
      		is_deterministic = true, is_sandboxed = true})
      box.func.sum
      ---
      - is_sandboxed: true
        is_deterministic: true
        id: 2
        setuid: false
        body: function(a, b) return a + b end
        name: sum
        language: LUA
      ...
      box.func.sum:call({1, 3})
      ---
      - 4
      ...
      4e3470ce
    • Kirill Shcherbatov's avatar
      box: refactor box_lua_find helper · 5035e3d5
      Kirill Shcherbatov authored
      The box_lua_find routine used to work with an empty stack only.
      It is unacceptable in following patches because this helper
      need to be reused in following patches for environment table
      construction.
      
      Needed for #4182, #1260
      5035e3d5
    • Kirill Shcherbatov's avatar
      box: export registered functions in box.func folder · abf5ef4f
      Kirill Shcherbatov authored
      Needed for #4182, #1260
      
      @TarantoolBot document
      Title: Export registered functions to box.func folder
      
      Now all registered with box.schema.func.create functions are
      exported in box.func folder.
      Each function have :call and :drop method. The :drop method
      just a shortcut for box.schema.func.drop interface.
      The :call method is similar to net.box connection:call method
      and allows to call a registered function directly.
      All access checks are performed on each function call.
      
      Example:
      function sum(a, b) return a + b end
      box.schema.func.create('sum')
      box.func.sum
      ---
      - language: LUA
        setuid: false
        name: sum
        id: 2
      ...
      box.func.sum:call({1, 3})
      ---
      - 4
      ...
      box.func.sum:drop()
      abf5ef4f
    • Kirill Shcherbatov's avatar
      box: rework func object as a function frontend · 5b3e0551
      Kirill Shcherbatov authored
      The function func object used to provide a call method only for
      C functions. In scope of this patch it reworked to be a uniform
      function call frontend both for C and Lua functions.
      
      Introduced classes func_c and func_lua, that provide own
      constructors which produce implementation-specific object with
      call and destroy methods.
      
      Needed for #4182, #1260
      5b3e0551
    • Kirill Shcherbatov's avatar
      box: rework box_lua_{call, eval} to use input port · 707e58a3
      Kirill Shcherbatov authored
      Re-factor box_lua_call and box_lua_eval so that they don't take
      call_request. This approach is more scalable: in case of a
      functional index, the user expects to see a tuple with field
      names so we should be able to pass not only raw msgpack, but
      also a tuple to a Lua call so we need an universal way to pass
      arguments to _call methods.
      
      To pass a tuple msgpack introduced a new port_msgpack: the port
      class with dump_lua method.
      A new method get_msgpack returns a content of a port as a
      msgpack data. The lifecycle of the returned value is
      implementation-specific: it may either be returned directly from
      the port, in which case the data will stay alive as long as the
      port is alive, or it may be allocated on the fiber()->gc, in
      which case the caller is responsible for cleaning up.
      
      Needed for #4182, #1260
      707e58a3
    • Kirill Shcherbatov's avatar
      box: rework func cache update machinery · d026b546
      Kirill Shcherbatov authored
      Tarantool used to assume that func_new call must not fail and
      it used to build a new func object by given definition just on
      func cache replace operation. We need to fix it to perform
      user-dependent risky actions like Lua function assemble in
      further patches.
      
      The replace method is disallowed for _func space because it is
      redundant and difficult to maintain in case of functions that
      have pre-compiled runtime.
      
      Needed for #4182, #1260
      d026b546
  5. Jun 23, 2019
    • Vladislav Shpilevoy's avatar
      swim: introduce incarnation.generation · 44b9a605
      Vladislav Shpilevoy authored
      SWIM uses incarnation to refute old information, but it is not
      enough when restarts are possible. If an instance restarts, its
      incarnation is reset to 0. After several local and fast updates
      it gets N. But it is possible, that other instances also know
      incarnation of this instance as N, from its previous life, but
      with different information. They will never take new version of
      data, because their current version is also considered actual.
      
      As a result, incarnation is not enough. There was a necessity to
      create a persistent part of incarnation. This patch introduces it
      and calls 'generation'. As an additional profit, generation
      allows to react on instance restart in user defined triggers.
      
      Closes #4280
      
      @TarantoolBot document
      Title: SWIM generation
      
      Incarnation now is a two-part value {generation, version}.
      
      Version is exactly the same that is called 'incarnation' in the
      original SWIM paper, and before this patch. It is a volatile
      automatically managed number to refute false gossips and update
      information on remote nodes.
      
      Generation is a new persistent part of incarnation allowing users
      to refute old pieces of information left from previous lifes of an
      instance. It is a static attribute set when a SWIM instance is
      created, and can't be changed without restarting the instance.
      
      A one could think of incarnation as 128 bit unsigned integer,
      where upper 64 bits are static and persistent, while lower 64 bits
      are volatile.
      
      Generation not only helps with overriding old information, but
      also can be used to detect restarts in user defined triggers,
      because it can be updated only when a SWIM instance is recreated.
      
      How to set generation:
      ```Lua
      swim = require('swim')
      s = swim.new({generation = <value>})
      ```
      Generation can't be set in `swim:cfg`. If it is omitted, then 0
      is used by default. But be careful - if the instance is started
      not a first time, it is safer to use a new generation. Ideally it
      should be persisted somehow: in a file, in a space, in a global
      service. Each next `swim.new()` should take incremented value of
      generation.
      
      How is incarnation update changed:
      ```Lua
      swim = require('swim')
      s = swim.new()
      s:on_member_event(function(m, e)
          if e:is_new_incarnation() then
              if e:is_new_generation() then
                  -- Process restart.
              end
              if e:is_new_version() then
                  -- Process version update. It means
                  -- the member is somehow changed.
              end
          end
      end)
      ```
      
      Note, `is_new_incarnation` is now a shortcut for checking update
      of generation, or version, or both.
      
      Method `member:incarnation()` is changed. Now it returns cdata
      object with attributes `version` and `generation`. Usage:
      ```Lua
      incarnation = member:incarnation()
      tarantool> incarnation.version
      ---
      - 15
      ...
      tarantool> incarnation.generation
      ---
      - 2
      ...
      ```
      
      These objects can be compared using comparison operators:
      ```Lua
      member1:incarnation() < member2:incarnation
      member1:incarnation() >= member2:incarnation()
      -- Any operator works: ==, <, >, <=, >=, ~=.
      ```
      
      Being printed, incarnation shows a string with both generation
      and incarnation.
      
      Binary protocol is updated. Now Protocol Logic section looks like
      this:
      
      ```
      +-------------------Protocol logic section--------------------+
      | map {                                                       |
      |     0 = SWIM_SRC_UUID: 16 byte UUID,                        |
      |                                                             |
      |                 AND                                         |
      |                                                             |
      |     2 = SWIM_FAILURE_DETECTION: map {                       |
      |         0 = SWIM_FD_MSG_TYPE: uint, enum swim_fd_msg_type,  |
      |         1 = SWIM_FD_GENERATION: uint,                       |
      |         2 = SWIM_FD_VERSION: uint                           |
      |     },                                                      |
      |                                                             |
      |               OR/AND                                        |
      |                                                             |
      |     3 = SWIM_DISSEMINATION: array [                         |
      |         map {                                               |
      |             0 = SWIM_MEMBER_STATUS: uint,                   |
      |                                     enum member_status,     |
      |             1 = SWIM_MEMBER_ADDRESS: uint, ip,              |
      |             2 = SWIM_MEMBER_PORT: uint, port,               |
      |             3 = SWIM_MEMBER_UUID: 16 byte UUID,             |
      |             4 = SWIM_MEMBER_GENERATION: uint,               |
      |             5 = SWIM_MEMBER_VERSION: uint,                  |
      |             6 = SWIM_MEMBER_PAYLOAD: bin                    |
      |         },                                                  |
      |         ...                                                 |
      |     ],                                                      |
      |                                                             |
      |               OR/AND                                        |
      |                                                             |
      |     1 = SWIM_ANTI_ENTROPY: array [                          |
      |         map {                                               |
      |             0 = SWIM_MEMBER_STATUS: uint,                   |
      |                                     enum member_status,     |
      |             1 = SWIM_MEMBER_ADDRESS: uint, ip,              |
      |             2 = SWIM_MEMBER_PORT: uint, port,               |
      |             3 = SWIM_MEMBER_UUID: 16 byte UUID,             |
      |             4 = SWIM_MEMBER_GENERATION: uint,               |
      |             5 = SWIM_MEMBER_VERSION: uint,                  |
      |             6 = SWIM_MEMBER_PAYLOAD: bin                    |
      |         },                                                  |
      |         ...                                                 |
      |     ],                                                      |
      |                                                             |
      |               OR/AND                                        |
      |                                                             |
      |     4 = SWIM_QUIT: map {                                    |
      |         0 = SWIM_QUIT_GENERATION: uint,                     |
      |         1 = SWIM_QUIT_VERSION: uint                         |
      |     }                                                       |
      | }                                                           |
      +-------------------------------------------------------------+
      ```
      
      Note - SWIM_FD_INCARNATION, SWIM_MEMBER_INCARNATION, and
      SWIM_QUIT_INCARNATION disappeared. Incarnation is sent now in two
      parts: version and generation.
      
      SWIM_MEMBER_PAYLOAD got a new value.
      
      This changes are legal because 1) the SWIM is not released yet,
      so it is mutable, 2) I wanted to emphasize that 'generation' is
      first/upper part of incarnation, 'version' is second/lower part.
      44b9a605
    • Vladislav Shpilevoy's avatar
      swim: make incarnation struct · 3aecf9f0
      Vladislav Shpilevoy authored
      Traditional SWIM describes member version as incarnation -
      volatile monotonically growing number to refute false gossips.
      But it is not enough in the real world because of necessity to
      detect restarts and refute information from previous lifes of an
      instance.
      
      Incarnation is going to be a two-part value with persistent upper
      part and volatile lower part. This patch does preparations making
      incarnation struct instead of a number.
      
      Volatile part is called 'version.
      
      Part of #4280
      3aecf9f0
  6. Jun 21, 2019
    • Serge Petrenko's avatar
      decimal: add const qualifiers for decimal_pack() and decimal_len() · 88a23290
      Serge Petrenko authored
      The const qualifiers weren't there initially for some reason.
      Fix this.
      
      Part of #692
      88a23290
    • Serge Petrenko's avatar
      lib: update msgpuck library · 00d770a9
      Serge Petrenko authored
      The ability to encode/decode ext types is added to msgpuck.
      These types will be used for decimal encoding/decoding in tarantool.
      
      Needed for #692
      00d770a9
    • Mergen Imeev's avatar
      netbox: define formats for tuple from netbox · 97e8c658
      Mergen Imeev authored
      This patch creates tuple_formats for the tuples obtained through
      the netbox.
      
      Closes #2978
      
      @TarantoolBot document
      Title: Field names for tuples received from net.box
      
      It is possible now to access by field name for tuples received
      from net.box. For example:
      
      box.cfg{listen = 3302}
      box.schema.user.grant('guest','read, write, execute', 'space')
      box.schema.user.grant('guest', 'create', 'space')
      
      box.schema.create_space("named", {format = {{name = "id"}}})
      box.space.named:create_index('id', {parts = {{1, 'unsigned'}}})
      box.space.named:insert({1})
      require('net.box').connect('localhost', 3302).space.named:get(1).id
      
      Result:
      
      tarantool> require('net.box').connect('localhost', 3302).space.named:get(1).id
      ---
      - 1
      ...
      97e8c658
    • Mergen Imeev's avatar
      lua: internal function to parse space format · bb85acfd
      Mergen Imeev authored
      This patch defines a new function that parses space format and
      returns it to Lua as cdata. For this cdata destructor is included.
      
      Needed for #2978
      bb85acfd
    • Mergen Imeev's avatar
      netbox: store method_encoder args in request · 665fc3a1
      Mergen Imeev authored
      This patch changes the way arguments are passed to functions in
      the array method_encoder, and saves these arguments in REQUEST.
      This is necessary to establish a connection between netbox space
      objects and the tuples obtained through the netbox
      
      Needed for #2978
      665fc3a1
  7. Jun 20, 2019
Loading