Skip to content
Snippets Groups Projects
  1. Jun 23, 2019
    • Vladislav Shpilevoy's avatar
      swim: introduce incarnation.generation · 44b9a605
      Vladislav Shpilevoy authored
      SWIM uses incarnation to refute old information, but it is not
      enough when restarts are possible. If an instance restarts, its
      incarnation is reset to 0. After several local and fast updates
      it gets N. But it is possible, that other instances also know
      incarnation of this instance as N, from its previous life, but
      with different information. They will never take new version of
      data, because their current version is also considered actual.
      
      As a result, incarnation is not enough. There was a necessity to
      create a persistent part of incarnation. This patch introduces it
      and calls 'generation'. As an additional profit, generation
      allows to react on instance restart in user defined triggers.
      
      Closes #4280
      
      @TarantoolBot document
      Title: SWIM generation
      
      Incarnation now is a two-part value {generation, version}.
      
      Version is exactly the same that is called 'incarnation' in the
      original SWIM paper, and before this patch. It is a volatile
      automatically managed number to refute false gossips and update
      information on remote nodes.
      
      Generation is a new persistent part of incarnation allowing users
      to refute old pieces of information left from previous lifes of an
      instance. It is a static attribute set when a SWIM instance is
      created, and can't be changed without restarting the instance.
      
      A one could think of incarnation as 128 bit unsigned integer,
      where upper 64 bits are static and persistent, while lower 64 bits
      are volatile.
      
      Generation not only helps with overriding old information, but
      also can be used to detect restarts in user defined triggers,
      because it can be updated only when a SWIM instance is recreated.
      
      How to set generation:
      ```Lua
      swim = require('swim')
      s = swim.new({generation = <value>})
      ```
      Generation can't be set in `swim:cfg`. If it is omitted, then 0
      is used by default. But be careful - if the instance is started
      not a first time, it is safer to use a new generation. Ideally it
      should be persisted somehow: in a file, in a space, in a global
      service. Each next `swim.new()` should take incremented value of
      generation.
      
      How is incarnation update changed:
      ```Lua
      swim = require('swim')
      s = swim.new()
      s:on_member_event(function(m, e)
          if e:is_new_incarnation() then
              if e:is_new_generation() then
                  -- Process restart.
              end
              if e:is_new_version() then
                  -- Process version update. It means
                  -- the member is somehow changed.
              end
          end
      end)
      ```
      
      Note, `is_new_incarnation` is now a shortcut for checking update
      of generation, or version, or both.
      
      Method `member:incarnation()` is changed. Now it returns cdata
      object with attributes `version` and `generation`. Usage:
      ```Lua
      incarnation = member:incarnation()
      tarantool> incarnation.version
      ---
      - 15
      ...
      tarantool> incarnation.generation
      ---
      - 2
      ...
      ```
      
      These objects can be compared using comparison operators:
      ```Lua
      member1:incarnation() < member2:incarnation
      member1:incarnation() >= member2:incarnation()
      -- Any operator works: ==, <, >, <=, >=, ~=.
      ```
      
      Being printed, incarnation shows a string with both generation
      and incarnation.
      
      Binary protocol is updated. Now Protocol Logic section looks like
      this:
      
      ```
      +-------------------Protocol logic section--------------------+
      | map {                                                       |
      |     0 = SWIM_SRC_UUID: 16 byte UUID,                        |
      |                                                             |
      |                 AND                                         |
      |                                                             |
      |     2 = SWIM_FAILURE_DETECTION: map {                       |
      |         0 = SWIM_FD_MSG_TYPE: uint, enum swim_fd_msg_type,  |
      |         1 = SWIM_FD_GENERATION: uint,                       |
      |         2 = SWIM_FD_VERSION: uint                           |
      |     },                                                      |
      |                                                             |
      |               OR/AND                                        |
      |                                                             |
      |     3 = SWIM_DISSEMINATION: array [                         |
      |         map {                                               |
      |             0 = SWIM_MEMBER_STATUS: uint,                   |
      |                                     enum member_status,     |
      |             1 = SWIM_MEMBER_ADDRESS: uint, ip,              |
      |             2 = SWIM_MEMBER_PORT: uint, port,               |
      |             3 = SWIM_MEMBER_UUID: 16 byte UUID,             |
      |             4 = SWIM_MEMBER_GENERATION: uint,               |
      |             5 = SWIM_MEMBER_VERSION: uint,                  |
      |             6 = SWIM_MEMBER_PAYLOAD: bin                    |
      |         },                                                  |
      |         ...                                                 |
      |     ],                                                      |
      |                                                             |
      |               OR/AND                                        |
      |                                                             |
      |     1 = SWIM_ANTI_ENTROPY: array [                          |
      |         map {                                               |
      |             0 = SWIM_MEMBER_STATUS: uint,                   |
      |                                     enum member_status,     |
      |             1 = SWIM_MEMBER_ADDRESS: uint, ip,              |
      |             2 = SWIM_MEMBER_PORT: uint, port,               |
      |             3 = SWIM_MEMBER_UUID: 16 byte UUID,             |
      |             4 = SWIM_MEMBER_GENERATION: uint,               |
      |             5 = SWIM_MEMBER_VERSION: uint,                  |
      |             6 = SWIM_MEMBER_PAYLOAD: bin                    |
      |         },                                                  |
      |         ...                                                 |
      |     ],                                                      |
      |                                                             |
      |               OR/AND                                        |
      |                                                             |
      |     4 = SWIM_QUIT: map {                                    |
      |         0 = SWIM_QUIT_GENERATION: uint,                     |
      |         1 = SWIM_QUIT_VERSION: uint                         |
      |     }                                                       |
      | }                                                           |
      +-------------------------------------------------------------+
      ```
      
      Note - SWIM_FD_INCARNATION, SWIM_MEMBER_INCARNATION, and
      SWIM_QUIT_INCARNATION disappeared. Incarnation is sent now in two
      parts: version and generation.
      
      SWIM_MEMBER_PAYLOAD got a new value.
      
      This changes are legal because 1) the SWIM is not released yet,
      so it is mutable, 2) I wanted to emphasize that 'generation' is
      first/upper part of incarnation, 'version' is second/lower part.
      44b9a605
    • Vladislav Shpilevoy's avatar
      swim: make incarnation struct · 3aecf9f0
      Vladislav Shpilevoy authored
      Traditional SWIM describes member version as incarnation -
      volatile monotonically growing number to refute false gossips.
      But it is not enough in the real world because of necessity to
      detect restarts and refute information from previous lifes of an
      instance.
      
      Incarnation is going to be a two-part value with persistent upper
      part and volatile lower part. This patch does preparations making
      incarnation struct instead of a number.
      
      Volatile part is called 'version.
      
      Part of #4280
      3aecf9f0
  2. Jun 21, 2019
    • Serge Petrenko's avatar
      decimal: add const qualifiers for decimal_pack() and decimal_len() · 88a23290
      Serge Petrenko authored
      The const qualifiers weren't there initially for some reason.
      Fix this.
      
      Part of #692
      88a23290
    • Serge Petrenko's avatar
      lib: update msgpuck library · 00d770a9
      Serge Petrenko authored
      The ability to encode/decode ext types is added to msgpuck.
      These types will be used for decimal encoding/decoding in tarantool.
      
      Needed for #692
      00d770a9
    • Mergen Imeev's avatar
      netbox: define formats for tuple from netbox · 97e8c658
      Mergen Imeev authored
      This patch creates tuple_formats for the tuples obtained through
      the netbox.
      
      Closes #2978
      
      @TarantoolBot document
      Title: Field names for tuples received from net.box
      
      It is possible now to access by field name for tuples received
      from net.box. For example:
      
      box.cfg{listen = 3302}
      box.schema.user.grant('guest','read, write, execute', 'space')
      box.schema.user.grant('guest', 'create', 'space')
      
      box.schema.create_space("named", {format = {{name = "id"}}})
      box.space.named:create_index('id', {parts = {{1, 'unsigned'}}})
      box.space.named:insert({1})
      require('net.box').connect('localhost', 3302).space.named:get(1).id
      
      Result:
      
      tarantool> require('net.box').connect('localhost', 3302).space.named:get(1).id
      ---
      - 1
      ...
      97e8c658
    • Mergen Imeev's avatar
      lua: internal function to parse space format · bb85acfd
      Mergen Imeev authored
      This patch defines a new function that parses space format and
      returns it to Lua as cdata. For this cdata destructor is included.
      
      Needed for #2978
      bb85acfd
    • Mergen Imeev's avatar
      netbox: store method_encoder args in request · 665fc3a1
      Mergen Imeev authored
      This patch changes the way arguments are passed to functions in
      the array method_encoder, and saves these arguments in REQUEST.
      This is necessary to establish a connection between netbox space
      objects and the tuples obtained through the netbox
      
      Needed for #2978
      665fc3a1
  3. Jun 20, 2019
  4. Jun 19, 2019
Loading