Skip to content
Snippets Groups Projects
  1. Apr 25, 2022
    • Serge Petrenko's avatar
      raft: introduce Pre-Vote machinery · ba19ceaa
      Serge Petrenko authored
      Support Pre-Vote in Raft: when a node knows, that some of the peers see
      the leader of the current term, it doesn't start elections. Even when it
      doesn't have a connection to the leader.
      
      Don't use the machinery yet: nodes never share info on whether they see
      the leader.
      
      Part-of #6654
      
      NO_DOC=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      ba19ceaa
    • Serge Petrenko's avatar
      raft: add comments to raft_ev.h declarations · dd41408c
      Serge Petrenko authored
      Our checkpatch.pl script demands that every new function has a comment.
      One new function will be added to raft_ev.h in the following commits,
      and, according to the new rules, it has to have a comment.
      
      Having one function in a file commented looks stupid, so let's add
      comments to every such function in raft_ev.h
      
      NO_DOC=making lint happy
      NO_CHANGELOG=making lint happy
      NO_TEST=making lint happy
      dd41408c
    • Serge Petrenko's avatar
      raft: reuse raft_checkpoint_remote in broadcast · 74166f16
      Serge Petrenko authored
      raft_checkpoint_remote and raft_worker_handle_broadcast perform the same
      job of filling the raft message with a complete raft state.
      
      Let's fix that.
      
      Part-of #6654
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      74166f16
    • Serge Petrenko's avatar
      replication: count bidirectional replica connections · fbaa7a99
      Serge Petrenko authored
      Start counting replicas, whose connections are "healthy", e.g. there are
      both relay and applier in SUBSCRIBE state.
      
      Introduce triggers for the events of "healthy quorum" loss and gain.
      "healthy quorum" is the same as replication_synchro_quorum.
      
      The triggers will be used in raft pre-vote and synchro queue owner
      fencing. Both these features require tracking whether there is a quorum
      of nodes reachable from the local node.
      
      Pre-requisite #6654
      Pre-requisite #6661
      
      NO_DOC=no user-visible changes
      NO_CHANGELOG=no user-visible changes
      NO_TEST=preparatory patch
      fbaa7a99
    • Aleksandr Lyapunov's avatar
      mvcc: track nothing-found range read · 2f543937
      Aleksandr Lyapunov authored
      MVCC must track a read set of transaction, including cases when
      the select returns an empty result. In this case MVCC has two
      options: to store a point key if the select has a full key or
      to store a gap otherwise.
      
      By a mistake the second variant was not implemented. Fix it now.
      
      Closes #7025
      
      NO_DOC=bugfix
      2f543937
    • Vladimir Davydov's avatar
      box: add space upgrade stubs · 38b25832
      Vladimir Davydov authored
      To implement online space upgrade, we need to add stub calls to the
      following code paths and data structures:
      
       - struct space: add a pointer to space upgrade state.
       - struct space_opts: add a pointer to space upgrade definition.
       - CheckSpaceFormat::prepare: skip space check if the format is changed
         in the scope of space upgrade.
       - alter_space_do: check space upgrade state and fail alter if upgrade
         is in progress.
       - alter_space_commit: run background worker for space upgrade.
       - space_on_final_recovery_complete: restart upgrade after recovery.
       - result_processor: apply space upgrade transformations to tuples
         returned to the user by box functions.
      
      We also need to:
       - Add a new error code ER_WRONG_SPACE_UPGRADE_OPTIONS, which we will
         use on error decoding upgrade options, stored in space options.
       - Load space upgrade Lua modules. The modules are supposed to define
         box.internal.space.upgrade method, which if available is used by
         box.schema.space.upgrade.
       - Add check_param, check_param_table and normalize_format helpers to
         box.internal, because we will use them from space.upgrade Lua code.
      
      Note, the space upgrade state will be reference counted, because
      background space upgrade may complete while some fiber is reading
      from the upgraded space (there may be yields in Vinyl). For this fiber
      to process the result correctly, it has to increment the reference
      counter of the space upgrade state before reading from the space.
      
      NO_DOC=ee
      NO_TEST=ee
      NO_CHANGELOG=ee
      38b25832
    • Vladimir Davydov's avatar
      box: introduce result processor · 21e2def9
      Vladimir Davydov authored
      A convenient helper that is supposed to apply various transformations
      to a tuple fetched from a space before returning it to the user.
      Usage:
      
        struct result_processor res_proc;
        result_process_prepare(&res_proc, space);
        rc = index_get(index, key, part_count, result);
        result_process(&res_proc, &rc, result);
      
      We need to split the procedure into two parts, because index_get and
      similar methods may yield in case of Vinyl. The 'prepare' method is
      supposed to pin all data structures needed to process the result in the
      result_processor struct while the 'process' method is supposed to
      release them.
      
      Currently, the new methods do absolutely nothing. They will be used to
      convert tuples during space upgrade.
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      21e2def9
    • Vladimir Davydov's avatar
      index: add iterator_space helper · e00fd428
      Vladimir Davydov authored
      The new helper function returns the space the iterator was created for.
      It looks up the space by id and caches it in iterator struct. We need it
      for space upgrade: we will use it in box_iterator_next() to update
      tuples returned by iterator_next().
      
      NO_DOC=internal
      NO_TEST=internal
      NO_CHANGELOG=internal
      e00fd428
    • Vladimir Davydov's avatar
      index: drop unused xc wrappers · defe4213
      Vladimir Davydov authored
      NO_DOC=code cleanup
      NO_TEST=code cleanup
      NO_CHANGELOG=code cleanup
      defe4213
    • Vladimir Davydov's avatar
      schema: rename update_format to normalize_format · 49b84cd6
      Vladimir Davydov authored
      It's a more appropriate name, because the function doesn't update
      a space format - it just adjusts a format table to a normal form.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      49b84cd6
    • Mergen Imeev's avatar
      sql: arithmetic for DATETIME and INTERVAL · 0636ec43
      Mergen Imeev authored
      After this patch, it will be possible to perform permitted arithmetic
      operations on DATETIME and INTERVAL values.
      
      Closes #6773
      
      @TarantoolBot document
      Title: Field type interval is available in SQL
      
      The INTERVAL field type is now available in SQL. INTERVAL values cannot
      be compared. INTERVAL values can be explicitly and implicitly cast to
      ANY. There are currently no literals for the INTERVAL field type.
      0636ec43
    • Mergen Imeev's avatar
      sql: check left operand before right in arithmetic · e61f4f61
      Mergen Imeev authored
      Prior to this patch, the right operand was checked before the left. This
      can lead to some complications when arithmetic operations are added for
      the DATETIME and INTERVAL values. To avoid this, let's check the left
      operand before the right one. This will cause the description of the
      error to change, but nothing else.
      
      Needed for #6773
      
      NO_DOC=Not documented feature.
      e61f4f61
    • Mergen Imeev's avatar
      sql: allow to bind INTERVAL values · 95b9c87b
      Mergen Imeev authored
      Part of #6773
      
      NO_DOC=Will be added later.
      NO_CHANGELOG=Will be added later.
      95b9c87b
    • Mergen Imeev's avatar
      sql: introduce field type INTERVAL · 1c0fccbb
      Mergen Imeev authored
      This patch introduces basic INTERVAL support in SQL. After this patch,
      it will be allowed to select INTERVAL values from spaces, insert them
      into spaces, and use them in functions. CAST() from INTERVAL to STRING
      and ANY is also supported.
      
      Part of #6773
      
      NO_DOC=Will be added later.
      NO_CHANGELOG=Will be added later.
      1c0fccbb
    • Mergen Imeev's avatar
      sql: fix error description for comparison · 509598d9
      Mergen Imeev authored
      This patch fixes the error description for comparison in case the left
      operand is comparable but the right operand is not.
      
      Part of #6668
      Needed for #6668
      
      NO_DOC=Will be added in another commit.
      NO_CHANGELOG=Will be added in another commit.
      509598d9
    • Mergen Imeev's avatar
      sql: disallow incomparable values in GROUP BY · 36c36911
      Mergen Imeev authored
      After this patch, using MAP and ARRAY in GROUP BY will throw an error
      instead of an assertion.
      
      Part of #6668
      Needed for #6773
      
      NO_DOC=Will be added in another commit.
      NO_CHANGELOG=Will be added in another commit.
      36c36911
    • Mergen Imeev's avatar
      sql: fix error when DISTINCT applied to MAP/ARRAY · c93109b0
      Mergen Imeev authored
      After this patch, when DISTINCT is used with MAP or ARRAY, an error will
      be thrown instead of a segmentation fault.
      
      Part of #6668
      Needed for #6773
      
      NO_DOC=Will be added in another commit.
      NO_CHANGELOG=Will be added in another commit.
      c93109b0
  2. Apr 22, 2022
    • Igor Munkin's avatar
      luajit: bump new version · 60fe9d14
      Igor Munkin authored
      
      LuaJIT submodule is bumped to introduce the following changes:
      * memprof: enrich symtab with newly loaded symbols
      * memprof: extend symtab with C symbols
      
      Within this changeset the new Lua module providing minimalistic AVL tree
      implementation required for processing C symbols is introduced:
      * utils/avl.lua: minimalistic AVL tree implementation
      
      Closes #5813
      
      NO_DOC=LuaJIT submodule bump
      NO_TEST=LuaJIT submodule bump
      
      Signed-off-by: default avatarIgor Munkin <imun@tarantool.org>
      60fe9d14
    • Mergen Imeev's avatar
      box: introduce field type INTERVAL · a9e30fb3
      Mergen Imeev authored
      New field type INTERVAL is now supported by BOX.
      
      Part of #6773
      
      NO_DOC=INTERVAL has already been introduced earlier.
      a9e30fb3
    • Mergen Imeev's avatar
      lua: introduce MP_INTERVAL to module msgpackffi · e6b354f3
      Mergen Imeev authored
      Part of #6773
      
      NO_DOC=INTERVAL has already been introduced earlier.
      NO_CHANGELOG=It will be added in another commit.
      e6b354f3
    • Mergen Imeev's avatar
      lua: introduce MP_INTERVAL to module msgpack · 38f0c904
      Mergen Imeev authored
      Part of #6773
      
      NO_DOC=INTERVAL has already been introduced earlier.
      NO_CHANGELOG=It will be added in another commit.
      38f0c904
    • Mergen Imeev's avatar
      core: introduce MP_INTERVAL · ed861743
      Mergen Imeev authored
      Part of #6773
      
      NO_CHANGELOG=It will be added in another commit.
      
      @TarantoolBot document
      Title: New MsgPack extension type MP_INTERVAL
      
      MsgPack extension type MP_INTERVAL was created to serialize values of
      struct interval.
      
      Description of serialized value:
      
      ------------------------------------------------------------
      |MP_EXT|Size of packed interval|MP_INTERVAL|packed interval|
      ------------------------------------------------------------
      
      Packed interval - packed number of non-zero fields and packed non-null
      fields. Each packed field consists of two packed integer values - the
      field ID and its value. The number of non-null fields can be zero, in
      which case `packed interval` will be just packed integer `0`.
      
      List of IDs:
      0 - `year`
      1 - `month`
      2 - `week`
      3 - `day`
      4 - `hour`
      5 - `minute`
      6 - `second`
      7 - `nanosecond`
      8 - `adjust`
      
      Example of packed interval value:
      
      |C7|09|06|03|00|01|01|CCC8|03|D0B3|
      
      Where
      C7 - MP_EXT;
      09 - size of packed interval value, 9 bytes;
      06 - MP_INTERVAL;
      03 - number of non-zero fields;
      00 - ID of field `year`;
      01 - packed value `1`;
      01 - ID of field `month`;
      CCC8 - packed value `200`;
      03 - ID of field `day`;
      D0B3 - packed value `-77`.
      
      From there we can see, that the packed interval is actually
      `1 year, 200 months and -77 days`.
      ed861743
    • Mergen Imeev's avatar
      lua: introduce totable() method for INTERVAL · ec911fce
      Mergen Imeev authored
      This method is necessary to compare two INTERVAL values in Luatest.
      
      Part of #6773
      
      NO_DOC=Internal function.
      NO_CHANGELOG=INTERVAL has already been introduced earlier.
      ec911fce
    • Vladimir Davydov's avatar
      space_def: use xmalloc · 750f9f61
      Vladimir Davydov authored
      Half of allocation functions in space_def_dup and space_def_new already
      use xmalloc. Use xmalloc for the rest of allocations.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      750f9f61
    • Vladimir Davydov's avatar
      field_def: add field_def_array_dup · 342fe058
      Vladimir Davydov authored
      To implement space upgrade, we need a function that duplicates field_def
      array, because we will store a new space format in space options.
      
      To factor out field_def_array_dup from space_def_dup, let's allocate it
      with a separate malloc rather than after space_def struct.
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      342fe058
    • Vladimir Davydov's avatar
      grp_alloc: add helpers to work with null-terminated strings · d82d1083
      Vladimir Davydov authored
      NO_DOC=internal
      NO_CHANGELOG=internal
      d82d1083
    • Vladimir Davydov's avatar
      field_def: rename space_format_decode to field_def_array_decode · ad97637e
      Vladimir Davydov authored
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      ad97637e
    • Vladimir Davydov's avatar
      Drop space_opts_destroy · e0a30a64
      Vladimir Davydov authored
      The function isn't used since commit 3dcc3fd6 ("Move
      default_value_expr from field_def to tuple_field").
      
      NO_DOC=refactoring
      NO_TEST=refactoring
      NO_CHANGELOG=refactoring
      e0a30a64
    • Timur Safin's avatar
      test: workaround for flaky 301-basic.t · b2b77dbb
      Timur Safin authored
      We have found that 301-basic.t from Lua harness tests became
      flaky on macosx runners.
      Please see https://github.com/tarantool/tarantool/issues/7058
      
       for
      more details. Use simple workaround `collectgarbage('collect')`
      for temporary workaround of this problem.
      
      Co-authored-by: default avatarIgor Munkin <imun@tarantool.org>
      
      NO_DOC=infra
      NO_CHANGELOG=infra
      NO_TEST=infra
      b2b77dbb
    • Timur Safin's avatar
      datetime: interval changelog and doc request · 08d6f7fa
      Timur Safin authored
      Part of #6923
      
      NO_TEST=docrequest
      
      @TarantoolBot document
      Title: Document interval arithmetic
      
      Interval arithmetic
      -------------------
      
      If we need to shift date values by the given period of time we call either
      modifier methods (i.e. `:add` or `:sub`) or apply interval arithmetic
      using overloaded `+` (__add) or `-` (__sub) methods.
      
      `:add`/`:sub` modify current object, but `__add`/`__sub` create copy of
      object for operation result.
      
      In the interval operation we sequentially calculate each interval
      subcomponents
      heading from largest (year) to smallest (nanosecond):
      * `year` - year(s);
      * `month` - month(s);
      * `week` - week(s);
      * `day` - day(s);
      * `hour` - hour(s);
      * `min` - minute(s);
      * `sec` - second(s);
      * `nsec` - nanosecond(s).
      
      If results of operation exceed allowed range for any of component then
      exception to be raised.
      
      Modifier methods `:add`/`:sub` return `self` object thus it''s possible to
      chain their calls together, i.e.
      
      ```
      -- add 9000 years, 82 months, 5 weeks, 201 days, 183 hours, 292 minutes
      -- and 191.001239234 seconds to given date
      dt:add{
      	year  = 9000,
      	month = 82,
      	week  = 5,
      	day   = 201,
      
      	sec   = 191,
      	min   = 292,
      	hour  = 183,
      
      	nsec  = 1239234,
      }
      
      dt:sub{
      	year  = 9000,
      	month = 82,
      	week  = 5,
      	day   = 201,
      
      	sec   = 191,
      	min   = 292,
      	hour  = 183,
      
      	nsec  = 1239234,
      }
      
      -- chaining
      dt:add{year = 2}:add{month = 2}:sub{day = 2}
      
      -- create a copy of current date, while moving it to the next day
      local dt2 = datetime.new(dt:totable()):add{day = 1}
      ```
      
      Arithmetic operations
      =====================
      
      Date and interval objects may participate in arithmetic operations:
      * Sum of 2 intervals is interval object, which fields would be sum of each
        particular components of operands;
      * Subtraction is similar to summation above - result is interval object
        where each subcomponent would be result of subtraction of particular
        fields in original operands;
      * If you add date and interval then result is a date. Addition to be
        performed in a determined order from largest component (i.e. year) to
        smallest (nanosecond);
      * Subtraction of dates to produce an interval object. Difference of 2 time
        moments is performed not as difference of epoch seconds, but as
        difference of all subcomponents (i.e. years, months, days, hours,
        minutes and seconds);
      * Untyped table object may be used in each context where typed date or
        interval object used if left operand is typed object with overloaded
        operation of '+' or '-'.
      
      Matrix of *addition* operands eligibility and their result type:
      
      |                     | datetime | interval | table    |
      |---------------------|----------|----------|----------|
      | **datetime**        |          | datetime | datetime |
      | **interval**        | datetime | interval | interval |
      | **table**           |          |          |          |
      
      Matrix of *subtraction* operands eligibility and their result type:
      
      |                     | datetime | interval | table    |
      |---------------------|----------|----------|----------|
      | **datetime**        | interval | datetime | datetime |
      | **interval**        |          | interval | interval |
      | **table**           |          |          |          |
      
      Date adjustions and leap years
      ==============================
      
      It''s always tricky to operate with days if we move date between months of
      different lengths. The default mode is - the day number in month of
      original date should become the same day but not exceed the length of
      resultant month.
      
      * 28 February of non-leap year if added +1 year should become 28 february
        of a leap year;
      * Addition of year to the 29 February of a leap year would result with 28
        February of a non-leap year;
      
      The general rule is as following: addition of months to the date should
      produce (if possible) the same day number, if this number is not exceeding
      number of days in resultant month. Otherwise it produces the last day in
      month.
      
      * 31 january + 1 month = 28 or 29 february;
      * 30 january + 1 month = 28 or 29 february;
      * 29 february + 1 month = 29 march;
      * 31 march + 1 month = 30 april.
      
      Optionally you may prefer to use "accountant" logics of operations when
      month operations (addition or subtraction) should leave last day in month
      as last day, i.e.:
      
      * 31 january + 1 month = 28 or 29 february (depending on leap year);
      * 29 february + 1 month = 31 march;
      * 31 march + 1 month = 30 april;
      * 30 april + 1 month = 31 may.
      
      - 28 february 2001 + 1 month = 28 march 2001;
      - february 2004 + 1 month = 28 марта 2004;
      
      This adjustment option named `adjust` may be passed as additional argument
      to `:add`/`:sub` methods or additional argument to interval constructor
      call.
      
      There are 3 supported `adjust` values:
      
      * `none` last day mode is not maintained, if there is overflow then value
        is truncated to the maximum possible value in month (corresponding to
        the c-dt `DT_LIMIT`). This is default mode if not provided;
      
      ```
          -- 29.02.* -> 29.03.*
          dt:add( {month = 1, adjust = "none" } )
      ```
      
      * `last` last day mode ("accountant" mode) maintained. i.e. last day in
        original month should stay last day in resultant month. This corresponds
         to `c-dt` `DT_SNAP` mode;
      
      ```
      -- 28.02.2001 -> 31.03.2001 -- last day in Feb 2001
      -- 28.02.2004 -> 28.03.2004 -- not a last day in Feb => no abjustments
      -- 29.02.2004 -> 31.03.2001 -- last day in Feb 2004
      dt:add( {month = 1,  adjust = "last" } )
      ```
      
      * `excess` overflow to the next month performed. According to `c-dt`
        `DT_EXCESS` rules.
      
      Stringization
      =============
      
      Intervals show each their subcomponents as negative or positive value.
      Seconds and nanoseconds normalized to the human-interpretable form.
      
      * '+1 seconds';
      * '+12 hours, 10 minutes, 30 seconds';
      * '-20 months, -10 weeks, -8 hours, -10 minutes, -30 seconds';
      * '-5000000 years, -20 months, -10 weeks, -10 minutes, -30 seconds';
      08d6f7fa
    • Timur Safin's avatar
      datetime: check overflow after interval operations · b6f855e2
      Timur Safin authored
      We may overflow supported years range if we proceed
      (multiple times) interval addition or subtraction operations.
      At the end of add/sub we check all interval subcomponents
      on validity of their value. Result of this check is returned
      in the encoded value to the Lua side, where less than 0 means
      underflow in some component, and greater than 0 overflow in
      subcomponent.
      
      Part of #6923
      
      NO_DOC=fix
      NO_CHANGELOG=fix
      b6f855e2
    • Timur Safin's avatar
      datetime: Mons intervals · d69b3229
      Timur Safin authored
      Split single sec field in the datetime_interval into multiple subfields,
      i.e. week, day, hour, min, sec. Make sure that whatever we enter in the
      constructor we get similarly visualized as textual representation
      
      Lua implementation of datetime_totable has been converted to C, which
      allows to use it for datetime values decomposing in datetimes
      subtraction. It uses per component subtraction for calculating of a
      resultant interval.
      
      Part of #6923
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      d69b3229
    • Timur Safin's avatar
      datetime: convert all sub/add datetime ops to C · 29f57b05
      Timur Safin authored
      Created C implementation for the rest of datetime/interval operations
      and call them via FFI. We need it for later Olson integration.
      
      Part of #6923
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring, tests left intact
      29f57b05
    • Timur Safin's avatar
      datetime: datetime_increment_by in C from Lua · 5f3ea3b4
      Timur Safin authored
      Convert Lua implementation of datetime_increment_by to C from their
      Lua variant.
      
      Part of #6923
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring, tests left intact
      5f3ea3b4
    • Timur Safin's avatar
      datetime: convert interval_to_string to C from Lua · 84b91f31
      Timur Safin authored
      Part of #6923
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring, tests left intact
      84b91f31
    • Timur Safin's avatar
      datetime: define interval types in the luaL_cdef · 44ebf1cc
      Timur Safin authored
      Part of #6923
      
      NO_DOC=internal
      NO_CHANGELOG=internal
      NO_TEST=refactoring, tests left intact
      44ebf1cc
    • Timur Safin's avatar
      datetime: refactor lua interval data structures · e9be8681
      Timur Safin authored
      Simplify datetime interval data handling in Lua module:
      * everytime we used to pass year, month, day, hour and second we simply
        pass aggregate containing those values.
      
      Part of #6923
      
      NO_DOC=refactoring
      NO_CHANGELOG=refactoring
      NO_TEST=refactoring, tests left intact
      e9be8681
    • Mergen Imeev's avatar
      box: fix problem with SCALAR and UUID · 589bd6b6
      Mergen Imeev authored
      This patch fixes an error that occurred when a UUID index was created
      on a SCALAR field or a SCALAR index was created on a UUID field.
      
      Follow-up #6042
      
      NO_DOC=Follow up.
      NO_CHANGELOG=Follow up.
      589bd6b6
  3. Apr 21, 2022
Loading