Skip to content
Snippets Groups Projects
  1. Jan 30, 2018
    • IlyaMarkovMipt's avatar
      security: Change checks on usage access · 9e30f895
      IlyaMarkovMipt authored
      * Add following behavior:
      Owner of object can't utilize her own objects if she has not usage
      access.
      * Change access checks of space, sequence, function objects
      Similar checks of other objects are performed in alter.cc.
      
      Closes gh-3089
      9e30f895
    • IlyaMarkovMipt's avatar
      net.box: Fix typo · a0681659
      IlyaMarkovMipt authored
      * Fix typo in net_box.lua in rare error case
      a0681659
    • imarkov's avatar
      fix: Broken compilation with gcc 4.6 · 459d63fc
      imarkov authored
      * Delete contructor delegation in ClientError
      * Move code body from one contructor to another
      459d63fc
    • Vladimir Davydov's avatar
      relay: send heartbeat on subscribe if replica is uptodate · f0892e5e
      Vladimir Davydov authored
      Currently, a realy sends a heartbeat message to the replica only if
      there was no WAL events for 'replication_timeout' seconds. As a result,
      a replica that happens to be uptodate on subscribe will not update the
      lag until the timeout passes, which may delay configuration. Let's make
      relay send a heartbeat message right after subscribe in case the replica
      is uptodate.
      f0892e5e
    • Vladimir Davydov's avatar
      replication: add helpers to set and clear replica applier · 85310417
      Vladimir Davydov authored
      These operations are going to become more complicated than just setting
      a pointer so let's introduce helpers for them.
      85310417
    • Vladimir Davydov's avatar
      replication: gather all replicaset variables in struct · 580f1505
      Vladimir Davydov authored
      There is already a handful of global variables describing the replica
      set state and there is going to be more so let's consolidate them in
      a singleton struct:
      
        replicaset => replicaset.hash
        replica_pool => replicaset.pool
        anon_replicas => replicaset.anon
        replicaset_vclock => replicaset.vclock
      
      While we are at it, let's also move INSTANCE_UUID definition from
      xrow.c to replication.cc, where it truly belongs. The only reason
      I see for it to be defined in xrow.c is to compile vinyl unit tests
      without linking replication.o, but we can easily circumvent this by
      defining INSTANCE_UUID in vy_iterators_helpers.c.
      
      Suggested by @kostja
      580f1505
    • Vladimir Davydov's avatar
      replication: get rid of replica->pause_on_connect flag · 3adb9789
      Vladimir Davydov authored
      replicaset_connect() leaves appliers that failed to connect within the
      specified time period running. To prevent them from entering 'subscribe'
      stage prematurely (i.e. before replicaset_follow() is called), we set
      replica->pause_on_connect flag, which will force them to freeze upon
      successful connection. We clear this flag in replicaset_follow(). This
      juggling with flags looks ugly. Instead, let's stop failed appliers in
      replicaset_connect() and restart them in replicaset_follow().
      
      Follow-up #2958
      3adb9789
    • IlyaMarkovMipt's avatar
      fio: Add new methods · 70baba01
      IlyaMarkovMipt authored
      Introduce in fio new methods taken from Python os.path:
      
      * fio.path.exists()
      * fio.path.lexists()
      * fio.path.is_file()
      * fio.path.is_dir()
      * fio.path.is_mount()
      70baba01
    • Konstantin Osipov's avatar
      4cdafae2
    • Ilya Konyukhov's avatar
      Setup apkbuild spec file · d1a6e58d
      Ilya Konyukhov authored
      It builds the last stable tarantool release at the moment (1.7.6).
      It clones tarantool from github repo, then updates submodules, then
      compiles tarantool, small and msgpuck.
      
      Then it packs everything into a package and cleans everything after
      itself
      d1a6e58d
    • Vladimir Davydov's avatar
      vinyl: ignore quota timeout on replication · 4d648f3b
      Vladimir Davydov authored
      If vinyl fails to do memory dumps in time on a replica (e.g. it ran
      out of disk space), replication will stop forever with an error, and
      the admin will have to call box.cfg() to restart replication. Since
      replication is asynchronous anyway, we shouldn't stop it on vinyl
      timeout - it isn't critical as the replica will recover as soon as
      the admin fixes the problem (e.g. frees up some disk space). Let's
      ignore vinyl timeout altogether for applier fibers (currently, we
      ignore it only on join) - the admin can monitor how badly a replica
      lags behind the master via box.info.replication lag/idle.
      
      Closes #3087
      4d648f3b
    • Vladimir Davydov's avatar
      Introduce BEFORE trigger · 5a27b737
      Vladimir Davydov authored
      To register a BEFORE trigger for a space, call space:before_replace()
      function. Similarly to space:on_replace(), this function takes a new
      trigger callback as the first argument and a function to remove from
      the registered trigger list as the second optional argument.
      
      Trigger callbacks are executed from space_execute_dml(), right before
      passing down a request to the engine implementation, but after resolving
      the space sequence. Just like on_replace, a before_replace callback is
      passed old and new tuples, but it can also return a tuple or nil, which
      will affect the current statement as follows:
      
       - If a callback function returns the old tuple, the statement is
         ignored and IPROTO_NOP is written to xlog to bump LSN.
      
       - If a callback function returns the new tuple or doesn't return
         anything, the statement is executed as is.
      
       - If a callback function returns nil, the statement is turned into
         DELETE.
      
       - If a callback function returns a tuple, the statement is turned
         into REPLACE for this tuple.
      
      Other return values result in ER_BEFORE_REPLACE_RET error.
      
      Note, the trigger must not change the primary key of the old tuple,
      because that would require splitting the resulting statement into two -
      DELETE and REPLACE.
      
      The new trigger can be used to resolve asynchronous replication
      conflicts as illustrated by replication/before_replace test.
      
      Closes #2993
      5a27b737
  2. Jan 29, 2018
    • Vladimir Davydov's avatar
      iproto: add IPROTO_NOP request type · b73030f2
      Vladimir Davydov authored
      To implement space:before_replace trigger, we need to introduce a new
      request type for bumping LSN, because the new trigger may turn any DML
      operation into a no-op. Let's call it IPROTO_NOP. It is treated as DML
      (passed to apply_row, etc), but it is ignored by space_execute_dml() and
      so doesn't actually modify anyting, only bumps LSN on the server. The
      new request type has name "NOP" (for xlog reader), however it isn't
      reported via box.stat().
      
      Needed for #2993
      b73030f2
    • Vladimir Davydov's avatar
      Move helpers for updating request from space.c to request.c · 05f9d568
      Vladimir Davydov authored
      This patch moves helpers used to fix requests after certain DML
      operations to a separate source file. Currently, there are only
      two of them, but there are going to be more so it seems to be a
      good idea to isolate them. No functional changes.
      
      Suggested by @kostja
      05f9d568
    • Vladimir Davydov's avatar
      space: introduce space_execute_dml helper · 9a85ae76
      Vladimir Davydov authored
      I'm planning to call BEFORE triggers in space.c. Since a BEFORE trigger
      can change the request type, we can't call it from functions handling
      particular kinds of requests (space_execute_replace() and others).
      So let's move the switch-case statement that executes different space
      callbacks depending on the request type from process_rw() to a new
      function, space_execute_dml(), defined in space.c. We will execute
      BEFORE triggers from this new function, right before dispatching the
      request by its type.
      
      Needed for #2993
      9a85ae76
  3. Jan 27, 2018
  4. Jan 25, 2018
    • IlyaMarkovMipt's avatar
      say: Fix multiple loggers patch · 68fdf19f
      IlyaMarkovMipt authored
      * Fix changing the behavior of the syslog write:
      It used to log to stderr only if level was S_FATAL and file
      * descriptor is not stderr. After multi loggers patch it logs to
      stderr on every logger call.
      
      Relates #3066
      68fdf19f
  5. Jan 24, 2018
  6. Jan 23, 2018
    • Vladislav Shpilevoy's avatar
      alter: introduce ModifySpaceFormat alter operation · 9f6113ab
      Vladislav Shpilevoy authored
      ModifySpaceFormat operation sets new names into old tuple formats.
      
      Closes #3011
      9f6113ab
    • Vladislav Shpilevoy's avatar
      tuple: move tuple field names hash to a separate shared struct · c94034df
      Vladislav Shpilevoy authored
      Struct tuple_dictionary stores hash of field names, defined in
      a space format. The structure is refable and are going to be used
      to share new field names with old space formats.
      
      Part of #3011
      c94034df
    • Vladimir Davydov's avatar
      txn: fix on_replace trigger chaining · 653cbcec
      Vladimir Davydov authored
      If there are multiple on_replace triggers installed for the same space
      and one of them creates a new statement (by issuing a DML request), then
      the trigger that is called next will get the statement created by the
      previous trigger instead of the original statement. To fix that, let's
      patch txn_current_stmt() so as to return the first statement at the
      current transaction level instead of the last statement and use it in
      txn_commit_stmt(). This will also allow us to issue DML requests from a
      before_replace trigger without disrupting the current statement.
      
      Needed for #2993
      Follow-up #3020
      653cbcec
    • Vladimir Davydov's avatar
      txn: fix rollback of sub-statements · 5ea035ba
      Vladimir Davydov authored
      If space.on_replace callback fails (throws), we must rollback all
      statements inserted by the callback before failing and the statement
      that triggered the callback while currently we only rollback the last
      statement on txn->stmts list. To fix this, let's remember the position
      to rollback to in case of failure for each sub statement, similarly to
      how we do with savepoints.
      
      Note, there's a comment to txn_rollback_stmt() that says that it doesn't
      remove the last statement from the txn->stmts list, just clears it:
      
              /**
               * Void all effects of the statement, but
               * keep it in the list - to maintain
               * limit on the number of statements in a
               * transaction.
               */
              void
              txn_rollback_stmt()
      
      It isn't going to hold after this patch, because this patch reuses the
      savepoint code to rollback statements on failure. Anyway, I haven't
      managed to figure out why we would ever need to keep statements on the
      list after rollback. The comment is clearly misleading as we don't have
      any limits on the number of statements, and even if we had, a statement
      counter would suffice. I guess the real reason why we don't delete
      statements from the list is that it is simply impossible to do in case
      of a singly linked list like stailq, but now it isn't going to be a
      problem. So remove the comment and the test case of engine/savepoint
      which relies on it.
      
      Needed for #2993
      Closes #3020
      5ea035ba
    • Vladimir Davydov's avatar
      Replace stailq_splice with stailq_cut_tail · 86c19c32
      Vladimir Davydov authored
      stailq_splice(head1, item, head2) moves elements from list 'head1'
      starting from 'item' to list 'head2'. To follow the protocol, it needs
      to know the element previous to 'item' in 'head1' so as to make it the
      new last element of 'head1'. To achieve that, it has to loop over
      'head1', which is inefficient. Actually, wherever we use this function,
      we know in advance the element stailq_splice() has to look up, but we
      still pass the next one, making its life difficult and obscuring the
      code at the caller's side.
      
      For example, look at how stailq_splice() is used in txn.c:
      
              if (stmt == NULL) {
                      rollback_stmts = txn->stmts;
                      stailq_create(&txn->stmts);
              } else {
                      stailq_create(&rollback_stmts);
                      stmt = stailq_next_entry(stmt, next);
                      stailq_splice(&txn->stmts, &stmt->next, &rollback_stmts);
              }
      
      while under the hood stailq_splice() has the loop to find 'stmt':
      
              stailq_splice(struct stailq *head1, struct stailq_entry *elem,
                            struct stailq *head2)
              {
                      if (elem) {
                              *head2->last = elem;
                              head2->last = head1->last;
                              head1->last = &head1->first;
                              while (*head1->last != elem)
                                      head1->last = &(*head1->last)->next;
                              *head1->last = NULL;
                      }
              }
      
      This is utterly preposterous.
      
      Let's replace stailq_splice() with a new method with the same signature,
      but with a slightly different semantics: move all elements from list
      'head1' starting from the element *following* 'item' to list 'head2'; if
      'item' is NULL, move all elements from 'head1' to 'head2'. This greatly
      simplifies the code for both parties, as the callee doesn't have to loop
      any more while the caller doesn't have to handle the case when 'item' is
      NULL.
      
      Also, let's change the name of this function, because stailq_splice()
      sounds kinda confusing: after all, this function tears a list in two
      first and only then splices the tail with another list. Let's remove the
      'splice' part altogether (anyway, there's another function for splicing
      lists - stailq_concat()) and call it stailq_cut_tail().
      86c19c32
    • Vladimir Davydov's avatar
      txn: zap txn_savepoint->is_first flag · 2634306f
      Vladimir Davydov authored
      This flag isn't necessary as we can set txn_savepoint->stmt to NULL when
      a savepoint is created inside an empty transaction. Using a separate
      flag for this purpose obscures the code flow and complicates further
      progress so let's remove it.
      2634306f
    • IlyaMarkovMipt's avatar
      box: make box.NULL available before box.cfg · 7bd84cc5
      IlyaMarkovMipt authored
      * Add NULL key word to whitelist of keywords available before box.cfg.
      
      Closes #3032
      7bd84cc5
    • Georgy Kirichenko's avatar
      Cache proc names for backtrace · 0280771a
      Georgy Kirichenko authored
      libunwind get_proc_name consumes too much time, so use a hash for
      to cache mapping of ip (instruction pointer) to proc_name+offset.
      
      Fixes #2877
      0280771a
    • AKhatskevich's avatar
      box: allow printable and only printable characters in identifiers · 4789babc
      AKhatskevich authored
      Before this patch, we would only allow alphabetical characters
      plus underscore in identifier names. And we did not treat
      all identifiers the same way: column names were not checked
      at all.
      
      SQL ANSI ISO allow delimited identifiers cantain any character
      from source language character set.
      
      After this patch, checks for allowed characters in identifier
      names follow the same ruls for all identifiers: column names,
      function names, user names, space names, index names.
      
      In other words, this patch makes tarantool itentifier rules closer
      to ANSI ones.
      
      Closes #2914
      4789babc
    • Konstantin Osipov's avatar
      lua: fix a typoe in error message · 07653cf8
      Konstantin Osipov authored
      07653cf8
  7. Jan 22, 2018
    • Konstantin Belyavskiy's avatar
      Fix compilation warning in cmp_def.h · f09860de
      Konstantin Belyavskiy authored
      According to offsetof(3) type of its return value is size_t.
      Assigning result of offsetof() to a variable of type ptrdiff_t
      may result in compilation warnings on certain platforms:
      
        src/box/key_def.cc:57:2: error: non-constant-expression cannot be narrowed from type 'size_t' (aka 'unsigned long') to 'ptrdiff_t' (aka 'long') in initializer list [-Wc++11-narrowing]
                OPT_DEF_ENUM(PART_OPT_TYPE, field_type, struct key_part_def, type,
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        src/box/opt_def.h:76:19: note: expanded from macro 'OPT_DEF_ENUM'
                { key, OPT_ENUM, offsetof(opts, field), sizeof(int), #enum_name, \
                                 ^~~~~~~~~~~~~~~~~~~~~
        src/trivia/util.h:193:32: note: expanded from macro 'offsetof'
        #define offsetof(type, member) ((size_t) &((type *)0)->member)
                                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      
      Fix this by changing the type of opt_def::offset from ptrdiff_t to
      size_t.
      
      Closes #2930
      f09860de
    • Konstantin Osipov's avatar
  8. Jan 19, 2018
  9. Jan 18, 2018
    • Konstantin Osipov's avatar
      iproto: make compile under clang 3.8 · 359d3229
      Konstantin Osipov authored
      iproto_msg used to be a non-POD struct because of : public cmsg, and
      offsetof(), introduced in the fix for gh-946, couldn't be legally used
      with it.
      
      Make it a POD struct, in preparation for moving iproto to plain C.
      359d3229
    • Vladimir Davydov's avatar
      call: fail requests that do not close transaction · f003b489
      Vladimir Davydov authored
      Currently, if a CALL/EVAL request leaves an open transaction at return,
      we silently rollback it and print a warning to the log mentioning the
      function name or eval expression to facilitate further debugging. After
      issue #946 was fixed, we can't do that anymore, because request input,
      which stores CALL/EVAL parameters, may be discarded before request
      completion and hence be unavailable for logging. Without additional
      information pointing at the culprit, the log message is pointless (see
      issue #1100). We could copy the arguments, but that would slow down CALL
      execution, which can't be justified solely by the need of verbose
      logging. So let's stop being lenient and fail requests that do not close
      transaction at return. This should encourage negligent users to finally
      fix their code.
      
      Follow-up #946
      f003b489
    • Vladimir Davydov's avatar
      iproto: discard CALL/EVAL input on yield · 3bc7f4e6
      Vladimir Davydov authored
      Currently, a long polling request can stall all other connections,
      because the input buffer can't be reclaimed until it is completed.
      Funny thing is CALL/EVAL only needs input for a short time, to decode
      arguments from msgpack and push them to stack, after that point the
      input can be safely discarded.
      
      So this patch makes tx_process_call() setup a trigger before executing
      the CALL/EVAL request. The trigger is invoked on fiber yield. The
      trigger's callback sends a message back to the iproto thread notifying
      it that tx has processed the request input. Upon receiving such a
      message, iproto discards the request input and resumes suspended
      connections, if any.
      
      Closes #946
      3bc7f4e6
    • Vladimir Davydov's avatar
      call: separate function invocation from result encoding · d461caf3
      Vladimir Davydov authored
      The iproto subsystem switches between two output buffers once in a while
      in order to reclaim memory so passing a pointer to the output buffer
      directly to box_process_call() or box_process_eval() is incorrect in
      case the called function yields. To fix that, let's make these functions
      return the CALL/EVAL result in a port object, which then can then be
      encoded in msgpack with port_dump().
      
      Needed for #946
      d461caf3
    • Vladimir Davydov's avatar
      Make struct port abstract · d5e479ee
      Vladimir Davydov authored
      So that it can be used not only for serializing a list of tuples, but
      also for serializing a Lua stack that stores output of CALL/EVAL.
      
      Needed for #946
      d5e479ee
    • Roman Tsisyk's avatar
      Travis CI: remove Ubuntu Zesty · 176b5d07
      Roman Tsisyk authored
      Remove all non-LTS versions of Ubuntu.
      176b5d07
  10. Jan 17, 2018
    • Vladimir Davydov's avatar
      call: refactor lua CALL/EVAL handling · cef230bd
      Vladimir Davydov authored
      Currently, we execute a Lua function/expression and dump the result to
      an output buffer in the same function invoked under lua_cpcall().
      Although this allows us to use only one pcall to handle a call request,
      this also makes box_lua_call() and box_lua_eval() dependent on the
      iproto format (they have to use iproto_reply_select()), which is ugly.
      What is worse, the caller has to pass the output buffer right away while
      in case of iproto it can change if the invoked Lua function yields
      (iproto switches buffers once in a while to reclaim memory).
      
      That being said, we'd better decouple the call itself from the result
      dump. Let's start from using two pcalls - one for executing the Lua
      expression and another for dumping arguments - and moving iproto
      dependent code out of Lua callbacks.
      
      Needed for #946
      cef230bd
    • Konstantin Osipov's avatar
      033160ef
Loading