Skip to content
Snippets Groups Projects
  1. Jan 18, 2018
  2. Jan 17, 2018
    • Vladimir Davydov's avatar
      call: refactor lua CALL/EVAL handling · cef230bd
      Vladimir Davydov authored
      Currently, we execute a Lua function/expression and dump the result to
      an output buffer in the same function invoked under lua_cpcall().
      Although this allows us to use only one pcall to handle a call request,
      this also makes box_lua_call() and box_lua_eval() dependent on the
      iproto format (they have to use iproto_reply_select()), which is ugly.
      What is worse, the caller has to pass the output buffer right away while
      in case of iproto it can change if the invoked Lua function yields
      (iproto switches buffers once in a while to reclaim memory).
      
      That being said, we'd better decouple the call itself from the result
      dump. Let's start from using two pcalls - one for executing the Lua
      expression and another for dumping arguments - and moving iproto
      dependent code out of Lua callbacks.
      
      Needed for #946
      cef230bd
    • Vladimir Davydov's avatar
      iproto: separate call route from misc · 13d50af5
      Vladimir Davydov authored
      Currently, they look pretty much the same, but this is going to change
      soon so let's separate them.
      13d50af5
    • Vladimir Davydov's avatar
      Convert box/call.cc to C · 98189a54
      Vladimir Davydov authored
      We can do it for free now as all functions used by call.cc have already
      been converted to C and there's nothing in call.cc that really needs any
      C++ features.
      98189a54
    • Vladimir Davydov's avatar
      session: make access_check_universe usable from C code · 81ae6f9b
      Vladimir Davydov authored
      Replace tnt_raise() with diag_set() and add a wrapper that raises
      exception in case of error to be used in C++ code.
      
      While we are at it, let's also move access_check_session_xc() to
      the header file, because it's a trivial wrapper.
      81ae6f9b
    • Vladimir Davydov's avatar
      func: don't raise exception from func_load · 9d1a6159
      Vladimir Davydov authored
      It is a C function so it is supposed to return -1 on error, not throw
      an exception. This misbehavior doesn't result in any runtime errors,
      because its only user box_process_call() is called from a try-catch
      block, but this is going to change soon.
      9d1a6159
    • Vladimir Davydov's avatar
      Fix app-tap/logger_pipe test · b745fad9
      Vladimir Davydov authored
      'echo $TEST_VAR' exits almost instantly, which may be detected by
      log_pipe_init(), resulting in the test failure:
      
          app-tap/logger_pipe.test.lua                        [ fail ]
      
          Test failed! Result content mismatch:
          --- app-tap/logger_pipe.result  Wed Jan 17 14:15:24 2018
          +++ app-tap/logger_pipe.reject  Wed Jan 17 14:47:35 2018
          @@ -1 +1,3 @@
           48
           +IllegalParams: logger process died
           +failed to initialize logging subsystem
      
      Fix this by appending 'cat > /dev/null' to the pipe logger command.
      
      Fixes 0ab233cd Don't discard environment variables in pipe logger
      
      Closes #3048
      b745fad9
    • Vladimir Davydov's avatar
      say: fix crash say_logger_init on invalid configuration · 4cef3221
      Vladimir Davydov authored
      say_logger_init() zeroes the default logger object (log_default) before
      proceeding to logging subsystem configuration. If configuration fails
      for some reason (e.g. error opening the log file), the default logger
      will be left uninitialized, and we will crash trying to print the error
      to the console:
      
        #0  0x564065001af5 in print_backtrace+9
        #1  0x564064f0b17f in _ZL12sig_fatal_cbi+e2
        #2  0x7ff94519f0c0 in __restore_rt+0
        #3  (nil) in +0
        #4  0x564064ffc399 in say_default+d2
        #5  0x564065011c37 in _ZNK11SystemError3logEv+6d
        #6  0x5640650117be in exception_log+3d
        #7  0x564064ff9750 in error_log+1d
        #8  0x564064ff9847 in diag_log+50
        #9  0x564064ffab9b in say_logger_init+22a
        #10 0x564064f0bffb in load_cfg+69a
        #11 0x564064fd2f49 in _ZL13lbox_cfg_loadP9lua_State+12
        #12 0x56406502258b in lj_BC_FUNCC+34
        #13 0x564065045103 in lua_pcall+18e
        #14 0x564064fed733 in luaT_call+29
        #15 0x564064fe5536 in lua_main+b9
        #16 0x564064fe5d74 in run_script_f+7b5
        #17 0x564064f0aef4 in _ZL16fiber_cxx_invokePFiP13__va_list_tagES0_+1e
        #18 0x564064fff4e5 in fiber_loop+82
        #19 0x5640651a123b in coro_init+4c
        #20 (nil) in +4c
      
      Fix this by making say_logger_init() initialize the default logger
      object first and only assign it to log_default on success.
      
      See #3048
      4cef3221
  3. Jan 16, 2018
    • IlyaMarkovMipt's avatar
      Add on_access_denied trigger · cc3a18b7
      IlyaMarkovMipt authored
      * Add box_on_access_denied API method
      * Modify access error handlers in order to call the mentioned trigger
      * Add new type of error - AccessDeniedError
      
      Related #2911 "add audit log triggers"
      cc3a18b7
  4. Jan 13, 2018
    • Roman Tsisyk's avatar
      Don't discard environment variables in pipe logger · 0ab233cd
      Roman Tsisyk authored
      Before this patch `box.cfg { log = "|wrapper" }` started `wrapper` by
      invoking /bin/sh -c 'wrapper' with empty environment. This patch makes
      /bin/sh -c 'wrapper' invocation to inherit parent's environment.
      
      Closes #3041
      0ab233cd
  5. Jan 12, 2018
    • Ilya's avatar
      security: extend on_auth trigger · dfce09d2
      Ilya authored
      * Add calls to on_auth trigger in case of failed authentication.
      * Add bool parameter to the mentioned trigger.
      
      Closes #3039
      
      With review remarks by @kostja
      dfce09d2
    • Vladimir Davydov's avatar
      replication: cleanup timeouts · f45ae852
      Vladimir Davydov authored
      Currently, we have three variables related to replication timeouts:
      applier_timeout, relay_timeout, and replication_cfg_timeout. They are
      all set to the value of box.cfg.replication_timeout. We use these
      variables in four different cases:
      
       - Sending heartbeat messages periodically from master to slave and back
         (relay_timeout, applier_timeout).
      
       - Reconnecting applier after a disconnect (applier_timeout).
      
       - Disconnecting a replica if no hearbeat message has been receivied
         within the specified timeout (TIMEOUT_PERIODS * replication_timeout).
      
       - Waiting for box.cfg() to succeed (replication_connect_quorum_timeout).
      
      This is confusing. Let's keep just one variable, replication_timeout,
      that would determine the heartbeat interval and introduce the following
      helpers for the three other cases:
      
       - replication_reconnect_timeout()
       - replication_disconnect_timeout()
       - replication_connect_quroum_timeout()
      
      Also, let's make replication_connect_quorum_timeout() return 4 times the
      configured timeout in the scope of this patch, because, as pointed out by
      @kostja,
      
      > We need another replication_timeout variable, using the same variable
      > for everything doesn't work.  Please try setting a broken
      > box.cfg.replication second time, and you'll see that it doesn't try to
      > reconnect, because reconnect timeout = replication timeout. This is
      > broken, reconnect_timeout should be < replication_timeout, to allow for
      > at least a few reconnects.
      
      Suggested by @kostja
      
      Follow-up #2958
      f45ae852
    • Vladimir Davydov's avatar
      Fix replication freeze if slave bumps lsn while master is down · e755ad24
      Vladimir Davydov authored
      To avoid rescanning the last recovered xlog in case it has been properly
      finalized, recover_remaining_wals() skips xlogs whose signature is less
      than the signature of the current recovery position. This assumption is
      incorrect if this function is used for replication. For example consider
      the following scenario in case of master -> slave replication:
      
       1. Master temporarily shuts down.
       2. Slave bumps its LSN while master is down.
       3. Master is brought back online.
       4. Slave reconnects to master.
      
      In such a case the recovery vclock signature sent by slave on reconnect
      will be greater than the signature of the xlog file created after master
      restart, causing replication to silently freeze.
      
      Instead of comparing xlog signature to recovery position, we should
      compare it to the signature of the last scanned xlog. To do that, we
      need to remove TRASH() from xlog_cursor_close() so that xlog cursor
      meta isn't overwritten on close. To make sure nobody attempts to use a
      closed cursor, let's add corresponding assertions to each public xlog
      cursor function.
      
      Fixes b25c60f0 ("recovery: do not rescan last xlog")
      
      Closes #3038
      e755ad24
  6. Jan 11, 2018
    • Vladimir Davydov's avatar
      Rename box.cfg.replication_quorum to replication_connect_quorum · 7d843674
      Vladimir Davydov authored
      'replication_connect_quorum' seems to be a better name for this option
      is ignored once connections to remote masters have been established.
      
      Suggested by @kostja
      
      Follow-up #2958
      7d843674
    • Ilya's avatar
      security: add object name to "access denied" error messages · ad237aeb
      Ilya authored
      Delete specifc access denied error code (ER_FUNCTION_ACCESS_DENIED,
      ER_SPACE_ACCESS_DENIED, ER_FUNC_ACCESS_DENIED) and always
      ER_ACCESS_DENIED code, which now contains object name and type
      
      Pass operation type (create, drop, grant, revoke) to ER_ACCESS_DENIED.
      Add a helper function schema_find_name() to schema.[h,cc].
      
      In scope of gh-2911 "add triggers for audit log".
      
      Heavily edited by @kostja
      ad237aeb
  7. Jan 10, 2018
    • Ilya's avatar
      alter: Refactor access_check outside constructors · 5b32bb7f
      Ilya authored
      Move access_check_ddl outside of raw msgpack constructors
      of defs to make code more clear
      5b32bb7f
    • Ilya's avatar
      fio: improve pathjoin · 746ff9fc
      Ilya authored
      Add an ability to treat single '/' as no-op.
      Ignore duplicate '/' in pathjoin.
      
      Closes #2968
      746ff9fc
    • Vladimir Davydov's avatar
      replication: do not wait for all masters on recovery · f1167c42
      Vladimir Davydov authored
      If one cluster node is down permanently for some reason, no other node
      can restart - they will stall in box.cfg{} until all other nodes are up
      and running. This complicates a tarantool cluster deployment in real
      world scenarios.
      
      To address this issue, let's complete the configuration as soon as
      connections have been established connections to the number of hosts
      specified by the new configuration option, box.cfg.replication_quorum,
      assuming the rest will connect asynchronously. If the option is unset,
      it defaults to the number of entries in box.cfg.replication so this
      patch shouldn't affect the behavior of existing setups.
      
      Closes #2958
      f1167c42
    • Vladimir Davydov's avatar
      replication: minor refactoring · 19cb9b90
      Vladimir Davydov authored
       - Fold call to replicaset_update() in replicaset_connect().
         These two functions are always called one after another.
      
       - Factor out replicaset_follow() to resume all appliers
         registered with the replica set.
      
      Needed for #2958
      19cb9b90
    • Vladimir Davydov's avatar
      Rename applier_connect_all() to replicaset_connect() · c42a6411
      Vladimir Davydov authored
      The logic of applier_connect_all() does not really belong to applier.cc,
      because it operates on a set of appliers while applier.cc is all about
      an individual applier implementation. Let's move it to replication.cc
      and rename it to replicaset_connect().
      
      Needed for #2958
      c42a6411
    • Vladimir Davydov's avatar
      applier: refactor wait for replicaset configuration to complete · d166bb79
      Vladimir Davydov authored
      Do not use the generic applier_on_state infrastructure, which is used to
      advance a single applier to a particular state as it doesn't really fit
      the case when we need to wait for multiple appliers and it utterly fails
      when it comes to waiting for an arbitrary subset of all appliers.
      
      Needed for #2958
      d166bb79
    • Ilya's avatar
      msgpack: Fix segfault in ibuf_decode · d1c53754
      Ilya authored
      Fix segfault in case when ibuf.rpos is null
      Now error is raised in the case
      
      Closes #3005
      d1c53754
    • Vladimir Davydov's avatar
      vinyl: fix page_size and range_size defaults · f40ab7f1
      Vladimir Davydov authored
      If the value of range_size is absent in the _index system space, it will
      be initialized to 0 in struct index_def. This will lead to insane range
      splitting in vinyl and, as a result, file descriptor exhaustion. We ran
      into this problem after restoring memtx spaces as vinyl using tarantool
      dump utility (the latter simply replaces 'memtx' with 'vinyl' in the
      _space system space on restore). To avoid a debacle like this in future,
      let's use hardcoded defaults if vinyl options were omitted in the _index
      system space on insertion. The default values are the same we use for
      the corresponding box.cfg options. We already do it for run_size_ratio,
      run_count_per_level, and bloom_fpr so let's do it for range_size and
      page_size too.
      
      Closes #3019
      f40ab7f1
  8. Jan 09, 2018
  9. Dec 30, 2017
    • Konstantin Osipov's avatar
      security: add automatic upgrade provisioning a fix for gh-945 · 9a7c918f
      Konstantin Osipov authored
      Currently we requrie read and write on system spaces to be able to
      create objects, and only object definer can drop an object.
      
      Release 1.7.7 adds 'create' and 'drop' acls, which can
      be used to explicitly pass around create/drop privileges.
      
      Automatically grant 'create' privilege to all users created
      pre-1.7.7 who have global read and write privileges on universe
      during 1.7.7 automatic upgrade.
      9a7c918f
  10. Dec 29, 2017
    • Konstantin Osipov's avatar
      security: add 'super' role · d645f58d
      Konstantin Osipov authored
      Add a role which contains all ACLs.
      
      Fixes gh-3022.
      
      Useful for any quick start:
      
      box.schema.user.grant('guest', 'super')
      d645f58d
    • Konstantin Osipov's avatar
      test: update test-run · 05096303
      Konstantin Osipov authored
      * --gdbserver
      * --gdb was broken and is fixed (again)
      05096303
    • Konstantin Osipov's avatar
      security: implement box.session.effective_user() · 4403fe39
      Konstantin Osipov authored
      box.session:
      
      * change .user() to return the authenticated user
      * implement .effective.user()
      
      Extends gh-2994.
      4403fe39
    • Ilya's avatar
      box: introduce system privileges · 74ab44ae
      Ilya authored
      Add system privileges 'session' and 'usage'
      
      * 'session' privilege lets user connect to database server
      * 'usage' privilege lets user use his/her rights on database objects
      * Both privileges are assigned to all users by default.
      
      Implementation details:
      * system privileges are special grant rights to 'universe'.
      Therefore, they can be granted only by admin. Because of this fact,
      during creation or deletion of user, we have to switch to 'admin' to
      grant or revoke these rights.
      
      Important changes:
      * changed bootstrap.snap due to need to start admin with new privileges
      * added auto upgrade script for 1.7.7
      
      Fixes gh-2898.
      
      With contributions by @kostja.
      74ab44ae
    • Konstantin Osipov's avatar
      gh-2994 (effective user) · 2f5749cb
      Konstantin Osipov authored
      The original patch returned authenticated user for effective and vice
      versa.
      
      Reverse the meaning and update the patch.
      2f5749cb
  11. Dec 28, 2017
    • Konstantin Osipov's avatar
      security: add a test case fog gh-3023 · 56438fa6
      Konstantin Osipov authored
      box.session.su() changes both user and effective user right now.
      Changing only the session user seems to be rather difficult:
      we need to keep the object allocated somewhere, and keeping
      in mind request multiplexor in iproto, with which many requests
      can share the same session, it can only be Lua stack.
      
      While at it, change current_user() to effective_user() to
      make it less ambiguous.
      56438fa6
    • Ilya's avatar
      session: Resolve user and effective user · 2b163e20
      Ilya authored
      * Fix box.session.uid returning user id
      * Add function box.session.euid returning
      effective user id
      Closes #2994
      2b163e20
    • Konstantin Osipov's avatar
      security: introduce all ANSI SQL ACL, as well as session and usage · 6090846e
      Konstantin Osipov authored
      Introduce all the necessary ACL for ANSI SQL, as well as SESSION
      and USAGE.
      
      Change access storage type from uint8_t to a typedef.
      
      Necessary for gh-2898.
      6090846e
    • Ilya's avatar
      Error on wrong user on space creation · bfd31f8b
      Ilya authored
      Add error if user given in box.schema.space.create option
      was not found
      
      Closes #2068
      bfd31f8b
  12. Dec 26, 2017
    • Vladimir Davydov's avatar
      Add box.info.memory() to report aggregated memory statistics · 4240860f
      Vladimir Davydov authored
      This patch adds a new function, box.info.memory(). The functions returns
      a table with the following fields:
      
       - data - size of memory in bytes used for storing user data (i.e.
         tuples) in memtx and in vinyl level 0, without taking into account
         memory fragmentation.
      
       - index - size of memory in bytes used for indexing user data. This
         includes memtx and vinyl memory tree extents, vinyl page index, vinyl
         bloom filters.
      
       - cache - size of memory in bytes used for caching user data. Memtx
         doesn't have cache so basically this is the size of the vinyl tuple
         cache.
      
       - tx - size of memory in bytes used up by active transactions. For
         vinyl it is the total size of all allocated struct txv, struct vy_tx,
         struct vy_read_interval, plus tuples pinned by those objects. For
         memtx it is going to be 0 for now as memtx transaction manager
         shouldn't consume much memory. We may want to account struct txn and
         struct txn_stmt there too in future.
      
       - net - size of memory in bytes used up by network input and output
         buffers.
      
       - lua - size of memory used by the Lua runtime.
      
      It is supposed to be used by the admin to get a general knowledge about
      what's going on with a particular tarantool instance. For more info, per
      subsystem statistics are supposed to be used (e.g. box.info.vinyl()).
      
      Closes #934
      4240860f
    • Vladimir Davydov's avatar
      iproto: account memory used by network buffers · 8516cbc3
      Vladimir Davydov authored
      Add function iproto_mem_used() that returns the total amount of memory
      allocated for storing input and output buffers. It will be used by
      box.info.memory() implementation to show aggregated network statistics.
      
      Note, to account memory used by output buffers, we have to introduce a
      separate slab cache (currently, the cache of the tx cord is used).
      
      Needed for #934
      8516cbc3
    • Vladimir Davydov's avatar
      vinyl: account memory used by statements in read and write sets · 49b3f748
      Vladimir Davydov authored
      To be reported by box.info.memory().tx
      
      Needed for #934
      49b3f748
    • Vladimir Davydov's avatar
      vinyl: account total size of page index and bloom · 4fc2fbdf
      Vladimir Davydov authored
      To be reported by box.info.memory().index
      
      Needed for #934
      4fc2fbdf
    • Vladimir Davydov's avatar
      vinyl: account memory occupied by tree extents · 28a24984
      Vladimir Davydov authored
      To be reported by box.info.memory().index
      
      Needed for #934
      28a24984
Loading