Skip to content
Snippets Groups Projects
  1. Aug 29, 2019
    • Kirill Shcherbatov's avatar
      sql: get rid of FuncDef function hash · 45396002
      Kirill Shcherbatov authored
      Now it is possible to move all SQL builtin functions to
      Tarantool's function hash. An existent FuncDef function
      representation was replaced with func_sql_builtin class.
      It has a sql-specific method :call and :finalize, while
      port API call is not supported and protected with stubs.
      
      This patch removes FuncDef hash and sql_function_create endpoint,
      but doesn't introduce something instead. Therefore few affected
      tests are disabled. A required functionality would be fixed in
      the next patch.
      
      Following tests using sql_create_function are broken now.
      They are going to be fixed in the next commit:
       sql-tap/alias.test.lua sql-tap/check.test.lua
       sql-tap/func5.test.lua sql-tap/lua_sql.test.lua
       sql-tap/subquery.test.lua sql-tap/trigger9.test.lua
       sql/errinj.result sql/errinj.test.lua
       sql/func-recreate.test.lua
      
      Part of #2200, #4113, #2233
      
      @TarantoolBot document
      Title: SQL builtins priveleges
      
      All SQL built-ins are executed on SQL privilege level that is
      undefined yet.
      45396002
    • Kirill Shcherbatov's avatar
      sql: remove name overloading for SQL builtins · 5b8aebd6
      Kirill Shcherbatov authored
      Now all SQL builtins are unique, i.e. each function has the only
      hash table entry. This technique requires to remove static argc
      checks for some builtins(substr, round, like, trim, count,
      group_concat). Now they raise a runtime error in case of invalid
      usage.
      
      Updated error messages correspondingly, to provide more
      informative messages in such cases. Taking into account upcoming
      changes, all built-in functions names are uppercased.
      
      Needed for #2200, #4113, #2233
      5b8aebd6
    • Kirill Shcherbatov's avatar
      sql: replace flag MINMAX with flags MIN and MAX · ddc83e04
      Kirill Shcherbatov authored
      Replaced sql function flag SQL_FUNC_MINMAX with couple new flags
      SQL_FUNC_MIN and SQL_FUNC_MAX. This allows to distinguish MIN and
      MAX function by flag instead of using user_data context.
      
      This allows to delete user_data field in FundDef object in
      further refactoring.
      
      Needed for #2200, #4113, #2233
      ddc83e04
    • Kirill Shcherbatov's avatar
      sql: rename sql_vdbe_mem_alloc_region helper · 025f79bd
      Kirill Shcherbatov authored
      Changed sql_vdbe_mem_alloc_region routine name to the
      vdbe_mem_alloc_blob_region because we are going to introduce
      a new function with a really similar (but more appropriate) name
      vdbemem_alloc_on_region in following patch. Due to the fact that
      the function used only in vdbe.c module it has been moved there
      and has been marked as "static".
      
      Needed for #2200, #4113, #2233
      025f79bd
    • Nikita Pettik's avatar
      sql: make GREATEST/LEAST built-ins accept at least two args · 4125c4ef
      Nikita Pettik authored
      Before a46b5200 SQL implementation featured min()/max() functions
      overloading: if one argument was passed, then aggregate version would be
      invoked; otherwise - scalar one. We decided to get rid of it and rename
      scalar version to LEAST()/GREATEST() correspondingly. However, assertion
      inside their implementations has been remained: it verifies that number
      of passed arguments is greater than 1. On the other hand, now one can
      pass literally any number of arguments to this function, including one
      (which results in fired mentioned assertion) and zero (which leads to
      NULL dereference in expr.c: these functions are marked with
      SQL_FUNC_NEEDCOLL flag, and as a consequence they are assumed to have at
      least one argument). Firstly, let's place check that number of passed
      arguments more than one. Secondly, let's not assume that functions with
      SQL_FUNC_NEEDCOLL must have any arguments.
      
      Closes #4453
      4125c4ef
    • Alexander V. Tikhonov's avatar
      Set fragile option to flaky tests · 165f8ee6
      Alexander V. Tikhonov authored
      Added "fragile" option to the flaky tests that are
      not intended to be run in parallel with others.
      Option set at the suite.ini file at the appropriate
      suites with comments including the issue that stores
      the fail.
      165f8ee6
    • Alexander V. Tikhonov's avatar
      travis-ci: remove obsolete jobs on travis · 05477787
      Alexander V. Tikhonov authored
      Removed the jobs that is interesting to check on gitlab-ci
      instead of travis-ci:
        - osx 13 "Sierra"
        - all LTO jobs
        - ASAN job
      
      Part of #4410
      Unverified
      05477787
    • Georgy Kirichenko's avatar
      Destroy port after iproto eval if transaction isn't finished · ff14626d
      Georgy Kirichenko authored
      This is a followup for 7691154a
      ff14626d
    • Maria Khaydich's avatar
      Iproto call won't leak if transaction isn't committed · 7691154a
      Maria Khaydich authored
      In case of throwing client error because of inactive function
      we did not destroy used port. It could possibly cause huge
      memory leaks as could be seen with top or its analogues when
      performing net.box test run in a loop.
      
      Closes #4388
      7691154a
  2. Aug 28, 2019
    • Kirill Yukhin's avatar
      Add missing test for _say patch · c80e9416
      Kirill Yukhin authored
      It turned out that patch d0e38d59 wasn't accompanied
      w/ a test. Add proper check.
      c80e9416
    • Serge Petrenko's avatar
      replication: enter orphan mode on manual replication configuration chage · 5a0cfe02
      Serge Petrenko authored
      Currently we only enter orphan mode when instance fails to connect to
      replication_connect_quorum remote instances during local recovery.
      On bootstrap and manual replication configuration change an error is
      thrown. We better enter orphan mode on manual config change, and leave
      it only in case we managed to sync with replication_connect_quorum
      instances.
      
      Closes #4424
      
      @TarantoolBot document
      Title: document reaction on error in replication configuration change.
      
      Now when issuing `box.cfg{replication={uri1, uri2, ...}}` and failing to
      sync with replication_connect_quorum remote instances, the server will
      throw an error, if it is bootstrapping, and just set its state to orphan
      in all other cases (recovering from existing xlog/snap files or manually
      changing box.cfg.replication on the fly). To leave orphan mode, you may
      wait until the server manages to sync with replication_connect_quorum
      instances.
      In order to leave orphan mode you need to make the server sync with
      enough instances. To do so, you may either:
      1) set replication_connect_quorum to a lower value
      2) reset box.cfg.replication to exclude instances that cannot
         be reached or synced with
      3) just set box.cfg.replication to "" (empty string)
      5a0cfe02
    • Konstantin Osipov's avatar
      replication: do not abort replication on ER_UNKNOWN_REPLICA · a850acfd
      Konstantin Osipov authored
      In 3-node replica-set, registering with the leader node
      does not guarantee the registration record arrives to the second
      peer immediately. The joining node may bootstrap faster than
      the registration record arrives to the second peer, in which
      case replication will fail to create a full mesh.
      a850acfd
    • Konstantin Osipov's avatar
      systemd: Do nothing if NOTIFY_SOCKET env variable is not set. · 83ef5a17
      Konstantin Osipov authored
      A follow up on gh-4305, fix failing args.test.py
      83ef5a17
    • Konstantin Osipov's avatar
      Fix build failure on Linux. · ef14050f
      Konstantin Osipov authored
      ef14050f
  3. Aug 27, 2019
    • Max Melentiev's avatar
      Enable support for NOTIFY_SOCKET in envs without systemd · 1e509dde
      Max Melentiev authored
      To make it possible to develop and test related features on
      systems without systemd.
      
      WITH_SYSTEMD cmake flag is used to generate systemd related files:
      unit, generator script, etc. To keep this behavior and make it possible
      to use NOTIFY_SOCKET without other systemd-related stuff,
      I added WITH_NOTIFY_SOCKET cmake flag.
      
      It also required some changes to support other OS:
      
      SOCK_CLOEXEC (not available on macOS) flag for socket()
      is replaced with `fcntl(fd, F_SETFD, FD_CLOEXEC)` which has the same effect.
      
      MSG_NOSIGNAL flag for sendmsg is also not available on macOS.
      However it has SO_NOSIGPIPE flag for setsockopt which disables SIGPIPE.
      So it requires different solution for different OS. Inspired by
      https://nwat.xyz/blog/2014/01/16/porting-msg_more-and-msg_nosigpipe-to-osx/
      
      Have to reduce send-buffer size to 4MB because larger values
      are not supported on macOS by default. This value should be enough
      for all systems because notification messages are usually less than 1KB.
      
      Fixes #4436
      1e509dde
    • Alexander Turenko's avatar
      build: fix linking with static openssl library · 2cdfaf3b
      Alexander Turenko authored
      System-wide dynamic libraries usually (always?) have NEEDED and RUNPATH
      tags in a dynamic section (as `readelf -d /usr/lib/lib<...>.so` shows),
      so when we link, say, with libssl.so, which depends on libz.so, a linker
      does not complain against unresolved symbols that can be found in Z
      library (if it is installed within a system).
      
      Things are different when we linking with a static library. Say, when we
      linking with libssl.a, which contains an unresolved symbol from Z
      library, a linker reports an error. It is not possible to store an
      information where to find unresolved symbols (NEEDED / RUNPATH) in a
      static library (AFAIK).
      
      We depend on three libraries that are depend on Z library: libcurl,
      libssl and libcrypto (two latter are part of OpenSSL). When one of those
      libraries is linked statically we should link with libz.so or libz.a
      (depending on BUILD_STATIC flag). The patch doing exactly this.
      
      The patch changes OPENSSL_LIBRARIES variable to fix the issue with
      static linking of OpenSSL libraries. It also changes CURL_LIBRARIES in
      the same way, however this does not alter any visible behaviour, because
      OPENSSL_LIBRARIES is added to CURL_LIBRARIES. The latter change was made
      to unify the way to choose libraries to link with: it is pure
      refactoring part.
      
      Fixes #4437.
      2cdfaf3b
    • Vladislav Shpilevoy's avatar
      tuple: implement update by field name · 9ee5cf82
      Vladislav Shpilevoy authored
      Tuple fields can be named, accessed by name, indexed by name, but
      till this commit field names couldn't be used in update
      operations. Now it is possible.
      
      This patch is a teaser of updates by JSON path.
      
      Part of #1261
      9ee5cf82
    • Vladislav Shpilevoy's avatar
      int96: add a missing header · 6d4430d0
      Vladislav Shpilevoy authored
      6d4430d0
    • Vladislav Shpilevoy's avatar
      tuple: relax struct tuple_update dependency on rope · 74ae6780
      Vladislav Shpilevoy authored
      Rope will not be used by tuple_update directly in
      next patches.
      74ae6780
    • Vladislav Shpilevoy's avatar
      rope: make rope library macro template · baa4659c
      Vladislav Shpilevoy authored
      Rope library stores alloc, split and free functions by pointer,
      that is a huge slog at performance, as any other virtual
      function call. There is no reason, why the rope library may not
      become a template, except historical ones.
      
      The patch not only removes virtual functions, but also makes rope
      deletion almost no-op in case if free() function is not defined.
      
      A second motivation point of that patch is that original rope
      structure was too big. Again, because it stored several pointers
      to functions. In forthcoming patches on #1261 multiple ropes can
      be created per each update, so it makes sense to reduce size of
      this structure.
      baa4659c
    • Vladislav Shpilevoy's avatar
      tuple: remove alloc and alloc_ctx args from update() · dba9dba7
      Vladislav Shpilevoy authored
      They are always region_aligned_alloc and region pointer. Lets use
      them always inside tuple_update.c, with no necessity to pass
      explicitly.
      
      The patch is motivated by forthcoming updates by JSON path, which
      will strongly complicate and perhaps slow down the tuple_update.c
      code. The present patch as well as some next ones should smooth
      these problems.
      dba9dba7
    • Alexander Turenko's avatar
      Use forked tarantool/curl repository · 5a46639f
      Alexander Turenko authored
      A manual action is needed after pulling of this commit to actually use
      the downstream repository instead of the upstream one:
      
      sed -i -e 's@https://github.com/curl/curl.git@https://github.com/tarantool/curl.git@' .git/config
      
      It is part of our processes to use forked repositories for submodules.
      
      Suggested by Konstantin Osipov.
      5a46639f
    • Alexander V. Tikhonov's avatar
      gitlab-ci: clean up .gitlab-ci.yml · 421d440b
      Alexander V. Tikhonov authored
      Cleaned up the .gitlab-ci.yml file from duplicating
      code - added templates that storres the needed
      configuration values for different jobs.
      Also moved the static_build from 'deploy' tag to
      'deploy_test' tag to be sure that tests will not
      be run under high load.
      421d440b
    • Serge Petrenko's avatar
      lua: fix decimal comparison with box.NULL · 24e75aa8
      Serge Petrenko authored
      This problem is similar to the one fixed in commit
      3c6c1cc9 (lua:fix decimal comparison
      with nil)
      We should handle box.NULL the same way.
      
      Closes #4454
      24e75aa8
    • Alexander Turenko's avatar
      test: update test-run · 418ab172
      Alexander Turenko authored
      * Added "fragile" option with a list of tests that are not intended to
        be run in parallel with others. The format of the list is the same as
        for "disabled" option (#187, PR #188).
      * Ensured that non-parallel test suites and fragile tests will be run
        only when all parallel ones will be finished (PR #188).
      * Fixed reporting in case of lack of temporary result (PR #172).
      Unverified
      418ab172
  4. Aug 26, 2019
    • Alexander Turenko's avatar
      lua: pwd: fix passwd and group traversal · 7753d350
      Alexander Turenko authored
      CentOS 6 and FreeBSD 12 implementations of getpwuid() rewind
      setpwent()-{getpwent()}-endpwent() loop to a start that leads to a hang
      during pwd.getpwall() invoke. The same is true for a getgrgid() call
      during setgrent()-{getgrent()}-endgrent() loop.
      
      The commit modifies pwd module to avoid getpwuid() calls during passwd
      database traversal and to avoid getgrgid() calls when traversing groups.
      
      The commit also fixes the important regression on CentOS 6 after
      f5d8331e ('lua: workaround
      pwd.getpwall() issue on Fedora 29'): tarantool hungs during startup due
      to added getpwall() call. This made tarantool unusable on CentOS 6 at
      all.
      
      Aside of that the commit fixes another pwd.getgrall() problem: the
      function gaves password entries instead of group entries.
      
      Fixes #4428.
      Fixes #4447.
      Part of #4271.
      7753d350
  5. Aug 23, 2019
  6. Aug 22, 2019
    • Yaroslav Dynnikov's avatar
      feedback: unify payload generation logic · 5b9f207d
      Yaroslav Dynnikov authored
      This change is related to #4391. The objective was to collect additional
      information about modules, but it's hard to do without changing API.
      
      This patch will allow to monkey-patch report generation and achieve the
      same results without interfering the daemon behavior.
      5b9f207d
    • Max Melentiev's avatar
      systemd: replace sendmsg with sendto shortcut · 89aae30c
      Max Melentiev authored
      There is a problem with calculating .msg_namelen field
      of msghdr struct. Instead of
      
          .msg_name   = &sa,
          .msg_namelen = sizeof(sa.sun_family) + strlen(sd_unix_path),
      
      it must set as
      
          .msg_namelen = sizeof(sa) // larger value than current invalid one
      
      It works on linux but when I tried to enable this feature for macOS
      it didn't (maybe because of different order of fields in the struct).
      
      Instead of fixing calculation, I've replaced original sendmsg call
      with sendto, because it's a convenient shortcut which
      simplifies code and can prevent such mistakes.
      
      Required for #4436
      89aae30c
    • Max Melentiev's avatar
      tarantoolctl: allow to start instances with delayed box.cfg{} · 17df9edf
      Max Melentiev authored
      `tarantoolctl start` patches box.cfg two times:
      1) before the init script to set default values and enforce some others,
      2) after the init script to prevent changing a pid_file in runtime.
      
      The second patching fails if an init file does not call
      box.cfg{} before it's finished. This can take a place in apps with
      managed instances which receive configuration from external server.
      
      This patch moves the second patching into the box.cfg
      wrapper created during the first patching. So the second patching
      is performed only after box.cfg{} was invoked, so it does not fail anymore.
      
      However there is relatively minor flaw for applications that
      invoke box.cfg{} after init script is finished:
      `tarantoolctl start` goes to background only when box.cfg{} is called.
      Though this is not the case for daemon management systems like systemd,
      as they handle backgrounding on their side
      
      Fixes #4435
      
      @TarantoolBot document
      Title: tarantoolctl allows to start instances without a box.cfg{}
      
      tarantoolctl now works for instances without box.cfg{} or
      with delayed box.cfg{} call. This can be managed instances which receive
      configuration from external server.
      
      For such instances `tarantoolctl start` goes to background when
      box.cfg{} is called, so it will wait until options for box.cfg are received.
      However this is not the case for daemon management systems like systemd,
      as they handle backgrounding on their side.
      17df9edf
    • Nikita Pettik's avatar
      Move mp_compare_double_uint64() to trivia/util.h · 72dbeb0e
      Nikita Pettik authored
      This function implements common way of precise comparison between
      unsigned integer and floating point values (doubles). Currently, it is
      used in tuple comparators, but we need the same thing in SQL. Hence,
      let's move it to header containing set of utilities.
      72dbeb0e
    • Nikita Pettik's avatar
      sql: use double_compare_uint64() for int<->float cmp · 73a4a525
      Nikita Pettik authored
      To compare floating point values and integers in SQL functions
      compare_uint_float() and compare_int_float() are used. Unfortunately,
      they contain bug connected with checking border case: that's not correct
      to cast UINT64_MAX (2^64 - 1) to double. Proper way is to use exp2(2^64)
      or predefined floating point constant. To not bother fixing function
      which in turn may contain other tricky places, let's use instead already
      verified double_compare_uint64(). So that we have unified way of
      integer<->float comparison.
      73a4a525
    • Alexander Turenko's avatar
      gitlab-ci: fix building of Debian Buster image · 45c2576d
      Alexander Turenko authored
      `apt-get update <...>` fails on Debian Buster on docker_bootstrap goal
      (see #4331 for the similar issue).
      
      Added a description how to change dependencies in .travis.mk.
      45c2576d
    • Nikita Pettik's avatar
      sql: allow to specify engine in CREATE TABLE stmt · 1013a744
      Nikita Pettik authored
      Closes #4422
      
      @TarantoolBot document
      Title: Introduce <WITH ENGINE> clause for CREATE TABLE statement
      
      To allow user to specify engine as per table option, CREATE TABLE
      statement has been extended with optional <WITH ENGINE = engine_name>
      clause. This clause comes at the end of CREATE TABLE statement.
      For instance:
      
      CREATE TABLE t_vinyl (id INT PRIMARY KEY) WITH ENGINE = 'vinyl';
      
      Name of engine is considered to be string literal ergo should be
      enclosed in single quotation marks and be lower-cased. Note that engine
      specified in WITH ENGINE clause overwrites default engine, which is set
      via 'pragma sql_default_engine'.
      1013a744
    • Nikita Pettik's avatar
      Do not log error in engine_find() · 35177fe0
      Nikita Pettik authored
      Error logging in engine_find() seems to be redundant: error message is
      displayed twice (since its callers always push error on the top).  Let's
      remove this duplication.
      35177fe0
    • Nikita Pettik's avatar
      txn: erase old savepoint in case of name collision · 8b8b6895
      Nikita Pettik authored
      Name duplicates are allowed for savepoints (both in our SQL
      implementation and in ANSI specification). ANSI SQL states that previous
      savepoint should be deleted. What is more, our doc confirms this fact
      and says that "...it is released before the new savepoint is set."
      Unfortunately, it's not true - currently old savepoint remains in the
      list. For instance:
      
      SAVEPOINT t;
      SAVEPOINT t;
      RELEASE SAVEPOINT t;
      RELEASE SAVEPOINT t; -- no error is raised
      
      Let's fix this and remove old savepoint from the list.
      8b8b6895
    • Nikita Pettik's avatar
      sql: use struct txn_savepoint as anonymous savepoint · 0a92ec7e
      Nikita Pettik authored
      This allows us to completely remove SQL specific struct Savepoint and
      use instead original struct txn_savepoint.
      0a92ec7e
    • Nikita Pettik's avatar
      txn: merge struct sql_txn into struct txn · 56096ff2
      Nikita Pettik authored
      This procedure is processed in several steps. Firstly, we add name
      to struct txn_savepoint since we should be capable of operating on named
      savepoints (which in turn is SQL feature). Still, anonymous (in the sense
      of name absence) savepoints are also valid. Then, we add list (as
      implementation of stailq) of savepoints to struct txn: it allows us to
      find savepoint by its name. Finally, we patch rollback to/release
      savepoint routines: for rollback tail of the list containing savepoints
      is cut (but subject of rollback routine remains in the list); for
      release routine we cut tail including node being released.
      56096ff2
    • Nikita Pettik's avatar
      txn: move fk_deferred_count from psql_txn to txn · 5259274d
      Nikita Pettik authored
      We are going to merge struct psql_txn with struct txn as a part of SQL
      integration into NoSQL, so let's move counter of deferred foreign key
      violations directly to struct txn.
      5259274d
    • Serge Petrenko's avatar
      box: add support for decimals in update ops · 8317189a
      Serge Petrenko authored
      Closes #4413
      
      @TarantoolBot document
      Title: update operations on decimal fields.
      
      tuple:update and space:update now support deicmal operands for
      arithmetic operations ('+' and '-'). The syntax is as usual:
      ```
      d = box.tuple.new(decimal.new('1'))
      ---
      ...
      d:update{{'+', 1, decimal.new('0.5')}}
      ---
      - [1.5]
      ...
      ```
      
      Insertion ('!') and assignment ('=') are also supported:
      ```
      a = decimal.new('1')
      ---
      ...
      b = decimal.new('1e10')
      ---
      ...
      c = decimal.new('1e-10')
      ---
      ...
      d = box.tuple.new{5, a, 6, b, 7, c, "string"}
      ---
      ...
      d
      ---
      - [5, 1, 6, 10000000000, 7, 0.0000000001, 'string']
      ...
      
      d:update{{'!', 3, dec.new('1234.5678')}}
      ---
      - [5, 1, 1234.5678, 6, 10000000000, 7, 0.0000000001, 'string']
      ...
      d:update{{'=', -1, dec.new('0.12345678910111213')}}
      ---
      - [5, 1, 6, 10000000000, 7, 0.0000000001, 0.12345678910111213]
      
      When performing an arithmetic operation ('+', '-'), where either the
      updated field or the operand is decimal, the result will be decimal.
      
      When both the updated field and the operand are decimal, the result
      will, of course, be decimal.
      
      ...
      ```
      8317189a
Loading