Skip to content
Snippets Groups Projects
  1. May 28, 2020
  2. May 27, 2020
    • Vladislav Shpilevoy's avatar
      build: link bit library to vclock library · 74ac4263
      Vladislav Shpilevoy authored
      Vclock depends on bit library, but wasn't linked to it.
      74ac4263
    • Nikita Pettik's avatar
      vinyl: unthrottle scheduler on manual checkpoint · 7305fe5a
      Nikita Pettik authored
      Before this patch box.snapshot() bails out immediately if it sees
      that the scheduler is throttled due to errors. For instance:
      
      box.error.injection.set('ERRINJ_VY_RUN_WRITE', true)
      snapshot() -- fails due to ERRINJ_VY_RUN_WRITE
      box.error.injection.set('ERRINJ_VY_RUN_WRITE', false)
      snapshot() -- still fails despite the fact that injected error is unset
      
      As a result, one has to wait up to a minute to make a snapshot. The
      reason why throttling was introduced was to avoid flooding the log
      in case of repeating disk errors.
      What is more, to deal with schedule throttling in tests, we had to
      introduce a new error injection (ERRINJ_VY_SCHED_TIMEOUT). It reduces
      time duration during which the scheduler remains throttled, which is
      ugly and race prone.  So, let's unthrottle scheduler when checkpoint
      process is launched via manual box.snapshot() invocation.
      
      Closes #3519
      7305fe5a
    • Nikita Pettik's avatar
      engine: add is_scheduled arg to engine->begin_checkpoint · 00c582f0
      Nikita Pettik authored
      In some cases it may turn out to be useful to know whether checkpoint
      process was launched manually (explicitly calling box.snapshot()) or
      automatically via checkpoint daemon. In particular, to unthrottle vinyl
      scheduler when it comes for manual checkpoints. So let's extend engine's
      vtab method begin_checkpoint() with corresponding argument.
      
      Needed for #3519
      00c582f0
    • Nikita Pettik's avatar
      vinyl: fix assert in vy_tx_write() · f6cb9289
      Nikita Pettik authored
      Assert in vy_tx_write() validates that upsert applied on the top of
      cached statement is always replace. In fact, it is not always so. If
      vy_apply_upsert() fails due to the fact that PK is modified, it returns
      old statement (i.e. statement which upsert is applied on). In this
      regard, if tuple cache contains insert and invalid upsert is executed,
      vy_apply_upsert() will return insert. As a result, assert will fire.
      Let's fix it and take into account that vy_apply_upsert() is able to
      return inserts as well.
      
      Closes #5005
      f6cb9289
  3. May 25, 2020
    • Vladislav Shpilevoy's avatar
      vinyl: add missing diag_set in space creation · 2cf4450f
      Vladislav Shpilevoy authored
      vinyl_engine_create_space() didn't set an error in the
      diagnostics area when region_alloc() failed. This
      could lead to a crash, although was almost impossible
      to happen.
      2cf4450f
    • Vladislav Shpilevoy's avatar
      build: turn off LTO for exports.c · 36927e54
      Vladislav Shpilevoy authored
      There were lots of errors of kind:
      
      /builds/M4RrgQZ3/0/tarantool/tarantool/src/exports.h:395:1: error: variable ‘uuid_nil’ redeclared as function
        EXPORT(uuid_nil)
        ^
       /builds/M4RrgQZ3/0/tarantool/tarantool/src/lib/uuid/tt_uuid.c:39:22: note: previously declared here
        const struct tt_uuid uuid_nil;
      
      when LTO was enabled. That happened because exports.c file, to
      take symbol addresses, declared lots of functions and variables
      from all the code base as
      
          extern void <symbol>(void);
      
      This is crazy, but it worked for normal builds. Because symbol is
      symbol. The compilers couldn't find conflicting declarations,
      because they never met in one compilation unit.
      
      However the lie was revealed by linker with LTO enabled. It could
      see, that actual symbol definitions didn't match their exports in
      exports.c. It could live with mismatching function-function or
      variable-variable cases, but couldn't withstand function-variable
      mismatches. When a symbol was declared as a variable in one place
      and as a function in another.
      
      This was the case for variables:
      - uuid_nil
      - tarantool_lua_ibuf
      - log_pid
      - log_format
      - crc32_calc
      - _say
      - log_level
      
      The errors were false positive, because the symbols were never
      used for anything except taking their addresses. To calm the
      linker down exports.c now does not participate in LTO.
      
      Closes #5001
      36927e54
  4. May 22, 2020
  5. May 21, 2020
  6. May 20, 2020
    • Alexander Turenko's avatar
      test: update test-run · b7b9a59f
      Alexander Turenko authored
      Support 'use_unix_sockets_iproto' suite.ini option in 'core = app' test
      suites. It'll allow to workaround TCP port choosing race (see more in
      [1]) by using Unix sockets instead of TCP ones: we did the same for
      'core = tarantool' test suites in [2].
      
      [1]: https://github.com/tarantool/test-run/issues/141
      [2]: 60f84cbf ('test: use unix sockets
           for iproto connections')
      
      Part of #4459.
      b7b9a59f
    • Nikita Pettik's avatar
      vinyl: fix comment to vy_read_iterator_cmp_stmt() · 4a98a9bb
      Nikita Pettik authored
      vy_read_iterator_cmp_stmt() uses tuple_compare_with_key() under the hood
      which follows next convention:
      (a) it returns 0 if key_fields of tuple equal to given key;
      (b) < 0 if key_fields less than given key;
      (c) > 0 otherwise.
      However, comment to vy_read_iterator_cmp_stmt() says that:
      (a) it returns -1 if first statement precedes second one;
      (b) 0 if they at the same position;
      (c) +1 if first statement supersedes second one.
      so, since there's no "normalization" of return code from
      tuple_compare_with_key() it would be correct to say that
      vy_read_iterator_cmp_stmt() returns:
      (a) it returns arbitrary integer value < 0 if first statement precedes
      second one;
      (b) 0 if they at the same position;
      (c) arbitrary integer value > 0 if first statement supersedes second one.
      4a98a9bb
    • Kirill Yukhin's avatar
      small: bump new version · bb7c3d16
      Kirill Yukhin authored
      Add MAP_STACK to mmap() flags.
      bb7c3d16
    • Alexander V. Tikhonov's avatar
      test: add more tests to packaging testing · d48406d5
      Alexander V. Tikhonov authored
      Changed number of tests to packaging testing jobs from
      suites: "unit/ app/ app-tap/ box/ box-tap/ engine/ vinyl/",
      to all suites except 'replication/' suite which has special
      issue for it's enabling:
        https://github.com/tarantool/tarantool/issues/4798
      Also added testing to the all available packing jobs except
      'CentOS 6'.
      
      Removed test-run option that run testing inline to be
      able to run it in parallel. Also changed 'test-run' to
      'make test-force' command.
      
      Closes #4599
      d48406d5
    • Alexander V. Tikhonov's avatar
      test: return tests to packaging testing · f5880666
      Alexander V. Tikhonov authored
      Found that issues #1227 and #1322 were closed, returned
      the tests blocked by it into the testing.
      
      Part of #4599
      f5880666
    • Alexander V. Tikhonov's avatar
      build: fix rpm packages build on Fedora 32 · b304a78d
      Alexander V. Tikhonov authored
      Tarantool packages build scenario at rpm/tarantool.spec
      installs Python packages for Tarantool testing. In some
      latest OS like Fedora 32 old Python packages based on
      Python 2 version were deprecated and changed to Python 3.
      Currently Tarantool testing based on Python 2 which
      packages stored at local backport repository and it
      should be used from there calling it as python2-* packages.
      
      Issue:
      ...
      No matching package to install: 'python-yaml >= 3.0.9'
      ...
      Not all dependencies satisfied
      Error: Some packages could not be found.
      
      Needed for #4599
      Closes #4965
      b304a78d
    • Alexander.V Tikhonov's avatar
      Enable script for saving RPM packages in S3 for modules · 64d1336a
      Alexander.V Tikhonov authored
      Found that modules may have only sources packages w/o binaries
      packages. Script updated to be able to work with only binaries
      either sources RPM packages. The same fix was already done for
      DEB packages at commit 4527a4da.
      
      Close #4839
      64d1336a
    • Alexander V. Tikhonov's avatar
      test: mark tests as fragile, 2nd group · 430c0e80
      Alexander V. Tikhonov authored
      Fragiled flaky tests from parallel runs to avoid
      of flaky fails in regular testing:
      
        app/fiber.test.lua                        ; gh-4987
        app/fiber_channel.test.lua                ; gh-4961
        app/socket.test.lua                       ; gh-4978
        box/alter_limits.test.lua                 ; gh-4926
        box/misc.test.lua                         ; gh-4982
        box/role.test.lua                         ; gh-4998
        box/rtree_rect.test.lua                   ; gh-4994
        box/sequence.test.lua                     ; gh-4996
        box/tuple.test.lua                        ; gh-4988
        engine/ddl.test.lua                       ; gh-4353
        replication/box_set_replication_stress    ; gh-4992
        replication/recover_missing_xlog.test.lua ; gh-4989
        replication/replica_rejoin.test.lua       ; gh-4985
        replication/wal_rw_stress.test.lua        ; gh-4977
        replication-py/conflict.test.py           ; gh-4980
        vinyl/errinj_ddl.test.lua                 ; gh-4993
        vinyl/misc.test.lua                       ; gh-4979
        vinyl/snapshot.test.lua                   ; gh-4984
        vinyl/write_iterator.test.lua             ; gh-4572
        xlog/panic_on_broken_lsn.test.lua         ; gh-4991
      
      Part of #4953
      430c0e80
    • Sergey Bronnikov's avatar
      Mention OpenBSD as supported platorm · 33cc961c
      Sergey Bronnikov authored
      Part of #4967
      33cc961c
    • Sergey Bronnikov's avatar
      Add README.OpenBSD · 54d682e2
      Sergey Bronnikov authored
      Part of #4967
      54d682e2
    • Sergey Bronnikov's avatar
      test: mute broken tests on OpenBSD · c244c0b2
      Sergey Bronnikov authored
      Part of #4967
      c244c0b2
    • Sergey Bronnikov's avatar
      say: fix compilation on OpenBSD · 64308484
      Sergey Bronnikov authored
      - define macros LOG_MAKEPRI() on OpenBSD as it is absent
      - replace sigtimedwait() by sigwait() as latter is unsupported on OpenBSD
      
      Part of #4967
      64308484
    • Sergey Bronnikov's avatar
      Include libgen.h when building on OpenBSD · fd653200
      Sergey Bronnikov authored
      Part of #4967
      fd653200
    • Sergey Bronnikov's avatar
      sql: use mremap() on OpenBSD · 43ffbba6
      Sergey Bronnikov authored
      Part of #4967
      43ffbba6
    • Sergey Bronnikov's avatar
      Fix building of tt_pthread_attr_getstack() on OpenBSD · 15601e09
      Sergey Bronnikov authored
      Part of #4967
      15601e09
    • Sergey Bronnikov's avatar
      build: skip linking with -ldl on OpenBSD · 291c392a
      Sergey Bronnikov authored
      OpenBSD includes DL library in a base system
      
      Part of #4967
      291c392a
    • Sergey Bronnikov's avatar
      build: test pthread_stackseg_np() on OpenBSD · 378e48aa
      Sergey Bronnikov authored
      Part of #4967
      378e48aa
    • Sergey Bronnikov's avatar
      build: introduce openbsd build target · 0b516c10
      Sergey Bronnikov authored
      Part of #4967
      0b516c10
    • Alexander Turenko's avatar
      test: popen: fix popen test 'hang' under test-run · 0afba959
      Alexander Turenko authored
      killpg() on Mac OS may don't deliver a signal to a process: it seems
      that there is a race when a process is just forked. It means that
      popen_handle:close() may leave a process alive, when `opts.setsid` and
      `opts.group_signal` are set.
      
      There is simple reproducer, which does not leave alive `sleep 120`
      processes on Linux, but does it on Mac OS (within three-four runs in
      rows):
      
       | #include <signal.h>
       | #include <unistd.h>
       | #include <fcntl.h>
       |
       | int
       | main()
       | {
       | 	char *child_argv[] = {
       | 		"/bin/sh",
       | 		"-c",
       | 		"sleep 120",
       | 		NULL,
       | 	};
       | 	pid_t pid;
       | 	int fd[2];
       | 	pipe(fd);
       | 	fcntl(fd[0], F_SETFD, FD_CLOEXEC);
       | 	fcntl(fd[1], F_SETFD, FD_CLOEXEC);
       |
       | 	if ((pid = fork()) == 0) {
       | 		/* Child. */
       | 		close(fd[0]);
       | 		setpgrp();
       | 		for (int i = 0; i < 10; ++i) {
       | 			/* Proceed with killpg. */
       | 			if (i == 5)
       | 				close(fd[1]);
       | 			if (fork() == 0) {
       | 				/* Child. */
       | 				execve("/bin/sh", child_argv, NULL);
       | 			}
       | 		}
       | 	} else {
       | 		/* Parent. */
       | 		close(fd[1]);
       | 		char c;
       | 		read(fd[0], &c, 1);
       | 		killpg(pid, SIGKILL);
       | 	}
       | 	return 0;
       | }
      
      Compile it (`cc test.c -o test`) and run several times:
      
      $ for i in $(seq 1 1000); do                     \
          echo $i;                                     \
          ./test                                       \
          && ps -o pid,pgid,command -ax | grep [s]leep \
          && break;                                    \
      done
      
      This is the reason why `sleep 120` process may be alive even when the
      whole test passes.
      
      test-run captures stdout and stderr of a 'core = app' test and waits EOF
      on them. If a child process inherit one of them or both, the fd is still
      open for writing and so EOF situation will not appear until `sleep 120`
      will exit.
      
      This commit doesn't try to overcome the root of the problem, but close
      stdout and stderr for the child process that may not be killed / exited
      in time.
      
      Aside of this, updated found Mac OS peculiars in API comments of C and
      Lua popen modules.
      
      Fixes #4938
      
      @TarantoolBot document
      Title: popen: add note re group signaling on Mac OS
      
      Copyed from the popen_handle:signal() updated description:
      
      > Note: Mac OS may don't deliver a signal to a process in a group when
      > opts.setsid and opts.group_signal are set. It seems there is a race
      > here: when a process is just forked it may be not signaled.
      
      Copyed from the popen_handle:close() updated description:
      
      > Details about signaling:
      >
      > <...>
      > - There are peculiars in group signaling on Mac OS,
      >   @see popen_handle:signal() for details.
      
      Follows up https://github.com/tarantool/doc/issues/1248
      0afba959
    • Alexander Turenko's avatar
      popen: fix access to freed memory after :close() · 01211e8e
      Alexander Turenko authored
      popen_delete() always frees a handle memory even when it reports a
      failure to send SIGKILL, see [1]. We should reflect this contract in
      popen_handle:close() and mark the handle as closed despite
      popen_delete() return value.
      
      There are cases, when killpg() fails with EPERM on Mac OS, so
      popen_delete() reports a failure. See [1] for more information.
      
      [1]: 01657bfb ('popen: always free
      resources in popen_delete()')
      
      Fixes #4995
      01211e8e
  7. May 19, 2020
    • Vladislav Shpilevoy's avatar
      session: remove box.session.push() 'sync' · a38c861e
      Vladislav Shpilevoy authored
      Closes #4689
      
      @TarantoolBot document
      Title: box.session.push() 'sync' is deprecated
      
      box.session.push() had two parameters - data to push and 'sync'.
      The sync was a request ID with which the out of bound data should
      be pushed into a socket.
      
      This was introduced as a workaround for #3450, and is useless
      since its resolution.
      
      A user anyway can't push to different sessions, where that
      parameter could be useful. And pushing into requests of the same
      session, on the contrary, is something not really needed anywhere,
      not portable to non-binary session types (console, background),
      and is just dangerous since it is easy to add a bug here.
      
      The patch removes the parameter. Now there will be thrown a
      'Usage' error at attempt to use 'sync' parameter. In version 2.4
      it is deprecated, prints warnings into logs, but still works. In
      2.5 it is removed completely.
      a38c861e
    • Kirill Yukhin's avatar
      small: revert previous bump · 61d2cb64
      Kirill Yukhin authored
      Revert 03790ac5 commit in small library part.
      61d2cb64
Loading