Skip to content
Snippets Groups Projects
  1. Jun 21, 2017
    • Roman Tsisyk's avatar
      lua: fix lua_tofield() for 2**64 value · 3a13be1d
      Roman Tsisyk authored
      exp2(64) <= UINT64_MAX is true due to type conversion.
      Use exp2(64) instead of UINT64_MAX (optimized out even on -O0).
      
      Fixes buggy box/indices_any_type.test.lua on x86_64
      
      Closes #2472
      3a13be1d
    • Roman Tsisyk's avatar
      lua: remove unsupported tonumber64() test cases · 15a9af26
      Roman Tsisyk authored
      ffi.new('long', x) and ffi.new('unsigned long', x) uses arbitratry
      user-defined ctypeid instead of built-in CTID_INT32 and CTID_UINT32.
      Our tonumber64() implementation is not ready for that. Remove these
      two test cases and move forward. Nobody cares.
      
      Fixes box/misc.test.lua on i386 and armhf
      
      Follow up #2459
      15a9af26
    • Roman Tsisyk's avatar
      lua: fix lua_tointeger() usage on 32-bit platforms · 90ef664e
      Roman Tsisyk authored
      lua_tointeger() is equivalent to (ptrdiff_t)lua_tonumber(L),
      which causes precision loss on 32-bit systems when you expect
      to get a proper uint32_t value. LuaJIT stores numbers as doubles,
      so lua_tointeger() makes absolutely no sense.
      
      Replace lua_tointeger() with lua_tonumber() everywhere except
      cases when result is converted to int or lua_Integer.
      
      Fixes box/indices_any_type.test.lua on i386 and armhf
      
      Closes #2459
      90ef664e
    • Vladimir Davydov's avatar
      alter: disallow truncation of system spaces · 3e1b734b
      Vladimir Davydov authored
      It is unsafe, because system spaces use triggers to keep records in sync
      with internal objects while space truncation doesn't invoke triggers.
      
      Closes #2523
      3e1b734b
    • Vladimir Davydov's avatar
      alter: initialize truncate system space lazily · 1f0b74e4
      Vladimir Davydov authored
      Currently, space.create must insert an entry into 'truncate' system
      space - if it does not, space.truncate won't work. This is incontinent,
      as it makes it impossible to create spaces by simply inserting a tuple
      into the 'space' system space via iproto API. So let's insert entries
      into truncate space lazily, on the first space truncation.
      
      Closes #2524
      1f0b74e4
    • Vladimir Davydov's avatar
      alter: proscribe space truncate in transaction · 50a4a87a
      Vladimir Davydov authored
      Space truncation is implemented as recreation of a space and all its
      indexes. On success the original space is deleted by the commit trigger,
      while on failure the new space is deleted by the rollback trigger. The
      commit/rollback triggers are always called before commit/rollback engine
      methods, which makes space truncation incompatible with transactions:
      engine commit/rollback may access a space that have already been deleted
      by commit/rollback trigger installed by space truncation, resulting in a
      crash. So let's forbid to use space.truncate from a transaction until
      triggers are made transaction-friendly.
      
      Closes #2525
      50a4a87a
  2. Jun 20, 2017
  3. Jun 19, 2017
  4. Jun 16, 2017
    • Roman Tsisyk's avatar
      Add a test case for string.hex() · 73a39ad3
      Roman Tsisyk authored
      Closes #2522
      73a39ad3
    • Veniamin Gvozdikov's avatar
      Add string.hex() method · 0ca6cb87
      Veniamin Gvozdikov authored
      * Add string.hex() method
      * Add hmac.*_hex to crypto.lua
      * Update crypto/digest to use string.hex()
      
      Closes #2510
      0ca6cb87
    • Vladimir Davydov's avatar
      vinyl: report memory and disk totals in index.info · 47c98418
      Vladimir Davydov authored
      Replace the following index.info fields
      
        memory_used           # size of statements in memory
        size                  # size of statements on disk
        count                 # number of statements on disk and in memory
        page_count            # number of pages on disk
      
      with
      
        memory
          rows                # number of statements in memory
          bytes               # size of statements in memory
      
        disk
          rows                # number of statements on disk
          bytes               # size of statements on disk (unpacked)
          bytes_compressed    # size of statements on disk (packed)
          pages               # number of pages on disk
      
        rows                  # total number of all statements
        bytes                 # total size of all statements
      
      To achieve that, this patch introduces new classes that can be used for
      accounting statements on disk and in memory, vy_stmt_disk_counter and
      vy_stmt_counter, and makes vy_slice, vy_run, vy_range, and vy_index use
      them instead of counting rows, bytes, and pages directly. The difference
      between the two classes is that vy_stmt_counter only accounts rows and
      bytes, while vy_stmt_disk_counter also includes bytes_compressed and
      pages. The classes will be reused for accounting reads and writes later.
      
      Needed for #1662
      47c98418
    • Vladimir Davydov's avatar
      vinyl: cleanup run and page info · dd6bd264
      Vladimir Davydov authored
       - Delete unused vy_page_info->min_key_offset.
       - Move vy_run_info->{size,keys,page_infos} to vy_run, because
         they are not a part of run info stored in .index file.
       - Rename vy_run->page_infos to page_info.
       - Rename vy_page_info->count, vy_run->keys, and vy_slice->keys
         to row_count.
       - Update comments.
      dd6bd264
    • Vladimir Davydov's avatar
      box: replace info_append_{u32,u64} with info_append_int · ab4e9e9b
      Vladimir Davydov authored
      It's no use having a separate method for every kind of integer we want
      to append to box info - int64_t should suit everyone.
      ab4e9e9b
    • Vladimir Davydov's avatar
      vinyl: fix dump bandwidth calculation · 0b173065
      Vladimir Davydov authored
      We compute dump bandwidth basing on the time it takes a run writing task
      to complete. While it used to work when we didn't have data compression
      and indexes didn't share in-memory tuples, today the logic behind dump
      bandwidth calculation is completely flawed:
      
       - Due to data compression, the amount of memory we dump may be much
         greater than the amount of data we write to disk, in which case dump
         bandwidth will be underestimated.
      
       - If a space has several indexes, dumping it may result in writing more
         data than is actually stored in-memory, because tuples of the same
         space are shared among its indexes in-memory, but stored separately
         when written to disk. In this case, dump bandwidth will be
         overestimated.
      
      This results in quota watermark being set incorrectly and, as a result,
      either stalling transactions or dumping memory non-stop.
      
      Obviously, to resolve both issues, we need to account memory freed per
      unit of time instead of data written to disk. So this patch makes
      vy_scheduler_trigger_dump() remember the time when dump was started and
      vy_scheduler_complete_dump() update dump bandwidth basing on the amount
      of memory dumped and the time dump took.
      0b173065
    • Vladimir Davydov's avatar
      vinyl: do not call vy_scheduler_complete_dump on index deletion · 824ceb32
      Vladimir Davydov authored
      Currently, vy_scheduler_remove_mem() calls vy_scheduler_complete_dump()
      if vy_scheduler_dump_in_progress() returns false, but the latter doesn't
      necessarily mean that the dump has just been completed. The point is
      that vy_scheduler_remove_mem() is called not only for a memory tree that
      has just been dumped to disk, but also for all memory trees of a dropped
      index, i.e. dropping an index when there's no dump in progress results
      in vy_scheduler_complete_dump() invocation. This doesn't do any harm
      now, but looks ugly. Besides, I'm planning to account dump bandwidth in
      vy_scheduler_complete_dump(), which must only be done on actual dump
      completion.
      824ceb32
    • Vladimir Davydov's avatar
      vinyl: factor out functions for memory dump start and completion · 9719b28a
      Vladimir Davydov authored
      Following patches will add more logic to them, so it's better to factor
      them out now to keep the code clean. No functional changes.
      9719b28a
    • Vladimir Davydov's avatar
      vinyl: fix crash if snapshot is called while dump is in progress · dbfd515f
      Vladimir Davydov authored
      Currently, to force dumping all in-memory trees, box.snapshot()
      increments scheduler->generation directly. If dump is in progress and
      there's a space that has more than one index and all its secondary
      indexes have been dumped by the time box.snapshot() is called and its
      primary index is being dumped, incrementing the generation will force
      the scheduler to start dumping secondary indexes of this space again
      (provided, of course, the space has fresh data). Then, creating a dump
      task for a secondary index will attempt to pin the primary index - see
      vy_task_dump_new() => vy_scheduler_pin_index() - which will crash,
      because the primary index is being dumped and hence can't be removed
      from the scheduler by vy_scheduler_pin_index():
      
        Segmentation fault
        #0  0x40c3a4 in sig_fatal_cb(int)+214
        #1  0x7f6ac7981890 in ?
        #2  0x4610bd in vy_scheduler_remove_index+46
        #3  0x4610fe in vy_scheduler_pin_index+49
        #4  0x45f93e in vy_task_dump_new+1478
        #5  0x46137e in vy_scheduler_peek_dump+282
        #6  0x461467 in vy_schedule+47
        #7  0x461bf8 in vy_scheduler_f+1143
      
      To fix that let's trigger dump (by bumping generation) only from the
      scheduler fiber, from vy_scheduler_peek_dump(). The checkpoint will
      force the scheduler to schedule dump by setting checkpoint_in_progress
      flag and setting checkpoint_generation.
      
      Closes #2508
      dbfd515f
    • Konstantin Osipov's avatar
      40d86fe6
    • Vladimir Davydov's avatar
      alter: init space truncate_count after recovering snapshot · 99d6a4f4
      Vladimir Davydov authored
      The replace trigger of _truncate system space (on_replace_dd_truncate)
      does nothing on insertion into or deletion from the space - it only
      updates space truncate_count when a tuple gets updated. As a result,
      space truncate_count isn't initialized properly after recovering
      snapshot. This does no harm to memtx, because it doesn't use space
      truncate_count at all, but it breaks the assumption made by vinyl that
      if space truncate_count is less than index truncate_count (which is
      loaded from vylog), the space will be truncated during WAL recovery and
      hence there's no point in applying statements to the space (see
      vy_is_committed_one). As a result, all statements inserted into a vinyl
      space after snapshot following truncation of the space, are ignored on
      WAL recovery. To fix that, we must initialize space truncate_count when
      a tuple is inserted into _truncate system space.
      
      Closes #2521
      99d6a4f4
    • Roman Tsisyk's avatar
      72be507f
    • Ilya's avatar
      Add HTTP client based on libcurl · 7e62ac79
      Ilya authored
      Inpspired by tarantool/curl module by Vasiliy Soshnikov.
      Reviewed and refactored by Roman Tsisyk.
      
      Closes #2083
      7e62ac79
    • Roman Tsisyk's avatar
      Fix name clash in reflection.h · 414635ed
      Roman Tsisyk authored
      Rename `struct type` to `struct type_info` and `struct method` to
      `struct method_info` to fix name clash with curl/curl.h
      414635ed
  5. Jun 15, 2017
    • Vladimir Davydov's avatar
      box: auto upgrade to 1.7.5 · 29a197cf
      Vladimir Davydov authored
      We added _truncate space to 1.7.5 and we are going to add new system
      spaces for storing sequences and triggers. Without upgrade, the
      corresponding operations won't work. Since 1.7.5 is a minor upgrade,
      users may not call box.schema.upgrade(), so we need to call it for them
      automatically. This patch introduces infrastructure for automatic
      upgrades and sets upgrade to 1.7.5 to be called automatically.
      
      While we are at it, rename schema version 1.7.4 to 1.7.5 (1.7.4 has
      already been released).
      
      Closes #2517
      29a197cf
    • Roman Tsisyk's avatar
      Rename snapshot_daemon to checkpoint_daemon · e126d26c
      Roman Tsisyk authored
      Follow up #2496
      e126d26c
    • Roman Tsisyk's avatar
      Enable snapshot_daemon by default · 92a8e0dc
      Roman Tsisyk authored
      Set checkpoint_count = 2, checkpoint_interval = 3600 by default.
      
      vinyl/layout.result is updated because checkpoint_count was changed
      from 6 to 2.
      
      Closes #2496
      92a8e0dc
Loading