diff --git a/doc/user/configuration-reference.xml b/doc/user/configuration-reference.xml index 162eb784224272441e18abe7c1bd54df92559595..5a39f48fa1255fdbcbf04c93867626947aa98a3e 100644 --- a/doc/user/configuration-reference.xml +++ b/doc/user/configuration-reference.xml @@ -364,11 +364,15 @@ tarantool: primary pri: 3301 adm: 3313</programlisting> <entry>0</entry> <entry>yes</entry> <entry> - Period to make new database snapshot. - Daemon is disabled by - <code>snapshot_period=0</code> value. - Use <code>snapshot_period=3600</code> to make - snapshot for each hour. + The interval between actions by the snapshot daemon, in seconds. + The snapshot daemon is a fiber which is constantly running. + If snapshot_period is set to a value greater than zero, + then the snapshot daemon + will call <olink targetptr="box.snapshot"/> every snapshot_period seconds, creating + a new snapshot file each time. + For example, <code>box.cfg{snapshot_period=3600}</code> + will cause the snapshot daemon to create a new database + snapshot once per hour. </entry> </row> @@ -379,15 +383,15 @@ tarantool: primary pri: 3301 adm: 3313</programlisting> <entry>yes</entry> <entry> <para> - The daemon creates new snapshot and then it - removes old snapshots (and their xlogs) using the - option. + The maximum number of snapshots that the snapshot daemon maintains. + For example, <code>box.cfg{snapshot_period=3600, snapshot_count=10}</code> + will cause the snapshot daemon + to create a new snapshot each hour until it has created + ten snapshots. After that, it will remove the oldest + snapshot (and any associated write-ahead-log files) after creating + a new one. If snapshot_count equals zero, then the snapshot + daemon does not remove old snapshots. </para> - - <para> - Daemon is disabled by - <code>snapshot_count=0</code> value. - </para> </entry> </row> </tbody> @@ -417,7 +421,7 @@ tarantool: primary pri: 3301 adm: 3313</programlisting> <entry>boolean</entry> <entry>true</entry> <entry>no</entry> - <entry>If there is an error reading the snapshot file (at + <entry>If there is an error while reading the snapshot file (at server start), abort.</entry> </row> @@ -426,7 +430,7 @@ tarantool: primary pri: 3301 adm: 3313</programlisting> <entry>boolean</entry> <entry>false</entry> <entry>no</entry> - <entry>If there is an error reading a write-ahead + <entry>If there is an error while reading a write-ahead log file (at server start), abort.</entry> </row> @@ -511,11 +515,17 @@ tarantool: primary pri: 3301 adm: 3313</programlisting> <entry>If replication_source is not an empty string, the server is considered to be a Tarantool replica. The replica server will try to connect to the master - which replication_source specifies with format ip:port. - For example, if replication_source = "1.2.3.4:55555" then - the replica server tries to connect to 1.2.3.4 port 55555. - A replica server does not accept updates - on <olink targetptr="primary_port">listen</olink>. This parameter is + which replication_source specifies with a URI. + The string format is similar to the + <link + xlink:href="http://en.wikipedia.org/wiki/URI_scheme#Generic_syntax">generic syntax for a URI schema</link> + So it may be a user name and password and ip:port address such as 'guest:password@127.0.0.1:3301', + or just ip:port address such as '127.0.0.1:3301', + or just port such as '3301'. + The default user name is 'guest'. + A replica server does not accept data-change requests + on the <olink targetptr="primary_port">listen</olink> port. + The replication_source parameter is dynamic, that is, to enter master mode, simply set replication_source to an empty string and issue "box.cfg{replication_source=new-value}".</entry> diff --git a/doc/user/databases.xml b/doc/user/databases.xml index 57d031b544a35c7c63906bb4f41ebcf4613ac137..cc92af0d232b553a19d9eb3f08959a40246e098d 100644 --- a/doc/user/databases.xml +++ b/doc/user/databases.xml @@ -638,7 +638,7 @@ tarantool> <userinput>box.space.space55.index.primary:rename('secondary')</useri <varlistentry> <term> <emphasis role="lua" xml:id="box.update"> - box.space.<replaceable>space-name</replaceable>:update{<replaceable>key, format, {field_no, value}...</replaceable>) + box.space.<replaceable>space-name</replaceable>:update({<replaceable>key {, operator, field_no, value}...</replaceable>}) </emphasis> </term> <listitem> @@ -647,46 +647,37 @@ tarantool> <userinput>box.space.space55.index.primary:rename('secondary')</useri </para> <para> The <code>update</code> function supports operations on fields — - assignment, arithmetic operations (the field must be numeric), - cutting and pasting fragments of a field, — as well as - operations on a tuple: push and pop of a field at the tail of - a tuple, deletion and insertion of a field. Multiple - operations can be combined into a single update, and in this - case they are performed atomically. Each operation expects - field number as its first argument. When a sequence of changes - is present, field identifier in each operation is assumed to - be relative to the most recent state of the tuple, i.e. as if + assignment, arithmetic (if the field is unsigned numeric), + cutting and pasting fragments of a field, + deletng or inserting a field. Multiple + operations can be combined in a single update request, and in this + case they are performed atomically and sequentially. Each operation requires + specification of a field number. When multiple operations + are present, the field number for each operation is assumed to + be relative to the most recent state of the tuple, that is, as if all previous operations in a multi-operation update have - already been applied. In other words, it's always safe to + already been applied. In other words, it is always safe to merge multiple <code>update</code> invocations into a single invocation, with no change in semantics. </para> <para> Parameters: <code>space-name</code>, <code>key</code> = primary-key field values, must be passed as a Lua table if key is multi-part, - <code>format</code> = a sequence of - pairs of characters, where the first character in each pair - is the operation specifier, and the second character in - each pair is the operation argument. - The <code>{field_no, value}</code> arguments are the - field numbers of affected fields and applicable values. + <code>{operator, field_no, value}</code> = a group of arguments + for each operation, indicating what the operation is, what field + the operation will apply to, and what value will be applied. For some operations the field number can be -1, meaning the last field in the tuple. - There must be a pair of {field_no, value} arguments - for each character pair in the format argument. - The format and {field_no, value} arguments are passed to - <code>pickle.pack()</code> and the result is sent - to <code>box.process()</code>. - Possible operation specifiers are: <quote>+</quote> + Possible operators are: <quote>+</quote> for addition, <quote>-</quote> for subtraction, <quote>&</quote> for bitwise AND, <quote>|</quote> for bitwise OR, <quote>^</quote> for bitwise exclusive OR (XOR), <quote>:</quote> - for string splice, <quote>!</quote> for insertion. - Possible operation arguments are: <quote>p</quote>. - Thus in the instruction <code>s:update{44,, {{'+p,1,55},{=p',3,'x'}})</code> + for string splice, <quote>!</quote> for insert, + <quote>#</quote> for delete. + Thus in the instruction <code>s:update{44,, {{'+,1,55},{=',3,'x'}})</code> the primary-key value is 44, - the formats are '+p' and '=p' + the operators are '+' and '=' meaning "add a value to a field and then assign a value to a field", the first affected field is field 1 and the value which will be added to it is 55, the second affected field @@ -764,11 +755,11 @@ box.space.tester:update({999}, {{'=', 2, 'XYZ'}}) #In the following update ... # The third argument is ':', that is, this is the example of splice. # The fourth argument is 2 because the change will occur in field[2]. -# The fifth argument is 1 because deletion will begin with the second byte. +# The fifth argument is 2 because deletion will begin with the second byte. # The sixth argument is 1 because the number of bytes to delete is 1. # The seventh argument is '!!' because '!!' is to be added at this position. # Therefore, after the following update, field[1] = 999, field[2] = 'X!!Z'. -box.space.tester:update({999}, {{':', 2, 1, 1, '!!'}}) +box.space.tester:update({999}, {{':', 2, 2, 1, '!!'}}) </programlisting> </para> @@ -2096,6 +2087,46 @@ tarantool> <userinput>tmp = ''; for k, v in t:pairs() do tmp = tmp .. v end</ tarantool> <userinput>tmp</userinput> --- - Fld#1Fld#2Fld#3Fld#4Fld#5 +...</programlisting> + </listitem> + </varlistentry> + + <varlistentry> + <term> + <emphasis role="lua"><replaceable>tuple-value</replaceable>:update(<replaceable>{{format, field_no, value}...}</replaceable>)</emphasis> + </term> + <listitem> + <para> + Update a tuple. + </para> + <para> + This function updates a tuple which is not in a space. + Compare the function + <code>box.space.<replaceable>space-name</replaceable>:update{<replaceable>key, format, {field_no, value}...</replaceable>)</code>, + which updates a tuple in a space. + </para> + <para> + Parameters: briefly: + <code>format</code> indicates the type of update operation such as '=' for 'assign new value', + <code>field_no</code> indicates the field number to change such as 2 for field number 2, + <code>value</code> indicates the string which operates on the field such as 'B' for a new assignable value = 'B'. + For details: see the description for <code>format</code>, <code>field_no</code>, and <code>value</code> + in the section <olink targetptr="box.update"><code>box.space.<replaceable>space-name</replaceable>:update{<replaceable>key, format, {field_no, value}...</replaceable>)</code></olink>. + </para> + <para> + Returns: (type = tuple) the new tuple. + </para> + <para> + In the following example, a tuple named t is created + and then its second field is updated to equal 'B'. + </para> + <bridgehead renderas="sect4">Example</bridgehead> +<programlisting>tarantool> <userinput>t = box.tuple.new({'Fld#1','Fld#2','Fld#3','Fld#4','Fld#5'})</userinput> +--- +... +tarantool> <userinput>t:update({{'=',2,'B'}})</userinput> +--- +- ['Fld#1', 'B', 'Fld#3', 'Fld#4', 'Fld#5'] ...</programlisting> </listitem> </varlistentry> diff --git a/doc/user/replication.xml b/doc/user/replication.xml index c2856b36e5874370e625a3534334f78b97963d6d..668a80bd6a1b800db0fe88d7a206c24af18c13d5 100644 --- a/doc/user/replication.xml +++ b/doc/user/replication.xml @@ -68,8 +68,8 @@ identifier which is unique within the cluster, known as the To prepare the master for connections from the replica, it's only necessary to include "listen" in the initial <code>box.cfg</code> request, for example <code>box.cfg{listen=3301}</code>. - A master with enabled "listen" uri can accept connections - from as many replicas as necessary on that uri. Each replica + A master with enabled "listen" URI can accept connections + from as many replicas as necessary on that URI. Each replica has its own replication state. </para> </section> @@ -120,7 +120,7 @@ identifier which is unique within the cluster, known as the <para> However, once a master failure is detected, the recovery is simple: declare that the replica is now the new master, - by saying <code>box.cfg{... listen=uri}</code>. + by saying <code>box.cfg{... listen=URI}</code>. Then, if there are updates on the old master that were not propagated before the old master went down, they would have to be re-applied manually. diff --git a/doc/user/stored-procedures.xml b/doc/user/stored-procedures.xml index 0aa7573a2dc6f05db9a127b0c7ffe4828e13707d..75f38d7bd02a37ff7a412f358714c67a1a3dae1b 100644 --- a/doc/user/stored-procedures.xml +++ b/doc/user/stored-procedures.xml @@ -288,7 +288,8 @@ administration with the built-in packages. <link xlink:href="https://en.wikipedia.org/wiki/Sha-2">SHA-2</link>) as well as a checksum function - (<link xlink:href="https://en.wikipedia.org/wiki/Cyclic_redundancy_check">CRC32</link>). + (<link xlink:href="https://en.wikipedia.org/wiki/Cyclic_redundancy_check">CRC32</link>) + and two functions for <link xlink:href="https://en.wikipedia.org/wiki/Base64">base64</link>. The functions in <code>digest</code> are: <informaltable> <tgroup cols="2" align="left" colsep="1" rowsep="0"> @@ -314,6 +315,8 @@ administration with the built-in packages. <row><entry><code>digest.md4_hex(<replaceable>string</replaceable>)</code></entry><entry> Returns hexadecimal of a digest calculated with md4.</entry></row> <row><entry><code>digest.md5(<replaceable>string</replaceable>)</code></entry><entry> Returns 256-bit digest made with MD5.</entry></row> <row><entry><code>digest.md5_hex(<replaceable>string</replaceable>)</code></entry><entry> Returns hexadecimal of a digest calculated with md5.</entry></row> + <row><entry><code>digest.base64_encode(<replaceable>string</replaceable>)</code></entry><entry> Returns base64 encoding from a regular string.</entry></row> + <row><entry><code>digest.base64_decode(<replaceable>string</replaceable>)</code></entry><entry> Returns a regular string from a base64 encoding.</entry></row> </tbody> </tgroup> </informaltable> @@ -2503,9 +2506,9 @@ end <listitem> <para> Listen on host:port. The primary way of listening for incoming - requests is via the host and port, or uri, specified in + requests is via the host and port, or URI, specified in <code>box.cfg{listen=...}</code>. The alternative way of - listening is via the host and port, or uri, specified in + listening is via the host and port, or URI, specified in <code>console.listen(...)</code>. This alternative way is called "administrative" or simply "admin port". </para> diff --git a/doc/www/pelicanconf.py b/doc/www/pelicanconf.py index 51305aa05c7b1c5bfc0f61d276b0f41c7a17f0e2..c27b653152dd7e6b55c04fadce786bff7cd651f3 100644 --- a/doc/www/pelicanconf.py +++ b/doc/www/pelicanconf.py @@ -39,13 +39,15 @@ TAG_SAVE_AS = '' STATIC_PATHS = [ 'robots.txt', - 'ycsb' + 'ycsb', + 'js/highcharts.js', + 'js/tabs.js' ] EXTRA_PATH_METADATA = { - 'robots.txt' : { 'path': 'robots.txt' }, - 'ycsb' : { 'path': 'ycsb' }, - 'js/highlight.js': { 'path': 'highlight.js'}, - 'js/tabs.js' : { 'path': 'tabs.js' }, + 'robots.txt' : { 'path': 'robots.txt' }, + 'ycsb' : { 'path': 'ycsb' }, + 'js/highcharts.js': { 'path': 'highcharts.js'}, + 'js/tabs.js' : { 'path': 'tabs.js' }, } # Uncomment following line if you want document-relative URLs when developing diff --git a/extra/dist/dist.lua b/extra/dist/dist.lua index 8a27a76e8cfd50aa0a39eb7451be975519699e7f..ccf1c4503140ad18a205c5687a1209ba3546d57f 100755 --- a/extra/dist/dist.lua +++ b/extra/dist/dist.lua @@ -61,14 +61,35 @@ Each instance can be controlled by C<dist.lua>: dist.lua logrotate instance_name +=head2 Enter instance admin console + + dist.lua enter instance_name + +=head2 status + + dist.lua status instance_name + +Check if instance is up. + +If pid file exists and control socket exists and control socket is alive +returns code C<0>. + +Return code != 0 in other cases. Can complain in log (stderr) if pid file +exists and socket doesn't, etc. + + +=head2 separate instances control + +If You use SysV init, You can use symlink from +C<dist.lua> to C</etc/init.d/instance_name[.lua]>. +C<dist.lua> detects if it is started by symlink and uses +instance_name as C<`basename $0 .lua`>. =head1 COPYRIGHT Copyright (C) 2010-2013 Tarantool AUTHORS: please see AUTHORS file. - - =cut ]] @@ -81,16 +102,56 @@ local console = require 'console' local socket = require 'socket' local ffi = require 'ffi' local os = require 'os' +local fiber = require 'fiber' ffi.cdef[[ int kill(int pid, int sig); ]] -if arg[1] == nil or arg[2] == nil then - log.error("Usage: dist.lua {start|stop|logrotate} instance") - os.exit(-1) + +local available_commands = { + 'start', + 'stop', + 'logrotate', + 'status', + 'enter', + 'restart' +} + +local function usage() + log.error("Usage: %s {%s} instance_name", + arg[0], table.concat(available_commands, '|')) + os.exit(1) end + local cmd = arg[1] -local instance = fio.basename(arg[2], '.lua') + +local valid_cmd = false +for _, vcmd in pairs(available_commands) do + if cmd == vcmd then + valid_cmd = true + break + end +end + +if not valid_cmd then + usage() +end + +local instance +if arg[2] == nil then + local istat = fio.lstat(arg[0]) + if istat == nil then + log.error("Can't stat %s: %s", arg[0], errno.strerror()) + os.exit(1) + end + if not istat:is_link() then + usage() + end + instance = fio.basename(arg[0], '.lua') + arg[2] = instance +else + instance = fio.basename(arg[2], '.lua') +end -- shift argv to remove 'tarantoolctl' from arg[0] for i = 0, 128 do @@ -200,20 +261,18 @@ wrapper_cfg = function(cfg) return res end -if cmd == 'start' then - box.cfg = wrapper_cfg - dofile(instance_lua) - -elseif cmd == 'stop' then +function stop() + log.info("Stopping instance...") if fio.stat(force_cfg.pid_file) == nil then log.error("Process is not running (pid: %s)", force_cfg.pid_file) - os.exit(-1) + return 0 end local f = fio.open(force_cfg.pid_file, 'O_RDONLY') if f == nil then log.error("Can't read pid file %s: %s", force_cfg.pid_file, errno.strerror()) + return -1 end local str = f:read(64) @@ -224,14 +283,34 @@ elseif cmd == 'stop' then if pid == nil or pid <= 0 then log.error("Broken pid file %s", force_cfg.pid_file) fio.unlink(force_cfg.pid_file) - os.exit(-1) + return -1 end if ffi.C.kill(pid, 15) < 0 then log.error("Can't kill process %d: %s", pid, errno.strerror()) fio.unlink(force_cfg.pid_file) + return -1 end - os.exit(-1) + return 0 +end + +function start() + log.info("Starting instance...") + box.cfg = wrapper_cfg + dofile(instance_lua) +end + + +if cmd == 'start' then + os.exit(start()) + +elseif cmd == 'stop' then + os.exit(stop()) + +elseif cmd == 'restart' then + stop() + fiber.sleep(1) + start() elseif cmd == 'logrotate' then if fio.stat(console_sock) == nil then @@ -253,6 +332,52 @@ elseif cmd == 'logrotate' then s:read({ '[.][.][.]' }, 2) os.exit(0) + +elseif cmd == 'enter' then + if fio.stat(console_sock) == nil then + log.error("Can't connect to %s (socket not found)", console_sock) + os.exit(-1) + end + + log.info('Connecting to %s', console_sock) + + local cmd = string.format( + "require('console').connect('%s')", console_sock) + + console.on_start( function(self) self:eval(cmd) end ) + console.on_client_disconnect( function(self) self.running = false end ) + console.start() + os.exit(0) +elseif cmd == 'status' then + if fio.stat(force_cfg.pid_file) == nil then + if errno() == errno.ENOENT then + os.exit(1) + end + log.error("Cant access pidfile %s: %s", + force_cfg.pid_file, errno.strerror()) + end + + if fio.stat(console_sock) == nil then + if errno() == errno.ENOENT then + log.warn("pidfile is exists, but control socket (%s) isn't", + console_sock) + os.exit(2) + end + end + + local s = socket.tcp_connect('unix/', console_sock) + if s == nil then + if errno() ~= errno.EACCES then + log.warn("Can't access control socket %s: %s", console_sock, + errno.strerror()) + os.exit(3) + else + os.exit(0) + end + end + + s:close() + os.exit(0) else log.error("Unknown command '%s'", cmd) os.exit(-1) diff --git a/src/box/access.h b/src/box/access.h index b0cbf366c4cafdc94261813ff62523d5fc363e50..385807aa4e37729031181450e945b76952ee21df 100644 --- a/src/box/access.h +++ b/src/box/access.h @@ -142,9 +142,12 @@ user_by_name(const char *name, uint32_t len); #define user() \ ({ \ struct session *s = session(); \ - uint8_t auth_token = s ? s->auth_token : (int) ADMIN; \ - struct user *u = &users[auth_token]; \ - assert(u->auth_token == auth_token); \ + struct user *u = &users[s->auth_token]; \ + if (u->auth_token != s->auth_token || \ + u->uid != s->uid) { \ + tnt_raise(ClientError, ER_NO_SUCH_USER, \ + int2str(s->uid)); \ + } \ u; \ }) diff --git a/src/box/box.cc b/src/box/box.cc index a4af9caa02bd1d899767e99a25a3db0a49dca52c..8c9f0b41c64e73de8d6a812b3c3bb5d456e4e713 100644 --- a/src/box/box.cc +++ b/src/box/box.cc @@ -90,10 +90,22 @@ static void process_ro(struct port *port, struct request *request) { if (!iproto_type_is_select(request->type)) - tnt_raise(LoggedError, ER_SECONDARY); + tnt_raise(LoggedError, ER_READONLY); return process_rw(port, request); } +void +box_set_ro(bool ro) +{ + box_process = ro ? process_ro : process_rw; +} + +bool +box_is_ro(void) +{ + return box_process == process_ro; +} + static void recover_row(void *param __attribute__((unused)), struct xrow_header *row) { @@ -237,7 +249,6 @@ box_leave_local_standby_mode(void *data __attribute__((unused))) stat_cleanup(stat_base, IPROTO_TYPE_DML_MAX); box_set_wal_mode(cfg_gets("wal_mode")); - box_process = process_rw; if (recovery_has_remote(recovery)) recovery_follow_remote(recovery); diff --git a/src/box/box.h b/src/box/box.h index 8bb7f8751759d976a8ceb68bf10399583b88fe08..970dad6d78e101a27876755f1fe77989e5535e66 100644 --- a/src/box/box.h +++ b/src/box/box.h @@ -62,7 +62,12 @@ void box_free(void); typedef void (*box_process_func)(struct port *port, struct request *request); /** For read-write operations. */ extern box_process_func box_process; -/** For read-only port. */ + +void +box_set_ro(bool ro); + +bool +box_is_ro(void); /** Non zero if snapshot is in progress. */ extern int snapshot_pid; diff --git a/src/box/cluster.cc b/src/box/cluster.cc index 5f142605a6ae0a8086165eb6824972c92bf60e2c..356f49ff313f4fa86dfe74d4322b4fbe4edf9c21 100644 --- a/src/box/cluster.cc +++ b/src/box/cluster.cc @@ -26,6 +26,7 @@ * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. */ +#include "box.h" #include "cluster.h" #include "recovery.h" #include "exception.h" @@ -65,5 +66,6 @@ cluster_set_server(const tt_uuid *server_uuid, uint32_t server_id) /* Assign local server id */ assert(r->server_id == 0); r->server_id = server_id; + box_set_ro(false); } } diff --git a/src/box/engine_memtx.cc b/src/box/engine_memtx.cc index ade14477d2fffcd524741bc0e59da8828150497f..24f30946565638c2b25bda4935a5ef1a06314337 100644 --- a/src/box/engine_memtx.cc +++ b/src/box/engine_memtx.cc @@ -120,7 +120,7 @@ MemtxFactory::dropIndex(Index *index) index->initIterator(it, ITER_ALL, NULL, 0); struct tuple *tuple; while ((tuple = it->next(it))) - tuple_ref(tuple, -1); + tuple_unref(tuple); } void diff --git a/src/box/engine_sophia.cc b/src/box/engine_sophia.cc index 42318828566ff01fe04f35096cd3a7d7e821c317..71cb5e9823b73249136dc9c3029db82ec1473cfb 100644 --- a/src/box/engine_sophia.cc +++ b/src/box/engine_sophia.cc @@ -217,6 +217,6 @@ SophiaFactory::txnFinish(struct txn *txn) #if 0 struct txn_stmt *stmt = txn_stmt(txn); if (stmt->new_tuple) - tuple_ref(stmt->new_tuple, -1); + tuple_unref(stmt->new_tuple); #endif } diff --git a/src/box/hash_index.cc b/src/box/hash_index.cc index fb3a4d48484dbb630f55133e903610b2aeeb5899..d8a8db0525af6f9b8e11c03fde870e73db9460f6 100644 --- a/src/box/hash_index.cc +++ b/src/box/hash_index.cc @@ -53,6 +53,40 @@ mh_index_eq_key(const char *key, struct tuple *const *tuple, key_def) == 0; } +static inline uint32_t +mh_hash_field(uint32_t *ph1, uint32_t *pcarry, const char **field, + enum field_type type) +{ + const char *f = *field; + uint32_t size; + + switch (type) { + case STRING: + /* + * (!) MP_STR fields hashed **excluding** MsgPack format + * indentifier. We have to do that to keep compatibility + * with old third-party MsgPack (spec-old.md) implementations. + * \sa https://github.com/tarantool/tarantool/issues/522 + */ + f = mp_decode_str(field, &size); + break; + default: + mp_next(field); + size = *field - f; /* calculate the size of field */ + /* + * (!) All other fields hashed **including** MsgPack format + * identifier (e.g. 0xcc). This was done **intentionally** + * for performance reasons. Please follow MsgPack specification + * and pack all your numbers to the most compact representation. + * If you still want to add support for broken MsgPack, + * please don't forget to patch tuple_compare_field(). + */ + break; + } + assert(size < INT32_MAX); + PMurHash32_Process(ph1, pcarry, f, size); + return size; +} static inline uint32_t mh_index_hash(struct tuple *const *tuple, const struct key_def *key_def) @@ -76,12 +110,7 @@ mh_index_hash(struct tuple *const *tuple, const struct key_def *key_def) for ( ; part < key_def->parts + key_def->part_count; part++) { const char *field = tuple_field(*tuple, part->fieldno); - const char *f = field; - mp_next(&f); - uint32_t size = f - field; - assert(size < INT32_MAX); - PMurHash32_Process(&h, &carry, field, size); - total_size += size; + total_size += mh_hash_field(&h, &carry, &field, part->type); } return PMurHash32_Result(h, carry, total_size); @@ -99,13 +128,15 @@ mh_index_hash_key(const char *key, const struct key_def *key_def) return ((uint32_t)((val)>>33^(val)^(val)<<11)); } - /* Calculate key size */ - const char *k = key; - for (uint32_t part = 0; part < key_def->part_count; part++) { - mp_next(&k); - } + uint32_t h = HASH_SEED; + uint32_t carry = 0; + uint32_t total_size = 0; + + /* Hash fields part by part (see mh_hash_field() comments) */ + for ( ; part < key_def->parts + key_def->part_count; part++) + total_size += mh_hash_field(&h, &carry, &key, part->type); - return PMurHash32(HASH_SEED, key, k - key); + return PMurHash32_Result(h, carry, total_size); } #define mh_int_t uint32_t diff --git a/src/box/lua/info.cc b/src/box/lua/info.cc index ec13926417ad5f5ae6fd001d809b03f2394dbe47..f1616df17f60b5bb94d5b3490b62e34f65a7ba72 100644 --- a/src/box/lua/info.cc +++ b/src/box/lua/info.cc @@ -59,11 +59,6 @@ lbox_info_recovery_last_update_tstamp(struct lua_State *L) static int lbox_info_server(struct lua_State *L) { - if (recovery->server_id == 0) { - lua_pushnil(L); - return 1; - } - lua_createtable(L, 0, 2); lua_pushliteral(L, "id"); lua_pushinteger(L, recovery->server_id); @@ -72,8 +67,11 @@ lbox_info_server(struct lua_State *L) lua_pushlstring(L, tt_uuid_str(&recovery->server_uuid), UUID_STR_LEN); lua_settable(L, -3); lua_pushliteral(L, "lsn"); - luaL_pushnumber64(L, vclock_get(&recovery->vclock, - recovery->server_id)); + luaL_pushinumber64(L, vclock_get(&recovery->vclock, + recovery->server_id)); + lua_settable(L, -3); + lua_pushliteral(L, "ro"); + lua_pushboolean(L, box_is_ro()); lua_settable(L, -3); return 1; diff --git a/src/box/lua/load_cfg.lua b/src/box/lua/load_cfg.lua index 47496a3e64d8c2c0834214b20cddab28acf7b028..126bc550ce075005651b18de318ada60f0b1f2a1 100644 --- a/src/box/lua/load_cfg.lua +++ b/src/box/lua/load_cfg.lua @@ -52,7 +52,7 @@ local default_cfg = { username = nil , coredump = false, - -- snap_daemon + -- snapshot_daemon snapshot_period = 0, -- 0 = disabled snapshot_count = 6, } @@ -99,9 +99,9 @@ local dynamic_cfg = { too_long_threshold = ffi.C.box_set_too_long_threshold, snap_io_rate_limit = ffi.C.box_set_snap_io_rate_limit, - -- snap_daemon - snapshot_period = box.internal.snap_daemon.set_snapshot_period, - snapshot_count = box.internal.snap_daemon.set_snapshot_count, + -- snapshot_daemon + snapshot_period = box.internal.snapshot_daemon.set_snapshot_period, + snapshot_count = box.internal.snapshot_daemon.set_snapshot_count, } local function prepare_cfg(table) @@ -163,11 +163,15 @@ local box = require('box') local box_configured = {} for k, v in pairs(box) do box_configured[k] = v - box[k] = nil + -- box.net.box uses box.error and box.internal + if k ~= 'error' and k ~= 'internal' then + box[k] = nil + end end setmetatable(box, { __index = function(table, index) + error(debug.traceback("Please call box.cfg{} first")) error("Please call box.cfg{} first") end }) @@ -194,7 +198,7 @@ function box.cfg(cfg) }) ffi.C.load_cfg() - box.internal.snap_daemon.start() + box.internal.snapshot_daemon.start() end jit.off(box.cfg) jit.off(reload_cfg) diff --git a/src/box/lua/schema.lua b/src/box/lua/schema.lua index 3e82ade171751248ee6336cf950bb9e7fe93b620..c8712ac8b8ac2aa5e02a1fdf7a6f068a2f095ac1 100644 --- a/src/box/lua/schema.lua +++ b/src/box/lua/schema.lua @@ -239,11 +239,9 @@ box.schema.space.drop = function(space_id) local v = keys[i] _index:delete{v[1], v[2]} end - local privs = _priv:select{} + local privs = _priv.index.object:select{'space', space_id} for k, tuple in pairs(privs) do - if tuple[3] == 'space' and tuple[4] == space_id then - box.schema.user.revoke(tuple[2], tuple[5], tuple[3], tuple[4]) - end + box.schema.user.revoke(tuple[2], tuple[5], tuple[3], tuple[4]) end if _space:delete{space_id} == nil then box.error(box.error.NO_SUCH_SPACE, '#'..tostring(space_id)) @@ -303,6 +301,7 @@ box.schema.index.create = function(space_id, name, options) parts = 'table', unique = 'boolean', id = 'number', + if_not_exists = 'boolean', } local options_defaults = { type = 'tree', @@ -315,6 +314,13 @@ box.schema.index.create = function(space_id, name, options) options.parts = update_index_parts(options.parts) local _index = box.space[box.schema.INDEX_ID] + if _index.index.name:get{space_id, name} then + if options.if_not_exists then + return box.space[space_id].index[name], "not created" + else + box.error(box.error.INDEX_EXISTS, name) + end + end local unique = options.unique and 1 or 0 local part_count = bit.rshift(#options.parts, 1) @@ -435,19 +441,24 @@ end -- Change one-based indexing in update commands to zero-based. -- local function normalize_update_ops(ops) + if type(ops) ~= 'table' then + return ops; + end for _, op in ipairs(ops) do - if op[1] == ':' then - -- fix offset for splice - if op[3] > 0 then - op[3] = op[3] - 1 - elseif op[3] == 0 then - box.error(box.error.SPLICE, op[2], "offset is out of bound") + if type(op) == 'table' then + if op[1] == ':' then + -- fix offset for splice + if op[3] > 0 then + op[3] = op[3] - 1 + elseif op[3] == 0 then + box.error(box.error.SPLICE, op[2], "offset is out of bound") + end + end + if op[2] > 0 then + op[2] = op[2] - 1 + elseif op[2] == 0 then + box.error(box.error.NO_SUCH_FIELD, op[2]) end - end - if op[2] > 0 then - op[2] = op[2] - 1 - elseif op[2] == 0 then - box.error(box.error.NO_SUCH_FIELD, op[2]) end end return ops @@ -933,11 +944,9 @@ box.schema.func.drop = function(name) local _func = box.space[box.schema.FUNC_ID] local _priv = box.space[box.schema.PRIV_ID] local fid = object_resolve('function', name) - local privs = _priv:select{} + local privs = _priv.index.object:select{'function', fid} for k, tuple in pairs(privs) do - if tuple[3] == 'function' and tuple[4] == fid then - box.schema.user.revoke(tuple[2], tuple[5], tuple[3], tuple[4]) - end + box.schema.user.revoke(tuple[2], tuple[5], tuple[3], tuple[4]) end _func:delete{fid} end diff --git a/src/box/lua/snapshot_daemon.lua b/src/box/lua/snapshot_daemon.lua index 6dc9cff6ee46bdec3935cb5a6696c63e855e87ca..8b8938d2b6196de60a856e7e2cf227d097d64579 100644 --- a/src/box/lua/snapshot_daemon.lua +++ b/src/box/lua/snapshot_daemon.lua @@ -1,4 +1,4 @@ --- snap_daemon.lua (internal file) +-- snapshot_daemon.lua (internal file) do local log = require 'log' @@ -7,7 +7,7 @@ do local yaml = require 'yaml' local errno = require 'errno' - local PREFIX = 'snap_daemon' + local PREFIX = 'snapshot_daemon' local daemon = { status = 'stopped' } @@ -239,6 +239,10 @@ do end, set_snapshot_count = function(snapshot_count) + if math.floor(snapshot_count) ~= snapshot_count then + box.error(box.error.PROC_LUA, + "snapshot_count must be integer") + end local daemon = box.internal[PREFIX] or daemon log.info("new snapshot count is %s", tostring(snapshot_count)) diff --git a/src/box/lua/tuple.cc b/src/box/lua/tuple.cc index a5057c1bec84f1109301c397d42044102ccd002f..f0c204c4bbdec6d62ebdb804a331cb86e46464d3 100644 --- a/src/box/lua/tuple.cc +++ b/src/box/lua/tuple.cc @@ -106,7 +106,7 @@ static int lbox_tuple_gc(struct lua_State *L) { struct tuple *tuple = lua_checktuple(L, 1); - tuple_ref(tuple, -1); + tuple_unref(tuple); return 0; } @@ -305,7 +305,7 @@ lbox_pushtuple(struct lua_State *L, struct tuple *tuple) *ptr = tuple; lua_pushcfunction(L, lbox_tuple_gc); luaL_setcdatagc(L, -2); - tuple_ref(tuple, 1); + tuple_ref(tuple); } else { return lua_pushnil(L); } diff --git a/src/box/lua/tuple.lua b/src/box/lua/tuple.lua index 515c792f47aeaa9ef511d531df7e13214676016f..cd0991f6a5ab6b0f66801bcd2065593fda80c0a1 100644 --- a/src/box/lua/tuple.lua +++ b/src/box/lua/tuple.lua @@ -16,8 +16,10 @@ struct tuple char data[0]; } __attribute__((packed)); +int +tuple_ref_nothrow(struct tuple *tuple); void -tuple_ref(struct tuple *tuple, int count); +tuple_unref(struct tuple *tuple); uint32_t tuple_field_count(const struct tuple *tuple); const char * @@ -50,13 +52,16 @@ local builtin = ffi.C local const_struct_tuple_ref_t = ffi.typeof('const struct tuple&') local tuple_gc = function(tuple) - builtin.tuple_ref(tuple, -1) + builtin.tuple_unref(tuple) end local tuple_bless = function(tuple) - -- update in-place, do not spent time calling tuple_ref + -- tuple_ref(..) may throw to prevent reference counter to overflow, + -- which is not allowed in ffi call, so we'll use nothrow version + if (builtin.tuple_ref_nothrow(tuple) ~= 0) then + box.error(); + end local tuple2 = ffi.gc(ffi.cast(const_struct_tuple_ref_t, tuple), tuple_gc) - tuple._refs = tuple._refs + 1 return tuple2 end diff --git a/src/box/recovery.cc b/src/box/recovery.cc index 6ce57146fc370d8f1e4ba1b46caa8942b9dcc464..e782e96bf8cd11a01a5713ca83a1f1e8be48ddfb 100644 --- a/src/box/recovery.cc +++ b/src/box/recovery.cc @@ -1154,7 +1154,7 @@ snapshot_write_row(struct log_io *l, struct xrow_header *row) if (l->rows % 100000 == 0) say_crit("%.1fM rows written", l->rows / 1000000.); - region_free_after(&fiber()->gc, 128 * 1024); + fiber_gc(); if (r->snap_io_rate_limit != UINT64_MAX) { if (last == 0) { diff --git a/src/box/sophia_index.cc b/src/box/sophia_index.cc index b50bf1c2c128acfe6a54323428d6387a01ac8bc0..1b2ee0536dc149d6449952008729a139bab9dcc4 100644 --- a/src/box/sophia_index.cc +++ b/src/box/sophia_index.cc @@ -128,7 +128,7 @@ sophia_gettuple(void *db, const char *key, size_t keysize, void *value = sp_get(result, "value", &valuesize); struct tuple *ret = tuple_new(format, (char*)value, (char*)value + valuesize); - tuple_ref(ret, 1); + tuple_ref(ret); return ret; } @@ -197,7 +197,7 @@ SophiaIndex::random(uint32_t rnd) const struct tuple *ret = tuple_new(space->format, (char*)value, (char*)value + valuesize); - tuple_ref(ret, 1); + tuple_ref(ret); return ret; } @@ -282,14 +282,14 @@ SophiaIndex::replace(struct tuple *old_tuple, struct tuple *new_tuple, sophia_check_dup(key_def, old_tuple, dup_tuple, mode); if (errcode) { if (dup_tuple) - tuple_ref(dup_tuple, -1); + tuple_unref(dup_tuple); tnt_raise(ClientError, errcode, index_id(this)); } void *o = sp_object(db); if (o == NULL) { if (dup_tuple) - tuple_ref(dup_tuple, -1); + tuple_unref(dup_tuple); tnt_raise(ClientError, ER_SOPHIA, sp_error(db)); } sp_set(o, "key", key, keysize); @@ -297,7 +297,7 @@ SophiaIndex::replace(struct tuple *old_tuple, struct tuple *new_tuple, int rc = sp_set(db, o); if (rc == -1) { if (dup_tuple) - tuple_ref(dup_tuple, -1); + tuple_unref(dup_tuple); tnt_raise(ClientError, ER_SOPHIA, sp_error(db)); } if (dup_tuple) @@ -352,7 +352,7 @@ sophia_iterator_next(struct iterator *ptr) const char *value = (const char*)sp_get(o, "value", &valuesize); struct tuple *ret = tuple_new(it->space->format, value, value + valuesize); - tuple_ref(ret, 1); + tuple_ref(ret); return ret; } diff --git a/src/box/tuple.cc b/src/box/tuple.cc index 69d2dff18976e64d295c490b30f27f2fb99023a1..4c656f84c8b1a98910f805380f6c73949eb54a63 100644 --- a/src/box/tuple.cc +++ b/src/box/tuple.cc @@ -284,19 +284,12 @@ tuple_delete(struct tuple *tuple) } /** - * Add count to tuple's reference counter. - * When the counter goes down to 0, the tuple is destroyed. - * - * @pre tuple->refs + count >= 0 + * Throw and exception about tuple reference counter overflow. */ void -tuple_ref(struct tuple *tuple, int count) +tuple_ref_exception() { - assert(tuple->refs + count >= 0); - tuple->refs += count; - - if (tuple->refs == 0) - tuple_delete(tuple); + tnt_raise(ClientError, ER_TUPLE_REF_OVERFLOW); } const char * diff --git a/src/box/tuple.h b/src/box/tuple.h index 6e79795feed8fb28c8fefc539ae4fd5cf0833744..959a75da265aa6dca1af5e4e3d2d34673e74b8a7 100644 --- a/src/box/tuple.h +++ b/src/box/tuple.h @@ -32,6 +32,7 @@ #include "key_def.h" /* for enum field_type */ enum { FORMAT_ID_MAX = UINT16_MAX - 1, FORMAT_ID_NIL = UINT16_MAX }; +enum { FORMAT_REF_MAX = INT32_MAX, TUPLE_REF_MAX = UINT16_MAX }; struct tbuf; @@ -123,6 +124,8 @@ static inline void tuple_format_ref(struct tuple_format *format, int count) { assert(format->refs + count >= 0); + assert((uint64_t)format->refs + count <= FORMAT_REF_MAX); + format->refs += count; if (format->refs == 0) tuple_format_delete(format); @@ -171,21 +174,73 @@ struct tuple * tuple_new(struct tuple_format *format, const char *data, const char *end); /** - * Change tuple reference counter. If it has reached zero, free the tuple. + * Free the tuple. + * @pre tuple->refs == 0 + */ +void +tuple_delete(struct tuple *tuple); + +/** + * Throw and exception about tuple reference counter overflow. + */ +void +tuple_ref_exception(); + +/** + * Increment tuple reference counter. + * Throws if overflow detected. * * @pre tuple->refs + count >= 0 */ -extern "C" void -tuple_ref(struct tuple *tuple, int count); +extern "C" inline void +tuple_ref(struct tuple *tuple) +{ + if (tuple->refs + 1 > TUPLE_REF_MAX) + tuple_ref_exception(); -void -tuple_delete(struct tuple *tuple); + tuple->refs++; +} + +/** + * Increment tuple reference counter. + * Returns -1 if overflow detected, 0 otherwise + * + * @pre tuple->refs + count >= 0 + */ +extern "C" inline int +tuple_ref_nothrow(struct tuple *tuple) +{ + try { + tuple_ref(tuple); + } catch (Exception *e) { + return -1; + } + return 0; +} + +/** + * Decrement tuple reference counter. If it has reached zero, free the tuple. + * + * @pre tuple->refs + count >= 0 + */ +extern "C" inline void +tuple_unref(struct tuple *tuple) +{ + assert(tuple->refs - 1 >= 0); + + tuple->refs--; + + if (tuple->refs == 0) + tuple_delete(tuple); +} /** Make tuple references exception-friendly in absence of @finally. */ struct TupleGuard { struct tuple *tuple; - TupleGuard(struct tuple *arg) :tuple(arg) {} - ~TupleGuard() { if (tuple->refs == 0) tuple_delete(tuple); } + TupleGuard(struct tuple *arg) :tuple(arg) { tuple_ref(tuple); } + ~TupleGuard() { tuple_unref(tuple); } + TupleGuard(const TupleGuard&) = delete; + void operator=(const TupleGuard&) = delete; }; /** diff --git a/src/box/txn.cc b/src/box/txn.cc index c350593c9d7ce40cfa64e91ff9b8e7a636d8780d..180980e2c37d06314ec7ef16dd4718009128a2d5 100644 --- a/src/box/txn.cc +++ b/src/box/txn.cc @@ -84,7 +84,7 @@ txn_replace(struct txn *txn, struct space *space, stmt->old_tuple = space_replace(space, old_tuple, new_tuple, mode); if (new_tuple) { stmt->new_tuple = new_tuple; - tuple_ref(stmt->new_tuple, 1); + tuple_ref(stmt->new_tuple); } stmt->space = space; /** @@ -225,7 +225,7 @@ txn_finish(struct txn *txn) struct txn_stmt *stmt; rlist_foreach_entry(stmt, &txn->stmts, next) { if (stmt->old_tuple) - tuple_ref(stmt->old_tuple, -1); + tuple_unref(stmt->old_tuple); if (stmt->space) stmt->space->engine->factory->txnFinish(txn); } @@ -254,7 +254,7 @@ txn_rollback_stmt() space_replace(stmt->space, stmt->new_tuple, stmt->old_tuple, DUP_INSERT); if (stmt->new_tuple) - tuple_ref(stmt->new_tuple, -1); + tuple_unref(stmt->new_tuple); } stmt->old_tuple = stmt->new_tuple = NULL; stmt->space = NULL; @@ -277,7 +277,7 @@ txn_rollback() trigger_run(&txn->on_rollback, txn); /* must not throw. */ rlist_foreach_entry(stmt, &txn->stmts, next) { if (stmt->new_tuple) - tuple_ref(stmt->new_tuple, -1); + tuple_unref(stmt->new_tuple); } /* if (!txn->autocommit && txn->n_stmts && engine_no_yield(txn->engine)) */ trigger_clear(&txn->fiber_on_yield); diff --git a/src/coeio.cc b/src/coeio.cc index 2d2a5af966d540135f5dedffcc231d8969c5e221..d9de772fac2ce61efb094e3947f3ef8241e1df04 100644 --- a/src/coeio.cc +++ b/src/coeio.cc @@ -42,7 +42,7 @@ * manner, when libeio is ready to process some requests it * calls coeio_poller callback. * - * Due to libeio design, want_pall callback is called while + * Due to libeio design, want_poll callback is called while * locks are being held, so it's not possible to call any libeio * function inside this callback. Thus coeio_want_poll raises an * async event which will be dealt with normally as part of the @@ -109,7 +109,6 @@ coeio_init(void) /** * ReInit coeio subsystem (for example after 'fork') - * */ void coeio_reinit(void) diff --git a/src/errcode.h b/src/errcode.h index 26cbd753527d5ea92dca2abe0a4f2fc3edeeb6c2..41436381a39e11f5cbbf1b8ed3aa02bbef6cd61f 100644 --- a/src/errcode.h +++ b/src/errcode.h @@ -56,7 +56,7 @@ enum { TNT_ERRMSG_MAX = 512 }; /* 4 */_(ER_TUPLE_NOT_FOUND, 2, "Tuple doesn't exist in index %u") \ /* 5 */_(ER_UNSUPPORTED, 2, "%s does not support %s") \ /* 6 */_(ER_NONMASTER, 2, "Can't modify data on a replication slave. My master is: %s") \ - /* 7 */_(ER_SECONDARY, 2, "Can't modify data upon a request on the secondary port.") \ + /* 7 */_(ER_READONLY, 2, "Can't modify data because this server in read-only mode.") \ /* 8 */_(ER_INJECTION, 2, "Error injection '%s'") \ /* 9 */_(ER_CREATE_SPACE, 2, "Failed to create space %u: %s") \ /* 10 */_(ER_SPACE_EXISTS, 2, "Space '%s' already exists") \ @@ -134,6 +134,8 @@ enum { TNT_ERRMSG_MAX = 512 }; /* 82 */_(ER_NO_SUCH_ROLE, 2, "Role '%s' is not found") \ /* 83 */_(ER_ROLE_EXISTS, 2, "Role '%s' already exists") \ /* 84 */_(ER_CREATE_ROLE, 2, "Failed to create role '%s': %s") \ + /* 85 */_(ER_INDEX_EXISTS, 2, "Index '%s' already exists") \ + /* 86 */_(ER_TUPLE_REF_OVERFLOW, 1, "Tuple reference counter is overflowed") \ /* * !IMPORTANT! Please follow instructions at start of the file diff --git a/src/ffisyms.cc b/src/ffisyms.cc index 6a6d8290d53af83637b5d45ac0b229ceaeb8099f..808d7af999b4c1ff84bdc87ea22c4d683cc4bea2 100644 --- a/src/ffisyms.cc +++ b/src/ffisyms.cc @@ -27,7 +27,8 @@ void *ffi_symbols[] = { (void *) tuple_rewind, (void *) tuple_seek, (void *) tuple_next, - (void *) tuple_ref, + (void *) tuple_ref_nothrow, + (void *) tuple_unref, (void *) boxffi_index_len, (void *) boxffi_index_random, (void *) boxffi_index_iterator, @@ -47,5 +48,7 @@ void *ffi_symbols[] = { (void *) bsdsocket_local_resolve, (void *) bsdsocket_nonblock, (void *) base64_decode, + (void *) base64_encode, + (void *) base64_bufsize, (void *) SHA1internal }; diff --git a/src/lua/box_net_box.lua b/src/lua/box_net_box.lua index 9642ce907ce87de7538d94d740b8019b62ff3192..b8169e71f58a09859f1952440edc848facfe3b1b 100644 --- a/src/lua/box_net_box.lua +++ b/src/lua/box_net_box.lua @@ -8,6 +8,7 @@ local errno = require 'errno' local ffi = require 'ffi' local digest = require 'digest' local yaml = require 'yaml' +local urilib = require 'uri' -- packet codes local OK = 0 @@ -42,6 +43,8 @@ local TIMEOUT_INFINITY = 500 * 365 * 86400 local sequence_mt = { __serialize = 'sequence'} local mapping_mt = { __serialize = 'mapping'} +local CONSOLE_FAKESYNC = 15121974 + local function request(header, body) -- hint msgpack to always encode header and body as a map @@ -345,14 +348,51 @@ local remote_methods = { setmetatable(self, getmetatable(remote)) end + + -- uri as the first argument + if opts == nil then + opts = {} + if type(port) == 'table' then + opts = port + port = nil + end + + if port == nil then + + local address = urilib.parse(tostring(host)) + if address == nil or address.service == nil then + box.error(box.error.PROC_LUA, + "usage: remote:new(uri[, opts] | host, port[, opts])") + end + + host = address.host + port = address.service + + opts.user = address.login or opts.user + opts.password = address.password or opts.password + end + end + + self.is_instance = true self.host = host self.port = port self.opts = opts + if self.opts == nil then self.opts = {} end + if self.opts.user ~= nil and self.opts.password == nil then + box.error(box.error.PROC_LUA, + "net.box: password is not defined") + end + if self.opts.user == nil and self.opts.password ~= nil then + box.error(box.error.PROC_LUA, + "net.box: user is not defined") + end + + if self.host == nil then self.host = 'localhost' end @@ -378,6 +418,9 @@ local remote_methods = { ping = function(self) + if type(self) ~= 'table' then + box.error(box.error.PROC_LUA, "usage: remote:ping()") + end if not self:is_connected() then return false end @@ -398,8 +441,37 @@ local remote_methods = { end, call = function(self, proc_name, ...) + if type(self) ~= 'table' then + box.error(box.error.PROC_LUA, "usage: remote:call(proc_name, ...)") + end + proc_name = tostring(proc_name) - local res = self:_request('call', true, proc_name, {...}) + + if not self.console then + local res = self:_request('call', true, proc_name, {...}) + return res.body[DATA] + end + + local eval_str = proc_name .. '(' + for i = 1, select('#', ...) do + if i > 1 then + eval_str = eval_str .. ', ' + end + local arg = select(i, ...) + + if arg == nil then + eval_str = eval_str .. 'nil' + elseif type(arg) == 'number' then + eval_str = eval_str .. tostring(arg) + else + arg = tostring(arg) + arg = string.gsub(arg, '"', '\\"') + eval_str = eval_str .. '"' .. arg .. '"' + end + end + eval_str = eval_str .. ")\n" + + local res = self:_console_request(eval_str, true) return res.body[DATA] end, @@ -497,7 +569,49 @@ local remote_methods = { end end, + + _check_console_response = function(self) + while true do + local resp = string.match(self.rbuf, '.-\n[.][.][.]\r?\n') + if resp == nil then + break + end + self.rbuf = string.sub(self.rbuf, #resp + 1) + + local result = yaml.decode(resp) + if result ~= nil then + result = result[1] + end + + local hdr = { [SYNC] = CONSOLE_FAKESYNC, [TYPE] = 0 } + local body = {} + + if type(result) ~= 'table' then + result = { result } + end + + + if result.error ~= nil then + hdr[TYPE] = bit.bor(ERROR_TYPE, box.error.PROC_LUA) + body[ERROR] = result.error + else + body[DATA] = { result } + end + + if self.ch.sync[CONSOLE_FAKESYNC] ~= nil then + self.ch.sync[CONSOLE_FAKESYNC]:put({hdr = hdr, body = body }) + self.ch.sync[CONSOLE_FAKESYNC] = nil + else + log.warn("Unexpected console response: %s", resp) + end + end + end, + _check_response = function(self) + if self.console then + return self:_check_console_response(self) + end + while true do if #self.rbuf < 5 then break @@ -599,6 +713,7 @@ local remote_methods = { end end, + _connect_worker = function(self) fiber.name('net.box.connector') while true do @@ -630,24 +745,29 @@ local remote_methods = { elseif string.len(self.handshake) ~= 128 then self:_fatal("Can't read handshake") else - self.wbuf = '' self.rbuf = '' - local s, e = pcall(function() - self:_auth() - end) - if not s then - self:_fatal(e) - end - - xpcall(function() self:_load_schema() end, - function(e) - log.info("Can't load schema: %s", tostring(e)) - end) - - if self.state ~= 'error' and self.state ~= 'closed' then + if string.match(self.handshake, '^Tarantool .*console') then + self.console = true self:_switch_state('active') + else + self.console = false + local s, e = pcall(function() + self:_auth() + end) + if not s then + self:_fatal(e) + end + + xpcall(function() self:_load_schema() end, + function(e) + log.info("Can't load schema: %s", tostring(e)) + end) + + if self.state ~= 'error' and self.state ~= 'closed' then + self:_switch_state('active') + end end end end @@ -886,16 +1006,20 @@ local remote_methods = { return self:_request_internal(name, raise, ...) end, - _request_internal = function(self, name, raise, ...) + _console_request = function(self, request_body, raise) + if raise == nil then + raise = true + end + return self:_request_raw(CONSOLE_FAKESYNC, request_body, raise) + end, + + _request_raw = function(self, sync, request, raise) local fid = fiber.id() if self.timeouts[fid] == nil then self.timeouts[fid] = TIMEOUT_INFINITY end - local sync = self.proto:sync() - local request = self.proto[name](sync, ...) - self.wbuf = self.wbuf .. request local wstate = self._to_wstate[self.state] @@ -931,8 +1055,11 @@ local remote_methods = { end if response.body[DATA] ~= nil then - for i, v in pairs(response.body[DATA]) do - response.body[DATA][i] = box.tuple.new(response.body[DATA][i]) + if rawget(box, 'tuple') ~= nil then + for i, v in pairs(response.body[DATA]) do + response.body[DATA][i] = + box.tuple.new(response.body[DATA][i]) + end end -- disable YAML flow output (useful for admin console) setmetatable(response.body[DATA], sequence_mt) @@ -941,6 +1068,14 @@ local remote_methods = { return response end, + _request_internal = function(self, name, raise, ...) + + local sync = self.proto:sync() + local request = self.proto[name](sync, ...) + return self:_request_raw(sync, request, raise) + + end, + -- private (low level) methods _select = function(self, spaceno, indexno, key, opts) local res = self:_request('select', true, spaceno, indexno, key, opts) @@ -976,6 +1111,9 @@ remote.self = { timeout = function(self) return self end, wait_connected = function(self) return true end, call = function(_box, proc_name, ...) + if type(_box) ~= 'table' then + box.error(box.error.PROC_LUA, "usage: remote:call(proc_name, ...)") + end proc_name = tostring(proc_name) local proc = { package.loaded['box.internal'] .call_loadproc(proc_name) } @@ -994,7 +1132,8 @@ remote.self = { result[i] = box.tuple.new(v) end return result - end + end, + console = false } diff --git a/src/lua/bsdsocket.lua b/src/lua/bsdsocket.lua index e7aa926cda7ada49175a24c5132020c859babdab..3b65bc04137cb3f88591bd588ceae8fba1c939e6 100644 --- a/src/lua/bsdsocket.lua +++ b/src/lua/bsdsocket.lua @@ -6,6 +6,8 @@ local ffi = require('ffi') local boxerrno = require('errno') local internal = require('socket') local fiber = require('fiber') +local fio = require('fio') +local log = require('log') ffi.cdef[[ struct socket { @@ -75,9 +77,9 @@ local socket_mt local function bless_socket(fd) -- Make socket to be non-blocked by default if ffi.C.bsdsocket_nonblock(fd, 1) < 0 then - local errno = box.errno() + local errno = boxerrno() ffi.C.close(fd) - box.errno(errno) + boxerrno(errno) return nil end @@ -506,7 +508,7 @@ socket_methods.accept = function(self) local cfd, from = internal.accept(fd) if cfd == nil then - self._errno = box.errno() + self._errno = boxerrno() return nil end return bless_socket(cfd), from @@ -950,6 +952,11 @@ local function tcp_connect(host, port, timeout) if dns == nil then return nil end + + if #dns == 0 then + boxerrno(boxerrno.EINVAL) + return nil + end for i, remote in pairs(dns) do timeout = stop - fiber.time() if timeout <= 0 then @@ -980,7 +987,7 @@ local function tcp_server_loop(server, s, addr) fiber.create(tcp_server_handler, server, sc, from) end if addr.family == 'AF_UNIX' and addr.port then - os.remove(addr.port) -- remove unix socket + fio.unlink(addr.port) -- remove unix socket end end @@ -988,6 +995,43 @@ local function tcp_server_usage() error('Usage: socket.tcp_server(host, port, handler | opts)') end +local function tcp_server_bind(s, addr) + if s:bind(addr.host, addr.port) then + return true + end + + if addr.family ~= 'AF_UNIX' then + return false + end + + if boxerrno() ~= boxerrno.EADDRINUSE then + return false + end + + local save_errno = boxerrno() + + local sc = tcp_connect(addr.host, addr.port) + if sc ~= nil then + sc:close() + boxerrno(save_errno) + return false + end + + if boxerrno() ~= boxerrno.ECONNREFUSED then + boxerrno(save_errno) + return false + end + + log.info("tcp_server: remove dead UNIX socket: %s", addr.port) + if not fio.unlink(addr.port) then + log.warn("tcp_server: %s", boxerrno.strerror()) + boxerrno(save_errno) + return false + end + return s:bind(addr.host, addr.port) +end + + local function tcp_server(host, port, opts, timeout) local server = {} if type(opts) == 'function' then @@ -1025,7 +1069,7 @@ local function tcp_server(host, port, opts, timeout) else s:setsockopt('SOL_SOCKET', 'SO_REUSEADDR', 1) -- ignore error end - if not s:bind(addr.host, addr.port) or not s:listen(backlog) then + if not tcp_server_bind(s, addr) or not s:listen(backlog) then local save_errno = boxerrno() s:close() boxerrno(save_errno) diff --git a/src/lua/console.lua b/src/lua/console.lua index fe04952f5b75c7ef940bb576a42e298be02df506..7a591139111ecb596d5a780cc007e187bb32b9db 100644 --- a/src/lua/console.lua +++ b/src/lua/console.lua @@ -70,6 +70,9 @@ end -- local function remote_eval(self, line) if not line then + if type(self.on_client_disconnect) == 'function' then + self:on_client_disconnect() + end pcall(self.remote.close, self.remote) self.remote = nil self.eval = nil @@ -151,7 +154,7 @@ local function client_print(self, output) elseif not output then -- disconnect peer local peer = self.client:peer() - log.info("console: client %s:%s disconnected", peer.host, peer.port) + log.info("client %s:%s disconnected", peer.host, peer.port) self.client:shutdown() self.client:close() self.client = nil @@ -179,7 +182,12 @@ local repl_mt = { -- REPL = read-eval-print-loop -- local function repl(self) + fiber.self().storage.console = self + if type(self.on_start) == 'function' then + self:on_start() + end + while self.running do local command = self:read() local output = self:eval(command) @@ -188,6 +196,22 @@ local function repl(self) fiber.self().storage.console = nil end +local function on_start(foo) + if foo == nil or type(foo) == 'function' then + repl_mt.__index.on_start = foo + return + end + error('Wrong type of on_start hook: ' .. type(foo)) +end + +local function on_client_disconnect(foo) + if foo == nil or type(foo) == 'function' then + repl_mt.__index.on_client_disconnect = foo + return + end + error('Wrong type of on_client_disconnect hook: ' .. type(foo)) +end + -- -- Set delimiter -- @@ -239,6 +263,14 @@ local function connect(uri) -- connect to remote host local remote = require('net.box'):new(u.host, u.service, { user = u.login, password = u.password }) + + -- run disconnect trigger + if remote.state == 'closed' then + if type(self.on_client_disconnect) == 'function' then + self:on_client_disconnect() + end + end + -- check permissions remote:call('dostring', 'return true') -- override methods @@ -249,15 +281,18 @@ local function connect(uri) end local function client_handler(client, peer) - log.info("console: client %s:%s connected", peer.host, peer.port) + log.info("client %s:%s connected", peer.host, peer.port) local state = setmetatable({ running = true; read = client_read; print = client_print; client = client; }, repl_mt) + state:print(string.format("%-63s\n%-63s\n", + "Tarantool ".. box.info.version.." (Lua console)", + "type 'help' for interactive help")) repl(state) - log.info("console: client %s:%s disconnected", peer.host, peer.port) + log.info("client %s:%s disconnected", peer.host, peer.port) end -- @@ -282,7 +317,7 @@ local function listen(uri) error(string.format('failed to create server %s:%s: %s', host, port, errno.strerror())) end - log.info("console: started on %s:%s", addr.host, addr.port) + log.info("started on %s:%s", addr.host, addr.port) return s end @@ -292,4 +327,6 @@ return { delimiter = delimiter; connect = connect; listen = listen; + on_start = on_start; + on_client_disconnect = on_client_disconnect; } diff --git a/src/lua/digest.lua b/src/lua/digest.lua index dd942480c2147b0cff7fe248642bfed856ae824f..3f5653be86145e88d8f376ed21873dc58ee1faef 100644 --- a/src/lua/digest.lua +++ b/src/lua/digest.lua @@ -27,6 +27,7 @@ ffi.cdef[[ extern crc32_func crc32_calc; /* base64 */ + int base64_bufsize(int binsize); int base64_decode(const char *in_base64, int in_len, char *out_bin, int out_len); int base64_encode(const char *in_bin, int in_len, char *out_base64, int out_len); ]] @@ -72,34 +73,25 @@ end local m = { base64_encode = function(bin) - if bin == nil then - error('Usage: base64.encode(str)') - else - -- note: need add check size of bin, bin might containe any binary data - if type(bin) == 'string' then - local blen = string.len(bin) - local slen = math.ceil(blen * 4 / 3) + 2 - local so = ffi.new('char[?]', slen) - local len = ffi.C.base64_encode(bin, blen, so, slen) - bin = ffi.string(so, len) - else - bin = '' - end + if type(bin) ~= 'string' then + error('Usage: digest.base64_encode(string)') end - return bin + local blen = #bin + local slen = ffi.C.base64_bufsize(blen) + local str = ffi.new('char[?]', slen) + local len = ffi.C.base64_encode(bin, blen, str, slen) + return ffi.string(str, len) end, - base64_decode = function(data) - if type(data) ~= 'string' then - data = '' - else - local slen = string.len(data) - local blen = math.ceil(slen * 3 / 4) - local bo = ffi.new('char[?]', blen) - local len = ffi.C.base64_decode(data, slen, bo, blen) - data = ffi.string(bo, len) + base64_decode = function(str) + if type(str) ~= 'string' then + error('Usage: digest.base64_decode(string)') end - return data + local slen = #str + local blen = math.ceil(slen * 3 / 4) + local bin = ffi.new('char[?]', blen) + local len = ffi.C.base64_decode(str, slen, bin, blen) + return ffi.string(bin, len) end, crc32 = function(str) diff --git a/src/lua/fiber.cc b/src/lua/fiber.cc index f8eda08a993a28ee4a8b822994d2b09435c9cb31..6d9e42201880a301fa87d2b898787edc349ffdb9 100644 --- a/src/lua/fiber.cc +++ b/src/lua/fiber.cc @@ -220,9 +220,13 @@ lbox_fiber_statof(struct fiber *f, void *cb_ctx) { struct lua_State *L = (struct lua_State *) cb_ctx; - lua_pushstring(L, fiber_name(f)); + lua_pushinteger(L, f->fid); lua_newtable(L); + lua_pushliteral(L, "name"); + lua_pushstring(L, fiber_name(f)); + lua_settable(L, -3); + lua_pushstring(L, "fid"); lua_pushnumber(L, f->fid); lua_settable(L, -3); diff --git a/src/lua/fio.cc b/src/lua/fio.cc index 375492d68d7674c722d61dc31349fb1f0f651898..721c68af8af0c3e8419c31bcdadadddf6a82d4ee 100644 --- a/src/lua/fio.cc +++ b/src/lua/fio.cc @@ -292,6 +292,32 @@ lbox_fio_pushtimespec(struct lua_State *L, const struct timespec *ts) lua_settable(L, -3); \ } +#define DEF_STAT_METHOD(method_name, macro_name) \ + static int \ + lbox_fio_stat_##method_name(struct lua_State *L) \ + { \ + if (lua_gettop(L) < 1 || !lua_istable(L, 1)) \ + luaL_error(L, "usage: stat:" #method_name "()"); \ + lua_pushliteral(L, "mode"); \ + lua_gettable(L, 1); \ + int mode = lua_tointeger(L, -1); \ + lua_pop(L, 1); \ + lua_pushboolean(L, macro_name(mode) ? 1 : 0); \ + return 1; \ + } + +DEF_STAT_METHOD(is_reg, S_ISREG); +DEF_STAT_METHOD(is_dir, S_ISDIR); +DEF_STAT_METHOD(is_chr, S_ISCHR); +DEF_STAT_METHOD(is_blk, S_ISBLK); +DEF_STAT_METHOD(is_fifo, S_ISFIFO); +#ifdef S_ISLNK +DEF_STAT_METHOD(is_link, S_ISLNK); +#endif +#ifdef S_ISSOCK +DEF_STAT_METHOD(is_sock, S_ISSOCK); +#endif + static int lbox_fio_pushstat(struct lua_State *L, const struct stat *stat) { @@ -316,6 +342,34 @@ lbox_fio_pushstat(struct lua_State *L, const struct stat *stat) PUSHTABLE("mtime", lbox_fio_pushtimespec, &stat->st_mtim); PUSHTABLE("atime", lbox_fio_pushtimespec, &stat->st_atim); #endif + + + int top = lua_gettop(L); + /* metatable for tables *stat */ + lua_newtable(L); + + lua_pushliteral(L, "__index"); + lua_newtable(L); + static const struct luaL_Reg stat_methods[] = { + { "is_reg", lbox_fio_stat_is_reg }, + { "is_dir", lbox_fio_stat_is_dir }, + { "is_chr", lbox_fio_stat_is_chr }, + { "is_blk", lbox_fio_stat_is_blk }, + { "is_fifo", lbox_fio_stat_is_fifo }, +#ifdef S_ISLNK + { "is_link", lbox_fio_stat_is_link }, +#endif +#ifdef S_ISSOCK + { "is_sock", lbox_fio_stat_is_sock }, +#endif + { NULL, NULL } + }; + luaL_register(L, NULL, stat_methods); + lua_settable(L, -3); + + lua_setmetatable(L, top); + lua_settop(L, top); + return 1; } @@ -525,6 +579,9 @@ lbox_fio_close(struct lua_State *L) return 1; } + + + void tarantool_lua_fio_init(struct lua_State *L) { @@ -649,6 +706,7 @@ tarantool_lua_fio_init(struct lua_State *L) lua_settable(L, -3); + lua_pushliteral(L, "seek"); lua_newtable(L); PUSHTABLE("SEEK_SET", lua_pushinteger, SEEK_SET); diff --git a/src/lua/fio.lua b/src/lua/fio.lua index 48687703bdfb83e2fc42e2debde74c4815949ab9..9f6b10d92fa7efefd86f443c2c1862829b144aa0 100644 --- a/src/lua/fio.lua +++ b/src/lua/fio.lua @@ -5,6 +5,7 @@ local ffi = require('ffi') ffi.cdef[[ int umask(int mask); + char *dirname(char *path); ]] local internal = fio.internal @@ -209,18 +210,8 @@ fio.dirname = function(path) return nil end path = tostring(path) - - while true do - if path == "" or string.sub(path, -1) == "/" then - break - end - path = string.sub(path, 1, -2) - end - if path == "" then - path = "." - end - - return path + path = ffi.new('char[?]', #path, path) + return ffi.string(ffi.C.dirname(path)) end fio.umask = function(umask) diff --git a/src/lua/init.cc b/src/lua/init.cc index 02ca2146cb47f870ab66c5b64c5115acda175cda..02fa51d7e6f357c9db52882451932aa0264b0a3b 100644 --- a/src/lua/init.cc +++ b/src/lua/init.cc @@ -94,11 +94,11 @@ static const char *lua_modules[] = { "uuid", uuid_lua, "log", log_lua, "uri", uri_lua, + "fio", fio_lua, "socket", bsdsocket_lua, "net.box", box_net_box_lua, "console", console_lua, "tap", tap_lua, - "fio", fio_lua, "help.en_US", help_en_US_lua, "help", help_lua, NULL @@ -293,9 +293,9 @@ tarantool_lua_init(const char *tarantool_bin, int argc, char **argv) tarantool_lua_fiber_init(L); tarantool_lua_ipc_init(L); tarantool_lua_errno_init(L); + tarantool_lua_fio_init(L); tarantool_lua_bsdsocket_init(L); tarantool_lua_pickle_init(L); - tarantool_lua_fio_init(L); luaopen_msgpack(L); lua_pop(L, 1); luaopen_yaml(L); diff --git a/src/lua/utils.cc b/src/lua/utils.cc index 7331bebc56ff7444be457a080d3e7ed422a8b37e..61f2786be6ec70ca79f08612844d5aad271f52b8 100644 --- a/src/lua/utils.cc +++ b/src/lua/utils.cc @@ -139,7 +139,7 @@ static struct { int type; int defvalue; } OPTIONS[] = { - OPTION(LUA_TBOOLEAN, encode_sparse_convert, 0), + OPTION(LUA_TBOOLEAN, encode_sparse_convert, 1), OPTION(LUA_TNUMBER, encode_sparse_ratio, 2), OPTION(LUA_TNUMBER, encode_sparse_safe, 10), OPTION(LUA_TNUMBER, encode_max_depth, 32), diff --git a/test/app/console.result b/test/app/console.result index c36eef7c724fd19eb6ed3b1f3a20fd4f865a97a3..40596ba7e469aa898ae91efc7b5cbc639a151dfa 100644 --- a/test/app/console.result +++ b/test/app/console.result @@ -1,6 +1,7 @@ TAP version 13 -1..25 +1..26 ok - console.listen started +ok - Handshake ok - connect to console ok - eval ok - state.socker:peer().host diff --git a/test/app/console.test.lua b/test/app/console.test.lua index 8ee97cbaf1dfe4bb67951b516a7ff692c324f803..5a611cc4e4b7c7961534b53c1d136311cdcef5f1 100755 --- a/test/app/console.test.lua +++ b/test/app/console.test.lua @@ -22,12 +22,14 @@ local EOL = "\n%.%.%.\n" test = tap.test("console") -test:plan(25) +test:plan(26) -- Start console and connect to it local server = console.listen(CONSOLE_SOCKET) test:ok(server ~= nil, "console.listen started") local client = socket.tcp_connect("unix/", CONSOLE_SOCKET) +local handshake = client:read{chunk = 128} +test:ok(string.match(handshake, '^Tarantool .*console') ~= nil, 'Handshake') test:ok(client ~= nil, "connect to console") -- Execute some command diff --git a/test/big/lua.result b/test/big/lua.result index 2d7a81ec4793d50821db2b144c028ca8c9c4494b..f460a36fb00831d5afa9a7c4c6ee4a50b48acfda 100644 --- a/test/big/lua.result +++ b/test/big/lua.result @@ -734,7 +734,7 @@ t:find(2, '2') ... t:find(89, '2') --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:findall(4, '3') --- diff --git a/test/box/access_bin.result b/test/box/access_bin.result index 796a085ac0370c0f898497aeb16e9c2a94c35cc7..e9f6c586f8e8277036ac79b9356e7add9c755b2a 100644 --- a/test/box/access_bin.result +++ b/test/box/access_bin.result @@ -102,3 +102,48 @@ setuid_space:drop() --- ... -- +-- gh-530 "assertion failed" +-- If a user is dropped, its session should not be usable +-- any more +-- +test = box.schema.space.create('test') +--- +... +test:create_index('primary') +--- +... +box.schema.user.create('test', {password='test'}) +--- +... +box.schema.user.grant('test', 'read,write', 'space','test') +--- +... +box.schema.user.grant('test', 'read', 'space', '_space') +--- +... +box.schema.user.grant('test', 'read', 'space', '_index') +--- +... +net = require('net.box') +--- +... +c = net.new(LISTEN.host, LISTEN.service, {user = 'test', password='test'}) +--- +... +c.space.test:insert{1} +--- +- [1] +... +box.schema.user.drop('test') +--- +... +c.space.test:insert{1} +--- +- error: User '3' is not found +... +c:close() +--- +... +test:drop() +--- +... diff --git a/test/box/access_bin.test.lua b/test/box/access_bin.test.lua index 18f3d167f28dac7446dbec4e7af1bd913a415274..7ff79d2bb53c448ae0f5df8cfdfe8b95481af5b1 100644 --- a/test/box/access_bin.test.lua +++ b/test/box/access_bin.test.lua @@ -36,3 +36,21 @@ c:close() box.schema.func.drop('setuid_func') setuid_space:drop() -- +-- gh-530 "assertion failed" +-- If a user is dropped, its session should not be usable +-- any more +-- +test = box.schema.space.create('test') +test:create_index('primary') +box.schema.user.create('test', {password='test'}) +box.schema.user.grant('test', 'read,write', 'space','test') +box.schema.user.grant('test', 'read', 'space', '_space') +box.schema.user.grant('test', 'read', 'space', '_index') +net = require('net.box') +c = net.new(LISTEN.host, LISTEN.service, {user = 'test', password='test'}) +c.space.test:insert{1} +box.schema.user.drop('test') +c.space.test:insert{1} +c:close() +test:drop() + diff --git a/test/box/alter_limits.result b/test/box/alter_limits.result index 8a64a0270eb9ea246df9d8a74aa6445d33acf323..10bf75b93b1f2adddb6b2fb14ffec0a384f0bba1 100644 --- a/test/box/alter_limits.result +++ b/test/box/alter_limits.result @@ -1189,6 +1189,21 @@ s_nil.index.secondary:count(1) --- - 0 ... +-- gh-503 if_not_exits option in create index +i1 = s_empty:create_index("test") +--- +... +i2 = s_empty:create_index("test") +--- +- error: Index 'test' already exists +... +i3 = s_empty:create_index("test", { if_not_exists = true } ) +--- +... +i3:select{} +--- +- [] +... -- cleanup s_empty:drop() --- diff --git a/test/box/alter_limits.test.lua b/test/box/alter_limits.test.lua index 1406fec524dd76b1c4e1e354fdb551d12712eb1f..73f87152d589d17a09ee24f953e0a326f21887aa 100644 --- a/test/box/alter_limits.test.lua +++ b/test/box/alter_limits.test.lua @@ -419,6 +419,12 @@ r_empty.index.secondary:count(1) r_full.index.secondary:count(1) s_nil.index.secondary:count(1) +-- gh-503 if_not_exits option in create index +i1 = s_empty:create_index("test") +i2 = s_empty:create_index("test") +i3 = s_empty:create_index("test", { if_not_exists = true } ) +i3:select{} + -- cleanup s_empty:drop() s_full:drop() diff --git a/test/box/box.net.box.result b/test/box/box.net.box.result index eff5cf47758c133d3d2552d1acfcc9a76cf90369..1c0b7af47d7b1574ee1aa4bb4903967f6061bd97 100644 --- a/test/box/box.net.box.result +++ b/test/box/box.net.box.result @@ -141,7 +141,7 @@ cn.space.net_box_test_space:insert{234, 1,2,3} ... cn.space.net_box_test_space.insert{234, 1,2,3} --- -- error: 'builtin/net.box.lua:226: Use space:method(...) instead space.method(...)' +- error: 'builtin/net.box.lua:229: Use space:method(...) instead space.method(...)' ... cn.space.net_box_test_space:replace{354, 1,2,3} --- @@ -518,3 +518,119 @@ remote.self:timeout(123).space.net_box_test_space:select{234} space:drop() --- ... +-- admin console tests +function console_test(...) return { ... } end +--- +... +function console_test_error(...) error(string.format(...)) end +--- +... +function console_unpack_test(...) return ... end +--- +... +ADMIN = require('uri').parse(os.getenv('ADMIN')) +--- +... +cn = remote:new(LISTEN.host, LISTEN.service) +--- +... +cnc = remote:new(ADMIN.host, ADMIN.service) +--- +... +cnc.console +--- +- true +... +cn:call('console_test', 1, 2, 3, 'string', nil) +--- +- - [1, 2, 3, 'string'] +... +cnc:call('console_test', 1, 2, 3, 'string', nil) +--- +- - [1, 2, 3, 'string'] +... +cn:call('console_test_error', 'error %d', 123) +--- +- error: '[string "function console_test_error(...) error(string..."]:1: error 123' +... +cnc:call('console_test_error', 'error %d', 123) +--- +- error: '[string "function console_test_error(...) error(string..."]:1: error 123' +... +cn:call('console_unpack_test', 1) +--- +- - [1] +... +cnc:call('console_unpack_test', 1) +--- +- - [1] +... +cn:call('123') +--- +- error: Procedure '123' is not defined +... +cnc:call('123') +--- +- error: '[string "123()"]:1: unexpected symbol near ''123''' +... +-- #545 user or password is not defined +remote:new(LISTEN.host, LISTEN.service, { user = 'test' }) +--- +- error: 'net.box: password is not defined' +... +remote:new(LISTEN.host, LISTEN.service, { password = 'test' }) +--- +- error: 'net.box: user is not defined' +... +-- #544 usage for remote[point]method +cn:call('console_test') +--- +- - [] +... +cn.call('console_test') +--- +- error: 'usage: remote:call(proc_name, ...)' +... +cn.ping() +--- +- error: 'usage: remote:ping()' +... +remote.self:call('console_test') +--- +- [] +... +remote.self.call('console_test') +--- +- error: 'usage: remote:call(proc_name, ...)' +... +-- uri as the first argument +uri = string.format('%s:%s@%s:%s', 'netbox', 'test', LISTEN.host, LISTEN.service) +--- +... +cn = remote.new(uri) +--- +... +cn:ping() +--- +- true +... +cn:close() +--- +... +uri = string.format('%s@%s:%s', 'netbox', LISTEN.host, LISTEN.service) +--- +... +remote.new(uri) +--- +- error: 'net.box: password is not defined' +... +cn = remote.new(uri, { password = 'test' }) +--- +... +cn:ping() +--- +- true +... +cn:close() +--- +... diff --git a/test/box/box.net.box.test.lua b/test/box/box.net.box.test.lua index 6dc3448aaf4362294bdf8be04f5bb286e707ce05..e1b7396b052abe0e74b7ac37f9759e8433b61377 100644 --- a/test/box/box.net.box.test.lua +++ b/test/box/box.net.box.test.lua @@ -195,3 +195,60 @@ remote.self:timeout(123).space.net_box_test_space:select{234} -- cleanup database after tests space:drop() + +-- admin console tests +function console_test(...) return { ... } end +function console_test_error(...) error(string.format(...)) end +function console_unpack_test(...) return ... end + + +ADMIN = require('uri').parse(os.getenv('ADMIN')) + +cn = remote:new(LISTEN.host, LISTEN.service) +cnc = remote:new(ADMIN.host, ADMIN.service) +cnc.console + +cn:call('console_test', 1, 2, 3, 'string', nil) +cnc:call('console_test', 1, 2, 3, 'string', nil) + +cn:call('console_test_error', 'error %d', 123) +cnc:call('console_test_error', 'error %d', 123) + + +cn:call('console_unpack_test', 1) +cnc:call('console_unpack_test', 1) + + + + +cn:call('123') +cnc:call('123') + + +-- #545 user or password is not defined +remote:new(LISTEN.host, LISTEN.service, { user = 'test' }) +remote:new(LISTEN.host, LISTEN.service, { password = 'test' }) + +-- #544 usage for remote[point]method +cn:call('console_test') +cn.call('console_test') + +cn.ping() + +remote.self:call('console_test') +remote.self.call('console_test') + + +-- uri as the first argument +uri = string.format('%s:%s@%s:%s', 'netbox', 'test', LISTEN.host, LISTEN.service) + +cn = remote.new(uri) +cn:ping() +cn:close() + +uri = string.format('%s@%s:%s', 'netbox', LISTEN.host, LISTEN.service) +remote.new(uri) +cn = remote.new(uri, { password = 'test' }) +cn:ping() +cn:close() + diff --git a/test/box/bsdsocket.result b/test/box/bsdsocket.result index 043ac1e95ce91274c569ca22e00fb7a3f63a7a39..b2122b38bd5b489c9c19d317676ca476041fd71d 100644 --- a/test/box/bsdsocket.result +++ b/test/box/bsdsocket.result @@ -279,7 +279,7 @@ s:getsockopt('SOL_SOCKET', 'SO_DEBUG') ... s:setsockopt('SOL_SOCKET', 'SO_ACCEPTCONN', 1) --- -- error: 'builtin/socket.lua:341: Socket option SO_ACCEPTCONN is read only' +- error: 'builtin/socket.lua:343: Socket option SO_ACCEPTCONN is read only' ... s:getsockopt('SOL_SOCKET', 'SO_RCVBUF') > 32 --- @@ -1433,3 +1433,50 @@ yaml.decode(yaml.encode(s)).fd == s:fd() s = nil --- ... +-- start AF_UNIX server with dead socket exists +path = '/tmp/tarantool-test-socket' +--- +... +s = socket('AF_UNIX', 'SOCK_STREAM', 0) +--- +... +s:bind('unix/', path) +--- +- true +... +s:close() +--- +- true +... +s = socket('AF_UNIX', 'SOCK_STREAM', 0) +--- +... +{ s:bind('unix/', path), errno.strerror() } +--- +- - false + - Address already in use +... +s:close() +--- +- true +... +s = socket.tcp_server('unix/', path, function() end) +--- +... +s ~= nil +--- +- true +... +s:close() +--- +- true +... +fio.stat(path) == nil +--- +- true +... +{ socket.tcp_connect('abrakadabra#123') == nil, errno.strerror() } +--- +- - true + - Invalid argument +... diff --git a/test/box/bsdsocket.test.lua b/test/box/bsdsocket.test.lua index faf0346c4099676777570ff8f70f3c88caa69d16..8fc124573ef7a22ad677a899eef160ccb1f812e0 100644 --- a/test/box/bsdsocket.test.lua +++ b/test/box/bsdsocket.test.lua @@ -482,3 +482,20 @@ s.waiters json.decode(json.encode(s)).fd == s:fd() yaml.decode(yaml.encode(s)).fd == s:fd() s = nil + +-- start AF_UNIX server with dead socket exists +path = '/tmp/tarantool-test-socket' +s = socket('AF_UNIX', 'SOCK_STREAM', 0) +s:bind('unix/', path) +s:close() + +s = socket('AF_UNIX', 'SOCK_STREAM', 0) +{ s:bind('unix/', path), errno.strerror() } +s:close() + +s = socket.tcp_server('unix/', path, function() end) +s ~= nil +s:close() +fio.stat(path) == nil + +{ socket.tcp_connect('abrakadabra#123') == nil, errno.strerror() } diff --git a/test/box/call.test.py b/test/box/call.test.py index cba95b6cf7dde82d65172ab3d44f74723c4b3092..bfb30d386a9509b7842ada91e683100b23fb9766 100644 --- a/test/box/call.test.py +++ b/test/box/call.test.py @@ -122,3 +122,6 @@ sql("call space:delete(4)") admin("space:drop()") admin("box.schema.user.drop('test')") + +# Re-connect after removing user +sql.py_con.close() diff --git a/test/box/cfg.result b/test/box/cfg.result index 60f359c31f8e870efd19e77d2601ffbeea567c33..6255ecf21bc9c468f64457ac52de01800a8b8e89 100644 --- a/test/box/cfg.result +++ b/test/box/cfg.result @@ -2,7 +2,7 @@ --# push filter 'admin: .*' to 'admin: <uri>' box.cfg.nosuchoption = 1 --- -- error: '[string "-- load_cfg.lua - internal file..."]:191: Attempt to modify a read-only +- error: '[string "-- load_cfg.lua - internal file..."]:195: Attempt to modify a read-only table' ... t = {} for k,v in pairs(box.cfg) do if type(v) ~= 'table' and type(v) ~= 'function' then table.insert(t, k..': '..tostring(v)) end end diff --git a/test/box/digest.result b/test/box/digest.result index 21588a9f96af172f0df0eb89b1489bb0991f6e90..0113ed92b6cd092d4543bde83331d1a09f28dce3 100644 --- a/test/box/digest.result +++ b/test/box/digest.result @@ -173,21 +173,39 @@ digest.base64_decode('MTEgMDAgMTEgMDAgYWJjZGVmIEFCQ0RFRiAwMCAxMSAwMCAxMQ==') --- - 11 00 11 00 abcdef ABCDEF 00 11 00 11 ... +s = string.rep('a', 54 * 2) -- two lines in base64 +--- +... +b = digest.base64_encode(s) +--- +... +b +--- +- 'YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh + + YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh + +' +... +digest.base64_decode(b) == s +--- +- true +... digest.base64_decode(nil) --- -- +- error: 'builtin/digest.lua:88: Usage: digest.base64_decode(string)' ... digest.base64_encode(nil) --- -- error: 'builtin/digest.lua:76: Usage: base64.encode(str)' +- error: 'builtin/digest.lua:77: Usage: digest.base64_encode(string)' ... digest.base64_encode(123) --- -- +- error: 'builtin/digest.lua:77: Usage: digest.base64_encode(string)' ... digest.base64_decode(123) --- -- +- error: 'builtin/digest.lua:88: Usage: digest.base64_decode(string)' ... digest = nil --- diff --git a/test/box/digest.test.lua b/test/box/digest.test.lua index f988abc98bdc3946c917f79ba5b620c204a25241..6475bf6b650136b9cd62cc81a0d1b134d9989b7d 100644 --- a/test/box/digest.test.lua +++ b/test/box/digest.test.lua @@ -50,6 +50,10 @@ digest.base64_encode('asdfl asdf adfa zxc vzxcvz llll') digest.base64_decode('YXNkZmwgYXNkZiBhZGZhIHp4YyB2enhjdnogbGxsbA==') digest.base64_encode('11 00 11 00 abcdef ABCDEF 00 11 00 11') digest.base64_decode('MTEgMDAgMTEgMDAgYWJjZGVmIEFCQ0RFRiAwMCAxMSAwMCAxMQ==') +s = string.rep('a', 54 * 2) -- two lines in base64 +b = digest.base64_encode(s) +b +digest.base64_decode(b) == s digest.base64_decode(nil) digest.base64_encode(nil) digest.base64_encode(123) diff --git a/test/box/fiber.result b/test/box/fiber.result index 58656e42e8ad628672c8a494ceafa4e72ecfe505..13a1aa6044ed9b8a51b2f6744deef3477548b8f8 100644 --- a/test/box/fiber.result +++ b/test/box/fiber.result @@ -726,6 +726,44 @@ done --- - true ... +-- # gh-536: fiber.info() doesn't list fibers with default names +-- +function loop() while true do fiber.sleep(10) end end +--- +... +f1 = fiber.create(loop) +--- +... +f2 = fiber.create(loop) +--- +... +f3 = fiber.create(loop) +--- +... +info = fiber.info() +--- +... +info[f1:id()] ~= nil +--- +- true +... +info[f2:id()] ~= nil +--- +- true +... +info[f3:id()] ~= nil +--- +- true +... +f1:cancel() +--- +... +f2:cancel() +--- +... +f3:cancel() +--- +... fiber = nil --- ... diff --git a/test/box/fiber.test.lua b/test/box/fiber.test.lua index 8c80a81c45e15cd3e8f1b1aa5c21a671947e5974..258561e06c04dc5b89bbe9f055cd63c2dcb2b8c4 100644 --- a/test/box/fiber.test.lua +++ b/test/box/fiber.test.lua @@ -296,4 +296,20 @@ end; f = fiber.create(test) done +-- # gh-536: fiber.info() doesn't list fibers with default names +-- +function loop() while true do fiber.sleep(10) end end +f1 = fiber.create(loop) +f2 = fiber.create(loop) +f3 = fiber.create(loop) + +info = fiber.info() +info[f1:id()] ~= nil +info[f2:id()] ~= nil +info[f3:id()] ~= nil + +f1:cancel() +f2:cancel() +f3:cancel() + fiber = nil diff --git a/test/box/fio.result b/test/box/fio.result index 1485c63ad1689a6263814b9a4919af3b1314692a..bd6d8634676e95a38c095640dae3c1ff6bd6119a 100644 --- a/test/box/fio.result +++ b/test/box/fio.result @@ -74,10 +74,45 @@ fh1 ~= nil --- - true ... -fh1:stat().size +f1s = fh1:stat() +--- +... +f1s.size --- - 0 ... +f1s.is_reg() +--- +- error: 'usage: stat:is_reg()' +... +f1s:is_reg() +--- +- true +... +f1s:is_dir() +--- +- false +... +f1s:is_link() +--- +- false +... +f1s:is_sock() +--- +- false +... +f1s:is_fifo() +--- +- false +... +f1s:is_chr() +--- +- false +... +f1s:is_blk() +--- +- false +... fh1:seek(121) --- - 121 @@ -286,3 +321,24 @@ fio.unlink(nil) --- - false ... +-- dirname +fio.dirname('abc') +--- +- . +... +fio.dirname('/abc') +--- +- / +... +fio.dirname('/abc/cde') +--- +- /abc +... +fio.dirname('/abc/cde/') +--- +- /abc +... +fio.dirname('/') +--- +- / +... diff --git a/test/box/fio.test.lua b/test/box/fio.test.lua index c97c15965a5d0f12d9d25e1c279a69bb2e5c0d54..8d5b7b5c4e39828c3d2af3aa4a0b775d59086f47 100644 --- a/test/box/fio.test.lua +++ b/test/box/fio.test.lua @@ -32,7 +32,18 @@ file4 = fio.pathjoin(tmpdir, 'file.4') fh1 = fio.open(file1, { 'O_RDWR', 'O_TRUNC', 'O_CREAT' }, 0777) fh1 ~= nil -fh1:stat().size +f1s = fh1:stat() +f1s.size + +f1s.is_reg() +f1s:is_reg() +f1s:is_dir() +f1s:is_link() +f1s:is_sock() +f1s:is_fifo() +f1s:is_chr() +f1s:is_blk() + fh1:seek(121) fh1:stat().size fh1:write("Hello, world") @@ -97,3 +108,11 @@ fio.rmdir(tmpdir) fio.unlink() fio.unlink(nil) + +-- dirname + +fio.dirname('abc') +fio.dirname('/abc') +fio.dirname('/abc/cde') +fio.dirname('/abc/cde/') +fio.dirname('/') diff --git a/test/box/iproto.result b/test/box/iproto.result index 92cd4fd8ae85c073a7b34fb16e74d933be4d14f7..9b6826200771788236f0f5e126eac12457059c5c 100644 --- a/test/box/iproto.result +++ b/test/box/iproto.result @@ -1,3 +1,6 @@ +box.schema.user.grant('guest', 'read,write,execute', 'universe') +--- +... # # iproto packages test @@ -106,3 +109,52 @@ box.schema.user.grant('guest', 'read,write,execute', 'space', 'test') space:drop() --- ... +space = box.schema.create_space('test') +--- +... +space:create_index('primary', { type = 'hash', parts = {1, 'str'}}) +--- +... +STR 1 +-- +0xa1 => ok ok ok ok ok ok +0xd901 => ok ok ok ok ok ok +0xda0001 => ok ok ok ok ok ok +0xdb00000001 => ok ok ok ok ok ok + +STR 31 +-- +0xbf => ok ok ok ok ok ok +0xd91f => ok ok ok ok ok ok +0xda001f => ok ok ok ok ok ok +0xdb0000001f => ok ok ok ok ok ok + +STR 32 +-- +0xd920 => ok ok ok ok ok +0xda0020 => ok ok ok ok ok +0xdb00000020 => ok ok ok ok ok + +STR 255 +-- +0xd9ff => ok ok ok ok ok +0xda00ff => ok ok ok ok ok +0xdb000000ff => ok ok ok ok ok + +STR 256 +-- +0xda0100 => ok ok ok ok +0xdb00000100 => ok ok ok ok + +STR 65535 +-- +0xdaffff => ok ok ok ok +0xdb0000ffff => ok ok ok ok + +STR 65536 +-- +0xdb00010000 => ok ok ok + +space:drop() +--- +... diff --git a/test/box/iproto.test.py b/test/box/iproto.test.py index d95af37e1e7f156fb18dfe925d637f7e29bb1c86..beae1a2071420f679482daa96524a5d97b971851 100644 --- a/test/box/iproto.test.py +++ b/test/box/iproto.test.py @@ -5,11 +5,12 @@ import socket import msgpack from tarantool.const import * from tarantool import Connection -from tarantool.request import RequestInsert -from tarantool.request import RequestSelect +from tarantool.request import Request, RequestInsert, RequestSelect from tarantool.response import Response from lib.tarantool_connection import TarantoolConnection +admin("box.schema.user.grant('guest', 'read,write,execute', 'universe')") + print """ # # iproto packages test @@ -149,3 +150,63 @@ c.close() admin("space:drop()") +# +# gh-522: Broken compatibility with msgpack-python for strings of size 33..255 +# +admin("space = box.schema.create_space('test')") +admin("space:create_index('primary', { type = 'hash', parts = {1, 'str'}})") + +class RawInsert(Request): + request_type = REQUEST_TYPE_INSERT + def __init__(self, conn, space_no, blob): + super(RawInsert, self).__init__(conn) + request_body = "\x82" + msgpack.dumps(IPROTO_SPACE_ID) + \ + msgpack.dumps(space_id) + msgpack.dumps(IPROTO_TUPLE) + blob + self._bytes = self.header(len(request_body)) + request_body + +class RawSelect(Request): + request_type = REQUEST_TYPE_SELECT + def __init__(self, conn, space_no, blob): + super(RawSelect, self).__init__(conn) + request_body = "\x83" + msgpack.dumps(IPROTO_SPACE_ID) + \ + msgpack.dumps(space_id) + msgpack.dumps(IPROTO_KEY) + blob + \ + msgpack.dumps(IPROTO_LIMIT) + msgpack.dumps(100); + self._bytes = self.header(len(request_body)) + request_body + +c = sql.py_con +space = c.space('test') +space_id = space.space_no + +TESTS = [ + (1, "\xa1", "\xd9\x01", "\xda\x00\x01", "\xdb\x00\x00\x00\x01"), + (31, "\xbf", "\xd9\x1f", "\xda\x00\x1f", "\xdb\x00\x00\x00\x1f"), + (32, "\xd9\x20", "\xda\x00\x20", "\xdb\x00\x00\x00\x20"), + (255, "\xd9\xff", "\xda\x00\xff", "\xdb\x00\x00\x00\xff"), + (256, "\xda\x01\x00", "\xdb\x00\x00\x01\x00"), + (65535, "\xda\xff\xff", "\xdb\x00\x00\xff\xff"), + (65536, "\xdb\x00\x01\x00\x00"), +] + +for test in TESTS: + it = iter(test) + size = next(it) + print 'STR', size + print '--' + for fmt in it: + print '0x' + fmt.encode('hex'), '=>', + field = '*' * size + c._send_request(RawInsert(c, space_id, "\x91" + fmt + field)) + tuple = space.select(field)[0] + print len(tuple[0])== size and 'ok' or 'fail', + it2 = iter(test) + next(it2) + for fmt2 in it2: + tuple = c._send_request(RawSelect(c, space_id, + "\x91" + fmt2 + field))[0] + print len(tuple[0]) == size and 'ok' or 'fail', + tuple = space.delete(field)[0] + print len(tuple[0]) == size and 'ok' or 'fail', + print + print + +admin("space:drop()") diff --git a/test/box/misc.result b/test/box/misc.result index b2397d2a88830c8510156437ed4eaf6ebc307ca9..5a65842d9dcf5f3287cc162a0fe7840e3e94167c 100644 --- a/test/box/misc.result +++ b/test/box/misc.result @@ -182,11 +182,11 @@ end; t; --- - - 'box.error.EXACT_MATCH : 19' - - 'box.error.SECONDARY : 7' + - 'box.error.NO_SUCH_TRIGGER : 34' - 'box.error.CLUSTER_ID_IS_RO : 65' - 'box.error.INDEX_TYPE : 13' - 'box.error.CLUSTER_ID_MISMATCH : 63' - - 'box.error.FIELD_TYPE : 23' + - 'box.error.MEMORY_ISSUE : 2' - 'box.error.KEY_PART_TYPE : 18' - 'box.error.CREATE_FUNCTION : 50' - 'box.error.SOPHIA : 60' @@ -208,12 +208,14 @@ t; - 'box.error.WAL_IO : 40' - 'box.error.CREATE_USER : 43' - 'box.error.CREATE_SPACE : 9' + - 'box.error.TUPLE_REF_OVERFLOW : 86' - 'box.error.UNKNOWN_SCHEMA_OBJECT : 49' + - 'box.error.PROC_LUA : 32' - 'box.error.CREATE_ROLE : 84' - 'box.error.ROLE_EXISTS : 83' - 'box.error.NO_SUCH_ROLE : 82' - 'box.error.NO_ACTIVE_TRANSACTION : 80' - - 'box.error.SPLICE : 25' + - 'box.error.TUPLE_FOUND : 3' - 'box.error.FIELD_TYPE_MISMATCH : 24' - 'box.error.UNSUPPORTED : 5' - 'box.error.INVALID_MSGPACK : 20' @@ -221,7 +223,7 @@ t; - 'box.error.ALTER_SPACE : 12' - 'box.error.ACTIVE_TRANSACTION : 79' - 'box.error.NO_CONNECTION : 77' - - 'box.error.DROP_SPACE : 11' + - 'box.error.FIELD_TYPE : 23' - 'box.error.INVALID_XLOG_NAME : 75' - 'box.error.INVALID_XLOG : 74' - 'box.error.REPLICA_MAX : 73' @@ -235,34 +237,34 @@ t; - 'box.error.INVALID_ORDER : 68' - 'box.error.CFG : 59' - 'box.error.SPACE_FIELD_COUNT : 38' - - 'box.error.SPACE_ACCESS_DENIED : 55' + - 'box.error.UNKNOWN : 0' - 'box.error.NO_SUCH_FIELD : 37' - 'box.error.LOCAL_SERVER_IS_NOT_ACTIVE : 61' - 'box.error.RELOAD_CFG : 58' - 'box.error.PROC_RET : 21' - 'box.error.INJECTION : 8' - - 'box.error.PROC_LUA : 32' + - 'box.error.FUNCTION_MAX : 54' - 'box.error.ILLEGAL_PARAMS : 1' - - 'box.error.TUPLE_NOT_ARRAY : 22' - 'box.error.TUPLE_FORMAT_LIMIT : 16' + - 'box.error.USER_MAX : 56' - 'box.error.INVALID_UUID : 64' - - 'box.error.UNKNOWN : 0' + - 'box.error.SPLICE : 25' - 'box.error.TIMEOUT : 78' - - 'box.error.TUPLE_FOUND : 3' - - 'box.error.MEMORY_ISSUE : 2' - - 'box.error.NO_SUCH_TRIGGER : 34' + - 'box.error.MORE_THAN_ONE_TUPLE : 41' + - 'box.error.NO_SUCH_SPACE : 36' + - 'box.error.INDEX_EXISTS : 85' - 'box.error.UPDATE_FIELD : 29' - 'box.error.ARG_TYPE : 26' - - 'box.error.NO_SUCH_SPACE : 36' - 'box.error.INDEX_FIELD_COUNT : 39' - - 'box.error.MORE_THAN_ONE_TUPLE : 41' + - 'box.error.READONLY : 7' - 'box.error.DROP_PRIMARY_KEY : 17' + - 'box.error.DROP_SPACE : 11' - 'box.error.UNKNOWN_REQUEST_TYPE : 48' - 'box.error.INVALID_XLOG_ORDER : 76' - - 'box.error.FUNCTION_MAX : 54' + - 'box.error.SPACE_ACCESS_DENIED : 55' - 'box.error.NO_SUCH_USER : 45' - - 'box.error.USER_MAX : 56' - 'box.error.UNKNOWN_UPDATE_OP : 28' + - 'box.error.TUPLE_NOT_ARRAY : 22' - 'box.error.NO_SUCH_PROC : 33' - 'box.error.FUNCTION_ACCESS_DENIED : 53' ... diff --git a/test/box/select.result b/test/box/select.result index 0d39be3522326db28d24d2dc3d10c6dab76a661b..9165a6e00019663278f530669fe6c2c5cb558cbf 100644 --- a/test/box/select.result +++ b/test/box/select.result @@ -538,3 +538,32 @@ s:select(2) s:drop() --- ... +s = box.schema.create_space('select', { temporary = true }) +--- +... +s:create_index('primary', { type = 'tree' }) +--- +... +local a s:insert{0} +--- +... +lots_of_links = {} +--- +... +ref_count = 0 +--- +... +while (true) do table.insert(lots_of_links, s:get{0}) ref_count = ref_count + 1 end +--- +- error: Tuple reference counter is overflowed +... +ref_count +--- +- 65534 +... +lots_of_links = {} +--- +... +s:drop() +--- +... diff --git a/test/box/select.test.lua b/test/box/select.test.lua index c09a1f7f16f781302c3537808b10ff93d36b2727..385a3c295bbec59e7f2b9ded48f92eacf18ea515 100644 --- a/test/box/select.test.lua +++ b/test/box/select.test.lua @@ -76,3 +76,13 @@ s.index[0]:select(1, { iterator = 'GE', offset = 10, limit = 2 }) s:select(2) s:drop() + +s = box.schema.create_space('select', { temporary = true }) +s:create_index('primary', { type = 'tree' }) +local a s:insert{0} +lots_of_links = {} +ref_count = 0 +while (true) do table.insert(lots_of_links, s:get{0}) ref_count = ref_count + 1 end +ref_count +lots_of_links = {} +s:drop() diff --git a/test/box/snap_daemon.result b/test/box/snapshot_daemon.result similarity index 93% rename from test/box/snap_daemon.result rename to test/box/snapshot_daemon.result index e411a87accc3ec5ed842a16873eb06664735ddf5..57e0fd3d52964518cf632195051ce6f36021c963 100644 --- a/test/box/snap_daemon.result +++ b/test/box/snapshot_daemon.result @@ -43,7 +43,7 @@ end --- ... -space = box.schema.create_space('snap_daemon') +space = box.schema.create_space('snapshot_daemon') --- ... space:create_index('pk', { type = 'tree', parts = { 1, 'num' }}) @@ -112,3 +112,7 @@ PERIOD --- - 0.03 ... +box.cfg{ snapshot_count = .2 } +--- +- error: snapshot_count must be integer +... diff --git a/test/box/snap_daemon.test.lua b/test/box/snapshot_daemon.test.lua similarity index 94% rename from test/box/snap_daemon.test.lua rename to test/box/snapshot_daemon.test.lua index bdfcc649274af25bdb9f564dfe1e17d884d62f4a..45aeb441fc5981d1527210571af289b486396613 100644 --- a/test/box/snap_daemon.test.lua +++ b/test/box/snapshot_daemon.test.lua @@ -23,7 +23,7 @@ end --# setopt delimiter '' -space = box.schema.create_space('snap_daemon') +space = box.schema.create_space('snapshot_daemon') space:create_index('pk', { type = 'tree', parts = { 1, 'num' }}) @@ -70,3 +70,5 @@ box.cfg{snapshot_period = 3600 * 4, snapshot_count = 4 } space:drop() PERIOD + +box.cfg{ snapshot_count = .2 } diff --git a/test/box/stat.result b/test/box/stat.result index ad1ea78af49b1ab993cefa331cfa834eb59e5d12..9a7d80f3ec8bc3c7ca54e63c41ba464e648c2433 100644 --- a/test/box/stat.result +++ b/test/box/stat.result @@ -50,7 +50,7 @@ box.stat.REPLACE.total ... box.stat.SELECT.total --- -- 2 +- 3 ... --# stop server default --# start server default diff --git a/test/box/tuple.result b/test/box/tuple.result index 3cf34faca9948f9068f1f645e04423bc4fd7596b..975be871b5ce53c4f92baf19cc6e47b17f4af263 100644 --- a/test/box/tuple.result +++ b/test/box/tuple.result @@ -377,15 +377,15 @@ t:next(3) ... t:next(4) --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:next(-1) --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:next("fdsaf") --- -- error: '[string "-- tuple.lua (internal file)..."]:69: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:74: error: invalid key to ''next''' ... box.tuple.new({'x', 'y', 'z'}):next() --- @@ -397,7 +397,7 @@ t=space:insert{1953719668} ... t:next(1684234849) --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:next(1) --- @@ -553,7 +553,7 @@ r = {} ... for _state, val in t:pairs(10) do table.insert(r, val) end --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... r --- @@ -639,19 +639,19 @@ t:findall(1, 'xxxxx') ... t:find(100, 'a') --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:findall(100, 'a') --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:find(100, 'xxxxx') --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... t:findall(100, 'xxxxx') --- -- error: '[string "-- tuple.lua (internal file)..."]:86: error: invalid key to ''next''' +- error: '[string "-- tuple.lua (internal file)..."]:91: error: invalid key to ''next''' ... --- -- Lua type coercion @@ -745,12 +745,12 @@ t = box.tuple.new({'a', 'b', 'c', 'd', 'e'}) ... t:update() --- -- error: '[string "-- tuple.lua (internal file)..."]:152: Usage: tuple:update({ { +- error: '[string "-- tuple.lua (internal file)..."]:157: Usage: tuple:update({ { op, field, arg}+ })' ... t:update(10) --- -- error: '[string "-- tuple.lua (internal file)..."]:152: Usage: tuple:update({ { +- error: '[string "-- tuple.lua (internal file)..."]:157: Usage: tuple:update({ { op, field, arg}+ })' ... t:update({}) diff --git a/test/box/update.result b/test/box/update.result index 3dfddd5c53f2c213fe5f11a75d6fbe8a4962007c..b2b49e328484118f8bc65f7d4222f463e871c4fe 100644 --- a/test/box/update.result +++ b/test/box/update.result @@ -330,3 +330,25 @@ s:truncate() s:drop() --- ... +-- #521: Cryptic error message in update operation +s = box.schema.create_space('tweedledum') +--- +... +s:create_index('pk') +--- +... +s:insert{1, 2, 3} +--- +- [1, 2, 3] +... +s:update({1}) +--- +- error: Tuple/Key must be MsgPack array +... +s:update({1}, {'=', 1, 1}) +--- +- error: Invalid MsgPack - expected an update operation name (string) +... +s:drop() +--- +... diff --git a/test/box/update.test.lua b/test/box/update.test.lua index 4723fef896c924637a630a79d4c83aa47270c102..010721ce170bed04660c16ed85d7e2a84c153bb4 100644 --- a/test/box/update.test.lua +++ b/test/box/update.test.lua @@ -110,3 +110,11 @@ s = box.space.tweedledum s:select{} s:truncate() s:drop() + +-- #521: Cryptic error message in update operation +s = box.schema.create_space('tweedledum') +s:create_index('pk') +s:insert{1, 2, 3} +s:update({1}) +s:update({1}, {'=', 1, 1}) +s:drop() diff --git a/test/lib/admin_connection.py b/test/lib/admin_connection.py index 14d2e908027773a4cc29607a5fb9077a097bbeb5..5841a12e2a2742630101a5bb8b15e2e24f0d21c5 100644 --- a/test/lib/admin_connection.py +++ b/test/lib/admin_connection.py @@ -56,3 +56,9 @@ class AdminConnection(TarantoolConnection): if not silent: sys.stdout.write(res.replace("\r\n", "\n")) return res + + def connect(self): + super(AdminConnection, self).connect() + handshake = self.socket.recv(128) + if not re.search(r'^Tarantool.*console.*', str(handshake)): + raise RuntimeError('Broken tarantool console handshake') diff --git a/test/lib/sql_ast.py b/test/lib/sql_ast.py index 3b0741745ef0322407fda3bda7c365f811ea66a9..f49a42ac6f8bb2ada158b1dcabb4cbfe01bcff38 100644 --- a/test/lib/sql_ast.py +++ b/test/lib/sql_ast.py @@ -25,7 +25,7 @@ ER = { 4: "ER_TUPLE_NOT_FOUND" , 5: "ER_UNSUPPORTED" , 6: "ER_NONMASTER" , - 7: "ER_SECONDARY" , + 7: "ER_READONLY" , 8: "ER_INJECTION" , 9: "ER_CREATE_SPACE" , 10: "ER_SPACE_EXISTS" , diff --git a/test/replication/hot_standby.result b/test/replication/hot_standby.result index 0f939412486f09b1b533d1173cc859b3281fafcb..d2b066d47a98c1c6fc952afcf7743f088c45fd28 100644 --- a/test/replication/hot_standby.result +++ b/test/replication/hot_standby.result @@ -12,7 +12,7 @@ box.schema.user.grant('guest', 'read,write,execute', 'universe') fiber = require('fiber'); --- ... -while box.info.server == nil do fiber.sleep(0.01) end; +while box.info.server.id == 0 do fiber.sleep(0.01) end; --- ... while box.space['_priv']:len() < 1 do fiber.sleep(0.001) end; diff --git a/test/replication/hot_standby.test.lua b/test/replication/hot_standby.test.lua index 82c5165dfdd6a7e70fe23595a3e160e58cf77671..560207fb6b5dd5f2b9495272dc252a75a29a32f8 100644 --- a/test/replication/hot_standby.test.lua +++ b/test/replication/hot_standby.test.lua @@ -9,7 +9,7 @@ box.schema.user.grant('guest', 'read,write,execute', 'universe') --# setopt delimiter ';' --# set connection default, hot_standby, replica fiber = require('fiber'); -while box.info.server == nil do fiber.sleep(0.01) end; +while box.info.server.id == 0 do fiber.sleep(0.01) end; while box.space['_priv']:len() < 1 do fiber.sleep(0.001) end; do local pri_id = '' diff --git a/test/replication/readonly.result b/test/replication/readonly.result new file mode 100644 index 0000000000000000000000000000000000000000..9a3b22b210fa4119b55ef7fd00086806a7e14693 --- /dev/null +++ b/test/replication/readonly.result @@ -0,0 +1,41 @@ +box.schema.user.grant('guest', 'read,write,execute', 'universe') +--- +... +box.info.server.id +--- +- 2 +... +box.info.server.ro +--- +- false +... +box.info.server.lsn +--- +- 0 +... +------------------------------------------------------------- +replica is read-only until receive self server_id in _cluster +------------------------------------------------------------- +box.cfg{replication_source = ""} +--- +... +box.info.server.id +--- +- 0 +... +box.info.server.ro +--- +- true +... +box.info.server.lsn +--- +- -1 +... +space = box.schema.create_space("ro") +--- +- error: Can't modify data because this server in read-only mode. +... +box.info.vclock[2] +--- +- null +... diff --git a/test/replication/readonly.test.py b/test/replication/readonly.test.py new file mode 100644 index 0000000000000000000000000000000000000000..24789c51a77e804d1b4b369e189a7d2e7bc5251c --- /dev/null +++ b/test/replication/readonly.test.py @@ -0,0 +1,46 @@ +import os +from glob import iglob as glob +from lib.tarantool_server import TarantoolServer + +# master server +master = server +master_id = master.get_param('server')['id'] + +master.admin("box.schema.user.grant('guest', 'read,write,execute', 'universe')") + +replica = TarantoolServer(server.ini) +replica.script = 'replication/replica.lua' +replica.vardir = os.path.join(server.vardir, 'replica') +replica.rpl_master = master +replica.deploy() +replica.wait_lsn(master_id, master.get_lsn(master_id)) +replica_id = replica.get_param('server')['id'] +replica.admin('box.info.server.id') +replica.admin('box.info.server.ro') +replica.admin('box.info.server.lsn') +replica.stop() + +print '-------------------------------------------------------------' +print 'replica is read-only until receive self server_id in _cluster' +print '-------------------------------------------------------------' + +# Remove xlog retrived by SUBSCRIBE +filename = str(0).zfill(20) + ".xlog" +wal = os.path.join(replica.vardir, filename) +os.remove(wal) + +# Start replica without master +server.stop() +replica.start() +replica.admin('box.cfg{replication_source = ""}') + +# Check that replica in read-only mode +replica.admin('box.info.server.id') +replica.admin('box.info.server.ro') +replica.admin('box.info.server.lsn') +replica.admin('space = box.schema.create_space("ro")') +replica.admin('box.info.vclock[%d]' % replica_id) + +replica.stop() +replica.cleanup(True) +server.deploy() diff --git a/test/replication/swap.result b/test/replication/swap.result index 5badc7052d0fff463c4f1fca6ffc8953d3e48e84..7264bc985d053c0fb122f80eeda232b9e9e736eb 100644 --- a/test/replication/swap.result +++ b/test/replication/swap.result @@ -4,7 +4,7 @@ box.schema.user.create('test', { password = 'pass123456'}) box.schema.user.grant('test', 'read,write,execute', 'universe') --- ... -while box.info.server == nil do require('fiber').sleep(0.01) end +while box.info.server.id == 0 do require('fiber').sleep(0.01) end --- ... while box.space['_priv']:len() < 1 do require('fiber').sleep(0.01) end diff --git a/test/replication/swap.test.py b/test/replication/swap.test.py index eb3021ebdad958602e7d0454f9309e03a5765cbc..74cd85a7b37e7d7042bb79fc03c6f29b04ba8c02 100644 --- a/test/replication/swap.test.py +++ b/test/replication/swap.test.py @@ -31,7 +31,7 @@ replica = TarantoolServer() replica.script = "replication/replica.lua" replica.vardir = os.path.join(server.vardir, 'replica') replica.deploy() -replica.admin("while box.info.server == nil do require('fiber').sleep(0.01) end") +replica.admin("while box.info.server.id == 0 do require('fiber').sleep(0.01) end") replica.uri = '%s:%s@%s' % (LOGIN, PASSWORD, replica.sql.uri) replica.admin("while box.space['_priv']:len() < 1 do require('fiber').sleep(0.01) end") replica.sql.py_con.authenticate(LOGIN, PASSWORD) diff --git a/test/sophia/index_random_test.lua b/test/sophia/index_random_test.lua index 5fb09f03c2b4c48723d45c62fd7181b1a5e0da73..00838e83e1d78527af6fbb6041899831fdcc368b 100644 --- a/test/sophia/index_random_test.lua +++ b/test/sophia/index_random_test.lua @@ -17,14 +17,11 @@ function index_random_test(space, index_no) error('too many iterations') return nil end - local tuple = space.index[index_no]:random(rnd) if tuple == nil then error('nil returned') return nil end - print(tuple) - local k = tuple[1] if tuples[k] == nil then found = found + 1 diff --git a/third_party/base64.c b/third_party/base64.c index e2fb9785c43ce65a76c2077455a1e18a0b4126a1..49a4f2492d28ab85fee030b550cd3c0f84978c38 100644 --- a/third_party/base64.c +++ b/third_party/base64.c @@ -35,6 +35,9 @@ /* {{{ encode */ +extern inline int +base64_bufsize(int binsize); + enum base64_encodestep { step_A, step_B, step_C }; struct base64_encodestate { diff --git a/third_party/base64.h b/third_party/base64.h index d590820ddb775f816cce7ef9941d47cd8cd70890..a52a5e91f7e767a3694f2efb9a70271279230187 100644 --- a/third_party/base64.h +++ b/third_party/base64.h @@ -39,7 +39,7 @@ extern "C" { #define BASE64_CHARS_PER_LINE 72 -static inline int +inline int base64_bufsize(int binsize) { int datasize = binsize * 4/3 + 4;