SQLite uses TCL shell to generate common headers.
Add dependency in Travis to TCL shell for linux targets.
OSX comes with TCL pre-installed - no need to add dependency.

	* .travis.mk (deps_linux): Add tcl.

sql: Add dependency to TCL in packages.

As far as SQLite sources depend on TCLsh we need to
add dependency to TCL in package info.

	* debian/control: Add dependency on TCL.
	* rpm/tarantool.spec: Ditto.
	* snapcraft.yaml: Ditto.

sql: Avoid multiline string to satisfy older cmake

RHEL uses some old cmake which doesn't work with long string
which span over few lines w/ '\' terminator. Make it single line.

	* src/box/CMakeLists.txt (set_source_files_properties): Use
	single line instead of vreaks w/ '\'.

	This test suite is supposed to contain tests converted from
	sqlite's TCL testsuite.

	test/
	* sql-tap/lua/sqltester.lua: New test wrapper.
	* sql-tap/suite.ini: New test suite initializer.

	* sql-tap/select3.[test.lua,result]: New test and expected result.

sql: [#2381] Convert `whereK.test`.

	test/sql-tap/
	* lua/sqltester.lua (flatten): Reindent.
	(do_test): Introduce matching. If expected result is single line
	which starts w/ '/' and ends w/ '/' then search for such string
	w/o '/'s in whole table. If found - test pass.
	(execsql): Reindent.
	* whereK.[test.lua,result]: New.

sql: [#2381] Convert `insert1.test`.

	test/sql-tap/
	* lua/sqltester.lua (header): Remove excessive includes.
	(fix_result): New routine intended to fix recieved result.
	Initially it simply convert values of boolean types to numbers.
	(do_test): Use fix_result.
	(execsql): Cleanup.
	(lsearch): New. Try to roughly mimic TCL's lsearch.
	* insert1.[test.lua,result]: New test.

sql: [#2381] Convert `update.test`.

	test/sql-tap/
	* lua/sqltester.lua (do_test): If result is nil - set it to empty table.
	(db): Use execsql instead of direct box.sql.execute.
	* update.[test.lua, result]: New.

sql: Add sql-tap/with1.test.

Function with1.test.lua:limit_test() completely re-written.

	src/box/
	* sql.h: Remove `lua_State` and `box_lua_sqlite_init` declarations.

	src/box/
	* sql.c (sql_init): Use `panic()` instead of `assert()`.

	src/lib/sqlite/src/
	* func.c (last_insert_rowid): Comment unused routine.
	(sqlite3RegisterBuiltinFunctions): Comment `last_insert_rowid` entry.

sql: Add forgotten tests.

	src/box/
	* sql.c (tarantoolSqlite3MovetoUnpacked): Add imaginary opcode (255) to
	signal Tarantool to re-seek the cursor.

	src/lib/sqlite/src/
	* btree.c (btreeMoveto): Pass imagbiary opcode (instead of garbage).
	(sqlite3BtreeNext): If Tarantool's cursor in invalid state - restore it.

	src/lib/sqlite/src/
	* expr.c (sqlite3SetHasNullFlag): Add column to check for null,
	as Tarantool doesn't change order of fields in index. So, first
	key field in index has no number 0.
	(sqlite3FindInIndex): Pass column number of first key in index to
	check for null.
	(sqlite3CodeSubselect): Pass 0, as first column of table is always
	first.
	(sqlite3ExprCodeIN): 1. Comment. Check that aiMap containst integers
	between 0 and nVector-1 is not relevant for Tarantool. As index keys are
	references to original table columns.
	2. Check for (aiMap[i] == i) is not relevant since order of column numbers
	are original. Comment whole loop, which allocates new register range.
	3. Fix column fetch: take into account Tarantool column order.

	src/lib/sqlite/src/
	* build.c (sqlite3PrimaryKeyIndex): Use `HadRowid` table predicate to
	understand if table wa created by Tarantool or SQLite, see #2376
	for details.

	test/sqlite-tcl/
	* transitive1.result: All test cases pass.

	src/lib/sqlite/src/
	* vdbe.c (sqlite3VdbeExec): Relax assertion by excluding Tarantool's
	backend. Tarantool stores tuples of arbitrary sizes in a single chunk,
	so assert makes no sense for Tarantool backend.

	test/sqlite-tcl/
	* types.result: All test cases pass.

	test/
	* sql/sql-persistency.[test|result]: New test.

	test/
	* box/sql-*: Move to ...
	* box/sql/sql--*: ... here.

     src/lib/sqlite/src/
     * expr.c (sqlite3FindInIndex): indroduce int* argument which is
     set to column number in case of single column query (if pointer is not
     null as well). This is temporary solution, all data can be stored in `aiMap`
     argument. Comments warn, that VM doesn't expect vector to be size of 1.
     So, play on the safe side right now.
     Store real column number in `aiMap`, sinnce Tarantool doesn't rearrange
     columns in index.
     (sqlite3ExprCodeIN): Pass dummy (0) value for new argumnet.
     * sqliteInt.h (sqlite3FindInIndex): Update decl.
     (wherecode.c): Fetch table column number in case (nExpr == 1) and use it
     while emit `OP_Column`.

     test/
     * box/sql-update-with-nested-select[.test.lua|result]: New regression test.
     * sqlite-tcl/update.result: All tests are pass now. Fix expected result.

     lib/sqlite/config.h.cmake: Block pread instead of pread64 on Darwin.
     lib/sqlite/src/CMakeLists.txt: Add `-flat_namespace` for Darwin.

     test/box-tap/
     * cfg.test.lua: Add check that invocation of SQL commands lead
     to error message.
     * cfg.result: Update expected result.

Due to the way LIKE operator is implemented, TEXT indices are looked up
with BLOB keys.

LIKE applies both to TEXT and BLOBs, hence a code is generated to lookup
a TEXT and then a BLOB even if column affinity is TEXT.

sql: btreeMoveto() should use MsgPack unpacker.

     * sqlite/src/btree (btreeMoveto): All data is stored in MsgPack
     format. This is yet another place where SQLite native format decoder
     is invoked for MsgPack data. Call MsgPack decoder instead.

sql: Fix some warnings and remove dead code

sql: Fix IdxKeyCompare once more

 * Incorrect format structure attributed was consulted for the field
   count.

 * Outdated formats were handled incorrectly - TUPLE_OFFSET_SLOT_NIL
   doesn't necessary designate a field immediately following the
   previously processed one.

 * Logic behind `OP_Next` allows to iterate over invalidated
   cursors. If cursor is invalid - `sqlite3BtreeNext()` will return
   non zero status to the engine. Respect this behaviour in
   `tarantoolSqlite3Next`.

 * Similar change in `tarantoolSqlite3Previous`.

     * While applying ONE_PASS[SINGLE|MULTI] optimizatoin positioning of
     one of the indexes is performed by another code, so it is not subject
     of `Seek`. Add explicit check that those optimizations are engaged.
     * Add regression test.

     * As far as Tarantool is unable to compare `integer` and `real` classes,
     make them generic `scalars`.
     * Fix `sqlite-tcl` test in `sort/nosort` part since `WITHOUT ROWID` is now
     set by default.

sql: Initialize `default_rc` in record unpack.

     * `MsgPack` conversion to `UnpackedRecord` doesn't initialize
     `default_rc`. This ends up with flakiness in where.test.lua:14.7.2
     * The same issue affects sort.test.lua, fixed by this patch.

     * Deleting tuple invalidates pointing iterator, so either iterator or
     byte-coded loop should be modified in future. Disable this optimization
     so far.
     * Add regression test.

sql: Fix check for view when enforcing W/O ROWID

sql: Fix comparator

BLOB vs non-BLOB case.

 * Introduce machinery to perform a task with a large stack; this
   workaround is necessary since fiber stack is only 64K.
 * Invoke sqlite3LockAndPrepare with a large stack.

sql: Allow larger index id

Allocate 10 bits in pageNo for index id. This is a temporary workaround
to allow some tests to pass.

sql: Fix issue in comparator

sql: Fix _space definition

Closes #2187

sql: Call Tarantool in OP_Clear explicitly

     * Restore sqlite3BtreeClearTable function since it's also used to
       truncate ephemeral tables which are backed by SQLite BTree-s.
       Ephemeral tables are needed to implement some SORT-s and JOIN-s.
     * OP_Clear truncates regular tables, backed by Tarantool storage
       layer. Invoke Tarantool truncate method explicitly from opcode
       handler.
     * Add regression test.

     * If `NULL` is allowed for column - demote column affinity
     to `scalar` in Tarantool.
     * Set `unique` Tarantool's property to false if any of
     index columns may contain `NULL`.
     * If column may not contain NULL then use OP_Rewind/OP_last
     to fetch first(last) indexed element during searchg for MAX/MIN.
     * Add tests for MIN() and MAX(), when value is indexed.

sql: Fix field extraction via field_map

Fix the crash when encountered a tuple with outdated field_map.

     * Allow NIL to belong to `scalar` class.
     * Extend comparator to always return false when comparing with `NIL`s.
     * Should be removed as fas as #1557 is implemented.

sql: Fix expected result in engine/indices_any_type.test.lua.

	test/engine
	* indices_any_type.result: Update expected result as comparison
	against nulls changed.

IdxDelete uses movetoUnpacked to position a cursor, followed by delete.
The implementation of movetoUnpacked for Tarantool cursors needs to know
the invoking opcode to accurately map expected semantics to Tarantool
iterator type. Handle OP_IdxDelete as well.

Copy a key into a buffer in ta_cursor structure for EQ/REQ iterators.
Those iterator types save a key pointer internally and will compare the
current element with the key for each next() invocation.

Previously, the key was deallocated immediately after positioning the
iterator.

sql: Fix several BtCursor assertions in sqlite

Tarantool cursors are disguised as BtCursor-s.  Those assertions
failed because some fields like a path taken down the BTree and BTree
page parser state weren't initialized.

Modify assertions to ignore Tarantool cursors.

sql: Fix sqlite3VdbeCompareMsgpack for MsgPack BIN

Instead of mp_decode_binl, mp_decode_strl was used. Fix the typo.

sql: Don't use sqlite3VdbeRecordUnpack in OP_Found

Record serialization format was switched to MsgPack, use
sqlite3VdbeRecordUpackMsgpack instead.

     * Make `Clear` btree method use direct box_truncate().
     * Add testcase.

sql: Decode NULL promoting the pointer (#2145).

     * MsgPack decode of NULL in SQL connector didn't promote parse pointer.
     Add proper promotion. Add test.

sql: Remove normalize_iter_type() from sql.c

Remove the function definition. Use a similar function from index.h

sql: Remove get_space_id() from sql.c

Use SQLITE_PAGENO_TO_SPACEID/SQLITE_PAGENO_TO_INDEXID instead.

Closes #2111.

To avoid asertion failure.

sql. Introduce TARANTOOL_INDEX.

  ... and use it instead of direct "_index".

sql. Introduce TARANTOOL_SYS_SPACE_NAME for "_space" ...

     ... also rename TARANTOOL_INDEX -> TARANTOOL_SYS_INDEX_NAME.

sql. Implement `DROP INDEX` statement (#2097).

     * lib/sqlite/src/build.c: Change DropIndex to remove from Tarantool's _index.
     Remove b-tree cleanup.
     * Add new test, fix existing to use `DROP INDEX`.

sql. Implement `DROP TABLE` SQL statement.

     * Add new test and fix existing to use `DROP TABLE` statement.
     * Remove now redundant destroyTable and destroyRootPage to avoid
     compiler warning.
     * Fix couple warning about unused var/arg in sqlite/src/build.c.

sql: Fix `DROP TABLE` to remove secondary indexes first.

Tarantool doesn't allow to remove primary index while secondary exist,
so remove secondary indexes first while droppping table.

sql: OP_IncMaxid

The opcode is specifically tailored for incrementing the max_id -
the internal key that track the maximum space id seen so far.

sql: Make CREATE TABLE work

sql: Use CREATE TABLE in tests, remove hack

Also no longer export sql_schema_put.

sql: Reject CREATE TABLE lacking WITHOUT ROWID clause

sql: Fix CREATE TABLE with implicit indices

sql: Several fixes to allow some tests to pass

 * Treat all CREATE TABLE as if WITHOUT ROWID clause was present.
 * Schema is loaded from Tarantool into additional SQLite instances
   (tcltestrunner creates additional database handles, doesn't use one
    provided by Tarantool.)

sql: Implement persistency for SQL schema (gh-2308).

	src/box/
	* alter.cc (opt_set): Support strings with length not limited
	to def->len.
	(key_def_new_from_tuple): Make external.
	(space_def_create_from_tuple): Ditto.
	(opt_encode): Skip for OPT_STRPTR.
	(key_opts_default): Use pointer instead of array for SQL,
	initialize sql_length variable.
	(space_opts_default): Ditto.
	(key_opts_reg): Use STRPTR instead of STR to store SQL value.
	(space_opts_reg): Ditto.
	* key_def.h (opt_type): add STRPTR.
	(key_opts): Replace array for storing SQL w/ pointer and its length.
	(space_opts): Ditto.
	* schema.cc (schema_init): Use update fields initialization.
	* sql.c (sql_schema_put): Move function.
	(tarantoolSqlite3LoadSchema): Move function. Remove rowid mentioning.
	Iterate over Tarantool `_space` and `_index` initializing SQL data
	structures.

	src/lib/sqlite/src/
	* prepare.c (sqlite3InitCallback): Detect SQL line length, pass it to
	sqlite3_prepare.
	(sqlite3InitCallback): Add sanity check for line length.
	(sqlite3InitOne): Initialize SQL length.
	* sqliteInt.h (struct InitData): Add SQL length to field.
	* vdbe.c (sqlite3VdbeExec): Initialize SQL length in OP_Prepare.

sql: Replace key_def w/ index_def after rebase.

	src/box/
	* alter.cc (index_def_new_from_tuple): Make "C" extern.
	(box_iterator_key_def): Rename to ...
	(box_iterator_index_def): ... this.
	* index.h (box_iterator_key_def): Update declaration to ...
	(box_iterator_key_def): ... this.
	* key_def.cc (index_opts_reg): Change argument from key_ to index_.
	* sql.c (tarantoolSqlite3Delete): Update key_def access.
	(tarantoolSqlite3IdxKeyCompare): Replace key_def w/ index_def.
	(key_def_new_from_tuple): Update declaration to ...
	(index_def_new_from_tuple): ... this.
	(tarantoolSqlite3LoadSchema): Use index_def instead of key_def.

sql: Privatize sql string in `index_opts`.

	src/box/
	* alter.cc (opt_set): if SQL passed is non empty - duplicate
	SQL query sting, set field to NULL otherwise.
	* key_def.cc (index_opts_destroy): New. Use free () to destroy
	privatized string.
	(index_opts_dup): New. Since sql is now privitely owned memory
	need to use malloc and memcpy in duplication of index_opts.
	(index_def_dup): Call index_opts duplicator.
	(index_def_delete): Call index_opts destructor.
	* key_def.h (struct index_opts): Remove `const` modifier from sql field.

sql: Remove explicit `sql_length` field.

	src/box/
	* alter.cc (opt_set): Insead of storing string length into `sql_length`
	simply use null terminated string. We won't allow UTF-16.
	* key_def.cc (index_opts_default): Remove `sql_length` initialization.
	(space_opts_default): Ditto.
	(index_opts_destroy): Ditto.
	(index_opts_dup): Use null-terminated string.
	* key_def.h (index_opts): Remove `sql_length`.
	(space_opts): Ditto.
	* schema.cc (schema_init): Remove `sql_length` initialization.
	* sql.c (tarantoolSqlite3LoadSchema): Remove `iSqlLength` initialization.

	src/lib/sqlite/src/
	* prepare.c (sqlite3InitCallback): Use checks for null-terminated strings.
	(sqlite3InitOne): Ditto.
	* sqliteInt.h (struct InitData): Remove `iSqlLength`.
	* vdbe.c (vdbeExec): Remove `iSqlLength` initialization.

Partially reverts 49293c4b

Closes #2420

testcase checked one behaviour, but ignores other

+ delete semicolons at end-of-line

`mkkeywordhash` is a program compiled from source, which
resides in extra/ dir. Put explicit dependency on this target for
mkkeywordhash.h generation rule.

sql. Add OP_ParseSchema2.

OP_ParseSchema adds new objects to the schema. A typical VDBE CREATE
program creates objects like BTree-s, updates sqlite-master and then
updates the schema with this opcode.

Originally, OP_ParseSchema fetched data from sqlite_master. Since we're
about to remove sqlite_master, OP_ParseSchema2 was created. The later
opcode perfoms similar to OP_ParseSchema; it extracts data from a range
of VDBE registers.

 * Introduce MsgPack subtype for Mem. A BLOB with MsgPack subtype is
   assumed to contain MsgPack-encoded data.
 * OP_Column sets MsgPack subtype when reading back arrays, maps or
   extensions from a row field.
 * OP_MakeRecord emits fields with MsgPack subtype verbatim (ordinary
   BLOBs are prefixed with MsgPack BIN header.)
 * Extend OP_Blob to support setting BLOB's subtype.

sql. Add functions to encode spaceid,iid in a PageNo

	* Tarantool handles indices update automatically during DELETE. Fix SQLite, add test.
	* Extract SQL-related hacks into separate module. Use it around SQL tests.

	* src/lib/sqlite/src/insert.c(sqlite3GenerateConstraintChecks): For primary key (and no rowid)
	put unchanged tuple into MakeRec.
	(sqlite3CompleteInsertion): Only do IdxInsert for primary key as it implies tuple add.
	* src/box/sql.c(tarantoolSqlite3MovetoUnpacked): Comment line which failed to compile.
	(tarantoolSqlite3MovetoUnpacked): Ditto.
	* test/box/sql-insert-unique.*: New test/output.

Data layout in Tarantool is very different from SQLite's.
Consider the following DDL statements:

CREATE TABLE foo (a, b PRIMARY KEY, c)
CREATE INDEX fI  ON foo(a)
INSERT INTO  foo VALUES (
  (1, "one", "junk1")
  (2, "two", "junk2")
)

In SQLite, two BTrees are created:

foo / primary key! other fields / -
 one!1|junk1
 two!2|junk2

fI / index key! primary key / -
 1!one
 2!two

In Tarantool, data is stored with the field order matching
the DDL -
 1|one|junk1
 2|two|junk2

Each index yields data in the same very format.

The following changes were made:

 * fix field number in Column-s at VDBE generation time;
 * fix IdxLE/IdxLT/IdxGE/IdxGT for index keys spanning non-adjacent
   tuple fields in an arbitrary order;
 * tell optimizer that every index can provide every column in order to
   eliminate lookups into the primary index.

Also update movetoUnpacked implementation to handle invocation from
several more opcodes (used in INSERT-s).

Handle tarantool error messages in SQL engine

The caller of SQLite API-s MUST receive Tarantool error messages intact.
They were formerly coerced to generic SQLITE_ERROR.

sql: Implement delete

Inconsistent CPP defines in sqlite:opcodes.c

As usual, there're plenty of ifdef-s in sqlite code. Since internal
sqlite source files are also used to build tarantool glue layer, it was
inconvenient to pass the defines via sqlite library target properties in
CMakeLists, so they are put into sqliteConfig.h instead.

The later file wasn't included by (generated) opcodes.c, causing misery.

sql: update the new iterator API to the latest 1.7. Report gh-2425