diff --git a/doc/sphinx/book/configuration/index.rst b/doc/sphinx/book/configuration/index.rst index 6853b42fb066e4174fb7eba26dfbb697f1786dae..6c0e359cafdc583db8d62375d14ce307d682ed54 100644 --- a/doc/sphinx/book/configuration/index.rst +++ b/doc/sphinx/book/configuration/index.rst @@ -88,7 +88,10 @@ If username:password is omitted, then 'guest' is assumed. Some examples: | username:password@host:port | notguest:sesame@mail.ru:3301 | +-----------------------------+------------------------------+ -In certain circumstances a Unix socket may be used where a URI is required. +In certain circumstances a Unix domain socket may be used +where a URI is expected, for example "unix/:/tmp/unix_domain_socket.sock" or +simply "/tmp/unix_domain_socket.sock". + ===================================================================== Initialization file diff --git a/doc/sphinx/dev_guide/building_from_source.rst b/doc/sphinx/dev_guide/building_from_source.rst index b5e8f612c862e4c56ca2180f69992c81f8199196..98cc8d426c132dfe27d5534abb8b6253c0d06316 100644 --- a/doc/sphinx/dev_guide/building_from_source.rst +++ b/doc/sphinx/dev_guide/building_from_source.rst @@ -38,12 +38,8 @@ explain what the steps are, then on the Internet you can look at some example sc 2. Set up python modules for running the test suite or creating documentation. This step is optional. Python modules are not necessary for building Tarantool itself, unless one intends to use the ``-DENABLE_DOC`` option in step 5 or the - "Run the test suite" option in step 7. Say: - - .. code-block:: bash - - python --version - + "Run the test suite" option in step 7. Say: |br| + :codenormal:`python --version` |br| You should see that the python version is greater than 2.6 -- preferably 2.7 -- and less than 3.0 @@ -108,12 +104,9 @@ explain what the steps are, then on the Internet you can look at some example sc sudo pip install -U sphinx 3. Use :code:`git` to download the latest source code from the - Tarantool 1.6 master branch on github.com. - - .. code-block:: bash - - cd ~ - git clone https://github.com/tarantool/tarantool.git ~/tarantool + Tarantool 1.6 master branch on github.com. |br| |br| + :code:`cd ~` |br| + :code:`git clone https://github.com/tarantool/tarantool.git ~/tarantool` 4. Use ``git`` again so that third-party contributions will be seen as well. This step is only necessary once, the first time you do a download. There @@ -123,7 +116,7 @@ explain what the steps are, then on the Internet you can look at some example sc .. code-block:: bash cd ~/tarantool - git submodule init --recursive + git submodule init git submodule update --recursive cd ../ @@ -132,7 +125,7 @@ explain what the steps are, then on the Internet you can look at some example sc 5. Use CMake to initiate the build. - .. code-block: bash + .. code-block:: bash cd ~/tarantool make clean # unnecessary, added for good luck @@ -140,21 +133,18 @@ explain what the steps are, then on the Internet you can look at some example sc cmake . # Start build with build type=Debug, no doc On some platforms it may be necessary to specify the C and C++ versions, - for example - - .. code-block: bash - - CC=gcc-4.8 CXX=g++-4.8 cmake . - + for example |br| + :code:`CC=gcc-4.8 CXX=g++-4.8 cmake .` |br| The option for specifying build type is :samp:`-DCMAKE_BUILD_TYPE={type}` where :samp:`{type} = None | Debug | Release | RelWithDebInfo | MinSizeRel` and a reasonable choice for production is ``-DCMAKE_BUILD_TYPE=RelWithDebInfo`` (``Debug`` is used only by project maintainers and ``Release`` is used only when the highest performance is required). - The option for asking to build documentation is ``-DENABLE_DOC=true|false`, + The option for asking to build documentation is :code:`-DENABLE_DOC=true|false`, which outputs HTML documentation (such as what you're reading now) to the - subdirectory doc/www/output/doc. Tarantool uses the Sphinx simplified markup system. + subdirectory doc/www/output/doc. Tarantool uses the `Sphinx <http://sphinx-doc.org/>`_ + simplified markup system. Since most users do not need to rebuild the documentation, the reasonable option is ``-DENABLE_DOC=false`` or just don't use the ``-DENABLE_DOC`` clause at all. @@ -183,7 +173,6 @@ explain what the steps are, then on the Internet you can look at some example sc # run tests using python PATH=~/tarantool/bin:$PATH ./test-run.py - The output should contain reassuring reports, for example: .. code-block:: bash @@ -209,24 +198,17 @@ explain what the steps are, then on the Internet you can look at some example sc rm ~/tarantool/bin/python rmdir ~/tarantool/bin - 8. Make an rpm. |br| This step is optional. It's only for people who want to redistribute Tarantool. Package maintainers who want to build with rpmbuild should consult the rpm-build instructions for the appropriate platform. This is the end of the list of steps to take for source downloads. -For your added convenience, github.com has README files with example scripts: - -* `README.CentOS <https://github.com/tarantool/tarantool/blob/master/README.CentOS>`_ - for CentOS 5.8, -* `README.FreeBSD <https://github.com/tarantool/tarantool/blob/master/README.FreeBSD>`_ - for FreeBSD 8.3, -* `README.MacOSX <https://github.com/tarantool/tarantool/blob/master/README.MacOSX>`_ - for Mac OS X `Lion`, -* `README.md <https://github.com/tarantool/tarantool/blob/master/README.md>`_ - for generic GNU/Linux. - +For your added convenience, github.com has README files with example scripts: |br| +`README.CentOS <https://github.com/tarantool/tarantool/blob/master/README.CentOS>`_ for CentOS 5.8, |br| +`README.FreeBSD <https://github.com/tarantool/tarantool/blob/master/README.FreeBSD>`_ for FreeBSD 10.1, |br| +`README.MacOSX <https://github.com/tarantool/tarantool/blob/master/README.MacOSX>`_ for Mac OS X `Lion`, |br| +`README.md <https://github.com/tarantool/tarantool/blob/master/README.md>`_ for generic GNU/Linux. |br| These example scripts assume that the intent is to download from the master branch, build the server (but not the documentation), and run tests after build. diff --git a/doc/sphinx/reference/console.rst b/doc/sphinx/reference/console.rst index 8949f15bddcd7446f3153bcce04ae47c5e04c693..ba1344ca5f0546b3e1d7bdc9878782fe94f4b727 100644 --- a/doc/sphinx/reference/console.rst +++ b/doc/sphinx/reference/console.rst @@ -10,64 +10,82 @@ host/port. .. module:: console -.. function:: connect(uri [, options]) +.. function:: connect(uri) - Connect to the server at :ref:`URI`, change the prompt from 'tarantool' to - 'host:port', and act henceforth as a client until the user ends the - session or types ``control-D``. + Connect to the server at :ref:`URI`, change the prompt from ':samp:`tarantool>`' to + ':samp:`{uri}>`', and act henceforth as a client until the user ends the + session or types :code:`control-D`. The console.connect function allows one Tarantool server, in interactive - mode, to access another Tarantool server over a TCP connection. Subsequent + mode, to access another Tarantool server. Subsequent requests will appear to be handled locally, but in reality the requests are being sent to the remote server and the local server is acting as a client. Once connection is successful, the prompt will change and subsequent requests are sent to, and executed on, the remote server. Results are displayed on the local server. To return to local mode, - enter ``control-D``. + enter :code:`control-D`. + + If the Tarantool server at :samp:`uri` requires authentication, the + connection might look something like: + :code:`console.connect('admin:secretpassword@distanthost.com:3301')`. There are no restrictions on the types of requests that can be entered, except those which are due to privilege restrictions -- by default the login to the remote server is done with user name = 'guest'. The remote server could allow for this by granting at least one privilege: - ``box.schema.user.grant('guest','execute','universe')``. + :code:`box.schema.user.grant('guest','execute','universe')`. - :param string uri: - :param table options: The options may be necessary if the Tarantool - server at host:port requires authentication. - In such a case the connection might look - something like: - ``console.connect('netbox:123@127.0.0.1'})`` + :param string uri: the URI of the remote server :return: nil :except: the connection will fail if the target Tarantool server - was not initiated with ``box.cfg{listen=...}``. - - .. code-block:: lua + was not initiated with :code:`box.cfg{listen=...}`. - tarantool> console = require('console') - --- - ... - tarantool> console.connect('198.18.44.44:3301') - --- - ... - 198.18.44.44:3301> -- prompt is telling us that server is remote + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`console = require('console')` + | :codenormal:`---` + | :codenormal:`...` + | :codenormal:`tarantool>` :codebold:`console.connect('198.18.44.44:3301')` + | :codenormal:`---` + | :codenormal:`...` + | :codenormal:`198.18.44.44:3301> -- prompt is telling us that server is remote` -.. function:: listen(URI) +.. function:: listen(uri) - Listen on URI. The primary way of listening for incoming requests - is via the connection-information string, or :ref:`URI`, specified in ``box.cfg{listen=...}``. + Listen on :ref:`URI`. The primary way of listening for incoming requests + is via the connection-information string, or URI, specified in :code:`box.cfg{listen=...}`. The alternative way of listening is via the URI - specified in ``console.listen(...)``. This alternative way is called + specified in :code:`console.listen(...)`. This alternative way is called "administrative" or simply "admin port". The listening is usually over a local host with a Unix socket, specified with host = 'unix/', port = 'path/to/something.sock'. - :param string uri: + :param string uri: the URI of the local server - The "admin" address is the :ref:`URI` to listen on for administrative + The "admin" address is the URI to listen on for administrative connections. It has no default value, so it must be specified if - connections will occur via telnet. It is not used unless assigned a - value. The parameters are expressed with :ref:`URI` = Universal Resource - Identifier format, for example "unix://unix_domain_socket", or as a + connections will occur via telnet. + The parameter is expressed with URI = Universal Resource + Identifier format, for example "/tmpdir/unix_domain_socket.sock", or as a numeric TCP port. Connections are often made with telnet. A typical port value is 3313. + + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`console = require('console')` + | :codenormal:`---` + | :codenormal:`...` + | + | :codenormal:`tarantool>` :codebold:`console.listen('unix/:/tmp/X.sock')` + | :codenormal:`... main/103/console/unix/:/tmp/X I> started` + | :codenormal:`---` + | :codenormal:`- fd: 9` + | |nbsp| |nbsp| :codenormal:`name:` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`host: unix/` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`family: AF_UNIX` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`type: SOCK_STREAM` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`protocol: 0` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`port: /tmp/X.sock` + | :codenormal:`...` + + + diff --git a/doc/sphinx/reference/fio.rst b/doc/sphinx/reference/fio.rst index ed459003be9a27e451cbbf59790153f940945b5d..9f9f39933835439c7f605c249d482e51ff181297 100644 --- a/doc/sphinx/reference/fio.rst +++ b/doc/sphinx/reference/fio.rst @@ -20,12 +20,11 @@ access the same file simultaneously. :return: path name :rtype: string - .. code-block:: lua - - tarantool> fio.pathjoin('/etc', 'default', 'myfile') - --- - - /etc/default/myfile - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.pathjoin('/etc', 'default', 'myfile')` + | :codenormal:`---` + | :codenormal:`- /etc/default/myfile` + | :codenormal:`...` .. function:: basename(path-name[, suffix]) @@ -38,12 +37,11 @@ access the same file simultaneously. :return: file name :rtype: string - .. code-block:: lua - - tarantool> fio.basename('/path/to/my.lua', '.lua') - --- - - my - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.basename('/path/to/my.lua', '.lua')` + | :codenormal:`---` + | :codenormal:`- my` + | :codenormal:`...` .. function:: dirname(path-name) @@ -54,12 +52,11 @@ access the same file simultaneously. :return: directory name, that is, path name except for file name. :rtype: string - .. code-block:: lua - - tarantool> fio.dirname('/path/to/my.lua') - --- - - /path/to/ - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.dirname('/path/to/my.lua')` + | :codenormal:`---` + | :codenormal:`- /path/to/` + | :codenormal:`...` ================================================= Common file manipulations @@ -74,12 +71,11 @@ access the same file simultaneously. :return: previous mask bits. :rtype: number - .. code-block:: lua - - tarantool> fio.umask(tonumber('755', 8)) -- pass 755 octal - --- - - 493 - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.umask(tonumber('755', 8)) -- pass 755 octal` + | :codenormal:`---` + | :codenormal:`- 493` + | :codenormal:`...` .. function:: lstat(path-name) stat(path-name) @@ -92,24 +88,23 @@ access the same file simultaneously. and other attributes. :rtype: table - .. code-block:: lua - - tarantool> fio.lstat('/etc') - --- - - inode: 1048577 - rdev: 0 - size: 12288 - atime: 1421340698 - mode: 16877 - mtime: 1424615337 - nlink: 160 - uid: 0 - blksize: 4096 - gid: 0 - ctime: 1424615337 - dev: 2049 - blocks: 24 - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.lstat('/etc')` + | :codenormal:`---` + | :codenormal:`- inode: 1048577` + | |nbsp| |nbsp| :codenormal:`rdev: 0` + | |nbsp| |nbsp| :codenormal:`size: 12288` + | |nbsp| |nbsp| :codenormal:`atime: 1421340698` + | |nbsp| |nbsp| :codenormal:`mode: 16877` + | |nbsp| |nbsp| :codenormal:`mtime: 1424615337` + | |nbsp| |nbsp| :codenormal:`nlink: 160` + | |nbsp| |nbsp| :codenormal:`uid: 0` + | |nbsp| |nbsp| :codenormal:`blksize: 4096` + | |nbsp| |nbsp| :codenormal:`gid: 0` + | |nbsp| |nbsp| :codenormal:`ctime: 1424615337` + | |nbsp| |nbsp| :codenormal:`dev: 2049` + | |nbsp| |nbsp| :codenormal:`blocks: 24` + | :codenormal:`...` .. function:: mkdir(path-name) rmdir(path-name) @@ -121,12 +116,11 @@ access the same file simultaneously. :return: true if success, false if failure. :rtype: boolean - .. code-block:: lua - - tarantool> fio.mkdir('/etc') - --- - - false - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.mkdir('/etc')` + | :codenormal:`---` + | :codenormal:`- false` + | :codenormal:`...` .. function:: glob(path-name) @@ -140,26 +134,36 @@ access the same file simultaneously. :rtype: table :except: nil on failure. - .. code-block:: lua - - tarantool> fio.glob('/etc/x*') - --- - - - /etc/xdg - - /etc/xml - - /etc/xul-ext - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.glob('/etc/x*')` + | :codenormal:`---` + | :codenormal:`- - /etc/xdg` + | |nbsp| |nbsp| :codenormal:`- /etc/xml` + | |nbsp| |nbsp| :codenormal:`- /etc/xul-ext` + | :codenormal:`...` .. function:: tempdir() Return the name of a directory that can be used to store temporary files. - .. code-block:: lua + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.tempdir()` + | :codenormal:`---` + | :codenormal:`- /tmp/lG31e7` + | :codenormal:`...` + + +.. function:: cwd() + + Return the name of the current working directory. + + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.cwd()` + | :codenormal:`---` + | :codenormal:`- /home/username/tarantool_sandbox` + | :codenormal:`...` - tarantool> fio.tempdir() - --- - - /tmp/lG31e7 - ... .. function:: link (src , dst) symlink (src , dst) @@ -176,16 +180,15 @@ access the same file simultaneously. success, false if failure. ``fio.readlink`` returns the link value if success, nil if failure. - .. code-block:: lua - - tarantool> fio.link('/home/username/tmp.txt', '/home/username/tmp.txt2') - --- - - true - ... - tarantool> fio.unlink('/home/pgulutzan/tmp.txt2') - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.link('/home/username/tmp.txt', '/home/username/tmp.txt2')` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` + | :codenormal:`tarantool>` :codebold:`fio.unlink('/home/pgulutzan/tmp.txt2')` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. function:: rename(path-name, new-path-name) @@ -197,12 +200,11 @@ access the same file simultaneously. :return: true if success, false if failure. :rtype: boolean - .. code-block:: lua - - tarantool> fio.rename('/home/username/tmp.txt', '/home/username/tmp.txt2') - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.rename('/home/username/tmp.txt', '/home/username/tmp.txt2')` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. function:: chown(path-name, owner-user, owner-group) chmod(path-name, new-rights) @@ -214,16 +216,15 @@ access the same file simultaneously. :param string owner-group: new group uid. :param number new-rights: new permissions - .. code-block:: lua - - tarantool> fio.chmod('/home/username/tmp.txt', tonumber('0755', 8)) - --- - - true - ... - fio.chown('/home/username/tmp.txt', 'username', 'username') - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.chmod('/home/username/tmp.txt', tonumber('0755', 8))` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` + | :codenormal:`tarantool>` :codebold:`fio.chown('/home/username/tmp.txt', 'username', 'username')` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. function:: truncate(path-name, new-size) @@ -235,12 +236,11 @@ access the same file simultaneously. :return: true if success, false if failure. :rtype: boolean - .. code-block:: lua - - tarantool> fio.truncate('/home/username/tmp.txt', 99999) - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.truncate('/home/username/tmp.txt', 99999)` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. function:: sync() @@ -249,12 +249,11 @@ access the same file simultaneously. :return: true if success, false if failure. :rtype: boolean - .. code-block:: lua - - tarantool> fio.sync() - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fio.sync()` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. function:: open(path-name [, flags]) @@ -269,15 +268,14 @@ access the same file simultaneously. :rtype: userdata :except: nil - .. code-block:: lua - - tarantool> fh = fio.open('/home/username/tmp.txt', {'O_RDWR', 'O_APPEND'}) - --- - ... - tarantool> fh -- display file handle returned by fio.open - --- - - fh: 11 - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh = fio.open('/home/username/tmp.txt', {'O_RDWR', 'O_APPEND'})` + | :codenormal:`---` + | :codenormal:`...` + | :codenormal:`tarantool>` :codebold:`fh -- display file handle returned by fio.open` + | :codenormal:`---` + | :codenormal:`- fh: 11` + | :codenormal:`...` .. class:: file-handle @@ -289,12 +287,11 @@ access the same file simultaneously. :return: true if success, false on failure. :rtype: boolean - .. code-block:: lua - - tarantool> fh:close() -- where fh = file-handle - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:close() -- where fh = file-handle` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. method:: pread(count, offset) pwrite(new-string, offset) @@ -310,14 +307,13 @@ access the same file simultaneously. :return: ``fh:pwrite`` returns true if success, false if failure. ``fh:pread`` returns the data that was read, or nil if failure. - .. code-block:: lua - - tarantool> fh:pread(25, 25) - --- - - |- - elete from t8// - insert in - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:pread(25, 25)` + | :codenormal:`---` + | :codenormal:`- |-` + | |nbsp| |nbsp| :codenormal:`elete from t8//` + | |nbsp| |nbsp| :codenormal:`insert in` + | :codenormal:`...` .. method:: read(count) write(new-string) @@ -338,12 +334,11 @@ access the same file simultaneously. :return: ``fh:write`` returns true if success, false if failure. ``fh:read`` returns the data that was read, or nil if failure. - .. code-block:: lua - - tarantool> fh:write('new data') - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:write('new data')` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. method:: truncate(new-size) @@ -354,12 +349,11 @@ access the same file simultaneously. :return: true if success, false if failure. :rtype: boolean - .. code-block:: lua - - tarantool> fh:truncate(0) - --- - - true - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:truncate(0)` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` .. method:: seek(position [, offset-from]) @@ -374,12 +368,11 @@ access the same file simultaneously. :rtype: number :except: nil - .. code-block:: lua - - tarantool> fh:seek(20, 'SEEK_SET') - --- - - 20 - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:seek(20, 'SEEK_SET')` + | :codenormal:`---` + | :codenormal:`- 20` + | :codenormal:`...` .. method:: stat() @@ -391,24 +384,23 @@ access the same file simultaneously. :return: details about the file. :rtype: table - .. code-block:: lua - - tarantool> fh:stat() - --- - - inode: 729866 - rdev: 0 - size: 100 - atime: 1409429855 - mode: 33261 - mtime: 1409430660 - nlink: 1 - uid: 1000 - blksize: 4096 - gid: 1000 - ctime: 1409430660 - dev: 2049 - blocks: 8 - ... + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:stat()` + | :codenormal:`---` + | :codenormal:`- inode: 729866` + | |nbsp| |nbsp| :codenormal:`rdev: 0` + | |nbsp| |nbsp| :codenormal:`size: 100` + | |nbsp| |nbsp| :codenormal:`atime: 1409429855` + | |nbsp| |nbsp| :codenormal:`mode: 33261` + | |nbsp| |nbsp| :codenormal:`mtime: 1409430660` + | |nbsp| |nbsp| :codenormal:`nlink: 1` + | |nbsp| |nbsp| :codenormal:`uid: 1000` + | |nbsp| |nbsp| :codenormal:`blksize: 4096` + | |nbsp| |nbsp| :codenormal:`gid: 1000` + | |nbsp| |nbsp| :codenormal:`ctime: 1409430660` + | |nbsp| |nbsp| :codenormal:`dev: 2049` + | |nbsp| |nbsp| :codenormal:`blocks: 8` + | :codenormal:`...` .. method:: fsync() fdatasync() @@ -420,9 +412,9 @@ access the same file simultaneously. :param userdata fh: file-handle as returned by ``fio.open()``. :return: true if success, false if failure. - .. code-block:: lua + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`fh:fsync()` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` - tarantool> fh:fsync() - --- - - true - ... diff --git a/doc/sphinx/reference/index.rst b/doc/sphinx/reference/index.rst index 602b48125c4abb3ecc7d1df1ecb51ae29dc0f6b9..9025ef6a01d36bf5d070729255023a5f70cde2ff 100644 --- a/doc/sphinx/reference/index.rst +++ b/doc/sphinx/reference/index.rst @@ -24,3 +24,5 @@ other expirationd shard + tarantool + diff --git a/doc/sphinx/reference/shard.rst b/doc/sphinx/reference/shard.rst index aa003ae0d6d956fbe8d91b92a53b43deb9a48f07..dbb9ec5ac625fcd54961c77b7c4f3836e59ea9bb 100644 --- a/doc/sphinx/reference/shard.rst +++ b/doc/sphinx/reference/shard.rst @@ -4,20 +4,276 @@ .. module:: shard -With `sharding`_, -the tuples of a tuple set are distributed -to multiple nodes, with a Tarantool database server on each node. With this arrangement, -each server is handling only a subset of the total data, so larger loads can be -handled by simply adding more computers to a network. +With sharding, the tuples of a tuple set are distributed to multiple nodes, +with a Tarantool database server on each node. With this arrangement, +each server is handling only a subset of the total data, +so larger loads can be handled by simply adding more computers to a network. -The Tarantool shard package has facilities -for creating or redistributing or cleaning up shards, as well as analogues for the -data-manipulation functions of the box library (select, insert, replace, update, delete). +The Tarantool shard package has facilities for creating shards, +as well as analogues for the data-manipulation functions of the box library +(select, insert, replace, update, delete). -Some details are `On the shard section of github`_. +First some terminology: |br| +**Consistent Hash** ... +The shard package distributes according to a hash algorithm, +that is, it applies a hash function to a tuple's primary-key value +in order to decide which shard the tuple belongs to. +The hash function is `consistent`_ +so that changing the number of tuples will not affect results for many keys. +The specific hash function that the shard package uses is +guava.digest in the :codeitalic:`digest` package. |br| +**Queue** ... +A temporary list of recent update requests. Sometimes called "batching". +Since updates to a sharded database can be slow, it may +speed up throughput to send requests to a queue rather +than wait for the update to finish on ever node. +The shard package has functions for adding requests to the queue, +which it will process without further intervention. +Queuing is optional. |br| +**Redundancy** ... +The number of replicas in each shard. |br| +**Replica** ... +A complete copy of the data. +The shard package handles both sharding and replication. +One shard can contain one or more replicas. +When a write occurs, the write is attempted on every replica in turn. +The shard package does not use the built-in replication feature. |br| +**Shard** ... +A subset of the tuples in the database partitioned according to the +value returned by the consistent hash function. Usually each shard +is on a separate node, or a separate set of nodes (for example if +redundancy = 3 then the shard will be on three nodes). |br| +**Zone** ... +A physical location where the nodes are closely connected, with +the same security and backup and access points. The simplest example +of a zone is a single computer with a single tarantool-server instance. +A shard's replicas should be in different zones. +The shard package is distributed separately from the main tarantool package. +To acquire it, do a separate install. For example on Ubuntu say |br| +sudo apt-get install tarantool-shard tarantool-pool |br| +Or, download from github tarantool/shard +and compile as described in the README. Then, before using the package, say +shard = require('shard') -.. _sharding: https://en.wikipedia.org/wiki/Sharding -.. _On the shard section of github: https://github.com/tarantool/shard +The most important function is |br| +:samp:`shard.init ({shard-configuration})` |br| +This must be called for every shard. +The shard-configuration is a table with these fields: |br| +* servers (a list of URIs of nodes and the zones the nodes are in) |br| +* login (the user name which applies for accessing via the shard package) |br| +* password (the password for the login) |br| +* redundancy (a number, minimum 1) |br| +* binary (a port number that this host is listening on, on the current host) +(distinguishable from the 'listen' port specified by box.cfg) |br| +Possible Errors: Redundancy should not be greater than the number of servers; +the servers must be alive; two replicas of the same shard should not be in the same zone. + | EXAMPLE: shard.init syntax for one shard + | :codenormal:`-- The number of replicas per shard (redundancy) is 3.` + | :codenormal:`-- The number of servers is 3.` + | :codenormal:`-- The shard package will conclude that there is only one shard.` + | :codenormal:`cfg = {` + | |nbsp| |nbsp| :codenormal:`servers = {` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33131', zone = '1' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33132', zone = '2' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33133', zone = '3' };` + | |nbsp| |nbsp| :codenormal:`};` + | |nbsp| |nbsp| :codenormal:`login = 'tester';` + | |nbsp| |nbsp| :codenormal:`password = 'pass';` + | |nbsp| |nbsp| :codenormal:`redundancy = 3;` + | |nbsp| |nbsp| :codenormal:`binary = 33131;` + | :codenormal:`}` + | :codenormal:`server.init(cfg)` + | + | :codenormal:`EXAMPLE: shard.init syntax for three shards` + | :codenormal:`-- This describes three shards.` + | :codenormal:`-- Each shard has two replicas.` + | :codenormal:`-- Since the number of servers is 7, and the number` + | :codenormal:`-- of replicas per server is 2, and dividing 7 / 2` + | :codenormal:`-- leaves a remainder of 1, one of the servers will` + | :codenormal:`-- not be used. This is not necessarily an error,` + | :codenormal:`-- because perhaps one of the servers in the list is` + | :codenormal:`-- not alive.` + | :codenormal:`cfg = {` + | |nbsp| |nbsp| :codenormal:`servers = {` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33131', zone = '1' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33132', zone = '2' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33133', zone = '1' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33134', zone = '2' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33135', zone = '1' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33136', zone = '2' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33137', zone = '1' };` + | |nbsp| |nbsp| :codenormal:`};` + | |nbsp| |nbsp| :codenormal:`login = 'tester';` + | |nbsp| |nbsp| :codenormal:`password = 'pass';` + | |nbsp| |nbsp| :codenormal:`redundancy = 2;` + | |nbsp| |nbsp| :codenormal:`binary = 33131;` + | :codenormal:`}` + | :codenormal:`server.init(cfg)` + +:samp:`shard.{space_name}.insert` :code:`{...}` etc. |br| +Every data-access function in the box package +has an analogue in the shard package, so (for +example) to insert in table T in a sharded database one +simply says "shard.T:insert{...}" instead of +"box.T:insert{...}". +A "shard.T:select{}" request without a primary key will search all shards. + +:samp:`q_shard.{space_name}.insert` {:code:`{...}` etc. |br| +Every queued data-access function has an analogue in the shard package. +The user must add an operation_id. The details of queued +data-access functions, and of maintenance-related functions, +are on `the shard section of github`_. + + | :codenormal:`EXAMPLE -- Shard, Minimal Configuration` + | :codenormal:`-- There is only one` + | :codenormal:`-- shard, and that shard contains only one replica.` + | :codenormal:`-- So this isn't illustrating the features of either` + | :codenormal:`-- replication or sharding, it's only illustrating`` + | :codenormal:`-- what the syntax is,` + | :codenormal:`-- and what the messages look like,` + | :codenormal:`-- that anyone could duplicate in a minute or two` + | :codenormal:`-- with the magic of cut-and-paste.` + | :codenormal:`mkdir ~/tarantool_sandbox_1` + | :codenormal:`cd ~/tarantool_sandbox_1` + | :codenormal:`rm -r *.snap` + | :codenormal:`rm -r *.xlog` + | :codenormal:`~/tarantool-master/src/tarantool` + | :codenormal:`box.cfg{listen = 3301}` + | :codenormal:`box.schema.create_space('tester')` + | :codenormal:`box.space.tester:create_index('primary', {})` + | :codenormal:`box.schema.user.passwd('admin', 'password')` + | :codenormal:`console = require('console')` + | :codenormal:`console.delimiter('!')` + | :codenormal:`cfg = {` + | |nbsp| |nbsp| :codenormal:`servers = {` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3301', zone = '1' };` + | |nbsp| |nbsp| :codenormal:`};` + | |nbsp| |nbsp| :codenormal:`login = 'admin';` + | |nbsp| |nbsp| :codenormal:`password = 'password';` + | |nbsp| |nbsp| :codenormal:`redundancy = 1;` + | |nbsp| |nbsp| :codenormal:`binary = 3301;` + | :codenormal:`}!` + | :codenormal:`shard = require('shard')!` + | :codenormal:`shard.init(cfg)!` + | :codenormal:`-- Now put something in ...!` + | :codenormal:`shard.tester:insert{1,'Tuple #1'}!` + +If one cuts and pastes the above, then the result, +showing only the requests and responses for shard.init +and shard.tester, should look approximately like this: + + | :codenormal:`tarantool>` :codebold:`shard.init(cfg)!` + | :codenormal:`2015-08-09 ... I> Sharding initialization started...` + | :codenormal:`2015-08-09 ... I> establishing connection to cluster servers...` + | :codenormal:`2015-08-09 ... I> - localhost:3301 - connecting...` + | :codenormal:`2015-08-09 ... I> - localhost:3301 - connected` + | :codenormal:`2015-08-09 ... I> connected to all servers` + | :codenormal:`2015-08-09 ... I> started` + | :codenormal:`2015-08-09 ... I> redundancy = 1` + | :codenormal:`2015-08-09 ... I> Zone len=1 THERE` + | :codenormal:`2015-08-09 ... I> Adding localhost:3301 to shard 1` + | :codenormal:`2015-08-09 ... I> Zone len=1 THERE` + | :codenormal:`2015-08-09 ... I> shards = 1` + | :codenormal:`2015-08-09 ... I> Done` + | :codenormal:`---` + | :codenormal:`- true` + | :codenormal:`...` + | + | :codenormal:`tarantool>` :codebold:`-- Now put something in ...!` + | :codenormal:`---` + | :codenormal:`...` + | + | :codenormal:`tarantool>` :codebold:`shard.tester:insert{1,'Tuple #1'}!` + | :codenormal:`---` + | :codenormal:`- - [1, 'Tuple #1']` + | :codenormal:`...` + + + + + | + | + | :codenormal:`EXAMPLE -- Shard, Scaling Out` + | :codenormal:`-- There are two shards, and each shard contains one replica.` + | :codenormal:`-- This requires two nodes. In real life the two nodes would` + | :codenormal:`-- be two computers, but for this illustration the requirement` + | :codenormal:`-- is merely: start two shells, which we'll call Terminal#1 and` + | :codenormal:`-- Terminal #2.` + | + | :codenormal:`-- On Terminal #1, say:` + | :codenormal:`mkdir ~/tarantool_sandbox_1` + | :codenormal:`cd ~/tarantool_sandbox_1` + | :codenormal:`rm -r *.snap` + | :codenormal:`rm -r *.xlog` + | :codenormal:`~/tarantool-master/src/tarantool` + | :codenormal:`box.cfg{listen = 3301}` + | :codenormal:`box.schema.create_space('tester')` + | :codenormal:`box.space.tester:create_index('primary', {})` + | :codenormal:`box.schema.user.passwd('admin', 'password')` + | :codenormal:`console = require('console')` + | :codenormal:`console.delimiter('!')` + | :codenormal:`cfg = {` + | |nbsp| |nbsp| :codenormal:`servers = {` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3301', zone = '1' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3302', zone = '2' };` + | |nbsp| |nbsp| :codenormal:`};` + | |nbsp| |nbsp| :codenormal:`login = 'admin';` + | |nbsp| |nbsp| :codenormal:`password = 'password';` + | |nbsp| |nbsp| :codenormal:`redundancy = 1;` + | |nbsp| |nbsp| :codenormal:`binary = 3301;` + | |nbsp| |nbsp| :codenormal:`}!` + | |nbsp| |nbsp| :codenormal:`shard = require('shard')!` + | |nbsp| |nbsp| :codenormal:`shard.init(cfg)!` + | |nbsp| |nbsp| :codenormal:`-- Now put something in ...!` + | |nbsp| |nbsp| :codenormal:`shard.tester:insert{1,'Tuple #1'}!` + | + | -- On Terminal #2, say: + | :codenormal:`mkdir ~/tarantool_sandbox_2` + | :codenormal:`cd ~/tarantool_sandbox_2` + | :codenormal:`rm -r *.snap` + | :codenormal:`rm -r *.xlog` + | :codenormal:`~/tarantool-master/src/tarantool` + | :codenormal:`box.cfg{listen = 3302}` + | :codenormal:`box.schema.create_space('tester')` + | :codenormal:`box.space.tester:create_index('primary', {})` + | :codenormal:`box.schema.user.passwd('admin', 'password')` + | :codenormal:`console = require('console')` + | :codenormal:`console.delimiter('!')` + | :codenormal:`cfg = {` + | |nbsp| |nbsp| :codenormal:`servers = {` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3301', zone = '1' };` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3302', zone = '2' };` + | |nbsp| |nbsp| :codenormal:`};` + | |nbsp| |nbsp| :codenormal:`login = 'admin';` + | |nbsp| |nbsp| :codenormal:`password = 'password';` + | |nbsp| |nbsp| :codenormal:`redundancy = 1;` + | |nbsp| |nbsp| :codenormal:`binary = 3302;` + | :codenormal:`}!` + | :codenormal:`shard = require('shard')!` + | :codenormal:`shard.init(cfg)!` + | :codenormal:`-- Now get something out ...!` + | :codenormal:`shard.tester:select{1}!` + +What will appear on Terminal #1 is: a loop of +error messages saying "Connection refused" and +"server check failure". This is normal. It will +go on until Terminal #2 process starts. + +What will appear on Terminal #2, at the end, +should look like this: + +| :codenormal:`tarantool>shard.tester:select{1}!` +| :codenormal:`---` +| :codenormal:`- - - [1, 'Tuple #1']` +| :codenormal:`...` + +This shows that what was inserted by Terminal #1 +can be selected by Terminal #2, via the shard package. + +Details are on `the shard section of github`_. + +.. _consistent: https://en.wikipedia.org/wiki/Consistent_hashing +.. _the shard section of github: https://github.com/tarantool/shard diff --git a/doc/sphinx/reference/tarantool.rst b/doc/sphinx/reference/tarantool.rst new file mode 100644 index 0000000000000000000000000000000000000000..526c922917200f9e7703209dba22113004ea8c5a --- /dev/null +++ b/doc/sphinx/reference/tarantool.rst @@ -0,0 +1,35 @@ +------------------------------------------------------------------------------- + Package `tarantool` +------------------------------------------------------------------------------- + +.. module:: tarantool + +By saying :code:`require('tarantool')`, one can answer +some questions about how the tarantool server was built, +such as "what flags were used", or "what was the version +of the compiler". Additionally one can see the uptime +and the server version and the process id, but those +information items are more commonly accessed with +:func:`box.info`. + + | EXAMPLE + | :codenormal:`tarantool>` :codebold:`tarantool = require('tarantool')` + | :codenormal:`---` + | :codenormal:`...` + | |nbsp| + | :codenormal:`tarantool>` :codebold:`tarantool` + | :codenormal:`---` + | :codenormal:`- build:` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`flags: ' -fno-omit-frame-pointer -fno-stack-protector` + | |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`-fopenmp -msse2 -std=c11 -Wall` + | |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`-fno-gnu89-inline'` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`target: Linux-x86_64-Release` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`compiler: /usr/bin/cc /usr/bin/c++` + | |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local` + | |nbsp| |nbsp| :codenormal:`uptime: 'function: 0x4109a670'` + | |nbsp| |nbsp| :codenormal:`version: 1.6.0-3142-g63af2df` + | |nbsp| |nbsp| :codenormal:`pid: 'function: 0x4109a690'` + | :codenormal:`...` + + +