Skip to content
Snippets Groups Projects
Commit 31d5f1c3 authored by Konstantin Osipov's avatar Konstantin Osipov
Browse files

Merge remote-tracking branch 'origin/master' into 1.6

parents f4ad19d3 b0eea2c7
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
doc/sphinx/book/configuration/index.rst
+ 4
− 1
View file @ 31d5f1c3
......@@ -88,7 +88,10 @@ If username:password is omitted, then 'guest' is assumed. Some examples:
| username:password@host:port | notguest:sesame@mail.ru:3301 |
+-----------------------------+------------------------------+
In certain circumstances a Unix socket may be used where a URI is required.
In certain circumstances a Unix domain socket may be used
where a URI is expected, for example "unix/:/tmp/unix_domain_socket.sock" or
simply "/tmp/unix_domain_socket.sock".
=====================================================================
Initialization file
......
doc/sphinx/dev_guide/building_from_source.rst
+ 17
− 35
View file @ 31d5f1c3
......@@ -38,12 +38,8 @@ explain what the steps are, then on the Internet you can look at some example sc
2. Set up python modules for running the test suite or creating documentation.
This step is optional. Python modules are not necessary for building Tarantool
itself, unless one intends to use the ``-DENABLE_DOC`` option in step 5 or the
"Run the test suite" option in step 7. Say:
.. code-block:: bash
python --version
"Run the test suite" option in step 7. Say: |br|
:codenormal:`python --version` |br|
You should see that the python version is greater than 2.6 --
preferably 2.7 -- and less than 3.0
......@@ -108,12 +104,9 @@ explain what the steps are, then on the Internet you can look at some example sc
sudo pip install -U sphinx
3. Use :code:`git` to download the latest source code from the
Tarantool 1.6 master branch on github.com.
.. code-block:: bash
cd ~
git clone https://github.com/tarantool/tarantool.git ~/tarantool
Tarantool 1.6 master branch on github.com. |br| |br|
:code:`cd ~` |br|
:code:`git clone https://github.com/tarantool/tarantool.git ~/tarantool`
4. Use ``git`` again so that third-party contributions will be seen as well.
This step is only necessary once, the first time you do a download. There
......@@ -123,7 +116,7 @@ explain what the steps are, then on the Internet you can look at some example sc
.. code-block:: bash
cd ~/tarantool
git submodule init --recursive
git submodule init
git submodule update --recursive
cd ../
......@@ -132,7 +125,7 @@ explain what the steps are, then on the Internet you can look at some example sc
5. Use CMake to initiate the build.
.. code-block: bash
.. code-block:: bash
cd ~/tarantool
make clean # unnecessary, added for good luck
......@@ -140,21 +133,18 @@ explain what the steps are, then on the Internet you can look at some example sc
cmake . # Start build with build type=Debug, no doc
On some platforms it may be necessary to specify the C and C++ versions,
for example
.. code-block: bash
CC=gcc-4.8 CXX=g++-4.8 cmake .
for example |br|
:code:`CC=gcc-4.8 CXX=g++-4.8 cmake .` |br|
The option for specifying build type is :samp:`-DCMAKE_BUILD_TYPE={type}` where
:samp:`{type} = None | Debug | Release | RelWithDebInfo | MinSizeRel` and a
reasonable choice for production is ``-DCMAKE_BUILD_TYPE=RelWithDebInfo``
(``Debug`` is used only by project maintainers and ``Release`` is used only
when the highest performance is required).
The option for asking to build documentation is ``-DENABLE_DOC=true|false`,
The option for asking to build documentation is :code:`-DENABLE_DOC=true|false`,
which outputs HTML documentation (such as what you're reading now) to the
subdirectory doc/www/output/doc. Tarantool uses the Sphinx simplified markup system.
subdirectory doc/www/output/doc. Tarantool uses the `Sphinx <http://sphinx-doc.org/>`_
simplified markup system.
Since most users do not need to rebuild the documentation,
the reasonable option
is ``-DENABLE_DOC=false`` or just don't use the ``-DENABLE_DOC`` clause at all.
......@@ -183,7 +173,6 @@ explain what the steps are, then on the Internet you can look at some example sc
# run tests using python
PATH=~/tarantool/bin:$PATH ./test-run.py
The output should contain reassuring reports, for example:
.. code-block:: bash
......@@ -209,24 +198,17 @@ explain what the steps are, then on the Internet you can look at some example sc
rm ~/tarantool/bin/python
rmdir ~/tarantool/bin
8. Make an rpm. |br| This step is optional. It's only for people who want to
redistribute Tarantool. Package maintainers who want to build with rpmbuild
should consult the rpm-build instructions for the appropriate platform.
This is the end of the list of steps to take for source downloads.
For your added convenience, github.com has README files with example scripts:
* `README.CentOS <https://github.com/tarantool/tarantool/blob/master/README.CentOS>`_
for CentOS 5.8,
* `README.FreeBSD <https://github.com/tarantool/tarantool/blob/master/README.FreeBSD>`_
for FreeBSD 8.3,
* `README.MacOSX <https://github.com/tarantool/tarantool/blob/master/README.MacOSX>`_
for Mac OS X `Lion`,
* `README.md <https://github.com/tarantool/tarantool/blob/master/README.md>`_
for generic GNU/Linux.
For your added convenience, github.com has README files with example scripts: |br|
`README.CentOS <https://github.com/tarantool/tarantool/blob/master/README.CentOS>`_ for CentOS 5.8, |br|
`README.FreeBSD <https://github.com/tarantool/tarantool/blob/master/README.FreeBSD>`_ for FreeBSD 10.1, |br|
`README.MacOSX <https://github.com/tarantool/tarantool/blob/master/README.MacOSX>`_ for Mac OS X `Lion`, |br|
`README.md <https://github.com/tarantool/tarantool/blob/master/README.md>`_ for generic GNU/Linux. |br|
These example scripts assume that the intent is to download from the master
branch, build the server (but not the documentation), and run tests after build.
......
doc/sphinx/reference/console.rst
+ 50
− 32
View file @ 31d5f1c3
......@@ -10,64 +10,82 @@ host/port.
.. module:: console
.. function:: connect(uri [, options])
.. function:: connect(uri)
Connect to the server at :ref:`URI`, change the prompt from 'tarantool' to
'host:port', and act henceforth as a client until the user ends the
session or types ``control-D``.
Connect to the server at :ref:`URI`, change the prompt from ':samp:`tarantool>`' to
':samp:`{uri}>`', and act henceforth as a client until the user ends the
session or types :code:`control-D`.
The console.connect function allows one Tarantool server, in interactive
mode, to access another Tarantool server over a TCP connection. Subsequent
mode, to access another Tarantool server. Subsequent
requests will appear to be handled locally, but in reality the requests
are being sent to the remote server and the local server is acting as a
client. Once connection is successful, the prompt will change and
subsequent requests are sent to, and executed on, the remote server.
Results are displayed on the local server. To return to local mode,
enter ``control-D``.
enter :code:`control-D`.
If the Tarantool server at :samp:`uri` requires authentication, the
connection might look something like:
:code:`console.connect('admin:secretpassword@distanthost.com:3301')`.
There are no restrictions on the types of requests that can be entered,
except those which are due to privilege restrictions -- by default the
login to the remote server is done with user name = 'guest'. The remote
server could allow for this by granting at least one privilege:
``box.schema.user.grant('guest','execute','universe')``.
:code:`box.schema.user.grant('guest','execute','universe')`.
:param string uri:
:param table options: The options may be necessary if the Tarantool
server at host:port requires authentication.
In such a case the connection might look
something like:
``console.connect('netbox:123@127.0.0.1'})``
:param string uri: the URI of the remote server
:return: nil
:except: the connection will fail if the target Tarantool server
was not initiated with ``box.cfg{listen=...}``.
.. code-block:: lua
was not initiated with :code:`box.cfg{listen=...}`.
tarantool> console = require('console')
---
...
tarantool> console.connect('198.18.44.44:3301')
---
...
198.18.44.44:3301> -- prompt is telling us that server is remote
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`console = require('console')`
| :codenormal:`---`
| :codenormal:`...`
| :codenormal:`tarantool>` :codebold:`console.connect('198.18.44.44:3301')`
| :codenormal:`---`
| :codenormal:`...`
| :codenormal:`198.18.44.44:3301> -- prompt is telling us that server is remote`
.. function:: listen(URI)
.. function:: listen(uri)
Listen on URI. The primary way of listening for incoming requests
is via the connection-information string, or :ref:`URI`, specified in ``box.cfg{listen=...}``.
Listen on :ref:`URI`. The primary way of listening for incoming requests
is via the connection-information string, or URI, specified in :code:`box.cfg{listen=...}`.
The alternative way of listening is via the URI
specified in ``console.listen(...)``. This alternative way is called
specified in :code:`console.listen(...)`. This alternative way is called
"administrative" or simply "admin port".
The listening is usually over a local host with a Unix socket,
specified with host = 'unix/', port = 'path/to/something.sock'.
:param string uri:
:param string uri: the URI of the local server
The "admin" address is the :ref:`URI` to listen on for administrative
The "admin" address is the URI to listen on for administrative
connections. It has no default value, so it must be specified if
connections will occur via telnet. It is not used unless assigned a
value. The parameters are expressed with :ref:`URI` = Universal Resource
Identifier format, for example "unix://unix_domain_socket", or as a
connections will occur via telnet.
The parameter is expressed with URI = Universal Resource
Identifier format, for example "/tmpdir/unix_domain_socket.sock", or as a
numeric TCP port. Connections are often made with telnet.
A typical port value is 3313.
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`console = require('console')`
| :codenormal:`---`
| :codenormal:`...`
|
| :codenormal:`tarantool>` :codebold:`console.listen('unix/:/tmp/X.sock')`
| :codenormal:`... main/103/console/unix/:/tmp/X I> started`
| :codenormal:`---`
| :codenormal:`- fd: 9`
| |nbsp| |nbsp| :codenormal:`name:`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`host: unix/`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`family: AF_UNIX`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`type: SOCK_STREAM`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`protocol: 0`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`port: /tmp/X.sock`
| :codenormal:`...`
doc/sphinx/reference/fio.rst
+ 155
− 163
View file @ 31d5f1c3
......@@ -20,12 +20,11 @@ access the same file simultaneously.
:return: path name
:rtype: string
.. code-block:: lua
tarantool> fio.pathjoin('/etc', 'default', 'myfile')
---
- /etc/default/myfile
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.pathjoin('/etc', 'default', 'myfile')`
| :codenormal:`---`
| :codenormal:`- /etc/default/myfile`
| :codenormal:`...`
.. function:: basename(path-name[, suffix])
......@@ -38,12 +37,11 @@ access the same file simultaneously.
:return: file name
:rtype: string
.. code-block:: lua
tarantool> fio.basename('/path/to/my.lua', '.lua')
---
- my
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.basename('/path/to/my.lua', '.lua')`
| :codenormal:`---`
| :codenormal:`- my`
| :codenormal:`...`
.. function:: dirname(path-name)
......@@ -54,12 +52,11 @@ access the same file simultaneously.
:return: directory name, that is, path name except for file name.
:rtype: string
.. code-block:: lua
tarantool> fio.dirname('/path/to/my.lua')
---
- /path/to/
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.dirname('/path/to/my.lua')`
| :codenormal:`---`
| :codenormal:`- /path/to/`
| :codenormal:`...`
=================================================
Common file manipulations
......@@ -74,12 +71,11 @@ access the same file simultaneously.
:return: previous mask bits.
:rtype: number
.. code-block:: lua
tarantool> fio.umask(tonumber('755', 8)) -- pass 755 octal
---
- 493
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.umask(tonumber('755', 8)) -- pass 755 octal`
| :codenormal:`---`
| :codenormal:`- 493`
| :codenormal:`...`
.. function:: lstat(path-name)
stat(path-name)
......@@ -92,24 +88,23 @@ access the same file simultaneously.
and other attributes.
:rtype: table
.. code-block:: lua
tarantool> fio.lstat('/etc')
---
- inode: 1048577
rdev: 0
size: 12288
atime: 1421340698
mode: 16877
mtime: 1424615337
nlink: 160
uid: 0
blksize: 4096
gid: 0
ctime: 1424615337
dev: 2049
blocks: 24
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.lstat('/etc')`
| :codenormal:`---`
| :codenormal:`- inode: 1048577`
| |nbsp| |nbsp| :codenormal:`rdev: 0`
| |nbsp| |nbsp| :codenormal:`size: 12288`
| |nbsp| |nbsp| :codenormal:`atime: 1421340698`
| |nbsp| |nbsp| :codenormal:`mode: 16877`
| |nbsp| |nbsp| :codenormal:`mtime: 1424615337`
| |nbsp| |nbsp| :codenormal:`nlink: 160`
| |nbsp| |nbsp| :codenormal:`uid: 0`
| |nbsp| |nbsp| :codenormal:`blksize: 4096`
| |nbsp| |nbsp| :codenormal:`gid: 0`
| |nbsp| |nbsp| :codenormal:`ctime: 1424615337`
| |nbsp| |nbsp| :codenormal:`dev: 2049`
| |nbsp| |nbsp| :codenormal:`blocks: 24`
| :codenormal:`...`
.. function:: mkdir(path-name)
rmdir(path-name)
......@@ -121,12 +116,11 @@ access the same file simultaneously.
:return: true if success, false if failure.
:rtype: boolean
.. code-block:: lua
tarantool> fio.mkdir('/etc')
---
- false
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.mkdir('/etc')`
| :codenormal:`---`
| :codenormal:`- false`
| :codenormal:`...`
.. function:: glob(path-name)
......@@ -140,26 +134,36 @@ access the same file simultaneously.
:rtype: table
:except: nil on failure.
.. code-block:: lua
tarantool> fio.glob('/etc/x*')
---
- - /etc/xdg
- /etc/xml
- /etc/xul-ext
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.glob('/etc/x*')`
| :codenormal:`---`
| :codenormal:`- - /etc/xdg`
| |nbsp| |nbsp| :codenormal:`- /etc/xml`
| |nbsp| |nbsp| :codenormal:`- /etc/xul-ext`
| :codenormal:`...`
.. function:: tempdir()
Return the name of a directory that can be used to store temporary files.
.. code-block:: lua
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.tempdir()`
| :codenormal:`---`
| :codenormal:`- /tmp/lG31e7`
| :codenormal:`...`
.. function:: cwd()
Return the name of the current working directory.
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.cwd()`
| :codenormal:`---`
| :codenormal:`- /home/username/tarantool_sandbox`
| :codenormal:`...`
tarantool> fio.tempdir()
---
- /tmp/lG31e7
...
.. function:: link (src , dst)
symlink (src , dst)
......@@ -176,16 +180,15 @@ access the same file simultaneously.
success, false if failure. ``fio.readlink`` returns the link value
if success, nil if failure.
.. code-block:: lua
tarantool> fio.link('/home/username/tmp.txt', '/home/username/tmp.txt2')
---
- true
...
tarantool> fio.unlink('/home/pgulutzan/tmp.txt2')
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.link('/home/username/tmp.txt', '/home/username/tmp.txt2')`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
| :codenormal:`tarantool>` :codebold:`fio.unlink('/home/pgulutzan/tmp.txt2')`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. function:: rename(path-name, new-path-name)
......@@ -197,12 +200,11 @@ access the same file simultaneously.
:return: true if success, false if failure.
:rtype: boolean
.. code-block:: lua
tarantool> fio.rename('/home/username/tmp.txt', '/home/username/tmp.txt2')
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.rename('/home/username/tmp.txt', '/home/username/tmp.txt2')`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. function:: chown(path-name, owner-user, owner-group)
chmod(path-name, new-rights)
......@@ -214,16 +216,15 @@ access the same file simultaneously.
:param string owner-group: new group uid.
:param number new-rights: new permissions
.. code-block:: lua
tarantool> fio.chmod('/home/username/tmp.txt', tonumber('0755', 8))
---
- true
...
fio.chown('/home/username/tmp.txt', 'username', 'username')
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.chmod('/home/username/tmp.txt', tonumber('0755', 8))`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
| :codenormal:`tarantool>` :codebold:`fio.chown('/home/username/tmp.txt', 'username', 'username')`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. function:: truncate(path-name, new-size)
......@@ -235,12 +236,11 @@ access the same file simultaneously.
:return: true if success, false if failure.
:rtype: boolean
.. code-block:: lua
tarantool> fio.truncate('/home/username/tmp.txt', 99999)
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.truncate('/home/username/tmp.txt', 99999)`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. function:: sync()
......@@ -249,12 +249,11 @@ access the same file simultaneously.
:return: true if success, false if failure.
:rtype: boolean
.. code-block:: lua
tarantool> fio.sync()
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fio.sync()`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. function:: open(path-name [, flags])
......@@ -269,15 +268,14 @@ access the same file simultaneously.
:rtype: userdata
:except: nil
.. code-block:: lua
tarantool> fh = fio.open('/home/username/tmp.txt', {'O_RDWR', 'O_APPEND'})
---
...
tarantool> fh -- display file handle returned by fio.open
---
- fh: 11
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh = fio.open('/home/username/tmp.txt', {'O_RDWR', 'O_APPEND'})`
| :codenormal:`---`
| :codenormal:`...`
| :codenormal:`tarantool>` :codebold:`fh -- display file handle returned by fio.open`
| :codenormal:`---`
| :codenormal:`- fh: 11`
| :codenormal:`...`
.. class:: file-handle
......@@ -289,12 +287,11 @@ access the same file simultaneously.
:return: true if success, false on failure.
:rtype: boolean
.. code-block:: lua
tarantool> fh:close() -- where fh = file-handle
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:close() -- where fh = file-handle`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. method:: pread(count, offset)
pwrite(new-string, offset)
......@@ -310,14 +307,13 @@ access the same file simultaneously.
:return: ``fh:pwrite`` returns true if success, false if failure.
``fh:pread`` returns the data that was read, or nil if failure.
.. code-block:: lua
tarantool> fh:pread(25, 25)
---
- |-
elete from t8//
insert in
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:pread(25, 25)`
| :codenormal:`---`
| :codenormal:`- |-`
| |nbsp| |nbsp| :codenormal:`elete from t8//`
| |nbsp| |nbsp| :codenormal:`insert in`
| :codenormal:`...`
.. method:: read(count)
write(new-string)
......@@ -338,12 +334,11 @@ access the same file simultaneously.
:return: ``fh:write`` returns true if success, false if failure.
``fh:read`` returns the data that was read, or nil if failure.
.. code-block:: lua
tarantool> fh:write('new data')
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:write('new data')`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. method:: truncate(new-size)
......@@ -354,12 +349,11 @@ access the same file simultaneously.
:return: true if success, false if failure.
:rtype: boolean
.. code-block:: lua
tarantool> fh:truncate(0)
---
- true
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:truncate(0)`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
.. method:: seek(position [, offset-from])
......@@ -374,12 +368,11 @@ access the same file simultaneously.
:rtype: number
:except: nil
.. code-block:: lua
tarantool> fh:seek(20, 'SEEK_SET')
---
- 20
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:seek(20, 'SEEK_SET')`
| :codenormal:`---`
| :codenormal:`- 20`
| :codenormal:`...`
.. method:: stat()
......@@ -391,24 +384,23 @@ access the same file simultaneously.
:return: details about the file.
:rtype: table
.. code-block:: lua
tarantool> fh:stat()
---
- inode: 729866
rdev: 0
size: 100
atime: 1409429855
mode: 33261
mtime: 1409430660
nlink: 1
uid: 1000
blksize: 4096
gid: 1000
ctime: 1409430660
dev: 2049
blocks: 8
...
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:stat()`
| :codenormal:`---`
| :codenormal:`- inode: 729866`
| |nbsp| |nbsp| :codenormal:`rdev: 0`
| |nbsp| |nbsp| :codenormal:`size: 100`
| |nbsp| |nbsp| :codenormal:`atime: 1409429855`
| |nbsp| |nbsp| :codenormal:`mode: 33261`
| |nbsp| |nbsp| :codenormal:`mtime: 1409430660`
| |nbsp| |nbsp| :codenormal:`nlink: 1`
| |nbsp| |nbsp| :codenormal:`uid: 1000`
| |nbsp| |nbsp| :codenormal:`blksize: 4096`
| |nbsp| |nbsp| :codenormal:`gid: 1000`
| |nbsp| |nbsp| :codenormal:`ctime: 1409430660`
| |nbsp| |nbsp| :codenormal:`dev: 2049`
| |nbsp| |nbsp| :codenormal:`blocks: 8`
| :codenormal:`...`
.. method:: fsync()
fdatasync()
......@@ -420,9 +412,9 @@ access the same file simultaneously.
:param userdata fh: file-handle as returned by ``fio.open()``.
:return: true if success, false if failure.
.. code-block:: lua
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`fh:fsync()`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
tarantool> fh:fsync()
---
- true
...
doc/sphinx/reference/index.rst
+ 2
− 0
View file @ 31d5f1c3
......@@ -24,3 +24,5 @@
other
expirationd
shard
tarantool
doc/sphinx/reference/shard.rst
+ 267
− 11
View file @ 31d5f1c3
......@@ -4,20 +4,276 @@
.. module:: shard
With `sharding`_,
the tuples of a tuple set are distributed
to multiple nodes, with a Tarantool database server on each node. With this arrangement,
each server is handling only a subset of the total data, so larger loads can be
handled by simply adding more computers to a network.
With sharding, the tuples of a tuple set are distributed to multiple nodes,
with a Tarantool database server on each node. With this arrangement,
each server is handling only a subset of the total data,
so larger loads can be handled by simply adding more computers to a network.
The Tarantool shard package has facilities
for creating or redistributing or cleaning up shards, as well as analogues for the
data-manipulation functions of the box library (select, insert, replace, update, delete).
The Tarantool shard package has facilities for creating shards,
as well as analogues for the data-manipulation functions of the box library
(select, insert, replace, update, delete).
Some details are `On the shard section of github`_.
First some terminology: |br|
**Consistent Hash** ...
The shard package distributes according to a hash algorithm,
that is, it applies a hash function to a tuple's primary-key value
in order to decide which shard the tuple belongs to.
The hash function is `consistent`_
so that changing the number of tuples will not affect results for many keys.
The specific hash function that the shard package uses is
guava.digest in the :codeitalic:`digest` package. |br|
**Queue** ...
A temporary list of recent update requests. Sometimes called "batching".
Since updates to a sharded database can be slow, it may
speed up throughput to send requests to a queue rather
than wait for the update to finish on ever node.
The shard package has functions for adding requests to the queue,
which it will process without further intervention.
Queuing is optional. |br|
**Redundancy** ...
The number of replicas in each shard. |br|
**Replica** ...
A complete copy of the data.
The shard package handles both sharding and replication.
One shard can contain one or more replicas.
When a write occurs, the write is attempted on every replica in turn.
The shard package does not use the built-in replication feature. |br|
**Shard** ...
A subset of the tuples in the database partitioned according to the
value returned by the consistent hash function. Usually each shard
is on a separate node, or a separate set of nodes (for example if
redundancy = 3 then the shard will be on three nodes). |br|
**Zone** ...
A physical location where the nodes are closely connected, with
the same security and backup and access points. The simplest example
of a zone is a single computer with a single tarantool-server instance.
A shard's replicas should be in different zones.
The shard package is distributed separately from the main tarantool package.
To acquire it, do a separate install. For example on Ubuntu say |br|
sudo apt-get install tarantool-shard tarantool-pool |br|
Or, download from github tarantool/shard
and compile as described in the README. Then, before using the package, say
shard = require('shard')
.. _sharding: https://en.wikipedia.org/wiki/Sharding
.. _On the shard section of github: https://github.com/tarantool/shard
The most important function is |br|
:samp:`shard.init ({shard-configuration})` |br|
This must be called for every shard.
The shard-configuration is a table with these fields: |br|
* servers (a list of URIs of nodes and the zones the nodes are in) |br|
* login (the user name which applies for accessing via the shard package) |br|
* password (the password for the login) |br|
* redundancy (a number, minimum 1) |br|
* binary (a port number that this host is listening on, on the current host)
(distinguishable from the 'listen' port specified by box.cfg) |br|
Possible Errors: Redundancy should not be greater than the number of servers;
the servers must be alive; two replicas of the same shard should not be in the same zone.
| EXAMPLE: shard.init syntax for one shard
| :codenormal:`-- The number of replicas per shard (redundancy) is 3.`
| :codenormal:`-- The number of servers is 3.`
| :codenormal:`-- The shard package will conclude that there is only one shard.`
| :codenormal:`cfg = {`
| |nbsp| |nbsp| :codenormal:`servers = {`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33131', zone = '1' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33132', zone = '2' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33133', zone = '3' };`
| |nbsp| |nbsp| :codenormal:`};`
| |nbsp| |nbsp| :codenormal:`login = 'tester';`
| |nbsp| |nbsp| :codenormal:`password = 'pass';`
| |nbsp| |nbsp| :codenormal:`redundancy = 3;`
| |nbsp| |nbsp| :codenormal:`binary = 33131;`
| :codenormal:`}`
| :codenormal:`server.init(cfg)`
|
| :codenormal:`EXAMPLE: shard.init syntax for three shards`
| :codenormal:`-- This describes three shards.`
| :codenormal:`-- Each shard has two replicas.`
| :codenormal:`-- Since the number of servers is 7, and the number`
| :codenormal:`-- of replicas per server is 2, and dividing 7 / 2`
| :codenormal:`-- leaves a remainder of 1, one of the servers will`
| :codenormal:`-- not be used. This is not necessarily an error,`
| :codenormal:`-- because perhaps one of the servers in the list is`
| :codenormal:`-- not alive.`
| :codenormal:`cfg = {`
| |nbsp| |nbsp| :codenormal:`servers = {`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33131', zone = '1' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33132', zone = '2' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33133', zone = '1' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33134', zone = '2' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33135', zone = '1' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33136', zone = '2' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:33137', zone = '1' };`
| |nbsp| |nbsp| :codenormal:`};`
| |nbsp| |nbsp| :codenormal:`login = 'tester';`
| |nbsp| |nbsp| :codenormal:`password = 'pass';`
| |nbsp| |nbsp| :codenormal:`redundancy = 2;`
| |nbsp| |nbsp| :codenormal:`binary = 33131;`
| :codenormal:`}`
| :codenormal:`server.init(cfg)`
:samp:`shard.{space_name}.insert` :code:`{...}` etc. |br|
Every data-access function in the box package
has an analogue in the shard package, so (for
example) to insert in table T in a sharded database one
simply says "shard.T:insert{...}" instead of
"box.T:insert{...}".
A "shard.T:select{}" request without a primary key will search all shards.
:samp:`q_shard.{space_name}.insert` {:code:`{...}` etc. |br|
Every queued data-access function has an analogue in the shard package.
The user must add an operation_id. The details of queued
data-access functions, and of maintenance-related functions,
are on `the shard section of github`_.
| :codenormal:`EXAMPLE -- Shard, Minimal Configuration`
| :codenormal:`-- There is only one`
| :codenormal:`-- shard, and that shard contains only one replica.`
| :codenormal:`-- So this isn't illustrating the features of either`
| :codenormal:`-- replication or sharding, it's only illustrating``
| :codenormal:`-- what the syntax is,`
| :codenormal:`-- and what the messages look like,`
| :codenormal:`-- that anyone could duplicate in a minute or two`
| :codenormal:`-- with the magic of cut-and-paste.`
| :codenormal:`mkdir ~/tarantool_sandbox_1`
| :codenormal:`cd ~/tarantool_sandbox_1`
| :codenormal:`rm -r *.snap`
| :codenormal:`rm -r *.xlog`
| :codenormal:`~/tarantool-master/src/tarantool`
| :codenormal:`box.cfg{listen = 3301}`
| :codenormal:`box.schema.create_space('tester')`
| :codenormal:`box.space.tester:create_index('primary', {})`
| :codenormal:`box.schema.user.passwd('admin', 'password')`
| :codenormal:`console = require('console')`
| :codenormal:`console.delimiter('!')`
| :codenormal:`cfg = {`
| |nbsp| |nbsp| :codenormal:`servers = {`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3301', zone = '1' };`
| |nbsp| |nbsp| :codenormal:`};`
| |nbsp| |nbsp| :codenormal:`login = 'admin';`
| |nbsp| |nbsp| :codenormal:`password = 'password';`
| |nbsp| |nbsp| :codenormal:`redundancy = 1;`
| |nbsp| |nbsp| :codenormal:`binary = 3301;`
| :codenormal:`}!`
| :codenormal:`shard = require('shard')!`
| :codenormal:`shard.init(cfg)!`
| :codenormal:`-- Now put something in ...!`
| :codenormal:`shard.tester:insert{1,'Tuple #1'}!`
If one cuts and pastes the above, then the result,
showing only the requests and responses for shard.init
and shard.tester, should look approximately like this:
| :codenormal:`tarantool>` :codebold:`shard.init(cfg)!`
| :codenormal:`2015-08-09 ... I> Sharding initialization started...`
| :codenormal:`2015-08-09 ... I> establishing connection to cluster servers...`
| :codenormal:`2015-08-09 ... I> - localhost:3301 - connecting...`
| :codenormal:`2015-08-09 ... I> - localhost:3301 - connected`
| :codenormal:`2015-08-09 ... I> connected to all servers`
| :codenormal:`2015-08-09 ... I> started`
| :codenormal:`2015-08-09 ... I> redundancy = 1`
| :codenormal:`2015-08-09 ... I> Zone len=1 THERE`
| :codenormal:`2015-08-09 ... I> Adding localhost:3301 to shard 1`
| :codenormal:`2015-08-09 ... I> Zone len=1 THERE`
| :codenormal:`2015-08-09 ... I> shards = 1`
| :codenormal:`2015-08-09 ... I> Done`
| :codenormal:`---`
| :codenormal:`- true`
| :codenormal:`...`
|
| :codenormal:`tarantool>` :codebold:`-- Now put something in ...!`
| :codenormal:`---`
| :codenormal:`...`
|
| :codenormal:`tarantool>` :codebold:`shard.tester:insert{1,'Tuple #1'}!`
| :codenormal:`---`
| :codenormal:`- - [1, 'Tuple #1']`
| :codenormal:`...`
|
|
| :codenormal:`EXAMPLE -- Shard, Scaling Out`
| :codenormal:`-- There are two shards, and each shard contains one replica.`
| :codenormal:`-- This requires two nodes. In real life the two nodes would`
| :codenormal:`-- be two computers, but for this illustration the requirement`
| :codenormal:`-- is merely: start two shells, which we'll call Terminal#1 and`
| :codenormal:`-- Terminal #2.`
|
| :codenormal:`-- On Terminal #1, say:`
| :codenormal:`mkdir ~/tarantool_sandbox_1`
| :codenormal:`cd ~/tarantool_sandbox_1`
| :codenormal:`rm -r *.snap`
| :codenormal:`rm -r *.xlog`
| :codenormal:`~/tarantool-master/src/tarantool`
| :codenormal:`box.cfg{listen = 3301}`
| :codenormal:`box.schema.create_space('tester')`
| :codenormal:`box.space.tester:create_index('primary', {})`
| :codenormal:`box.schema.user.passwd('admin', 'password')`
| :codenormal:`console = require('console')`
| :codenormal:`console.delimiter('!')`
| :codenormal:`cfg = {`
| |nbsp| |nbsp| :codenormal:`servers = {`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3301', zone = '1' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3302', zone = '2' };`
| |nbsp| |nbsp| :codenormal:`};`
| |nbsp| |nbsp| :codenormal:`login = 'admin';`
| |nbsp| |nbsp| :codenormal:`password = 'password';`
| |nbsp| |nbsp| :codenormal:`redundancy = 1;`
| |nbsp| |nbsp| :codenormal:`binary = 3301;`
| |nbsp| |nbsp| :codenormal:`}!`
| |nbsp| |nbsp| :codenormal:`shard = require('shard')!`
| |nbsp| |nbsp| :codenormal:`shard.init(cfg)!`
| |nbsp| |nbsp| :codenormal:`-- Now put something in ...!`
| |nbsp| |nbsp| :codenormal:`shard.tester:insert{1,'Tuple #1'}!`
|
| -- On Terminal #2, say:
| :codenormal:`mkdir ~/tarantool_sandbox_2`
| :codenormal:`cd ~/tarantool_sandbox_2`
| :codenormal:`rm -r *.snap`
| :codenormal:`rm -r *.xlog`
| :codenormal:`~/tarantool-master/src/tarantool`
| :codenormal:`box.cfg{listen = 3302}`
| :codenormal:`box.schema.create_space('tester')`
| :codenormal:`box.space.tester:create_index('primary', {})`
| :codenormal:`box.schema.user.passwd('admin', 'password')`
| :codenormal:`console = require('console')`
| :codenormal:`console.delimiter('!')`
| :codenormal:`cfg = {`
| |nbsp| |nbsp| :codenormal:`servers = {`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3301', zone = '1' };`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`{ uri = 'localhost:3302', zone = '2' };`
| |nbsp| |nbsp| :codenormal:`};`
| |nbsp| |nbsp| :codenormal:`login = 'admin';`
| |nbsp| |nbsp| :codenormal:`password = 'password';`
| |nbsp| |nbsp| :codenormal:`redundancy = 1;`
| |nbsp| |nbsp| :codenormal:`binary = 3302;`
| :codenormal:`}!`
| :codenormal:`shard = require('shard')!`
| :codenormal:`shard.init(cfg)!`
| :codenormal:`-- Now get something out ...!`
| :codenormal:`shard.tester:select{1}!`
What will appear on Terminal #1 is: a loop of
error messages saying "Connection refused" and
"server check failure". This is normal. It will
go on until Terminal #2 process starts.
What will appear on Terminal #2, at the end,
should look like this:
| :codenormal:`tarantool>shard.tester:select{1}!`
| :codenormal:`---`
| :codenormal:`- - - [1, 'Tuple #1']`
| :codenormal:`...`
This shows that what was inserted by Terminal #1
can be selected by Terminal #2, via the shard package.
Details are on `the shard section of github`_.
.. _consistent: https://en.wikipedia.org/wiki/Consistent_hashing
.. _the shard section of github: https://github.com/tarantool/shard
doc/sphinx/reference/tarantool.rst 0 → 100644
+ 35
− 0
View file @ 31d5f1c3
-------------------------------------------------------------------------------
Package `tarantool`
-------------------------------------------------------------------------------
.. module:: tarantool
By saying :code:`require('tarantool')`, one can answer
some questions about how the tarantool server was built,
such as "what flags were used", or "what was the version
of the compiler". Additionally one can see the uptime
and the server version and the process id, but those
information items are more commonly accessed with
:func:`box.info`.
| EXAMPLE
| :codenormal:`tarantool>` :codebold:`tarantool = require('tarantool')`
| :codenormal:`---`
| :codenormal:`...`
| |nbsp|
| :codenormal:`tarantool>` :codebold:`tarantool`
| :codenormal:`---`
| :codenormal:`- build:`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`flags: ' -fno-omit-frame-pointer -fno-stack-protector`
| |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`-fopenmp -msse2 -std=c11 -Wall`
| |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`-fno-gnu89-inline'`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`target: Linux-x86_64-Release`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`compiler: /usr/bin/cc /usr/bin/c++`
| |nbsp| |nbsp| |nbsp| |nbsp| :codenormal:`options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local`
| |nbsp| |nbsp| :codenormal:`uptime: 'function: 0x4109a670'`
| |nbsp| |nbsp| :codenormal:`version: 1.6.0-3142-g63af2df`
| |nbsp| |nbsp| :codenormal:`pid: 'function: 0x4109a690'`
| :codenormal:`...`
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment