The failover coordinator is a service provided by Tarantool Enterprise
Edition. It is going to be released in the 3.1.0 version. Let's add the
`--failover` option into the help message and the man page.

This reverts commit af4bad43 ("main: hide --failover CLI option").

Part of tarantool/tarantool-ee#564

NO_DOC=the documentation request for the failover coordinator at whole
       will be added manually
NO_CHANGELOG=this is EE feature, no reason to highlight it in CE release
             notes

This patch updates example for vshard usage in config module.

NO_DOC=example update
NO_TEST=example update
NO_CHANGELOG=example update

Fixes #9506

@TarantoolBot document
Title: config.context: define cluster config parts in env/file

Example:

```yaml
config:
  context:
    replicator_password:
      from: file
      file: secrets/replicator_password.txt
      rstrip: true
    client_password:
      from: file
      file: secrets/client_password.txt
      rstrip: true

credentials:
  users:
    replicator:
      password: '{{ context.replicator_password }}'
      roles: [replication]
    client:
      password: '{{ context.client_password }}'
      roles: [super]
```

The new `config.context` section allows a user to define its own
variables, whose content resides in an environment variable or a file.

The primary usage is to extract passwords from the configuration.

All the variables are defined in `config.context` section as keys. Each
key is a record with the following fields.

* `from`: either `env` or `file`
* `file`: a file path
* `env`: an environment variable name
* `rstrip`: whether to strip whitespace characters from the end of the
   data (it strips newlines too)

A relative file path is interpreted as relative to `process.work_dir`.

`from` is mandatory, `file` is mandatory when `from` is `file`, `env` is
mandatory when `from` is `env`, `rstrip` is optional.

If there is no given file or no given environment variable, an error of
a configuration applying is reported.

This patch moves example for config of config-storage to doc/examples
from enterprise doc/.

NO_DOC=added an example
NO_TEST=validation will be added later
NO_CHANGELOG=added an example

This patch fixes iproto.listen and iproto.advertise.peer in local.yaml.

Part of #8862

NO_DOC=changes in example
NO_TEST=will be tested in EE
NO_CHANGELOG=changes in example

This patch changes format of iproto.listen and introduces support for
TLS options in the iproto.listen config option.

Part of #8862

@TarantoolBot document
Title: new format of the `iproto.*` config options.

The config options `iproto.listen`, `iproto.advertise.peer` and
`iproto.advertise.sharding` now have a new format.

The `iproto.listen` config option now has the following format:
```
iproto:
  listen:
    - uri: <string>
      params:
        transport: <'plain' or 'ssl'>
        ssl_ca_file: <string>
        ssl_cert_file: <string>
        ssl_ciphers: <string>
        ssl_key_file: <string>
        ssl_password: <string>
        ssl_password_file: <string>
    - uri: <string>
      params:
        transport: <'plain' or 'ssl'>
        ssl_ca_file: <string>
        ssl_cert_file: <string>
        ssl_ciphers: <string>
        ssl_key_file: <string>
        ssl_password: <string>
        ssl_password_file: <string>
...
```
The `iproto.listen` config option is now an array, and its elements are
`records` that can only have a required `uri` field and an optional
`params` field.

The `uri` field is of type `string` and must be a single valid URI. The
URI cannot contain parameters, login and password.

The `params` field is a `record` with the fields `transport`,
`ssl_ca_file`, `ssl_cert_file`, `ssl_ciphers`, `ssl_key_file`,
`ssl_password`, `ssl_password_file`. The `transport` field can have one
of two values: `plain` or `ssl`, the default is `plain`. If `transport`
is set to `plain`, no additional fields are required. When `transport`
is set to `ssl`, the fields `ssl_key_file` and `ssl_cert_file` are
required, and the remaining fields are optional.

The `iproto.advertise.peer` and `iproto.advertise.sharding` now have the
following format:
```
iproto:
  advertise:
    <'peer' or 'sharding'>:
      uri: <string>
      login: <string>
      password: <string>
      params:
        transport: <'plain' or 'ssl'>
        ssl_ca_file: <string>
        ssl_cert_file: <string>
        ssl_ciphers: <string>
        ssl_key_file: <string>
        ssl_password: <string>
        ssl_password_file: <string>
```

The `iproto.advertise.peer` and `iproto.advertise.sharding` are now
`records`. All of their fields are optional, however, if the `params`
field can be set only if the `uri` field is set and the `password` field
can be set only if the `login` field is set.

The `uri` field is of type `string` and must be a single valid URI.
The URI cannot contain parameters, login and password. If the `uri` and
`params` fields are missing, the appropriate `iproto.listen` element is
selected. An element is eligible if, after parsing its `url` field using
`require('uri').parse()`, the `ipv4` result field is not equal to
`'0.0.0.0'`, the `ipv6` result field is not equal to `'::'`, and the
`service` result field is not equal to `'0'`.

The `login` and `password` fields are of type `string`.

For description of the `params` field see description of
`iproto.listen`.

This patch changes the structures of the iproto.advertise.peer and
iproto.advertise.sharding options and introduces TLS options for these
options.

There is no proper integration test for the new TLS options as our
iproto.listen does not currently support these options. The test will
be added along with the addition of TLS support in iproto.listen.

Part of #8862

NO_DOC=will be added later.

Fixes #9452

@TarantoolBot document
Title: config: conditional sections for upgrading

See https://github.com/tarantool/tarantool/issues/9452 for the problem
statement. In short: some upgrade scenarios may need to configure
tarantool instances differently depending on a tarantool version.

A new top level configuration block is added for this purpose:
`conditional`. Let's look on an example:

```yaml
conditional:
- if: tarantool_version >= 3.99.0 && tarantool_version < 4.0.0
  # This section shouldn't be validated and shouldn't be applied.
  replication:
    new_option: foo
- if: tarantool_version < 3.99.0
  # This section is to be applied.
  process:
    title: '{{ instance_name }} -- in upgrade'
```

The block contains an array of conditional sections, each accompanied by
`if` predicate to determine, whether to apply it on particular tarantool
version. (`if` is required.)

If a section is not to be applied on the given version, it is not
validated at all and may contain unknown options.

If a section is to be applied, it must match the cluster configuration
schema like the main config.

If the same option is set by several sections with true predicate, the
last section wins.

The `if` expression supports one data type: `version`. A value may be
referenced in two ways:

1. Version literal: `1.2.3` (three components, not less, not more).
2. Variable: `tarantool_version` (only this variable is supported).

`tarantool_version` is assumed as three components version, say, 3.0.0.

The operations are the following.

1. Logical OR: `||`
2. Logical AND: `&&`
3. Compare: `>`, `<`, `>=`, `<=`, `==`, `!=`
4. Parentheses: `(`, `)`

All the comparisons assume the versions as three component ones.

The failover agent is not ready at the moment. Let's hide the CLI option
from the help message, but still handle it: it is useful to continue the
development of the agent.

See tarantool/tarantool-ee#564

NO_DOC=The option is not present into the documentation.
NO_CHANGELOG=No behavior changes, just help message.

This commit aims several goals.

First, we're going to recommend `tt` as a tool to run and manage
tarantool instances. The tool is going to replace `tarantoolctl` and it
seems valuable to sync recipes across different parts of the
documentation (including this help message).

Second, the action options and modifier options are separated now. At
least one action option should be specified. I hope, it is more obvious
from the help message now.

Third, single line descriptions are replaced with a bit more extended
ones to give a user basic context to understand an influence of an
option.

Updated the man page accordingly: synchronized the options, removed
`tarantoolctl`.

Part of #8862

NO_TEST=the help message is not an API for a developer
NO_CHANGELOG=nothing new here, just better wording here and there

@TarantoolBot document
Title: Update CLI options documentation

https://www.tarantool.io/en/doc/latest/reference/configuration/#command-options
seems to need an update. See the linked commit for details and updated
`tarantool --help` message.

The instance config schema was changed in commit 4bb1eb0e ("config:
remove hashes from credentials.password"), but an example of a config
for etcd was not updated.

The example is tested on Tarantool EE, so we should update it to fix the
testing failure.

Part of #8967

NO_DOC=It is a fix of the testing problem.
NO_CHANGELOG=see NO_DOC
NO_TEST=It is a fix of a test in fact.

Follow-up #9007

NO_DOC=Will be described when full support for vshard is introduced.
NO_CHANGELOG=Addition of an example.

In the initial credentials schema, the hashes were supposed to give a
way to have passwords out of plain-text config file.
Later, it was decided to remove this feature, because this way of
authorisation is inferior to the one with auth service and tokens,
but the latter is out of scope for current config development.
This patch removes `credentials.password.{sha1,sha256}` and moves
plain password from `credentials.password.plain` to `credentials.password`.

Part of #8967

NO_DOC=tarantool/doc#3544 links the most actual schema,
       no need to update the issue.
NO_CHANGELOG=removed feature was not released yet

The following syntax variants are fobidden now:

* user@
* user:pass@
* user@host:port
* user:pass@host:port

Only host:port is allowed (inet URI or unix socket URI).

The idea is that we shouldn't distribute login/password information
using the configuration, so it is better to explicitly forbid such
usage of the option.

Closes #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This failover mode enables automatic leader election on a replicaset.
Assigning a leader manually (`leader` option) and assigning RO/RW mode
(`database.mode` option) are forbidden in this failover mode.

Configuration example:

```yaml
replication:
  failover: election         # !!

groups:
  group-001:
    replicasets:
      replicaset-001:
        instances:
          instance-001: {}
          instance-002: {}
          instance-003: {}
```

All the replicaset instances are so called candidates by default: they
can vote for a leader, they can be elected as a leader. However, it is
possible to set `replicaset.election_mode` to 'voter', 'off' or
'manual'. See more detailed description in the box_cfg applier comments
in the code and in the box.cfg() options documentation [1].

Note: Unlike box.cfg()'s option 'election_mode', the
replication.election_mode = "off" forces the instance to be read-only
(if failover = "election" is enabled). box.cfg() call has no cluster
configuration and it doesn't know, whether the given replicaset is
managed by the built-in election algorithm or an external coordinator.

The election failover may be used to improve cluster's availability.

[1]: https://www.tarantool.io/en/doc/latest/reference/configuration/#cfg-replication-election-mode

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This failover mode allows to set a leader for a replicaset by its
instance name instead of per-instance database.mode option. For example:

```yaml
replication:
  failover: manual           # !!

groups:
  group-001:
    replicasets:
      replicaset-001:
        leader: instance-001 # !!
        instances:
          instance-001: {}
          instance-002: {}
          instance-003: {}
```

The "manual" failover mode doesn't allow several leaders in a
replicaset. If it is desired, use the "off" failover mode.

The "manual" mode doesn't perform a proper leader switching at the
moment. An administrator should resign the old leader (by unsetting the
leader option or setting it to `null`), wait till all the data arrives
to the upcoming leader and then configure it as the new leader.

The proper leader switching is subject of a future work. Also, the next
commit will add replicaset.failover = "election", which handles such
problems.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

Enabling read-write by default is unsafe for an instance in a replicaset
with more than one instance. The only meaningful default here is
read-only.

On the other hand, a signleton instance (the only one in its replicaset)
almost always started in the read-write mode in practice.

Let's use these values as defaults for these situations.

The name of the option is changed from `rw` (boolean) to `mode` (enum
with allowed values `'ro'` and `'rw'`). We agreed on the enum after a
long controversy whether it should be `ro` or `rw`.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

In brief:

* client -- for external clients
* peer -- for connections within the cluster, in particular for replicas
* sharding -- for routers and a rebalancer

See the instance_config.lua file for the details.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

The following new syntax variants are introduced for `iproto.advertise`.

* `user@` -- use the given user, a password from the `credentials`
  section and `host:port` from `iproto.listen`
* `user:pass@` -- use the given user and password, use `host:port` from
  `iproto.listen`
* `user@host:port` -- use the given user, host and port, use a password
  from the `credentials` section

It allows to don't repeat the same information in different places of
the config.

The `test_no_advertise_*` test cases are generalized: now the same
boilerplate code is used for these and new cases.

The `test.luatest_helpers.server` util gains ability to parse all the
new `iproto.advertise` variants.

The replicaset and etcd configuration examples are updated to use the
`user@` syntax.

Part of #8810

NO_DOC=the old behavior was not released, the documentation request will
       be registered manually
NO_CHANGELOG=see NO_DOC

This patch adds configuration examples. Although the configuration
module will be introduced later, the examples can already be tested
using the instance_config and cluster_config modules.

Closes #8778

NO_DOC=will be added later
NO_CHANGELOG=will be added later

There are two new options introduced in Tarantool CLI:
* --name (-n) to specify instance name to be started. The option can be
  omitted in case TT_INSTANCE_NAME environment variable is set.
* --config (-c) to specify the path to the config file. If the option
  is not set the value of TT_CONFIG environment variable is considered.

Closes #8613

Co-authored-by: Sergey Bronnikov <sergeyb@tarantool.org>

@TarantoolBot document
Title: introduce new CLI options for conf module

There are two new options introduced in Tarantool CLI:
* --name (-n) to specify instance name to be started. The option can be
  omitted in case TT_INSTANCE_NAME environment variable is set.
* --config (-c) to specify the path to the config file. If the option
  is not set the value of TT_CONFIG environment variable is considered.

Synced with description in the README updated in #5679.

Show manual page: pod2man doc/man/tarantool.pod | mandoc -a

NO_CHANGELOG=internal
NO_DOC=internal
NO_TEST=internal

Follows up #7456
Follows up #5541

NO_CHANGELOG=doc
NO_DOC=doc
NO_TEST=doc

This commit adds the RFC for the sysprof that
was approved a year ago.

Part of #781

NO_DOC=RFC
NO_TEST=RFC
NO_CHANGELOG=RFC

Fixup of f06417 ("doc: publish autogenerated module api documentation").

https://tarantool.github.io/tarantool/api/html/module_8h.html

NO_CHANGELOG=ci
NO_DOC=ci
NO_TEST=ci

NO_DOC=internal
NO_TEST=internal

Copyright year in manual page is outdated: update copyright year to
2021.

Closes #5309

I think that our current process is better. But if someone will want to
understand why we stick with it, it is good to have explicit cons and
pros.

There is discussion in #5910 about all inconsistencies
we see between Lua and SQL worlds and possible future
directions of SQL development and new types additions.

This RFC is current state of this discussion.

Part of #5910

Co-authored-by: Igor Munkin <imun@tarantool.org>
Co-authored-by: Peter Gulutzan <pgulutzan@ocelot.ca>
Co-authored-by: Mergen Imeev <imeevma@tarantool.org>
Co-authored-by: Sergey Ostanevich <sergos@tarantool.org>

Resolves #5857

Reviewed-by: Igor Munkin <imun@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

See https://github.com/tarantool/tarantool/discussions/5634

Part of #5442

Reviewed-by: Sergey Ostanevich <sergos@tarantool.org>
Reviewed-by: Igor Munkin <imun@tarantool.org>
Signed-off-by: Igor Munkin <imun@tarantool.org>

Part of #5187

Part of #1148

As per request by Kostja, commit an RFC document with a brief history of
the vinyl metadata log infrastructure, issues it was intended to solve,
problems we are facing now, and possible ways to solve them.

Part of #4182

Part of #3271

Add description of possible redesigning of vector clocks.