Commits · 536ed0bb4d80b84e4d0a1e2bf96bab824c79955d · core / tarantool

Oct 14, 2021

build, lua: built-in module datetime · 43e10ed3

Timur Safin authored 3 years ago

Introduce a new builtin Tarantool module `datetime.lua` for timestamp
and interval types support.

New third_party module - c-dt
-----------------------------

* Integrated chansen/c-dt parser as 3rd party module to the
  Tarantool cmake build process;
* We use tarantool/c-dt instead of original chansen/c-dt to
  have an easier cmake build integration, as we have added some
  changes, which provide cmake support, and allow to rename symbols
  if necessary (this symbol renaming is similar to that we see
  with xxhash or icu).

New built-in module `datetime`
------------------------------

* created a new Tarantool built-in module `datetime`, which uses
  `struct datetime` data structure for keeping timestamp values;
* Lua module uses a number of `dt_*` functions from `c-dt` library,
  but they were renamed to `tnt_dt_*` at the moment of exporting
  from executable - to avoid possible name clashes with external
  libraries.

* At the moment we libc `strftime` for formatting of datetime
  values according to flags passed, i.e. `date:format('%FT%T%z')`
  will return something like '1970-01-01T00:00:00+0000', but
  `date:format('%A %d, %B %Y')` will return 'Thursday 01, January 1970'

* if there is no format provided then we use default
  `tnt_datetime_to_string()` function, which converts datetime
  to their default ISO-8601 output format, i.e.
  `tostring(date)` will return string like "1970-01-01T00:00:00Z"

* There are a number of simplified interfaces
  - totable() for exporting table with attributes names as provided
    by `os.date('*t')`
  - set() method provides unified interface to set values using
    the set of attributes as defined above in totable()

Example,

```
local dt = datetime.new {
	nsec      = 123456789,

	sec       = 19,
	min       = 29,
	hour      = 18,

	day       = 20,
	month     = 8,
	year      = 2021,

	tzoffset  = 180
}

local t = dt:totable()
--[[
{
	sec = 19,
	min = 29,
	wday = 6,
	day = 20,
	nsec = 123456789,
	isdst = false,
	yday = 232,
	tzoffset = 180,
	month = 8,
	year = 2021,
	hour = 18
}
--]]

dt:format()   -- 2021-08-21T14:53:34.032Z
dt:format('%Y-%m-%dT%H:%M:%S')   -- 2021-08-21T14:53:34

dt:set {
	usec      = 123456,

	sec       = 19,
	min       = 29,
	hour      = 18,

	day       = 20,
	month     = 8,
	year      = 2021,

	tzoffset  = 180,

}
dt:set {
	timestamp = 1629476485.124,

	tzoffset  = 180,
}

```

Coverage is

File                 Hits Missed Coverage
-----------------------------------------
builtin/datetime.lua 299  23     92.86%
-----------------------------------------
Total                299  23     92.86%

Part of #5941

@TarantoolBot document
Title: Introduced a new `datetime` module for timestamp and interval support

Create `datetime` module for timestamp and interval types support.
It allows to create date and timestamp values using either object interface,
or via parsing of string values conforming to iso-8601 standard.
One may manipulate (modify, subtract or add) timestamp and interval values.

Please refer to https://hackmd.io/@Mons/S1Vfc_axK#Datetime-in-Tarantool
for a more detailed description of module API.

43e10ed3

Oct 05, 2021

Libev update to version 4.33 · e6088a22
EvgenyMekhanik authored 3 years ago
```
Closes #4909
```
e6088a22

libev: fix error `ev_io_closing` implementation. · 1e543361

EvgenyMekhanik authored 3 years ago

Fixed error in implementation of `ev_io_closing`: this function
is called to do same work as `fd_kill` and should call it after
checking fd (in `coio_close` we pass invalid event value, that
leads to assertion in new libev version).

1e543361

Sep 17, 2021

lua/msgpack: fix crash while encoding error extension · 6dde8898

Vladimir Davydov authored 3 years ago

Whether errors are encoded as a msgpack extension or not is determined
by the serializer_opts::error_marshaling_enabled flag. Although an
instance of serizlier_opts is passed to luamp_encode(), it doesn't
propagate it to luamp_encode_extension_box(). The latter encodes an
error as a msgpack extension if the error_marshaling_enabled flag is set
in serializer_opts of the current session. This leads to a bug when
luamp_encode() is called with error_marshaling_enabled unset while the
current session has the flag set:

1. luaL_tofield() sets field->type to MP_EXT and field->ext_type to
MP_UNKNOWN_EXTENSION, because the error_marshaling_enabled flag is
unset:

https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/serializer.c#L548

2. Basing on the ext_type, luamp_encode_r() skips the MP_ERROR
swtich-case branch for the default branch and calls the
luamp_encode_extension callback:

https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/msgpack.c#L203

3. The callback implementation (luamp_encode_extension_box()) encodes
the error, because the error_marshaling_enabled flag is set in the
current session settings, and returns MP_EXT:

https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/box/lua/init.c#L420

4. luamp_encode_r() assumes that the callback didn't encode the
extension, because it returned MP_EXT, and encodes it again as
a string:

https://github.com/tarantool/tarantool/blob/b0431cf8f47e9d081f6a402bc18edb1d6ad49847/src/lua/msgpack.c#L209

This results in a broken msgpack content.

To fix this bug, let's do the following:
- luaL_tofield() now sets ext_type to MP_ERROR unconditionally,
irrespective of serializer_opts::error_marshaling_enabled.
- luamp_encode_r() invokes the luamp_encode_extension callback for
a MP_ERROR field only if error_marshaling_enabled is set. If the flag
is unset, it proceeds with converting the field to string.
- luamp_encode_extension_box() doesn't check serializer_opts anymore.
It doesn't need to, because it's called iff error_marshaling_enabled
is set.
- YAML and JSON encoders are patched to handle the MP_ERROR field type
by appending error::errmsg to the output (they use luaL_tofield()
internally to determine the field type so they have to handle
MP_ERROR).

This basically disables error encoding as msgpack extension
everywhere except returning an error from a Lua CALL/EVAL, in
particular:
- when creating a tuple with box.tuple.new(),
- when inserting an error into a space,
- when encoding an error with the msgpack module.

This is okay, because the functionality has always been broken anyway.
We will introduce a separate msgpack encoder option to enable encoding
errors as MP_ERROR msgpack extension.

Looking at the code links above, one is likely to wonder why error
encoding was implemented via the encode extension callback in the first
place. The lua/msgpack module knows about the MP_ERROR extension and
even partially handles it so it'd be only natural to call the error
encoder function directly, as we do with decimals and uuids.
Unfortunately, we can't do it, because the error encoder is (surprise!)
a part of the box library. I filed a ticket to move it to the core lib,
see #6432.

Closes #6431

6dde8898

Aug 19, 2021

luajit: bump new version · 847b096e

Igor Munkin authored 3 years ago

* ARM64: Fix exit stub patching.
* arm64: fix cur_L restoration on error throw

Closes #6098
Closes #6189
Part of #5629
Relates to #6323
Follows up #1516