Segmentation Fault при выполнении SELECT
При выполнении команды получил ошибку
$ psql postgres://postgres:Postgres13@127.0.0.1:5432?sslmode=disable -c 'SELECT * FROM pgbench_accounts ORDER BY aid DESC LIMIT 10 option(SQL_VDBE_MAX_STEPS=0, vtable_max_rows=0);' сервер неожиданно закрыл соединение Скорее всего сервер прекратил работу из-за сбоя до или в процессе выполнения запроса. подключение к серверу было потеряно
В окне picodata было сообщение:
2024-10-01 18:42:29.542 [45656] main/1180/pgproto::client otm.rs:321 E> failed to delete query with id: : more than 1 ref
picodata> Segmentation fault
code: SEGV_MAPERR
addr: 0x1000c
context: 0x20744440
siginfo: 0x20744570
rax 0xfffc 65532
rbx 0xffff 65535
rcx 0xfff53272 4294259314
rdx 0x239ecc4 37350596
rsi 0xffff40928a60fff7 -210476845694985
rdi 0x32726b36 846359350
rsp 0x7f9d1c47e540 140312761066816
rbp 0x7f9d1c47e690 140312761067152
r8 0x7f9d1c47ea80 140312761068160
r9 0x0 0
r10 0x10 16
r11 0x33a59010 866488336
r12 0x4c55414a49543204 5500374307216568836
r13 0x1 1
r14 0x32726b36 846359350
r15 0x7f9d1c47e5a0 140312761066912
rip 0xb3c5b1 11781553
eflags 0x10206 66054
cs 0x33 51
gs 0x0 0
fs 0x0 0
cr2 0x1000c 65548
err 0x4 4
oldmask 0x0 0
trapno 0xe 14
Current time: 1727772156
Please file a bug at support@picodata.io
Attempting backtrace... Note: since the server has already crashed,
this may fail as well
#1 0x8602d9 in fiber_parent_backtrace_disable+185
#2 0x7f9e3f64fd00 in __restore_rt+0
#3 0xb3c5b1 in luaM_sysprof_report+14465
#4 0xb3c89a in luaM_sysprof_report+15210
#5 0x7f9e40143077 in _Unwind_RaiseException+343
#6 0xb38651 in lua_gethookcount+577
#7 0xb3c359 in luaM_sysprof_report+13865
#8 0xb3c3e3 in luaM_sysprof_report+14003
#9 0xb45119 in luaopen_bit+10985
#10 0xb3dab0 in luaM_sysprof_report+19840
#11 0xb3db8c in luaM_sysprof_report+20060
#12 0xb830dd in lj_sysprof_start+127421
#13 0x13add40 in picodata::ForbiddenIprotoTypes::from_i64::h7a6d954b273015ad+16048
[supervisor:45655] no ipc message from child
[supervisor:45655] subprocess 45656 was signaled with SIGABRT
[supervisor:45655] core dumped
volody@fedora:~/picoinstances$
$ coredumpctl gdb
PID: 45656 (picodata)
UID: 1000 (volody)
GID: 1000 (volody)
Signal: 6 (ABRT)
Timestamp: Tue 2024-10-01 18:42:36 +10 (5min ago)
Command Line: picodata run --data-dir /home/volody/picoinstances/pgbench --listen 0.0.0.0:3301 --pg-listen 0.0.0.0:5432 --init-replication-factor 1 --failure-domain DC=MSK --http-listen 0.0.0.0:8001 -i --memtx-memory 4294967296
Executable: /usr/bin/picodata
Control Group: /user.slice/user-1000.slice/user@1000.service/app.slice/app-org.gnome.Terminal.slice/vte-spawn-2c7ef0c3-b58e-4f8f-b41a-bd44989160b4.scope
Unit: user@1000.service
User Unit: vte-spawn-2c7ef0c3-b58e-4f8f-b41a-bd44989160b4.scope
Slice: user-1000.slice
Owner UID: 1000 (volody)
Boot ID: 5780c490dbed4111ae6ade36f5e1a728
Machine ID: bca106169f65437ebaf06dd9bb8e7f70
Hostname: fedora
Storage: /var/lib/systemd/coredump/core.picodata.1000.5780c490dbed4111ae6ade36f5e1a728.45656.1727772156000000.zst (present)
Size on Disk: 1.1G
Package: picodata/24.5.1.0-1.fc39
build-id: 11203e571771906caa5c466a3192ec256f654310
Message: Process 45656 (picodata) of user 1000 dumped core.
Module legacy.so from rpm openssl-3.2.2-3.fc40.x86_64
Module libz.so.1 from rpm zlib-ng-2.1.7-2.fc40.x86_64
Module libcrypto.so.3 from rpm openssl-3.2.2-3.fc40.x86_64
Module libssl.so.3 from rpm openssl-3.2.2-3.fc40.x86_64
Module picodata from rpm picodata-24.5.1.0-1.fc39.x86_64
Stack trace of thread 45656:
#0 0x00007f9e3f6a8664 __pthread_kill_implementation (libc.so.6 + 0x99664)
#1 0x00007f9e3f64fc4e raise (libc.so.6 + 0x40c4e)
#2 0x00007f9e3f637902 abort (libc.so.6 + 0x28902)
#3 0x0000000000860341 crash_signal_cb (picodata + 0x460341)
#4 0x00007f9e3f64fd00 __restore_rt (libc.so.6 + 0x40d00)
#5 0x0000000000b3c5b1 err_unwind (picodata + 0x73c5b1)
#6 0x0000000000b3c89a lj_err_unwind_dwarf (picodata + 0x73c89a)
#7 0x00007f9e40143077 _Unwind_RaiseException (libgcc_s.so.1 + 0x23077)
#8 0x0000000000b38651 lj_err_throw (picodata + 0x738651)
#9 0x0000000000b3c359 lj_err_mem (picodata + 0x73c359)
#10 0x0000000000b3c3e3 lj_mem_realloc (picodata + 0x73c3e3)
#11 0x0000000000b45119 lj_tab_resize (picodata + 0x745119)
#12 0x0000000000b3dab0 gc_onestep.lto_priv.0 (picodata + 0x73dab0)
#13 0x0000000000b3db8c lj_gc_step (picodata + 0x73db8c)
#14 0x0000000000b830dd lj_trace_exit (picodata + 0x7830dd)
#15 0x00000000013add40 lj_vm_exit_handler (picodata + 0xfadd40)
#16 0x000000000000028e n/a (n/a + 0x0)
ELF object binary architecture: AMD x86-64
GNU gdb (Fedora Linux) 15.1-1.fc40
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/picodata...
This GDB supports auto-downloading debuginfo from the following URLs:
<https://debuginfod.fedoraproject.org/>
Enable debuginfod for this session? (y or [n]) y
Debuginfod has been enabled.
To make this setting permanent, add 'set debuginfod enabled on' to .gdbinit.
Downloading separate debug info for /usr/bin/picodata
Reading symbols from .gnu_debugdata for /usr/bin/picodata...
(No debugging symbols found in .gnu_debugdata for /usr/bin/picodata)
[New LWP 45656]
[New LWP 45658]
[New LWP 45657]
[New LWP 45659]
--Type <RET> for more, q to quit, c to continue without paging--
[New LWP 45660]
Downloading separate debug info for /lib64/libssl.so.3
Downloading separate debug info for /home/volody/.cache/debuginfod_client/8e576614999e00b3e599c6f5be7e453aed3a4d1f/debuginfo
Downloading separate debug info for /lib64/libstdc++.so.6
Downloading separate debug info for /home/volody/.cache/debuginfod_client/efdde18722aa72a0a779270371aeee5a9f6ddf87/debuginfo
Downloading separate debug info for /lib64/libcrypto.so.3
Downloading separate debug info for /lib64/libgcc_s.so.1
Downloading separate debug info for /lib64/libm.so.6
Downloading separate debug info for /lib64/libc.so.6
Downloading separate debug info for /lib64/libz.so.1
Downloading separate debug info for /home/volody/.cache/debuginfod_client/8afe247a17adfe53e47aff493b76878169a97fb1/debuginfo
Downloading separate debug info for /usr/lib64/ossl-modules/legacy.so
Downloading separate debug info for system-supplied DSO at 0x7f9e40243000
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `picodata run --data-dir /home/volody/picoinstances/pgbench --listen 0.0.0.0:330'.
Program terminated with signal SIGABRT, Aborted.
--Type <RET> for more, q to quit, c to continue without paging--
Downloading source file /usr/src/debug/glibc-2.39-22.fc40.x86_64/nptl/pthread_kill.c
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
44 return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;
[Current thread is 1 (Thread 0x7f9e400f9300 (LWP 45656))]
warning: Missing auto-load script at offset 0 in section .debug_gdb_scripts
of file /usr/bin/picodata.
Use `info auto-load python-scripts [REGEXP]' to list them.
Missing debuginfo, try: dnf debuginfo-install picodata-24.5.1.0-1.fc39.x86_64
(gdb)Инстанс запускал командой:
picodata run --data-dir ~/picoinstances/pgbench --listen 0.0.0.0:3301 --pg-listen 0.0.0.0:5432 --init-replication-factor 1 --failure-domain DC=MSK --http-listen 0.0.0.0:8001 -i --memtx-memory 4294967296
ССылка на coredump https://fs.picodata.io/index.php/f/32173
Activity
Файл с приведёнными текстами ошибок 241001-segmentation-fault.txt
Edited by Владимир Паршинadded bug label
-
Общедоступная ссылка на coredump https://fs.picodata.io/index.php/s/bySwWUlw6ZlzuIM
Edited by Владимир Паршин changed milestone to %25.1
assigned to @kbezuglyi
added triaged label
- Resolved by Егор Ивков
Я не знаю, связано это с Segmentation Fault или нет, но при запуске этого инстанса этой командой отхватил:
2024-10-15 19:26:43.401 [39903] main/104/interactive I> became leader at term 14, term: 14, raft_id: 1 2024-10-15 19:26:43.401 [39903] main/117/governor_loop C> panicked at src/storage.rs:2227:43: entry should decode correctly: Decode { error: Syntax("invalid length 3, expected struct Tier with 6 elements"), expected_type: "picodata::tier::Tier", actual_msgpack: [147, 167, 100, 101, 102, 97, 117, 108, 116, 1, 195] } 2024-10-15 19:26:43.401 [39903] main/117/governor_loop C> backtrace: disabled backtrace 2024-10-15 19:26:43.401 [39903] main/117/governor_loop C> aborting due to panic [supervisor:39902] no ipc message from child [supervisor:39902] subprocess 39903 was signaled with SIGABRT [supervisor:39902] core dumped
Полный листинг в приложенном файле 241015-coredumped.txt. Если нужна будет информация, говори. Чтоб повторить Segmentation fault, наверное, надо пересоздать базу. Не знаю, что сделать. Может быть, тебе передать файлы БД для экспериментов?
picodata --version picodata 24.6.0-181-gc9aee752
- Last reply by Владимир Паршин
- Resolved by Егор Ивков
Не смогу проверить на новой версии из-за невылеченного:
$ psql postgres://postgres:T0psecret@127.0.0.1:5432?sslmode=disable -c 'SELECT * FROM pgbench_accounts ORDER BY aid DESC LIMIT 10 option(SQL_VDBE_MAX_STEPS=0, vtable_max_rows=0);' ERROR: picodata error: sbroad: rule parsing error: --> 1:66 | 1 | SELECT * FROM pgbench_accounts ORDER BY aid DESC LIMIT 10 option(SQL_VDBE_MAX_STEPS=0, vtable_max_rows=0); | ^--- | = expected VdbeMaxSteps or VTableMaxRows
- Last reply by Кирилл Безуглый
Resolved by Егор ИвковЭто, похоже, не связано с пгпрото. Проблема воспроизводится из консоли пикодаты?
Возможно, это уже исправлено на текущем мастере после выпиливания трейсинга, который вел к перерасходу памяти.
Edited by Егор Ивков- Last reply by Егор Ивков
assigned to @e-ivkov and unassigned @kbezuglyi
-
сегфолт из-за ООМ пофикшен в луаджите и в тарантуле - нам нужно черрипикнуть https://github.com/tarantool/tarantool/issues/10290
