Skip to content

Draft: feat: respond to picodata heartbeats over iproto in iproto thread

Summary

This MR adds necessary plumbing to parse & respond to picodata-specific heartbeat messages fully in the iproto thread, without hitting the TX thread. A iproto_picodata_hb_handler hook is added to let picodata define its own logic for generating the heartbeat response message based on its raft state.

To allow responding to the heartbeat messages, a static pri_obuf buffer is added, which, if not empty, will be sent out before any responses buffered from the TX thread. Instead, writing to the socket is done directly in the message handler, producing limitations 3 & 4.

Still a WIP, because I am not sure whether it has everything needed to stuff picodata raft heartbeats into the message or not.

Limitations

  1. This approach will not allow us to create new iproto connections while the TX thread is busy, since that still requires a roundtrip to the TX thread. This only allows sending (and getting responses to) heartbeats over existing connections.

  2. Requests other than heartbeats can fill up the per-connection request queue (which is sized according to iproto_net_msg_max picodata config parameter). After this happens, no responses to pings will be sent out.

  3. The heartbeat response is dropped if the connection is busy writing a response (waiting for the socket to become available for writing). This is done to prevent heartbeat responses being inserted into the middle of a response. This might be an issue for the picodata client, which usually expects a response to all of its messages.

  4. If a write of the heartbeat response could not be completed in one go without blocking, the connection will be closed. This is unlikely to happen, unless the being actively written to. This, however, is itself unlikely due to limitation 3.

Meta

Necessary for picodata#989 Docs follow-up: not necessary

Edited by Nikita Strygin

Merge request reports

Loading