config: introduce droppable alerts

For now it's impossible to drop created alert in any way except
manual searching for _alerts table. However, we need to drop alerts
on missing names, when the names are set.

Let's introduce simple key-value alerts in order to easily drop them
by key.

Needed for #8978

NO_DOC=internal
NO_TEST=following commit
NO_CHANGELOG=internal