sptree: don't use variable length arrays

This causes warnings if compiled with clang-18. Let's define a sane
upper limit for the max tree depth and use it for allocating arrays
on stack. Note that we don't really care about performance because
sptree is used only in unit tests.

Closes #10354

NO_DOC=internal
NO_TEST=internal
NO_CHANGELOG=internal