sql: replace MEM-type flags by enum mem_type

This patch moves MEM types from the 'u32 flags' field to the new
'enum mem_type type' field. Now, we can be sure that only one type is
set for MEM. In addition, it is now easier to distinguish MAP and ARRAY
from VARBINARY, and this makes it easier to add extension types - UUID
and DECIMAL.

Closes #4906