txn: change limbo rollback check in the trigger

A transaction on rollback used to check if it was already rolled
back inside of the limbo by looking at its signature as

    signature != TXN_SIGNATURE_ROLLBACK

It meant the transaction is already completed.
TXN_SIGNATURE_ROLLBACK was used as a default value of the
signature. Therefore if it is not default, it is completed.

This is going to break if normal (not synchronous) transactions
would have more rollback codes except just TXN_SIGNATURE_ROLLBACK.
Also treatment of TXN_SIGNATURE_ROLLBACK as a default value looks
confusing.

Next patches are going to rework the codes and render the
assumptions above incorrect.

This patch makes the transaction use a correct way to check
whether it is in the limbo still - look at TXN_WAIT_SYNC flag. It
is set for all txns in the limbo and is not set for all the
others.

Part of #6027