net_box: don't use integer implementation defined

When calculating `rows_index` in `netbox_decode_execute` etc we use
implementation defined behavior of compiler. `i < mapsize` and this is
fine according to standard as unsigned arithmetic is well defined on
overflows. But then we cast the result to int and this is implementation
defined as the result can not be represented by int.

Closes https://github.com/tarantool/security/issues/98

NO_DOC=minor
NO_TEST=minor
NO_CHANGELOG=minor