Binary transaction layout

In the current mainnet a transaction consists of 2,673 tryte-encoded characters and is composed of various fields of different tryte leghts (See https://docs.iota.org/docs/getting-started/0.1/transactions/transactions).

On a binary system this effectively leads to many binary-trinary conversion in order to parse/create transactions. In this topic we will discuss the possibility of replacing the transaction layout with a binary structure, without requiring a transition for existing addresses or private keys.

It is important to note, that with the currently used Kerl hash function a hash (and thus also and address) natevely does not consist of 243 trits but of 48 bytes. This means for every address or hash the 48 byte representation is absolutely equivalent to the currently used 243 trits representation, which is only the result of an additional conversion to base 3.

First, we need to represent a transaction as a binary structure. The current trinary transaction layout consists of the following types:

  • Integers
  • Hashes (hash (Curl), address (Kerl), bundle (Kerl)
  • signatureMessageFragment

Integers are trivial to encode in binary.

Trinary strings can be encoded as

  • 1 byte per tryte (3 trits),
  • 1 byte per 5 trits or
  • native 48 bytes for Kerl hashes.

For Kerl hashes (addresses, bundle hashes, signatures) the representation as 48 bytes is by far the fastest and most efficient as it does not require a rather costly (and in this case useless conversion) to trinary. However, this only works for Kerl.

Packing 5 trits into one byte leads to a better compression ratio, but it is also computationally more expensive. As such it seems like the best option to use 81 chars/bytes for the transaction hash, while all the other string fields are represented as 48 bytes or 1296 bytes for the signature fragment.

The bundle essence can now be defined analogously from the transaction data as a binary structure. The final resulting structure should be a multiple of 48 bytes in order to match the chunk size of keccak384.

The bundle hash is now defined as the keccak384 48 bytes hash of the bundle essence.

For the actual signing, this 48 byte hash is then converted into a regular 81 tryte hash using this algorithm.

This hash can then be signed using the curret W-OTS signatures scheme, assuring 100% compatibility with the current addresses and private keys.

This approach allows us to work on binary data, for validation, IO and other processing without the need to have many binary-ternary conversions as in current software. However, since the bundle hash can still be represented as 243 trits the actual signing scheme stays the same and no transition of funds is necessary.

Even if using LUTs ?

I just recently did some experiments and optimizations with the go library. Even when LUTs are used (and they do a lot for byte->trits not so much the other way) you still have to convert to trits as 5 doesn’t play nicely with the tryte boundary of 3 trits. And this adds further overhead.
Heaving 1 byte per 1 tryte is just a memcpy which is basically as fast as it goes.

2 Likes

And how do you suggest we can get the transaction’s hash?
we are currently doing it by using Curl which performs on trits

Also, from what i understand, the gain is that we reduce the size of the packet that is being sent over the network,
but you would have to convert a bundle data when signing/validating a tx,
how much we save then exactly?