catnotes

Adding a Simple Transaction

December 9, 2021

If you've been around Symbol for a few months, you might remember the Cyprus fork. As part of that fork, a new MosaicSupplyRevocationTransaction was added to the protocol. Have you ever wondered what client changes were required to add that new functionality? Well, you're in luck because we're going to go over them here! 😄

⚠️ This is one of the simplest transactions in the Symbol protocol, so please don't expect all new functionality to be this easy.

Be prepared for a lot of C++ in this guide! With that out of the way, grab a ☕ or 🍺 and let's get started!

Architecture Overview

In this post, we will be focusing on the starred parts of the diagram. 💪

┌─────────────────┐          ┌─────────────────┐
│                 │          │                 │    1 *   ┌──────────────┐
│ Pipeline        ◄──────────┤  Block          ◄──────────┤ Consumer     │
│                 │          │  Disruptor      │          └──────────────┘
│                 ◄──┐     ┌►│                 │
└─────────▲───────┘  │     │ └─────────────────┘
          │          │     │ ┌─────────────────┐
          │1         └─────┼─┤                 │    1 *   ┌──────────────┐
          │*               │ │  Transaction    ◄──────────┤ Consumer     │
┌*********┴**********┐     │ │  Disruptor      │          └──────────────┘
* Transaction Plugin *     │ │                 ◄─┐
*  ┌──────────────┐  *     │ └─────────────────┘ │
*  │ Observers    │  *     │                     │
*  └──────────────┘  *     │  ┌───────────────┐  │
*  ┌──────────────┐  *     └──┤ Packet        ├──┘
*  │ Validators   │  *        │ Handlers      │
*  └──────────────┘  *        └───────▲───────┘
*  ┌**************┐  *                │
*  * Transactions *  *       ┌────────┴────────┐
*  * Definitions  *  *       │ NETWORK TRAFFIC │
*  └**************┘  *       └─────────────────┘
└********************┘

Procedure

Catapult is designed to be very composable and configurable, sometimes to its detriment. 😬 Nonetheless, this composability has a huge advantage in adding new features. The core processing pipeline knows nothing about specific transactions. Adding new transactions merely requires writing new code and wiring it up properly. Minimal modifications need to be made to existing code. It really is a great example of the benefits of the open-closed principle in practice.

Core Processing Pipeline is what processes blocks and transactions and modifies global blockchain state. It learns about transactions by loading one or more transaction plugins.

Transaction Plugin defines one or more transactions and specific instructions for interacting with them. These instructions include defining how to validate each new transaction and the state transitions triggered by each one. In addition, a plugin can define a new sub cache, which will increase the number of hashes in the SubCacheMerkleRoots. Typically, a plugin adds an entire feature like mosaics (the mosaic plugin) or namespaces (the namespace plugin).

Transaction defines a binary data layout that can be included in blocks. In addition, each transaction can be broken up into Notifications.

Notification is the smallest unit that can be processed by the Core Processing Pipeline. These tend to be single actions, like transferring X units of Mosaic M from Account S to Account R (BalanceTransferNotification). This allows a single handler to be used to validate and execute these actions instead of needing to reimplement them for each different transaction.

┌─────────────────┐          ┌─────────────────┐
│                 │   1 *    │                 │
│ Core Processing ├─────────►│  Transaction    │
│ Pipeline        │          │  Plugin         │
│                 │          │                 │
└─────────────────┘          └────────┬────────┘
                                     1│
                                      │
                                     *│
┌─────────────────┐          ┌────────▼────────┐
│                 │   * 1    │                 │
│  Notification   │◄─────────┤  Transaction    │
│                 │          │                 │
│                 │          │                 │
└─────────────────┘          └─────────────────┘

Since the MosaicSupplyRevocationTransaction is related to the handling of mosaics, we'll add it to the mosaic plugin.

Step 1 - Define Binary Transaction Format

Catapult uses a custom DSL called catbuffer to define the layout of binary data structures. Since we're adding a new transaction, we first need to define the binary layout. We will define it in catbuffer CATS format.

First we define the body, which defines two custom fields - source_address and mosaic:

# Shared content between MosaicSupplyRevocationTransaction and EmbeddedMosaicSupplyRevocationTransaction.
struct MosaicSupplyRevocationTransactionBody
    # Address from which tokens should be revoked.
    source_address = UnresolvedAddress

    # Revoked mosaic and amount.
    mosaic = UnresolvedMosaic

We add a transaction wrapper around the body, which indicates it can be used as a top-level transaction (i.e. not part of an aggregate):

# Revoke mosaic.
struct MosaicSupplyRevocationTransaction
    TRANSACTION_VERSION = make_const(uint8, 1)
    TRANSACTION_TYPE = make_const(TransactionType, MOSAIC_SUPPLY_REVOCATION)

    inline Transaction
    inline MosaicSupplyRevocationTransactionBody

The key directive is inline Transaction, which prepends the standard transaction header to the custom body.

We'll also add an embedded transaction wrapper around the body, which will allow it to be used within an aggregate:

# Embedded version of MosaicSupplyRevocationTransaction.
struct EmbeddedMosaicSupplyRevocationTransaction
    TRANSACTION_VERSION = make_const(uint8, 1)
    TRANSACTION_TYPE = make_const(TransactionType, MOSAIC_SUPPLY_REVOCATION)

    inline EmbeddedTransaction
    inline MosaicSupplyRevocationTransactionBody

The key directive is inline EmbeddedTransaction, which prepends the standard embedded transaction header to the custom body.

The full CATS source can be found here.

Ideally, we'd be able to autogenerate the C++ transaction model directly from the CATS schema. We hope to achieve that in the (near) future and/or in a galaxy far far away. For now, since we live in the present, we need to write the C++ model by hand.

Thankfully, it should look pretty similar to the CATS version:

/// Binary layout for a mosaic supply revocation transaction body.
template<typename THeader>
struct MosaicSupplyRevocationTransactionBody : public THeader {
private:
    using TransactionType = MosaicSupplyRevocationTransactionBody<THeader>;

public:
    DEFINE_TRANSACTION_CONSTANTS(Entity_Type_Mosaic_Supply_Revocation, 1)

public:
    /// Address from which tokens should be revoked.
    UnresolvedAddress SourceAddress;

    /// Revoked mosaic.
    UnresolvedMosaic Mosaic;

public:
    /// Calculates the real size of a mosaic supply revocation \a transaction.
    static constexpr uint64_t CalculateRealSize(const TransactionType&) noexcept {
        return sizeof(TransactionType);
    }
};

DEFINE_EMBEDDABLE_TRANSACTION(MosaicSupplyRevocation)

DEFINE_EMBEDDABLE_TRANSACTION indicates that this transaction can be used both inside and outside of aggregate transactions.

Entity_Type_Mosaic_Supply_Revocation is something we have to define in the plugin's Entity Types file. Luckily, it's only one line:

DEFINE_TRANSACTION_TYPE(Mosaic, Mosaic_Supply_Revocation, 0x3);

In the definition above, Mosaic specifies the plugin in order to encode the plugin's facility code in the type. Mosaic_Supply_Revocation is the friendly name. 0x3 is the local identifier that has to be unique within a plugin (i.e. it is the third transaction exposed by the mosaic plugin).

In the catapult client, links follow for the model and entity type.

Step 2 - Writing the Transaction Plugin

Since this new transaction is leveraging existing functionality, it doesn't need to add any new Validators (used for validating state changes before they happen) or Observers (used for executing state changes). Instead, all that we need to do is decompose the transaction actions down into preexisting notifications.

Importantly, we only need to raise notifications that are unique to the new transaction. Standard notifications that are common to all transactions are raised separately in the NotificationPublisher.

ℹ️ The notification publisher is what loads the transaction plugins via this code:

const auto& plugin = *m_transactionRegistry.findPlugin(transaction.Type);

In the above snippet, the transaction plugin corresponding to the specified transaction type (transaction.Type) is being retrieved from the transaction registry.

First, we need to raise MosaicRequiredNotification in order to indicate that the mosaic must have the Revokable flag set and it is owned by the signer.

ℹ️ In order to allow the Cyprus fork to pass, we relax this setting for the Nemesis signer since the XYM mosaic doesn't have the Revokable flag set.

Second, we issue a balance transfer request from the source address to the signer (mosaic owner) for the amount specified in the transaction.

template<typename TTransaction>
auto CreatePublisher(const Address& nemesisAddress) {
    return [nemesisAddress](const TTransaction& transaction, const PublishContext& context, NotificationSubscriber& sub) {
        auto isNemesisSigner = nemesisAddress == context.SignerAddress;
        auto requiredMosaicFlags = utils::to_underlying_type(isNemesisSigner ? MosaicFlags::None : MosaicFlags::Revokable);

        // MosaicFlagsValidator prevents any mosaics from being created with Revokable flag prior to fork block
        // consequently, MosaicSupplyRevocation transactions will be rejected until then because of Revokable flag requirement
        sub.notify(MosaicRequiredNotification(context.SignerAddress, transaction.Mosaic.MosaicId, requiredMosaicFlags));

        sub.notify(BalanceTransferNotification(
                transaction.SourceAddress,
                context.SignerAddress,
                transaction.Mosaic.MosaicId,
                transaction.Mosaic.Amount));
    };
}

Step 3 - Wiring It Up

Finally, we need to tell the mosaic plugin about the new transaction plugin. Yes, there are two levels of plugins. 😱 Don't leave, we're almost done!

In the MosaicPlugin, we just have to add this one line:

manager.addTransactionSupport(CreateMosaicSupplyRevocationTransactionPlugin(
    model::GetNemesisSignerAddress(manager.config().Network)));

By registering the MosaicSupplyRevocationTransactionPlugin, we're simply informing the core processing pipeline that the MosaicPlugin includes a MosaicSupplyRevocationTransaction. When transactions of that type are received, they should be forwarded to the MosaicPlugin for handling instead of being rejected outright.

Postscript

That wasn't so bad, was it? ... WAS IT?

There are a lot more things that we can go into. As mentioned before, the levels of configurability are very very high. We could talk about how to define custom validators, observers and notifications (oh my!). Maybe some of that could even be the topic of future articles. :thinking_face:

Hopefully, you got all the way to the end. Although, now you might really need a ☕ and/or 🍺! If you can't decide, I guess you could always have an Irish coffee!