Using Substrate API Sidecar¶

Introduction¶

Substrate API Sidecar allows applications to access blocks, account balance, and other information of Substrate-based blockchains through a REST API. This can be useful for exchanges, wallets or other types of applications that need to keep track of account balance and other state changes on a Tanssi-powered network. This page will describe how to install and run a Substrate API Sidecar for a Tanssi network, and the commonly used API endpoints.

Installing and Running Substrate API Sidecar¶

There are multiple ways of installing and running the Substrate API Sidecar. This guide will describe the steps for installing and running it locally through NPM. For running Substrate API Sidecar through Docker, or building and running it from source, please refer to the Substrate API Sidecar Github Repository.

Installing the Substrate API Sidecar¶

To install the Substrate API Sidecar service locally in the current directory, run this from the command line:

npm install @substrate/api-sidecar@17.1.2

Substrate API Sidecar v17.1.2 is the current stable version that has been tested to work with Tanssi networks. You can verify the installation was successful by typing from the installation directory root:

node_modules/.bin/substrate-api-sidecar --version

Setting up the Substrate API Sidecar¶

In the terminal that Sidecar will run, export the environmental variable for the WS endpoint of the network you want to connect to. For example, the WSS endpoint of your Tanssi network. Some examples:

export SAS_SUBSTRATE_URL=wss://dancebox.tanssi-api.network

export SAS_SUBSTRATE_URL=wss://fraa-dancebox-3001-rpc.a.dancebox.tanssi.network

export SAS_SUBSTRATE_URL=INSERT_NETWORK_WSS_ENDPOINT

After setting the environmental variable, you can use the echo command to check that the environmental variable has been set correctly, by typing:

echo $SAS_SUBSTRATE_URL

And it should display the network endpoint you have just set.

Running Substrate API Sidecar¶

With the network endpoint environmental variable set, and from the installation directory root, run:

node_modules/.bin/substrate-api-sidecar

If the installation and configuration are successful, you should see this output in the console:

Substrate API Sidecar Endpoints¶

Some of the commonly used Substrate API Sidecar endpoints include:

• GET /blocks​/head — Get the most recently finalized block. The optional parameter finalized can be set to false to the get the newest known block, which may not be finalized
• GET /blocks/head/header — Get the most recently finalized block header. The optional parameter finalized can be set to false to the get the newest known block header, which may not be finalized
• GET /blocks/{blockId} — Get a block by its height or hash
• GET /accounts/{accountId}/balance-info — Get balance information for an account
• GET /node/version — Get information about the Substrates node's implementation and versioning
• GET /runtime/metadata — Get the runtime metadata in decoded, JSON form

For a full list of API endpoints available on Substrate API Sidecar, please refer to the official documentation.

Field Mapping in Block JSON Object¶

Substrate API Sidecar returns blocks as a JSON object. Part of this JSON object is a nesting structure for individual extrinsics processed in a specific block. Each extrinsic calls a specific method of a given module. Generally speaking, for individual extrinsics, the nesting structure is as following:

RESPONSE JSON Block Object:
    |--extrinsics
        |--{extrinsic_number}
            |--method
                |--pallet: "MODULE_NAME"
                |--method: "METHOD_NAME"
            |--signature
            |--nonce
            |--args
                |--transaction
                    |--{transaction_type}
            |--hash
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "MODULE_NAME"
                        |--method: "METHOD_EVENT_NAME"
                    |--data
                        |--0
                        |--1
                        |--2
                        |--3
    ...

Consequently, information from specific extrinsics (like balance transfers) can be extracted by knowing the module and method called by the extrinsic.

EVM Field Mapping in Block JSON Object¶

For Tanssi EVM networks, the information related to EVM execution of each Tanssi EVM network transaction can be identified by the method field under the current extrinsic object, where it is set to:

{extrinsic_number}.method.pallet = "ethereum"
{extrinsic_number}.method.method = "transact"

The nesting structure for EVM transactions is as following:

RESPONSE JSON Block Object:
    |--extrinsics
        |--{extrinsic_number}
            |--method
                |--pallet: "ethereum"
                |--method: "transact"
            |--signature
            |--nonce
            |--args
                |--transaction
                    |--{transaction_type}
            |--hash
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "ethereum"
                        |--method: "Executed"
                    |--data
                        |--0
                        |--1
                        |--2
                        |--3
    ...

For example, for Substrate transactions, the "Nonce" and "Signature" fields are under:

extrinsics[extrinsic_number]

EVM Transaction Types and Payload¶

Tanssi EVM networks currently support three transaction standards: legacy, eip1559, and eip2930. These correspond to the transaction type field in the above JSON object diagram. For each transaction type, the transaction payload contains the following fields:

    ...
    |--eip1559
        |--chainId
        |--nonce
        |--maxPriorityFeePerGas
        |--maxFeePerGas
        |--gasLimit
        |--action
        |--value
        |--input
        |--accessList
        |--oddYParity
        |--r
        |--s
    ...

    ...
    |--legacy
        |--nonce
        |--gasPrice
        |--gasLimit
        |--action
        |--value
        |--input
        |--signature
    ...

    ...
    |--eip2930
        |--chainId
        |--nonce
        |--gasPrice
        |--gasLimit
        |--action
        |--value
        |--input
        |--accessList
        |--oddYParity
        |--r
        |--s
    ...

For more information on the new EIP1559 and EIP2930 transaction types and what each field means, please refer to the respective official Ethereum proposal specs.

Transaction Field Mappings¶

To obtain the EVM sender address, recipient address, and EVM hash of any EVM transaction type, check the events field under the current extrinsic object, and identify the event where the method field is set to:

{event_number}.method.pallet: "ethereum"
{event_number}.method.method: "Executed"

The EVM field mappings are then summarized as the following:

For example, for EVM transactions, the "Nonce" and "Signature" fields are under:

extrinsics[extrinsic_number].args.transaction[transaction_type]

Consequently, this leaves the "Nonce" and "Signature" for the Substrate-level field extrinsics[extrinsic_number] to be null.

A successfully executed EVM transaction will return either succeed: "Stopped" or succeed: "Returned" under the "EVM Execution Status" field.

Monitor Token Balance Transfers¶

The following code samples will demonstrate how to listen to both native token transfers, sent via Substrate or Ethereum API, and ERC-20 token transfers sent via the Ethereum API, using Substrate API Sidecar. Transfers via the Ethereum API are only applicable to Tanssi EVM networks.

Native Token Transfers¶

Both Tanssi non-EVM networks and EVM networks can perform Substrate-based native token balance transfers.

The following code snippet uses the Axios HTTP client to query the Sidecar endpoint /blocks/head for the latest finalized block, and then decodes the block for the from, to, value, tx hash and transaction status of native token transfers at both the EVM and Substrate API level.

import axios from 'axios';

// This script will decode all native token transfers (Substrate & Ethereum)
// in a given Sidecar block, and extract the tx hash. It can be adapted for
// any Tanssi-powered network.

// Endpoint to retrieve the latest block
const endpoint = 'http://127.0.0.1:8080/blocks/head';

async function main() {
  try {
    // Retrieve the block from the Sidecar endpoint
    const response = await axios.get(endpoint);
    // Retrieve the block height of the current block
    console.log('Block Height: ' + response.data.number);

    // Iterate through all extrinsics in the block
    response.data.extrinsics.forEach((extrinsic) => {
      // Retrieve Ethereum Transfers
      if (
        extrinsic.method.pallet === 'ethereum' &&
        extrinsic.method.method === 'transact'
      ) {
        // Get the value for any of the three EIP transaction standards supported
        const value =
          (extrinsic.args.transaction.legacy &&
            extrinsic.args.transaction.legacy.value) ||
          (extrinsic.args.transaction.eip1559 &&
            extrinsic.args.transaction.eip1559.value) ||
          (extrinsic.args.transaction.eip2930 &&
            extrinsic.args.transaction.eip2930.value);

        // Iterate through the events to get transaction details
        extrinsic.events.forEach((event) => {
          if (
            event.method.pallet === 'ethereum' &&
            event.method.method === 'Executed'
          ) {
            console.log('From: ' + event.data[0]);
            console.log('To: ' + event.data[1]);
            console.log('Tx Hash: ' + event.data[2]);
            console.log('Value: ' + value);
            // Check the execution status
            if (event.data[3].succeed) {
              console.log('Status: Success');
            } else {
              console.log('Status: Failed');
            }
          }
        });
      }

      // Retrieve Substrate Transfers
      if (
        extrinsic.method.pallet === 'balances' &&
        (extrinsic.method.method === 'transferKeepAlive' ||
          extrinsic.method.method === 'transfer')
      ) {
        // Iterate through the events to get transaction details
        extrinsic.events.forEach((event) => {
          if (
            event.method.pallet === 'balances' &&
            event.method.method === 'Transfer'
          ) {
            console.log('From: ' + event.data[0]);
            console.log('To: ' + event.data[1]);
            console.log('Tx Hash: ' + extrinsic.hash);
            console.log('Value: ' + event.data[2]);
            // Check the execution status
            if (extrinsic.success) {
              console.log('Status: Success');
            } else {
              console.log('Status: Failed');
            }
          }
        });
      }
    });
  } catch (err) {
    console.log(err);
  }
}

main();

ERC-20 Token Transfers¶

Events emitted by smart contracts such as an ERC-20 token contract deployed on Tanssi EVM networks can be decoded from Sidecar block JSON objects. The nesting structure is as following:

RESPONSE JSON Block Object:
    |--extrinsics
        |--{extrinsic_number}
            |--method
                |--pallet: "ethereum"
                |--method: "transact"
            |--signature:
            |--nonce:
            |--args
                |--transaction
                    |--{transaction_type}
            |--hash
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "evm"
                        |--method: "Log"
                    |--data
                        |--0
                            |-- address
                            |-- topics
                                |--0
                                |--1
                                |--2
                            |-- data
            ...
    ...

ERC-20 token transfers will emit the Transfer event which can be decoded as the following:

Other events emitted by EVM smart contracts can be decoded in a similar fashion, but the content of the topics and data fields will change depending on the definition of the specific event.

Substrate API Transaction Fees¶

For Tanssi non-EVM networks and EVM networks, all the information around fee data for transactions sent via the Substrate API can be extracted from the following block endpoint:

GET /blocks/{blockId}

The block endpoints will return data relevant to one or more blocks. You can read more about the block endpoints on the official Sidecar documentation.

Read as a JSON object, for a given pallet (module) and method, the transaction fee is provided by an associated event with the following extructure:

{event_number}.method.pallet: "transactionPayment"
{event_number}.method.method: "TransactionFeePaid"

The relevant nesting structure is as follows:

RESPONSE JSON Block Object:
    ...
    |--number
    |--extrinsics
        |--{extrinsic_number}
            |--method
            |--signature
            |--nonce
            |--args
            |--tip
            |--hash
            |--info
            |--era
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "transactionPayment"
                        |--method: "TransactionFeePaid"
                    |--data
                        |--0
                        |--1
                        |--2
    ...

The object mappings are summarized as follows:

Then, the total transaction fee paid for this extrinsic is mapped to the following field of the block JSON object:

extrinsics[extrinsic_number].events[event_number].data[1]

Ethereum API Transaction Fees¶

For Tanssi EVM networks, users could also send funds via the Ethereum API. To calculate the fee incurred on transactions sent via the Ethereum API, the following formula can be used:

GasPrice = BaseFee + MaxPriorityFeePerGas < MaxFeePerGas ?
            BaseFee + MaxPriorityFeePerGas : 
            MaxFeePerGas;
Transaction Fee = (GasPrice * TransactionWeight) / 25000

Transaction Fee = (GasPrice * TransactionWeight) / 25000

Transaction Fee = (GasPrice * TransactionWeight) / 25000

The following sections describe in more detail each of the components needed to calculate the transaction fee.

Base Fee¶

The BaseFee is the minimum amount charged to send a transaction and is a value set by the network itself. It was introduced in EIP-1559. Tanssi EVM networks have a dynamic fee mechanism that aims to replicate the EIP-1559 fee market mechanism, where the base fee is adjusted based on block congestion.

For example, for the Tanssi EVM network template the minimum gas price is 1 GWei.

The BaseFee can be directly retrieved from the baseFeePerGas storage found in the baseFee module (pallet), using the following endpoint:

GET /pallets/baseFee/storage/baseFeePerGas?at={blockId}

Read as a JSON object, the relevant nesting structure is as follows:

RESPONSE JSON Storage Object:
    |--at
        |--hash
        |--height
    |--pallet
    |--palletIndex
    |--storageItem
    |--keys
    |--value

The relevant data will be stored in the value key of the JSON object. This value is a fixed point data type, hence the real value is found by dividing the value by the decimals.

GasPrice, MaxFeePerGas, and MaxPriorityFeePerGas¶

The GasPrice is used to specify the gas price of legacy transactions prior to EIP-1559. The MaxFeePerGas and MaxPriorityFeePerGas were both introduced in EIP-1559 alongside the BaseFee. The MaxFeePerGas defines the maximum fee permitted to be paid per unit of gas and is the sum of the BaseFee and the MaxPriorityFeePerGas. The MaxPriorityFeePerGas is the maximum priority fee configured by the sender of a transaction that is used to incentive the prioritization of a transaction in a block.

Although Tanssi EVM networks are Ethereum-compatible, they are also Substrate-based chains at their core, and priorities work differently in Substrate than in Ethereum. In Substrate, transactions are not prioritized by gas price. To address this, Tanssi EVM networks uses a modified prioritization system that reprioritizes Substrate transactions using an Ethereum-first solution. A Substrate transaction still goes through the validity process, where it is assigned transaction tags, longevity, and a priority. The original priority is then overwritten with a new priority based on the transaction's fee per gas, which is derived from the transaction's tip and weight. If the transaction is an Ethereum transaction, the priority is set according to the priority fee.

It's important to note that priority is not the sole component responsible for determining the order of transactions in a block. Other components, such as the longevity of a transaction, also play a role in the sorting process.

The values of GasPrice, MaxFeePerGas and MaxPriorityFeePerGas for the applicable transaction types can be read from the block JSON object according to the structure described in the Sidecar API page.

The data for an Ethereum transaction in a particular block can be extracted from the following block endpoint:

GET /blocks/{blockId}

The paths to the relevant values have also truncated and reproduced below:

Transaction Weight¶

TransactionWeight is a Substrate mechanism used to measure the execution time a given transaction takes to be executed within a block. For all transactions types, TransactionWeight can be retrieved under the event of the relevant extrinsic where the method field is set to:

pallet: "system", method: "ExtrinsicSuccess"

And then TransactionWeight is mapped to the following field of the block JSON object:

extrinsics[extrinsic_number].events[event_number].data[0].weight