Skip to content

Using Substrate API Sidecar

Introduction

Substrate API Sidecar allows applications to access blocks, account balance, and other information of Substrate-based blockchains through a REST API. This can be useful for exchanges, wallets or other types of applications that need to keep track of account balance and other state changes on a Tanssi ContainerChain. This page will describe how to install and run a Substrate API Sidecar for a Tanssi ContainerChain, and the commonly used API endpoints.

Installing and Running Substrate API Sidecar

There are multiple ways of installing and running the Substrate API Sidecar. This guide will describe the steps for installing and running it locally through NPM. For running Substrate API Sidecar through Docker, or building and running it from source, please refer to the Substrate API Sidecar Github Repository.

Note

The examples in this guide are based on a MacOS or Ubuntu 20.04 environment. If you're using Windows, you'll need to adapt them accordingly.

Furthermore, please ensure that you have Node.js and a package manager (such as npm or yarn) installed. Also, make sure you've initialized a package.json file for ES6 modules. To learn how to install Node.js, please check their official documentation.

Installing the Substrate API Sidecar

To install the Substrate API Sidecar service locally in the current directory, run this from the command line:

npm install @substrate/api-sidecar@17.1.2

Note

If the current folder does not already have a Node.js project structure, you need to manually created the node_modules directory by typing mkdir node_modules.

Substrate API Sidecar v17.1.2 is the current stable version that has been tested to work with ContainerChains. You can verify the installation was successful by typing from the installation directory root:

node_modules/.bin/substrate-api-sidecar --version

Setting up the Substrate API Sidecar

In the terminal that Sidecar will run, export the environmental variable for the WS endpoint of the network you want to connect to. For example, the WSS endpoint of your ContainerChain. Some examples:

export SAS_SUBSTRATE_URL=wss://fraa-dancebox-rpc.a.dancebox.tanssi.network
export SAS_SUBSTRATE_URL=wss://fraa-dancebox-3001-rpc.a.dancebox.tanssi.network
export SAS_SUBSTRATE_URL=INSERT_CONTAINERCHAIN_WSS_ENDPOINT

After setting the environmental variable, you can use the echo command to check that the environmental variable has been set correctly, by typing:

echo $SAS_SUBSTRATE_URL

And it should display the network endpoint you have just set.

Running Substrate API Sidecar

With the network endpoint environmental variable set, and from the installation directory root, run:

node_modules/.bin/substrate-api-sidecar

If the installation and configuration are successful, you should see this output in the console:

Successful Output

Substrate API Sidecar Endpoints

Some of the commonly used Substrate API Sidecar endpoints include:

  • GET /blocks​/head — Get the most recently finalized block. The optional parameter finalized can be set to false to the get the newest known block, which may not be finalized
  • GET /blocks/head/header — Get the most recently finalized block header. The optional parameter finalized can be set to false to the get the newest known block header, which may not be finalized
  • GET /blocks/{blockId} — Get a block by its height or hash
  • GET /accounts/{accountId}/balance-info — Get balance information for an account
  • GET /node/version — Get information about the Substrates node's implementation and versioning
  • GET /runtime/metadata — Get the runtime metadata in decoded, JSON form

For a full list of API endpoints available on Substrate API Sidecar, please refer to the official documentation.

Field Mapping in Block JSON Object

Substrate API Sidecar returns blocks as a JSON object. Part of this JSON object is a nesting structure for individual extrinsics processed in a specific block. Each extrinsic calls a specific method of a given module. Generally speaking, for individual extrinsics, the nesting structure is as following:

RESPONSE JSON Block Object:
    |--extrinsics
        |--{extrinsic_number}
            |--method
                |--pallet: "MODULE_NAME"
                |--method: "METHOD_NAME"
            |--signature
            |--nonce
            |--args
                |--transaction
                    |--{transaction_type}
            |--hash
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "MODULE_NAME"
                        |--method: "METHOD_EVENT_NAME"
                    |--data
                        |--0
                        |--1
                        |--2
                        |--3
    ...

Consequently, information from specific extrinsics (like balance transfers) can be extracted by knowing the module and method called by the extrinsic.

EVM Field Mapping in Block JSON Object

For EVM ContainerChains, the information related to EVM execution of each EVM ContainerChain transaction can be identified by the method field under the current extrinsic object, where it is set to:

{extrinsic_number}.method.pallet = "ethereum"
{extrinsic_number}.method.method = "transact"

The nesting structure for EVM transactions is as following:

RESPONSE JSON Block Object:
    |--extrinsics
        |--{extrinsic_number}
            |--method
                |--pallet: "ethereum"
                |--method: "transact"
            |--signature
            |--nonce
            |--args
                |--transaction
                    |--{transaction_type}
            |--hash
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "ethereum"
                        |--method: "Executed"
                    |--data
                        |--0
                        |--1
                        |--2
                        |--3
    ...

For example, for Substrate transactions, the "Nonce" and "Signature" fields are under:

extrinsics[extrinsic_number]

EVM Transaction Types and Payload

EVM ContainerChains currently support three transaction standards: legacy, eip1559, and eip2930. These correspond to the transaction type field in the above JSON object diagram. For each transaction type, the transaction payload contains the following fields:

    ...
    |--eip1559
        |--chainId
        |--nonce
        |--maxPriorityFeePerGas
        |--maxFeePerGas
        |--gasLimit
        |--action
        |--value
        |--input
        |--accessList
        |--oddYParity
        |--r
        |--s
    ...
    ...
    |--legacy
        |--nonce
        |--gasPrice
        |--gasLimit
        |--action
        |--value
        |--input
        |--signature
    ...
    ...
    |--eip2930
        |--chainId
        |--nonce
        |--gasPrice
        |--gasLimit
        |--action
        |--value
        |--input
        |--accessList
        |--oddYParity
        |--r
        |--s
    ...

For more information on the new EIP1559 and EIP2930 transaction types and what each field means, please refer to the respective official Ethereum proposal specs.

Transaction Field Mappings

To obtain the EVM sender address, recipient address, and EVM hash of any EVM transaction type, check the events field under the current extrinsic object, and identify the event where the method field is set to:

{event_number}.method.pallet: "ethereum"
{event_number}.method.method: "Executed"

The EVM field mappings are then summarized as the following:

EVM Field Block JSON Field
Chain ID extrinsics[extrinsic_number].args.transaction.eip1559.chainId
Nonce extrinsics[extrinsic_number].args.transaction.eip1559.nonce
Max priority fee per gas extrinsics[extrinsic_number].args.transaction.eip1559.maxPriorityFeePerGas
Max fee per gas extrinsics[extrinsic_number].args.transaction.eip1559.maxFeePerGas
Gas limit extrinsics[extrinsic_number].args.transaction.eip1559.gasLimit
Access list extrinsics[extrinsic_number].args.transaction.eip1559.accessList
Signature extrinsics[extrinsic_number].args.transaction.eip1559.oddYParity/r/s
Sender address extrinsics[extrinsic_number].events[event_number].data[0]
Recipient address extrinsics[extrinsic_number].events[event_number].data[1]
EVM hash extrinsics[extrinsic_number].events[event_number].data[2]
EVM execution status extrinsics[extrinsic_number].events[event_number].data[3]
EVM Field Block JSON Field
Nonce extrinsics[extrinsic_number].args.transaction.legacy.nonce
Gas price extrinsics[extrinsic_number].args.transaction.legacy.gasPrice
Gas limit extrinsics[extrinsic_number].args.transaction.legacy.gasLimit
Value extrinsics[extrinsic_number].args.transaction.legacy.value
Signature extrinsics[extrinsic_number].args.transaction.legacy.signature
Sender address extrinsics[extrinsic_number].events[event_number].data[0]
Recipient address extrinsics[extrinsic_number].events[event_number].data[1]
EVM hash extrinsics[extrinsic_number].events[event_number].data[2]
EVM execution status extrinsics[extrinsic_number].events[event_number].data[3]
EVM Field Block JSON Field
Chain ID extrinsics[extrinsic_number].args.transaction.eip2930.chainId
Nonce extrinsics[extrinsic_number].args.transaction.eip2930.nonce
Gas price extrinsics[extrinsic_number].args.transaction.eip2930.gasPrice
Gas limit extrinsics[extrinsic_number].args.transaction.eip2930.gasLimit
Value extrinsics[extrinsic_number].args.transaction.eip2930.value
Access list extrinsics[extrinsic_number].args.transaction.eip2930.accessList
Signature extrinsics[extrinsic_number].args.transaction.eip2930.oddYParity/r/s
Sender address extrinsics[extrinsic_number].events[event_number].data[0]
Recipient address extrinsics[extrinsic_number].events[event_number].data[1]
EVM hash extrinsics[extrinsic_number].events[event_number].data[2]
EVM execution status extrinsics[extrinsic_number].events[event_number].data[3]

For example, for EVM transactions, the "Nonce" and "Signature" fields are under:

extrinsics[extrinsic_number].args.transaction[transaction_type]

Consequently, this leaves the "Nonce" and "Signature" for the Substrate-level field extrinsics[extrinsic_number] to be null.

A successfully executed EVM transaction will return either succeed: "Stopped" or succeed: "Returned" under the "EVM Execution Status" field.

Monitor Token Balance Transfers

The following code samples will demonstrate how to listen to both native token transfers, sent via Substrate or Ethereum API, and ERC-20 token transfers sent via the Ethereum API, using Substrate API Sidecar. Transfers via the Ethereum API are only applicable to EVM ContainerChains.

Native Token Transfers

Both Tanssi, non-EVM ContainerChains and EVM ContainerChains can perform Substrate-based native token balance transfers.

The following code snippet uses the Axios HTTP client to query the Sidecar endpoint /blocks/head for the latest finalized block, and then decodes the block for the from, to, value, tx hash and transaction status of native token transfers at both the EVM and Substrate API level.

import axios from 'axios';

// This script will decode all native token transfers (Substrate & Ethereum)
// in a given Sidecar block, and extract the tx hash. It can be adapted for
// any ContainerChain.

// Endpoint to retrieve the latest block
const endpoint = 'http://127.0.0.1:8080/blocks/head';

async function main() {
  try {
    // Retrieve the block from the Sidecar endpoint
    const response = await axios.get(endpoint);
    // Retrieve the block height of the current block
    console.log('Block Height: ' + response.data.number);

    // Iterate through all extrinsics in the block
    response.data.extrinsics.forEach((extrinsic) => {
      // Retrieve Ethereum Transfers
      if (
        extrinsic.method.pallet === 'ethereum' &&
        extrinsic.method.method === 'transact'
      ) {
        // Get the value for any of the three EIP transaction standards supported
        const value =
          (extrinsic.args.transaction.legacy &&
            extrinsic.args.transaction.legacy.value) ||
          (extrinsic.args.transaction.eip1559 &&
            extrinsic.args.transaction.eip1559.value) ||
          (extrinsic.args.transaction.eip2930 &&
            extrinsic.args.transaction.eip2930.value);

        // Iterate through the events to get transaction details
        extrinsic.events.forEach((event) => {
          if (
            event.method.pallet === 'ethereum' &&
            event.method.method === 'Executed'
          ) {
            console.log('From: ' + event.data[0]);
            console.log('To: ' + event.data[1]);
            console.log('Tx Hash: ' + event.data[2]);
            console.log('Value: ' + value);
            // Check the execution status
            if (event.data[3].succeed) {
              console.log('Status: Success');
            } else {
              console.log('Status: Failed');
            }
          }
        });
      }

      // Retrieve Substrate Transfers
      if (
        extrinsic.method.pallet === 'balances' &&
        (extrinsic.method.method === 'transferKeepAlive' ||
          extrinsic.method.method === 'transfer')
      ) {
        // Iterate through the events to get transaction details
        extrinsic.events.forEach((event) => {
          if (
            event.method.pallet === 'balances' &&
            event.method.method === 'Transfer'
          ) {
            console.log('From: ' + event.data[0]);
            console.log('To: ' + event.data[1]);
            console.log('Tx Hash: ' + extrinsic.hash);
            console.log('Value: ' + event.data[2]);
            // Check the execution status
            if (extrinsic.success) {
              console.log('Status: Success');
            } else {
              console.log('Status: Failed');
            }
          }
        });
      }
    });
  } catch (err) {
    console.log(err);
  }
}

main();

ERC-20 Token Transfers

Events emitted by smart contracts such as an ERC-20 token contract deployed on EVM ContainerChains can be decoded from Sidecar block JSON objects. The nesting structure is as following:

RESPONSE JSON Block Object:
    |--extrinsics
        |--{extrinsic_number}
            |--method
                |--pallet: "ethereum"
                |--method: "transact"
            |--signature:
            |--nonce:
            |--args
                |--transaction
                    |--{transaction_type}
            |--hash
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "evm"
                        |--method: "Log"
                    |--data
                        |--0
                            |-- address
                            |-- topics
                                |--0
                                |--1
                                |--2
                            |-- data
            ...
    ...

ERC-20 token transfers will emit the Transfer event which can be decoded as the following:

Tx Information Block JSON Field
ERC-20 contract address extrinsics[extrinsic_number].events[event_number].data[0].address
Event signature hash extrinsics[extrinsic_number].events[event_number].data[0].topics[0]
Sender address extrinsics[extrinsic_number].events[event_number].data[0].topics[1]
Recipient address extrinsics[extrinsic_number].events[event_number].data[0].topics[2]
Amount extrinsics[extrinsic_number].events[event_number].data[0].data

Other events emitted by EVM smart contracts can be decoded in a similar fashion, but the content of the topics and data fields will change depending on the definition of the specific event.

Note

The amount transferred is given in accounting for decimals and in hexadecimal format.

Substrate API Transaction Fees

For Tanssi, non-EVM Continaer chains and EVM ContainerChains, all the information around fee data for transactions sent via the Substrate API can be extracted from the following block endpoint:

GET /blocks/{blockId}

The block endpoints will return data relevant to one or more blocks. You can read more about the block endpoints on the official Sidecar documentation.

Read as a JSON object, for a given pallet (module) and method, the transaction fee is provided by an associated event with the following extructure:

{event_number}.method.pallet: "transactionPayment"
{event_number}.method.method: "TransactionFeePaid"

The relevant nesting structure is as follows:

RESPONSE JSON Block Object:
    ...
    |--number
    |--extrinsics
        |--{extrinsic_number}
            |--method
            |--signature
            |--nonce
            |--args
            |--tip
            |--hash
            |--info
            |--era
            |--events
                |--{event_number}
                    |--method
                        |--pallet: "transactionPayment"
                        |--method: "TransactionFeePaid"
                    |--data
                        |--0
                        |--1
                        |--2
    ...

The object mappings are summarized as follows:

Tx Information Block JSON Field
Fee paying account extrinsics[extrinsic_number].events[event_number].data[0]
Total fees paid extrinsics[extrinsic_number].events[event_number].data[1]
Tip extrinsics[extrinsic_number].events[event_number].data[2]

Then, the total transaction fee paid for this extrinsic is mapped to the following field of the block JSON object:

extrinsics[extrinsic_number].events[event_number].data[1]

Ethereum API Transaction Fees

For EVM ContainerChains, users could also send funds via the Ethereum API. To calculate the fee incurred on transactions sent via the Ethereum API, the following formula can be used:

GasPrice = BaseFee + MaxPriorityFeePerGas < MaxFeePerGas ?
            BaseFee + MaxPriorityFeePerGas : 
            MaxFeePerGas;
Transaction Fee = (GasPrice * TransactionWeight) / 25000
Transaction Fee = (GasPrice * TransactionWeight) / 25000
Transaction Fee = (GasPrice * TransactionWeight) / 25000

The following sections describe in more detail each of the components needed to calculate the transaction fee.

Base Fee

The BaseFee is the minimum amount charged to send a transaction and is a value set by the network itself. It was introduced in EIP-1559. EVM ContainerChains have a dynamic fee mechanism that aims to replicate the EIP-1559 fee market mechanism, where the base fee is adjusted based on block congestion.

For example, for the Dancebox EVM ContainerChain template the minimum gas price is 1 GWei.

The BaseFee can be directly retrieved from the baseFeePerGas storage found in the baseFee module (pallet), using the following endpoint:

GET /pallets/baseFee/storage/baseFeePerGas?at={blockId}

Read as a JSON object, the relevant nesting structure is as follows:

RESPONSE JSON Storage Object:
    |--at
        |--hash
        |--height
    |--pallet
    |--palletIndex
    |--storageItem
    |--keys
    |--value

The relevant data will be stored in the value key of the JSON object. This value is a fixed point data type, hence the real value is found by dividing the value by the decimals.

GasPrice, MaxFeePerGas, and MaxPriorityFeePerGas

The GasPrice is used to specify the gas price of legacy transactions prior to EIP-1559. The MaxFeePerGas and MaxPriorityFeePerGas were both introduced in EIP-1559 alongside the BaseFee. The MaxFeePerGas defines the maximum fee permitted to be paid per unit of gas and is the sum of the BaseFee and the MaxPriorityFeePerGas. The MaxPriorityFeePerGas is the maximum priority fee configured by the sender of a transaction that is used to incentive the prioritization of a transaction in a block.

Although EVM ContainerChains are Ethereum-compatible, they are also Substrate-based chains at their core, and priorities work differently in Substrate than in Ethereum. In Substrate, transactions are not prioritized by gas price. To address this, EVM ContainerChains uses a modified prioritization system that reprioritizes Substrate transactions using an Ethereum-first solution. A Substrate transaction still goes through the validity process, where it is assigned transaction tags, longevity, and a priority. The original priority is then overwritten with a new priority based on the transaction's fee per gas, which is derived from the transaction's tip and weight. If the transaction is an Ethereum transaction, the priority is set according to the priority fee.

It's important to note that priority is not the sole component responsible for determining the order of transactions in a block. Other components, such as the longevity of a transaction, also play a role in the sorting process.

The values of GasPrice, MaxFeePerGas and MaxPriorityFeePerGas for the applicable transaction types can be read from the block JSON object according to the structure described in the Sidecar API page.

The data for an Ethereum transaction in a particular block can be extracted from the following block endpoint:

GET /blocks/{blockId}

The paths to the relevant values have also truncated and reproduced below:

EVM Field Block JSON Field
MaxFeePerGas extrinsics[extrinsic_number].args.transaction.eip1559.maxFeePerGas
MaxPriorityFeePerGas extrinsics[extrinsic_number].args.transaction.eip1559.maxPriorityFeePerGas
EVM Field Block JSON Field
GasPrice extrinsics[extrinsic_number].args.transaction.legacy.gasPrice
EVM Field Block JSON Field
GasPrice extrinsics[extrinsic_number].args.transaction.eip2930.gasPrice

Transaction Weight

TransactionWeight is a Substrate mechanism used to measure the execution time a given transaction takes to be executed within a block. For all transactions types, TransactionWeight can be retrieved under the event of the relevant extrinsic where the method field is set to:

pallet: "system", method: "ExtrinsicSuccess"

And then TransactionWeight is mapped to the following field of the block JSON object:

extrinsics[extrinsic_number].events[event_number].data[0].weight
The information presented herein has been provided by third parties and is made available solely for general information purposes. Tanssi does not endorse any project listed and described on the Tanssi Doc Website (https://docs.tanssi.network/). Tanssi Foundation does not warrant the accuracy, completeness or usefulness of this information. Any reliance you place on such information is strictly at your own risk. Tanssi Foundation disclaims all liability and responsibility arising from any reliance placed on this information by you or by anyone who may be informed of any of its contents. All statements and/or opinions expressed in these materials are solely the responsibility of the person or entity providing those materials and do not necessarily represent the opinion of Tanssi Foundation. The information should not be construed as professional or financial advice of any kind. Advice from a suitably qualified professional should always be sought in relation to any particular matter or circumstance. The information herein may link to or integrate with other websites operated or content provided by third parties, and such other websites may link to this website. Tanssi Foundation has no control over any such other websites or their content and will have no liability arising out of or related to such websites or their content. The existence of any such link does not constitute an endorsement of such websites, the content of the websites, or the operators of the websites. These links are being provided to you only as a convenience and you release and hold Tanssi Foundation harmless from any and all liability arising from your use of this information or the information provided by any third-party website or service.
Last update: January 24, 2024
| Created: August 17, 2023