On-Chain Monitoring
Concrete protocol internal documentation
Introduction
first principles
Blockchain data for EVM networks is accessible by looking at confirmed or pending transactions, smart contract states, and emitted events. By synthesizing these data structures together, an accurate picture of the on-chain state can be modeled and monitored.
value prop.
Realtime on-chain monitoring (OCM) is essential for acute risk management, e.g. following and tracking liquidations or price updates as they happen, as well as accurate pricing, e.g. understanding historical on chain liquidity.
function
OCM involves retrieving, processing, and storing blockchain data directly from a network's node in an efficient, scalable, and accessible manner. This includes both "real-time" data from recently from a chain's head or pending in the mempool as well as historical data at randomly accessed time ranges.
System Architecture
nodes
Ethereum- we utilize Erigon (and are exploring reth) instead of Geth for archive performance reasons
Arbitrum- we utilize official client implementations to run both a nitro node (which launched 2022-08-31) and a classic node to serve pre nitro data
liquidation bot
V1 The liquidation bot listens to targeted protocols' smart contract's events that indicate a liquidation has occured and sends the resultant data to the backend for consumption for pricing and risk analysis. The OCM also tracks Chainlink price updates to provide DeFi integrated pricing to other Blueprint services.
v2 The liquidation bot tracks potential liquidation events in the mempool and alerts that they may be fired as well as continues to alert on actually emitted events
liquidity monitoring
Current- We generate datasets based on historically emitted dex Swap events for a variety of different liquidity pools and, at each event block, calculate the price impact of swapping a varying levels of requested quantities to gather some proxy of "generally" available liquidity and spreads over time.
The data is current shared and exported as a report using Jupyter Lab
Future- This approach can be generalized for any kind of EVM event as well as expanded to more blockchains depending on where we expect to execute risk management swaps.
We can also track, with the events, the tx originator plus other metadata that can be used for additional market counter intelligence
Common Understandings
mempool utility
At a high level, a mempool (memory pool) is a public, meaning accessible to all nodes, but not consistent, meaning different nodes may have different mempool representations, method for a blockchain's nodes to gossip and share pending transactions before they have been included in a proposed block.
By analyzing and tracking the mempool, we can get a predictive view of what transactions may be included before they are actually confirmed
EVM blockchains with a centralized sequencer, like most currrent L2 implementations, do not have the concept of a mempool because there is a single actor responsible for proposing blocks
evm event
An EVM Event is a data structure used by smart contracts that is much easier to index and thus track from an end user perspective. There is no standard or requirement, besides general best practices, as to when an Event must be emitted but are generally used by smart contract developers to indicate when something important happened or to provide an easier way to track state
Integration within the Product Ecosystem
The on chain data service will initially run on a separate machine and connect to the Pricing and Risk Engine servers via a SSH tunnel. The data will be fed to the backends using a Websocket connection where each respective service can subscribe to to different Event or Address alerts
The raw blockchain data will be pre-processed before being sent to the other Blueprint components to ensure it is human readable so token decimals are converted, timestamps converted from unix timestamps, and addresses are checksummed
The data may also be fed to our future alerting system, whether that be a slack bot or email notifications, with the exact format and requirements to be determined
The messaging protocol is defined using Msgspec https://jcristharif.com/msgspec/ and the schema can be exported/defined in the OpenApi format
Sample Communication format:
{
"anyOf": [
{
"type": "array",
"items": {
"anyOf": [
{
"$ref": "#/$defs/Sub"
},
{
"$ref": "#/$defs/USub"
}
],
"discriminator": {
"propertyName": "type",
"mapping": {
"sub": "#/$defs/Sub",
"usub": "#/$defs/USub"
}
}
}
},
{
"anyOf": [
{
"$ref": "#/$defs/Sub"
},
{
"$ref": "#/$defs/USub"
}
],
"discriminator": {
"propertyName": "type",
"mapping": {
"sub": "#/$defs/Sub",
"usub": "#/$defs/USub"
}
}
}
],
"$defs": {
"Sub": {
"title": "Sub",
"description": "New Subscription Request",
"type": "object",
"properties": {
"type": {
"enum": [
"sub"
]
},
"data": {
"anyOf": [
{
"$ref": "#/$defs/Event"
},
{
"$ref": "#/$defs/Address"
}
],
"discriminator": {
"propertyName": "sType",
"mapping": {
"event": "#/$defs/Event",
"address": "#/$defs/Address"
}
}
}
},
"required": [
"type",
"data"
]
},
"Event": {
"title": "Event",
"description": "Event monitoring subscription request, addr is the address of the contract and eName is the abi defined event name",
"type": "object",
"properties": {
"sType": {
"enum": [
"event"
]
},
"addr": {
"type": "string"
},
"eName": {
"type": "string"
},
"opts": {
"anyOf": [
{
"type": "object"
},
{
"type": "null"
}
],
"default": {}
}
},
"required": [
"sType",
"addr",
"eName"
]
},
"Address": {
"title": "Address",
"description": "Address monitoring subscription request, dir indicates monitoring to/from or both directions for the given address",
"type": "object",
"properties": {
"sType": {
"enum": [
"address"
]
},
"addr": {
"type": "string"
},
"dir": {
"$ref": "#/$defs/AddressSubDir"
},
"opts": {
"anyOf": [
{
"type": "object"
},
{
"type": "null"
}
],
"default": {}
}
},
"required": [
"sType",
"addr",
"dir"
]
},
"AddressSubDir": {
"title": "AddressSubDir",
"enum": [
"both",
"from",
"to"
]
},
"USub": {
"title": "USub",
"description": "Unsubscribe from a specific subscription id",
"type": "object",
"properties": {
"type": {
"enum": [
"usub"
]
},
"subId": {
"type": "string",
"format": "uuid"
}
},
"required": [
"type",
"subId"
]
}
}
}
Server Schema
{
"anyOf": [
{
"$ref": "#/$defs/Init"
},
{
"$ref": "#/$defs/Notify"
},
{
"$ref": "#/$defs/Error"
}
],
"discriminator": {
"propertyName": "type",
"mapping": {
"init": "#/$defs/Init",
"notify": "#/$defs/Notify",
"error": "#/$defs/Error"
}
},
"$defs": {
"Init": {
"title": "Init",
"description": "Initial server response providing server's internal client id",
"type": "object",
"properties": {
"type": {
"enum": [
"init"
]
},
"cid": {
"type": "integer"
}
},
"required": [
"type",
"cid"
]
},
"Notify": {
"title": "Notify",
"description": "Notify message for a given subscription id, provided in the Subscription confirmation message and actual subscription event data",
"type": "object",
"properties": {
"type": {
"enum": [
"notify"
]
},
"subId": {
"type": "string",
"format": "uuid"
},
"data": {
"type": "object"
}
},
"required": [
"type",
"subId",
"data"
]
},
"Error": {
"title": "Error",
"description": "Server response indicating some sort of error state",
"type": "object",
"properties": {
"type": {
"enum": [
"error"
]
},
"code": {
"$ref": "#/$defs/ErrorCodes"
},
"rMsg": {
"type": "object"
},
"eMsg": {
"type": "string"
}
},
"required": [
"type",
"code",
"rMsg",
"eMsg"
]
},
"ErrorCodes": {
"title": "ErrorCodes",
"enum": [
-1,
0,
1,
2
]
}
}
}
Last updated