avax-python Network Message Pipeline

Here’s a short overview of the avax-python Avalanche message pipeline.

We’ve followed the reference Go implementation as closely as possible, but the Python code is still in very early development, so it’s lacking a lot of details. For example, right now the Python message pipeline is synchronous, whereas the Go implementation uses asynchronous channels and parallel goroutines.

If you’re trying to learn how the Avalanche Go node implementation works, the avax-python code may actually be useful in this respect, because it’s barebones and straight to the point. The ideas are the same. Since Avalanche is a multi-blockchain platform, where new chains and DAGs can be added dynamically, the system requires a routing system which delivers messages to the correct consensus engine and VM instance.

For example, messages to/from the P-Chain must go to the blockchain based consensus engine and VM, while X-Chain messages must go to the Avalanche DAG-based subsystem. But the network interface sends and receives all messages together – after all we usually only have one pathway to the Internet. So, most the work is performed in the routing part, which requires us to correctly decode binary network data, find the correct chain ID and route the message to the right consensus subsystem.

So here’s a quick summary of the path taken by an Avalanche message arriving from the network and into the consensus sytem:

• An avaxpython.network.network.Network object reads messages from peers.
• Messages are read and passed to the network_handler. You can implement different network handlers for different purposes. The default handler is avaxpython.network.handlers.AVAX.py
• The AVAX handler will then separate network-specific messages from consensus/validator messages.
• Network-specific messages are handled at AVAX.py directly.
• Consensus/validator messages are passed through to peer.net.router which is a ChainRouter instance.
• ChainRouter is implemented in avaxpython.snow.networking.router.chain_router.py and will route messages to the correct blockchain using the chain’s encoded 32 byte ID
• The blockchain-specific message receiver is implemented in avaxpython.snow.networking.router.handler.Handler which then routes the message to the consensus engine.
• Currently, there are two Avalanche consensus engine implementations: the blockchain Snowman engine and the DAG-based Avalanche engine. These are implemented in avaxpython.snow.engine.snowman.transitive.Transitive and avaxpython.snow.engine.avalanche.transitive.Transitive respectively.
• The engine will then parse the message bytes into the chain-specific structures. Avalanche will turn incoming payloads into Vertex and Tx objects, while Snowman will unmarshal Block and Tx objects.
• The engine works with the correct VM implementation on each blockchain’s specific consensus protocol. The Platform chain (P-Chain) uses the avaxpython.vms.platformvm implementation, while the Avalanche engine uses avaxpython.vms.avm.

So, this is a short summary of the path taken by network messages from the wire to the consensus engines and VM’s.

If you’re trying to learn the Go implementation, this summary is pretty close to what the Golang code does. Except the Go implementation is asynchronous and uses advanced multitasking constructs like channels and parallel goroutines to achieve high throughput.

Links

avax-python Github Project Page

Reference Avalanche Golang Implementation