crypto.bi

Scrape AVAX network peers using avax-python

P2P networking is the single most important technical functionality in all cryptocurrency implementations. The decentralized nature of Bitcoin, AVAX, Ethereum and others requires peers to talk to each other 24×7 in order to find out about transactions, new blocks, vertices (for DAG-based cryptos like AVAX) and so on.

Even if you had the world’s most powerful mining infrastructure, you wouldn’t be able to get the blocks to everyone’s knowledge without a fast and stable P2P network. Understanding the way each P2P system works is crucial to making the most of any cryptocurrency, especially 100% network-based ones like Avalanche AVAX.

Networking is even more critical in Avalanche and Proof of Stake cryptocurrencies, because the network conversation itself is used for transaction validation. While Bitcoin simply burns through gigawatts of electricity to produce blocks, next generation cryptos like AVAX require intense network conversation in order to establish consensus.

In such a scenario, the more peers you find, the more chances you have to get transactions out quickly and to receive transactions efficiently. This is especially true if you’re running some infrastructure service like a payments solution or transaction explorer.

In this article we describe a Python script which listens to the AVAX P2P network and tries to discover peers. It prints out lists of peers it is able to find as it listens to the Avalanche network.

avax-python Peer Scraper

As you probably know, avax-python is one of our favorite hobbies here at crypto.bi.

Recently, we decided to implement a network listener which could scrape the AVAX P2P network looking for peers.

It’s a kind of dumbed-down AVAX node which simply processes network peer gossip messages looking for more peers to connect to.

AVAX Protocol

The AVAX protocol specs tells us that nodes will send a Peers message to other nodes upon establishing a valid connection. It will also send a peers message in response to a GetPeers request, though this is not the only case.

The reference Golang implementation will automatically send a peer list once a peer is connected and sent it a version message (part of the default handshake), as you can see in line 660 of network/peer.go:

func (p *peer) version(msg Msg) {
	if p.gotVersion.GetValue() {
		p.net.log.Verbo("dropping duplicated version message from %s", p.id)
		return
	}

// ...snip....

	p.SendPeerList()

As you can see, when the Golang implementation receives a valid version message, its final command in that subroutine is to send back a PeerList. This makes perfect sense : in trying to build the most widely connected P2P network, trading Peer lists should be done up front (and periodically throughout the network conversation).

Getting a List of Peers

So, based on how the Go code works, what we have to do in order to build a list of AVAX peers is:

  • Connect to random AVAX nodes, starting with a list of trusted beacons.
  • Perform a proper handshake.
  • Wait for PeerList messages.

Sounds cool. Let’s try!

Below is how we implemented this in Python. First, we implemented a dummy protocol handler which ignores all messages except for PeerList.

    def peerlist(self, msg: Msg, peer: Peer):
        """On receiving a Peers message, a node should compare the nodes appearing in the message
        to its own list of neighbors, and forge connections to any new nodes."""
        self.avax_config.logger().debug("Handling peerlist : Msg {} Peer {}".format(msg, peer))    
        for p in msg.fields[5]:
            print(p)

We called this handler class HostLister(Handler). You may be wondering why we printed out the 5th field in the received message in msg.fields[5]?

Here’s the definition of a PeerList:

		Op.PeerList:    [Field.Peers],

Wait! It only has ONE field. Why do we request number 5 then?

Well, it turns out AVAX messages are not arrays. They are maps! We’re not requesting the 6th field from a zero-based array! We’re requesting a Field which happens to be encoded as number 5, as you can see here:

class Field:

    # Fields that may be packed. These values are not sent over the wire.
    VersionStr = 0
    NetworkID = 1
    NodeID = 2
    MyTime = 3
    IP = 4
    Peers = 5

So there it is. We need to read each peer in a Peers field within the message, which is number five. (We probably shouldn’t have hard-coded that number and used Field.Peers instead.). So, moving on.

Next, we instantiate an avax-python Node while passing it HostLister as the network handler:

hl = HostLister(avax_config)
avax_config.set("network_handler", hl)
# [...snip...]
node = Node(avax_config=avax_config)

When we run this, the HostLister handler will ignore all network messages except for PeerList.

When we run bin/list_peers.py it will print out a bunch of IP:port entries as soon as PeerList messages randomly arrive:

$ python3 bin/list_peers.py
139.59.44.108:9651
95.179.221.40:9651
3.236.27.81:9651
51.81.80.185:9651
88.157.23.186:9651
13.244.63.181:9651
144.91.115.220:9651
167.99.245.2:9651
51.137.131.148:9651
143.110.174.29:9651
54.184.27.142:9651
95.111.255.251:9651
54.251.208.21:9651
168.119.87.192:9651
208.190.138.53:9651
134.209.146.7:9651
174.138.6.31:9651
144.172.120.104:9651
95.111.241.166:9651
167.86.93.251:9651
184.72.11.192:9651
178.128.23.199:9651
78.141.208.143:9651
188.166.157.92:9651
223.25.78.232:9651
34.71.3.127:9651
.....

So, there it is! An AVAX Peer finder! The script will print out peers whenever a new PeerList message is received. This might take a while, so if you just leave the script running, it’ll probably collect a few hundred peers every hour or so.

We hope you like exploring Avalanche AVAX as much as we do. Stay tuned for more experiments!

Links

avax-python AVAX exploration tools in Python

Complete list_peers.py script

About the Author
Published by @rektbuildr - Software developer and technical writer. In charge of fixing the coffee machine at crypto.bi. Interest include Java, Avalanche AVAX, C and C++, Spark, Hadoop, Scala language, golang, RedHat OpenShift and kubernetes, container tech, DevOps and related techs. Learn More About Us