Generate AVAX addresses and keys from a mnemonic phrase in Python

@rektbuildr

4 years ago

While playing with AVAX programming in Python, I wanted to have some wallet functionality on the Linux command line.

It’s easily done from Javascript, since Ava Labs provides an official library, but this specific application required Python and I really didn’t wanna mix the two languages (although it’s perfectly fine if you do).

You could also do this by running your own full node and importing your keys into it. But where’s the fun in that!?

Project Goals

Given a mnemonic pass phrase, I’d like to list a certain number of X-chain addresses. (Update: I’ve updated this post to include private keys and seed as well.)

The main motivation is so that I can later plug those addresses into an online tool to check their balances and transactions.

I’d like the mnemonic phrase process to be performed by an offline script, outside a web browser and possibly run it using an air gapped computer.

These are our main goals for this project!

Let’s see what we can come up with.

Into the Void

So I dug into the official AVAX wallet source code and traced the Vue.js front-end handlers to this subroutine:

// The master key from avalanche.js
    constructor(mnemonic: string) {
        let seed: globalThis.Buffer = bip39.mnemonicToSeedSync(mnemonic)
        let masterHdKey: HDKey = HDKey.fromMasterSeed(seed)
        let accountHdKey = masterHdKey.derive(AVA_ACCOUNT_PATH)

        super(accountHdKey, false)

        // Derive EVM key and address
        let ethAccountKey = masterHdKey.derive(ETH_ACCOUNT_PATH + '/0/0')
        let ethPrivateKey = ethAccountKey.privateKey
        this.ethKey = ethPrivateKey.toString('hex')
        this.ethAddress = privateToAddress(ethPrivateKey).toString('hex')
        this.ethBalance = new BN(0)

        let cPrivKey = `PrivateKey-` + bintools.cb58Encode(Buffer.from(ethPrivateKey))
        this.ethKeyBech = cPrivKey

        let cKeyChain = new KeyChain(ava.getHRP(), 'C')
        this.ethKeyChain = cKeyChain

        let cKeypair = cKeyChain.importKey(cPrivKey)
        this.ethAddressBech = cKeypair.getAddressString()

        this.type = 'mnemonic'
        this.seed = seed.toString('hex')
        this.hdKey = masterHdKey
        this.mnemonic = mnemonic
        this.isLoading = false
    }

That’s the code which generates a HD wallet from a mnemonic phrase.

That constructor is part of this class:

export default class AvaHdWallet extends HdWalletCore implements IAvaHdWallet

This same TS source file defines a few constants that we’ll copy as well:

const AVA_TOKEN_INDEX: string = '9000'
export const AVA_ACCOUNT_PATH: string = `m/44'/${AVA_TOKEN_INDEX}'/0'` // Change and index left out
const ETH_ACCOUNT_PATH: string = `m/44'/60'/0'`

const INDEX_RANGE: number = 20 // a gap of at least 20 indexes is needed to claim an index unused
const SCAN_SIZE: number = 70 // the total number of utxos to look at initially to calculate last index
const SCAN_RANGE: number = SCAN_SIZE - INDEX_RANGE // How many items are actually scanned

We’ll need those in a while. For now, let’s review how this process works.

BIP39 Derivation

As you probably know from BIP 39, a 256 bit key requires a 24 word mnemonic phrase.

In the wallet’s src/store/index.ts file we can see how this class is used:

async addWalletMnemonic(
            { state, dispatch },
            mnemonic: string
        ): Promise<AvaHdWallet | null> {
            // Cannot add mnemonic wallets on ledger mode
            if (state.activeWallet?.type === 'ledger') return null

            // Make sure wallet doesnt exist already
            for (var i = 0; i < state.wallets.length; i++) {
                let w = state.wallets[i] as WalletType
                if (w.type === 'mnemonic') {
                    if ((w as AvaHdWallet).mnemonic === mnemonic) {
                        throw new Error('Wallet already exists.')
                    }
                }
            }

            let wallet = new AvaHdWallet(mnemonic)
            state.wallets.push(wallet)
            state.volatileWallets.push(wallet)
            return wallet
        },

This dispatch handler is called when you click on the “Access Wallet” button after filling in the mnemonic phrase:

BTW: That’s my secret phrase, where I keep my grandsons** savings. Please don’t copy it.

Here’s the relevant TypeScript:

            let wallet = new AvaHdWallet(mnemonic)
            state.wallets.push(wallet)
            state.volatileWallets.push(wallet)
            return wallet

AvaHdWallet*

The above snippet instantiates the AvaHdWallet class, passing it the mnemonic phrase as a parameter. It then adds that wallet to two collections within the current browser’s memory. It looks like the first push goes into some form of durable local storage? (I assume it’ll forget the wallet when we leave, but with all the browser malware out there I wouldn’t trust this assumption at all.) Lastly, it returns the newly instantiated wallet.

*Notice how it’s called AvaHdWallet and not AvaxHdWallet? This is due to historical reasons. Until Q3 2020, the Avalanche project’s currency used to be called AVA. This collided with the pre-existing Travala AVA coin, so Ava Labs rebranded everything to AVAX. You’ll still occasionally find references to Ava in the Avalanche source code (e.g. AVA_ACCOUNT_PATH below). Just read AVAX and you’ll be fine!

So, this is exactly what we want to do in Python. Let’s continue tracing this TS code to see if it leads us to the HD wallet address derivation routines.

Back to the wallet construction code:

// The master key from avalanche.js
    constructor(mnemonic: string) {
        let seed: globalThis.Buffer = bip39.mnemonicToSeedSync(mnemonic)
        let masterHdKey: HDKey = HDKey.fromMasterSeed(seed)
        let accountHdKey = masterHdKey.derive(AVA_ACCOUNT_PATH)

        super(accountHdKey, false)

        // Derive EVM key and address
        let ethAccountKey = masterHdKey.derive(ETH_ACCOUNT_PATH + '/0/0')
        let ethPrivateKey = ethAccountKey.privateKey
        this.ethKey = ethPrivateKey.toString('hex')
        this.ethAddress = privateToAddress(ethPrivateKey).toString('hex')
        this.ethBalance = new BN(0)

        let cPrivKey = `PrivateKey-` + bintools.cb58Encode(Buffer.from(ethPrivateKey))
        this.ethKeyBech = cPrivKey

        let cKeyChain = new KeyChain(ava.getHRP(), 'C')
        this.ethKeyChain = cKeyChain

        let cKeypair = cKeyChain.importKey(cPrivKey)
        this.ethAddressBech = cKeypair.getAddressString()

        this.type = 'mnemonic'
        this.seed = seed.toString('hex')
        this.hdKey = masterHdKey
        this.mnemonic = mnemonic
        this.isLoading = false
    }

As far as I can tell, these were the steps performed by the constructor:

Derive a seed from the mnemonic phrase
Get a master HD key from the seed
Derive an account HD key from the master HD key
Do the same for an Ethereum subsystem wallet
Store the derived structures to the local object’s state

But there’s something missing here. We want the addresses! And I don’t see those getting generated anywhere in this particular code section… There must be an additional component missing!

Enter HdHelper

Diving deeper into AvaHdWallet.ts, we find that it uses a helper class to do most of its HD wallet related heavy lifting: HdHelper.ts. So let’s take a closer look at what it does.

(BTW reading these TS source files, I finally understand why the cooler fans sound like a jet engine whenever I access the AVAX wallet. There’s some heavy duty work happening in the background when you enter that mnemonic keyphrase!)

This is the screen we wish to replicate on the command line :

Digging a bit into the Vue code that renders this window, we find this:

<HdDerivationListRow
                        v-for="(addr, i) in addrsInternal"
                        :key="addr"
                        :index="i"
                        :address="addr"
                        :balance="keyBalancesInternal[i]"
                        :path="1"
                        class="list_row"
                    ></HdDerivationListRow>

What it does is iterate over several tuples made up of (address, index) where index is used to find the balance in a keyBalancesInternal array.

So, who generates keyBalancesInternal? I’m no Vue expert, but the only other place this string shows up is a function in HDDerivationList.vue :

    get keyBalancesInternal(): DerivationListBalanceDict[] {
        let wallet = this.wallet
        let utxoSet = wallet.internalHelper.utxoSet
        let addrs = this.addrsInternal
        return this.utxoSetToBalanceDict(utxoSet, addrs)
    }

I guess this is what we’re looking for. It loads the current wallet into an aptly named variable called …. wallet. Then it gets the set of unspent TX’s (UTXO) from its internalHelper, which happens to be an instance of HdHelper.

Great! Now we have the TypeScript recipe figured out, let’s do it in Python.

But first, a Word About Pizza

I really feel like this should be a legal mnemonic phrase. Not a single upper case character was used.

So, uh…. Back to Python.

Python Implementation

First things first, we need to turn a mnemonic phrase into an HD seed. We do this using the Mnemonic module.

from mnemonic import Mnemonic
# [....]
seed = Mnemonic("english").to_seed(" ".join(sys.argv[1:]), passphrase="")

This assumes you’ve passed the secret phrase as a list of parameters to the script. E.g. :

python3 offline/wallet/key_from_phrase.py shoulder man day worry sweet clip outdoor little matter interest option eyebrow asset visa snake find toddler labor puzzle danger quit secret flip foil

Now that we have a seed, we need to instantiate an HD wallet and generate some addresses using it!

HD Wallet Generation

The following Python snippet generates an account key that we’ll use to derive all future addresses:

accountHdKey = Bip32.FromSeedAndPath(seed, WalletConfig.AVA_ACCOUNT_PATH)

The AVA account path has been refactored to an external module for convenience:

AVA_TOKEN_INDEX = '9000'
AVA_ACCOUNT_PATH = f"m/44'/{AVA_TOKEN_INDEX}'/0'" # Change and index left out

We now have an account key, we can start generating some X-chain addresses!

ADDRESS_COUNT = 10
for index in range(ADDRESS_COUNT):
    addr = BIP32.get_address_for_index(accountHdKey, '0', index, "X", "avax")
    print(addr)

There you go! A 100% offline AVAX address generator starting from a mnemonic seed!

You can find the complete script (offline/wallet/addresses_from_mnemonic.py) and lots of other AVAX Python tools on the avax-python project (written by one of our contributors).

Generating Private Key From Mnemonic Phrase

What if you wanted the first private key derived from a HD wallet, so you could log into that specific address’ wallet?

In the TypeScript AVAX implementation this concept is known as a SingletonWallet.

It’s a self-explanatory term that means it’s a single address wallet. Since a private key is not a seed to an HD wallet, it can’t generate multiple addresses. When you use AVAX by loggin into your wallet using a single private key, it becomes an old school cryptocurrency wallet where you only had one address.

To achieve this using AVAX, go to the root of your avax-python installation and run offline/wallet/private_key_from_mnemonic.py passing it a mnemonic phrase:

python3 offline/wallet/private_key_from_mnemonic.py drill abstract solar magic crash derive chief fish mention sausage tenant drum violin enroll excess wife capable special tent venue predict captain museum question
{"masterHdKeyEncoded": "PrivateKey-2Mwsyw84uFiNUMS9cvUS2MaTv7RJ4yi8vd15gv3jvD9YZkN8kY", "accountHdKeyEncoded": "PrivateKey-2EwAnQcnH2RByKqGiv5zy3incJbgfTH623E9huhGKJrSK7qjcc", "masterHdKey": "b341c982f97fedefdef4929c2dc969defa0e8411edfa84932d9afcd06991cdea", "accountHdKey": "a355b689a9fe6d4ca6514f47c080dd1b951d301ac75258b46165a3fcd610d578", "firstHdKey": "cbe75b5f1abcb317c9e087e7ba51bf503a91423fd1716fd89981965b7726a880", "firstHdKeyEncoded": "PrivateKey-2YoSwnEeSK3W1Nhq5G1uDtVRKr3ECJZNMYJKw5Hqv1e9DihL7R", "seed": "0149a27f9564ccc8807d3f87a7a1108031f06493ec126dc799e6351928597dd52a8e62c55406e45c23e70c77a506c07bd8d9ca23e053dbb84f81cd0c9f12f8f7"}

Now take the firstHdKeyEncoded (or firstHdKey) values and use them to log into the AVAX wallet via the Private Key option.

Try logging in using the mnemonic phrase and then log in using firstHdKeyEncoded – you’ll see that the first address generated by the mnemonic phrase wallet is the same as the address generated by firstHdKeyEncoded. If you send funds to this address, the funds will show up on your mnemonic phrase wallet (well, duh, it’s the same address!).

A Word About Security

Keep in mind that you’re dealing with private keys directly using these commands and low level programming interfaces. More importantly : the private keys are visible on your screen throughout.

While we’ve been playing around with disposable wallets throughout this article, you should never, ever, post this data anywhere, nor keep windows open where this data is visible.

For instance, if you happen to take part in a live video session somewhere like Zoom or Teams and you have a shell window open with an avax-python session showing, then anyone who’s been recording the session can rewind the video and see your private keys if they want to. Note how the end-user AVAX wallet hides all private keys most the time (unless you explicitly click on show private key). As you can see the avax-python commands are low level and it’s assumed you know what you’re doing security-wise.

Last week I took part in a video session where the instructor had a Notepad window open with recently received credit card information on it. The PIN was clearly visible on one of the screens. This information was forever recorded in the session video. If the PIN we saw were a cryptocurrency private key, the damage could be irreversible.

avax-python commands are meant for experts and should be run offline whenever possible.