crypto.bi

Bitcoin Blockchain to MySQL ETL Tool

Crypto.BI Toolbox comes with an ETL tool that allows you to extract data from the block DAT files and import the blockchain into MySQL.

If you got the blocks directory configuration set up correctly, then you can run the ETL tool. Go to the Crypto.BI Toolbox installation directory and run:

$ ./toolbox/db/blocks_to_sql > <output_file.sql>

This will start the ETL tool which will produce SQL output on the console by default. The above command redirects output to an output file.

Note that only raw SQL will be output.

To configure the import of this dump file, you’ll need to use your database’s import tool with the right database schema and other connection parameters set up.

This tool does not access the database server, so no related configuration is necessary.

Disk Usage

Each block data file will produce around 700 MB (!!!) of SQL.

The resulting SQL file will likely be over 1.5 TB in size (~2000 files).

ETA

The program can run from several hours to several days depending on your system’s IO capabilities.

Our tests completed in approximately 8 hours.

Links

Blockchain ETL

Exporting and Analyzing Ethereum Blockchain

Coursera : Extract, transform and load data

About the Author
Published by Toolbox Team - Cryptography and cryptocurrency software development specialists. In-house nerds, stay at home aficionados, anti-pandemic crew of coffee addict devs. Learn More About Us