Building Daedalus Cardano wallet without nix

Building Daedalus Cardano wallet without nix

This is a 2 part tutorial showing how you can create your own sovereign cryptocurrency using existing Cardano software.

  • Part I explains how to set up a genesis block and get your own network of nodes to start producing blocks and bootstrap a new cryptocurrency.
  • Part II (this post) shows you how to tinker with Daedalus to customize your own wallet for the new cryptocurrency.

Part II – Wallet Implementation

In our previous post, we set up our own cryptocurrency using the Haskell implementation of cardano-node.

For those not familiar with the details of Cardano software development, there’s also an official Cardano full node written in the Rust programming language.

Since Haskell is going to be the future official node implementation, it makes sense to to focus on it rather than the Rust version.

Fakedalus

Our main goal is to compile and run a bare-bones version of Daedalus without all the nix dependencies so we can play with the Javascript without having to build the Haskell side of cardano-node.

Throughout the config files I’ve called it Fakedalus Wallet :p to make sure nobody uses this in a production setting using real ADA.

The original Daedalus source code includes several pre-configured node configurations, such as selfnode, jormungandr and of course mainnet cardano-node.

In order to remove the nix dependencies, I forked the Daedalus source tree so we could tinker with it and keep a record of the changes somewhere.

By removing the dependency on nix, the electron code base can be tested outside the usual chroot environment.

Work Setup

As in our previous articles, I’ll be using Linux throughout these experiments. Here’s my lsb_release output :

$ lsb_release -a
No LSB modules are available.
Distributor ID:Ubuntu
Description:Ubuntu 19.10
Release:19.10
Codename:eoan

I use Jetbrains IDE’s most the time, but for the JS editing in this project I chose VS Code. Since VS Code itself is an electron app, its Javascript support is pretty awesome.

Quick Start

If you’re in a hurry, this section will get you up and running in minutes. (Keep reading below for the nitty gritty details.)

First, install the latest version of nodejs and npm. Here are the nodejs download and install instructions.

You will also need electron, yarn and gulp in order to build and run Daedalus. After you have nodejs properly configured, can easily install these using npm:

$ npm i -D electron@latest
$ npm install -g yarn
$ npm install --global gulp-cli

All the files we use in this text are located in this repo. In case you want to skip all the details and give it a try, just :

$ git clone https://github.com/cryptobi/daedalus.git
$ XDG_DATA_HOME=. LAUNCHER_CONFIG=config/myTN-launcher.yaml yarn build
$ XDG_DATA_HOME=. LAUNCHER_CONFIG=config/myTN-launcher.yaml yarn start

If everything works ok, you should see the main Daedalus screen open up and connect to your custom node.

The Details

Here you’ll find a detailed account of the changes I made to get the node to compile and run without nix. I hope this will help others when customizing Daedalus for their own applications.

Let’s take a look at the official documentation for the build entry point:

Run nix-shell with correct list of arguments or by using existing package.json scripts to load a shell with all the correct versions of all the required dependencies for development.

There are tons of scripts on package.json. I guess the very first one is a good starting point:

  "scripts": {
    "build": "gulp build",

Apparently build will call gulp build, so let’s try that.

$ gulp build
[ .. some output ..]
SyntaxError: 'import' and 'export' may appear only with 'sourceType: module'
[ .. lots of output ..]
[16:58:53] 'build' errored after 4.38 s

After chasing down the problem for hours, I finally thought of removing the transform from webpack.config.js. The code below caused the problem with EcmaScript (ES) module transforms:

      {
        test: /(pdfkit|linebreak|fontkit|unicode|brotli|png-js).*\.js$/,
        use: {
          loader: 'transform-loader?brfs',
        },
      },

Long story, short: when this code was written, some previous version of the modules which match the test regex used to require the transform-loader transformation. Now they don’t require this anymore and transforming the modules caused problems.

First Run

With a successful gulp build we now have a dist/ folder containing an electron app. Let’s try to run it.

$ gulp start

Daedalus improperly started!
Daedalus was launched without needed configuration. Please start Daedalus using the shortcut provided by the installer.

We’ll be developing our own configuration options later. For now, I’ll remove this requirement from config.js(see fork commits for details). Retrying after the edit, we get an error while trying to load some configuration file.

Apparently, it’s looking for the LAUNCHER_CONFIG environment variable, which is passed via const { LAUNCHER_CONFIG } = process.env. Let’s see how nix or whatever other build tool passes this variable in. In shell.nix we find:

LAUNCHER_CONFIG = launcherConfig';
# WHERE
launcherConfig = "${daedalusPkgs.daedalus.cfg}/etc/launcher-config.yaml";

Webpack Follies

Searching for launcher-config.yaml reveals that it’s not bundled in the source code. Instead, it’s downloaded by nix from IOHK servers. We don’t want that, so we’ll extract this file from inside the nix chroot store and into our source tree:

$ mkdir config/
$ cp /home/cryptobi/.daedalus/nix/store/n2ix31gayjz2si7sshgnbdyqikb8yna7-launcher-config/etc/launcher-config.yaml config/

That particular launcher configuration file is meant for jormungandr and the ITN. I also included a cardano-node launcher config for mainnet in config/.

So let’s give it a shot :

$ LAUNCHER_CONFIG=config/myTN-launcher.yaml gulp start

[15:43:07] Using gulpfile ~/cryptos/cardano/daedalus/gulpfile.js
[15:43:07] Starting 'start'...
(electron) 'getName function' is deprecated and will be removed. Please use 'name property' instead.
readLauncherConfig: warning var undefined: XDG_DATA_HOME
readLauncherConfig: warning var undefined: XDG_DATA_HOME
readLauncherConfig: warning var undefined: XDG_DATA_HOME
readLauncherConfig: warning var undefined: XDG_DATA_HOME
readLauncherConfig: warning var undefined: XDG_DATA_HOME
readLauncherConfig: warning var undefined: XDG_DATA_HOME
App threw an error during load
Error: ENOENT: no such file or directory, open '/home/cryptobi/cryptos/cardano/daedalus/dist/main/data.trie'
    at Object.openSync (fs.js:440:3)
    at Object.func (electron/js2c/asar.js:140:31)
    at Object.func [as openSync] (electron/js2c/asar.js:140:31)
    at Object.readFileSync (fs.js:342:35)
    at Object.fs.readFileSync (electron/js2c/asar.js:542:40)
    at Object.fs.readFileSync (electron/js2c/asar.js:542:40)
    at Module../node_modules/unicode-properties/unicode-properties.es.js (/home/cryptobi/cryptos/cardano/daedalus/dist/main/index.js:150805:110)
    at __webpack_require__ (/home/cryptobi/cryptos/cardano/daedalus/dist/main/index.js:20:30)
    at Object../node_modules/fontkit/index.js (/home/cryptobi/cryptos/cardano/daedalus/dist/main/index.js:71719:31)
    at __webpack_require__ (/home/cryptobi/cryptos/cardano/daedalus/dist/main/index.js:20:30)

Here we’re greeted by something caused by an incomplete webpack configuration, since there are missing external files. At least 3 different modules depend on a file called data.trie:

node_modules/fontkit/src/opentype/shapers/data.trie
node_modules/fontkit/data.trie
node_modules/unicode-properties/data.trie

In fact, manual testing revealed that several other files are missing as well. I’ll add each missing file extension to the file loader in webpack.config.js and see what happens:

test: /\.(woff2?|eot|ttf|otf|png|jpe?g|gif|svg|trie|afm)(\?.*)?$/,

Nothing happened – the file are still missing. I guess because the .trie files aren’t loaded via require(), so webpack doesn’t detect them as dependencies. Bummer. Javascript bundling isn’t my favorite kind of software deployment (this is such a hack TBH) and I don’t really know how to handle this in idiomatic Javascript.

Static File Copy

A reply on StackOverflow suggests manually copying the files as part of the build process. So I’ll create a deployment script specifically to get all the files into the right directories under dist/:

# install copy module
$ npm install --save copy

# See addLibDistFiles() on scripts/package.js

This worked, but now I need to call addLibDistFiles() from 2 gulp tasks (dev and build), so I’ll refactor it into its own file (scripts/copystatics.js). Let’s try running the dev task now:

# REF: dev launch command
XDG_DATA_HOME=. LAUNCHER_CONFIG=config/myTN-launcher.yaml yarn dev

Boom! Habemus Fakedalus!

Now we need to get the wallet talking to our node. Remember, for our new cryptocurrency net we have 2 block producing nodes running on ports 3001 and 3002. I’ll run a third common node on port 3005 to emulate a real client full node (not minting blocks).

But first, let’s make sure this setup depends 100% on Javascript build tools. The time to remove nix has come.

Removing nix

First, let’s find all filenames containing .nix :

$ find | grep \\.nix
./default.nix
./installers/daedalus-installer.nix
./installers/default.nix
./installers/dhall-haskell.nix
./installers/dhall-json.nix
./installers/dhall/shell.nix
./installers/nix/electron.nix
./installers/nix/linux.nix
./installers/nix/nix-installer.nix
./installers/overlays/add-test-stubs.nix
./installers/overlays/dhall-json.nix
./installers/overlays/dhall.nix
./installers/overlays/nsis.nix
./installers/overlays/required.nix
./installers/overlays/universum.nix
./installers/shell.nix
./lib.nix
./nix/cardano-bridge.nix
./nix/fastlist.nix
./nix/jormungandr-bridge.nix
./nix/launcher-config.nix
./nix/nsis-inner.nix
./nix/nsis.nix
./nix/sources.nix
./release-build.nix
./release.nix
./shell.nix
./tests/flow.nix
./tests/lint.nix
./tests/shellcheck.nix
./yarn2nix.nix

Done removed all these files, now let’s see what’s going to blow up. Using dev launch command (see above), we get:

And, there it is. Fakedalus’ first run! (Note the new window title.). 100% free of allnix scripts. Now let’s clear nix from the JS build scripts.

Done and, as expected, the build and dev gulp tasks run smoothly.

Connecting

We can finally focus on the meat and potatoes of all this work! Everything we’ve done until now was meant to enable us to run Daedalus (I mean Fakedalus) without depending on the Haskell node build process (which uses nix). We now have a 100% electron based project that we can tweak.

Let’s see how Daedalus talks to the node under normal operation. After agreeing to the EULA and selecting date/language, I’m greeted by this helpful message:

I’m sure this screen would contain something relevant in a production version. Clicking on that mystery button opens an URL on the default WWW brower (I use Brave Browser):

Sigh. That’s very helpful, thank you.

Back at the wallet, click on the X at the top right hand corner of the orange message and it’ll go away. Now we’re taken to this screen:

The Download logs message is all we could’ve asked for! The Daedalus.json log file contains all the errors found while trying to connect to the node. Let’s see what these errors look like.

"Copying mainnet genesis file failed"

"node: setStatus NotStarted -> Starting"

Here we have some strings to get us started, but the next message may be our best bet to search for the wallet-node interface:

{"errno":"ECONNREFUSED","code":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":30080}},"app":["daedalus"],"msg":"uncaughtException","pid":"","sev":"error","thread":""}

Searching for these error messages in the source code leads me to where they originated. By CTRL+clicking definitions one after the other on my IDE, I end up….at main/index.js [sigh]. Would’ve been easier to just start there.

Daedalus Entry Point

Daedalus starts running here:

if (!isSingleInstance) {
  app.quit();
} else {
  app.on('second-instance', () => {
    if (mainWindow) {
      if (mainWindow.isMinimized()) mainWindow.restore();
      mainWindow.focus();
    }
  });
  app.on('ready', onAppReady);
}

As we can see, the app tests whether another instance of Daedalus is already running. If it is, then it exits the system at once. Else, a handler for electron’s app.ready is responsible for triggering onAppReady. This is the main program’s entry point.

As we can see, Daedalus is a non reentrant application. Only one instance can run on a system at a given time. So much for all the functional concepts introduced by nix, such as isolation and immutability. The sole purpose of these concepts is to allow separate calls to the same function (the program) to not interfere with one another.

Node Startup

Here’s where things start to get interesting:

cardanoNode = setupCardanoNode(launcherConfig, mainWindow);

A cardanoNode instance is created by setupCardanoNode (defined in main/cardano/setup.js), based on the parameters contained in launcherConfig. (We won’t use the mainWindow parameter for now.)

These params are passed to startCardanoNode, which then destructures it into the individual fields used to fire up a Cardano full node:

startCardanoNode(cardanoNode, launcherConfig);

[ ... stepping into startCardanoNode ... ]

const {
    logsPrefix,
    nodeImplementation,
    nodeConfig,
    tlsPath,
    stateDir,
    cluster,
    block0Path,
    block0Hash,
    secretPath,
    configPath,
    syncTolerance,
    cliBin,
  } = launcherConfig;

launcherConfig is then expanded into a config object, with additional launcher parameters that get passed to the start()method of CardanoNode (defined in main/cardano/CardanoNode.js):

  const config = {
    logFilePath,
    nodeImplementation,
    nodeConfig,
    tlsPath,
    stateDir,
    cluster,
    block0Path,
    block0Hash,
    secretPath,
    configPath,
    syncTolerance,
    cliBin,
    startupTimeout: NODE_STARTUP_TIMEOUT,
    startupMaxRetries: NODE_STARTUP_MAX_RETRIES,
    shutdownTimeout: NODE_SHUTDOWN_TIMEOUT,
    killTimeout: NODE_KILL_TIMEOUT,
    updateTimeout: NODE_UPDATE_TIMEOUT,
  };

  return node.start(config);

The start() method spans over 150 lines. First it tests whether the node can be started, then counts how many times startup has been unsuccessfully retried and gives up if there’s been too many tries. It then destructures the config parameter once more into its many fields:

    // Setup
    const {
      // startupTimeout,
      nodeConfig,
      stateDir,
      cluster,
      tlsPath,
      block0Path,
      block0Hash,
      secretPath,
      configPath,
      syncTolerance,
      cliBin,
    } = config;

Next it sets up nodeLogFile and walletLogFile. Finally, we get to the code we’ve been looking for:

        const node = await CardanoWalletLauncher({
          nodeImplementation,
          nodeConfig,
          cluster,
          stateDir,
          tlsPath,
          block0Path,
          block0Hash,
          secretPath,
          configPath,
          syncTolerance,
          nodeLogFile,
          walletLogFile,
          cliBin,
        });

[... skip a few lines ...]

        node
          .start()
          .then(api => {
            const processes: {
              wallet: ChildProcess,
              node: ChildProcess,
            } = {
              wallet: node.walletService.getProcess(),
              node: node.nodeService.getProcess(),
            };

There it is. The startup code we’ve been looking for. To get our custom cryptocurrency running inside Fakedalus, we need to find a way to get all those parameters to CardanoWalletLauncher.

The Long and Winding Code

So now, our quest is to rewind and trail the road back to index.js making sure we can pass the necessary parameters to Fakedalus using the command line. This way we can easily test the wallet with any customizations we make to cardano-node.

Let’s take a look at each parameter passed to CardanoWalletLauncherand see where it originates :

Field NameTypeDescription
nodeImplementationStringIdentifies the node implementation. Currently valid values are ‘jormungandr’ or ‘cardano’.
nodeConfigNodeConfigA NodeConfiginstance. See below for NodeConfig fields.
clusterStringIdentifies the networkName . Possible values: ‘mainnet’ , ‘mainnet_flight’, ‘selfnode’, ‘staging’ and ‘testnet’
stateDirStringPath to a directory where these configuration files live: ‘config.yaml’ and ‘genesis.json’. Block data and node state will also be stored here.
tlsPathStringPath to a directory where these certificates live: ‘server/ca.crt’, ‘server/server.crt’, ‘server/server.key’
block0PathStringPath to a genesis.json genesis block file.
block0HashStringHex string containing the genesis block hash. Passed to jormungandr via --genesis-block-hash
secretPathStringPath to node secret file.
configPathStringPath to config.yaml file
syncToleranceStringA string in the format “XYZs”, such as “300s” for 300 seconds tolerance.
nodeLogFilefs.WriteStreamStream where node messages are logged.
walletLogFilefs.WriteStreamStream where wallet messages are logged.
cliBinStringCLI binary program. Can be “jcli” for jormungandr or “cardano-cli” for cardano-node.

NodeConfig has the following fields:

Field NameTypeDescription
configurationDirStringPath to a config dir. Seems to be left blank in all Daedalus node implementations.
delegationCertificateStringPath to a cert, passed to cardano-node via --delegation-certificate parameter
kindStringThe nodeConfig kind field. Possible values: ‘byron’, ‘shelley’ or ‘jormungandr’
network.configFileStringName of config file, normally called configuration.yaml
network.genesisFileStringName of genesis file, usually genesis.json
network.genesisHashStringHex string containing genesis block hash.
network.topologyFileStringName of file containing network topology. Normally called topology.json
signingKeyStringPath to cardano-node signing key, passed via --signing-key

The Trail

index.js gets launcherConfig from config.js by calling readLauncherConfig.

readLauncherConfig reads configPath from process.env

configPath contains YAML which is interpreted by readLauncherConfig and yields a LauncherConfig

LauncherConfig has all fields documented above.

All we have to do now is edit our own launcher config and run it! I’ve placed an example launcher config under config/ in our forked repo. For running instructions, see quick start section earlier in this article.

Conclusion

In this post, we’ve forked Daedalus and removed nixso we could play with the Javascript / electron code. Working with Javascript is always fun, because it’s a very high level language where there’s no concern for memory management and pointer arithmetic.

GUI’s built with electron, like Daedalus, are especially fun to tinker with, since it’s just like Javascript for browsers. Anyone familiar with basic WWW development will feel at home with electron.

References

Part I – Create Your Own Cryptocurrency using Cardano Technology – Node setup.

[Fakedalus] – crypto.bi Daedalus Fork – Working Tree

[Fakedalus] – crypto.bi Daedalus Fork

About the Daedalus Wallet

Meta