Or, how I built an Ethereum client designed for resource-restricted devices on a resource-restricted device.
Nimbus's tag line is "an Ethereum 2.0 Sharding Client for Resource-Restricted Devices." My thought when reading this is if it's designed for resource-restricted devices, let's see if we can build it on a resource-restricted device. And when I say "resource-restricted device," I'm not talking about a Raspberry Pi, though you can certainly build and run Nimbus on a Raspberry Pi. What I'm talking about is building and running Nimbus on an Android device, in my case, a Oneplus 6T. What follows is a guided tour of how to get a development environment up and running on your Android device and how to leverage that to build and run Nimbus.
Before you can build Nimbus, you need something more foundational, a development environment.
First stop on our road to building Nimbus is Termux. Termux is a complete Linux distribution packaged up inside an Android app and gives you access to a complete operating system, including a terminal emulator, and even any of several Linux GUIs if you want, no root required. Once you've installed Termux and fire it up, you'll be greeted with a very familiar site, the Bash terminal prompt. If you've ever used Linux, this is the same Bash shell you've used on any other distribution under the sun. So far, so good.
Now that we have a Linux environment available to us, we need to get all the Nimbus prerequisite packages. As a reminder, Nimbus requires RocksDB, PCRE, GNU Make, Git, etc. The issue is that Termux has a relatively limited set of packages available to it. While you can install Make, Git, and maybe some of the other needed Posix utilities, any attempt to
apt install RocksDB will result in an unfriendly
package not found error message from the Termux package manager. What's a would-be phone coder to do?
Ubuntu on Termux
Our second stop is to leverage one of Termux's many awesome features, the ability to support chroot. Since Termux's default package manager doesn't have the packages we need, we're going to have to somehow get access to a package manager that does, which means setting up a guest Linux distribution in a chroot on Termux. There are a variety of options for other Linux distributions that are supported but we're going with Ubuntu. Use the below commands on Termux:
wget https://raw.githubusercontent.com/MFDGaming/ubuntu-in-termux/master/ubuntu.sh bash ubuntu.sh ./startubuntu.sh
You'll find it in a subdirectory named something like
~/jails/ubuntu.Now, we've got the basic OS setup that we need to be able to build Nimbus.
Okay, if you've made it this far, we're finally at the point where we ready to actually build an Ethereum client on our phone.
- First, load up your Ubuntu chroot
- Install the prerequisites (It may take a while, enter
apt update apt install librocksdb-dev libpcre3-dev git build-essential
- Clone the Nimbus repo:
git clone https://github.com/status-im/nimbus.git
make nimbusto build the Nimbus binary.
cd nimbus make nimbus
If all goes well, you should see output like below:
[email protected]:~/nimbus# make nimbus Building: build/nimbus [NimScript] exec: nim c --out:build/nimbus -d:chronicles_log_level=TRACE --verbosity:0 --hints:off --warnings:off -d:usePcreHeader --passL:"-lpcre" nimbus/nimbus.nim [email protected]:~/nimbus#
Now, let's see what we can do!
This is the moment we've been waiting for. Run
build/nimbus --nodiscover --log-level=TRACE
to fire up the client.
Note, I've turned off discovery and set the log level to TRACE so I can see what the client is doing. Nimbus defaults to a log-level of error in the build script so you won't see much output if you don't adjust it to something like TRACE/DEBUG/INFO.!
[email protected]:~/nimbus# build/nimbus --nodiscover --log-level=TRACE Nimbus Version 0.0.1 [linux: arm64, rocksdb, e651975c] Copyright (c) 2018-2020 Status Research & Development GmbH DBG 2020-01-04 01:30:33+00:00 UPnP topics="nat" tid=23766 file=nat.nim:64 msg="Internet Gateway Device found." DBG 2020-01-04 01:30:33+00:00 UPnP: added port mapping topics="nat" tid=23766 file=nat.nim:122 externalPort=30303 internalPort=30303 protocol=TCP DBG 2020-01-04 01:30:33+00:00 UPnP: added port mapping topics="nat" tid=23766 file=nat.nim:122 externalPort=30303 internalPort=30303 protocol=UDP INF 2020-01-04 01:30:33+00:00 RLPx listener up tid=23766 file=p2p.nim:87 self=enode://89e925220f113521a1d6ced3f5ad45575ad7127c7e6307501c00874208e0d8010[email protected]184.108.40.206:30303 INF 2020-01-04 01:30:33+00:00 Discovery disabled tid=23766 file=p2p.nim:109 TRC 2020-01-04 01:30:33+00:00 Waiting for more peers tid=23766 file=p2p.nim:112 peers=0
Now, yea, we've finally built Nimbus and gotten it up and running. That's cool and all but I want to write some actual code!
nano but it'd be nice to have a nice code editor to work with like VSCode. That isn't possible on your phone, or is it?
I use Visual Studio Code as my editor of choice and was hunting around the web for something similar to use on my phone. Sure, there's Stackblitz and Code Sandbox that will theoretically work in your phone's browser, but that's not going to work for building Nimbus locally. What I stumbled across was a little tool called BLACKICECoder. BlackIceCoder is essentially a stripped down version of VSCode that runs in your phone's browser that is specifically designed to run on Termux. Follow the instructions on the repo to get it up and running.
One important note: BlackIceCoder should be installed in your base Termux install and not from within the Ubuntu chroot.
Below is a simple script that makes starting BlackIceCoder easier.
#!/bin/bash php -S 127.0.0.1:1028 -t ~/BLACKICEcoder
When you run this script in Termux, open your phone's browser and go to localhost:1028 and the IDE will be ready and waiting for you to get down to coding.
Throwing BlackIceCoder into the mix is where the magic happens. Let's get down to coding!
./black.sh and BlackIceCoder will start.
Open your browser and go to localhost:1028.
Find the directory where the Nimbus repo lives (probably something like jails->ubuntu->ubuntu-fs->root->nimbus) in the file drawer on the right.
Once you've finished your edits, save your changes.
Switch back to Termux and open a new terminal session by swiping over from the left side of the screen and tapping on "New Session"
~/jails/ubuntu/start_ubuntu.sh to get to your Ubuntu chroot and navigate to your nimbus repo directory. As a reminder, adjust the above directory to match wherever the
start_ubuntu.sh script is located.
make nimbusagain, run
build/nimbusand test your changes.
And that's it. Now we're actually coding, building, and running Nimbus entirely within a virtual Linux environment on an Android device. As of this writing, I've actually used this set up for a few commits on a PR I'm making for the Nimbus repo so it actually does work. It's not a replacement for my main development setup but if I'm away from my PC and have a sudden inspiration or the urge to code, it lets me scratch that itch.
There are a few things to be aware of when working on Nimbus with the above setup.
- The above should theoretically work on any reasonably modern Android device though your results will vary if you have a low powered device without a lot of space or RAM. That said, this works pretty well on a reasonably specced device like my Oneplus 6T.
- On occasion, the
make nimbuscommand will fail inexplicably and throws an "OSError". This is probably just a resource constraint issue so don't have lots of other apps running while you're doing this. Remember, we are running an Ubuntu emulator inside of a Linux emulator inside of an Android app, so just be mindful of the limitations of your device.
make nimbuswill probably take a lot longer to compile Nimbus on your phone than on your high-powered workstation so don't get frustrated if it takes a couple of minutes to compile. I've seen it take more than 5 minutes to compile.
- When running Nimbus, remember that you're running this on your phone, so probably best to always run it with the
--nodiscoverargument if you're not connected to Wifi or you might chew up a lot data while it tries to sync whatever chain you're connected to.
- If you run Nimbus, it doesn't really respect the
Ctrl+zcommands to shut down gracefully. The only way I've found to kill Nimbus is to open a new terminal window and then run
from the Termux terminal.
- BlackIceCoder doesn't support syntax highlighting and color for Nim. That said, it still does autocomplete and parentheses matching like a champ.
- Sometimes, after you've compiled and run Nimbus and then switch back to BlackIceCoder in the browser, you'll find that it's completely refreshed and showing the default screen again rather than still being open to the file you were editing last. I think this is just a system limitation with how much Android can keep in memory at any one time. As such, always save your changes before switching between the browser and Termux.
One thing I did to simplify the start-up process for getting ready to code is a simple bash script called
ub.sh that I placed in my home Termux directory to simplify getting into the Ubuntu environment.
Put the above in a shell script and then you can get to Ubuntu by running:
./ub.shwhen the Termux terminal first comes up.