We now dive into one of the most exciting components of the Bitcoin Core source code.
The RPC subsystem interconnects the connection manager, wallets, the storage subsystem and, most importantly, the front-end with which users interact.
It’s basically the Bitcoin command interpreter embedded in the system and, from a developer’s perspective, the application programming interface.
Most block explorers and similar applications actually interface with the Bitcoin Core RPC subsystem. Since the reference implementation is not meant to be a block explorer, but a fully verifying node instead, developers must customize the way Bitcoin Core works in order to develop special apps on top of it. Therefore understanding the RPC system is crucial to good Bitcoin application development.
RPC is also a great hub for our Bitcoin source code exploratory purposes!
Since RPC offers an interface into some of the main components of Bitcoin Core, as well as the most important functionality from the end user’s perspective (creating wallets, checking transactions, sending transactions and so on) it offers us a great starting point for many routines.
How does a transaction get created and sent? There’s an entry point for that in RPC. How do wallets get created? There’s an entry point for that too and so on for most Bitcoin Core functionalities.
So let’s get started because, as you can imagine, it’s a big source file.
rpc/server.h
rpc/server.h defines four classes and one constant variable, namely static const unsigned int DEFAULT_RPC_SERIALIZE_VERSION = 1
DEFAULT_RPC_SERIALIZE_VERSION alters the value of the -rpcserialversion command line option in init.cpp both in SetupServerArgs and in AppInitParameterInteraction subroutines. It’s used as a default value for comparison with a user-provided version.
Next up we have the declaration of two event handlers within namespace RPCServer : OnStarted and OnStopped. As the names imply, these are called when the RPC server starts and stops. The handlers are wired up to signals on init.cpp
Next up 4 subroutines are declared in the global namespace: IsRPCRunning, SetRPCWarmupStatus, SetRPCWarmupFinished and RPCIsInWarmup. The names are self-explanatory but we’ll go over these in detail when we discuss rpc/server.cpp
class RPCTimerBase is a trivial interface which only defines destructor. RPCTimerBase is used from RPCTimerInterface which is an interface declared immediately after it on rpc/server.h
Two concrete classes implement RPCTimerBase : HTTPRPCTimer and QtRPCTimerBase
Three global functions then take a RPCTimerInterface as parameter in order to set which concrete class will generate NewTimer‘s :
/** Set the factory function for timers */
void RPCSetTimerInterface(RPCTimerInterface *iface);
/** Set the factory function for timer, but only, if unset */
void RPCSetTimerInterfaceIfUnset(RPCTimerInterface *iface);
/** Unset factory function for timers */
void RPCUnsetTimerInterface(RPCTimerInterface *iface);Timers are organized in a map which relates name to timer function. RPCSetTimerInterface sets the factory function and replaces name, RPCSetTimerInterfaceIfUnset only sets the factory if name isn’t set yet and lastly RPCUnsetTimerInterface clears any previously registered factory.
A global function is defined to schedule functions to be run at a later time:
/**
* Run func nSeconds from now.
* Overrides previous timer <name> (if any).
*/
void RPCRunLater(const std::string& name, std::function<void()> func, int64_t nSeconds);As with most RPC-related functions, it takes a timer name, a function object and a delta in seconds after which func should run.
Next up we have a convenient typedef to a function pointer that’ll be used as a parameter to CRPCCommand::CRPCCommand() constructor.
typedef UniValue(*rpcfn_type)(const JSONRPCRequest& jsonRequest);rpcfn_type takes a JSONRPCRequest and returns a UniValue. We’ll go over Univalue‘s later, for now you can think of it as a “variant” type that can take any value from a JSON structure (scalars, arrays, etc).
We now find the class CRPCCommand declaration.
CRPCCommand is a simple wrapper around a RPC command. It contains what should be done (function pointer), a category, name, arguments and so on.
The CRPCTable class implements a dispatch table that maps names to RPC commands. Notice how the data types slowly build on one another. CRPCTable takes std::string‘s and CRPCCommand‘s and maps the strings to the commands so they can be called via RPC. CRPCCommand itself uses all the definitions we’ve looked at earlier. Having these definitions in a single file is really handy. You can clearly see how the RPC subsystem takes shape.
CRPCTable implements an execute method that takes a JSONRPCRequest. The method name passed to execute via the JSON request is mapped to a CRPCCommand using the dispatch table. If the name matches, it becomes a CRPCCommand that is executed. In essence this is like a little interpreted language inside Bitcoin Core. Most scripting languages use this kind of dispatch table mechanism to implement dynamic functions.
Lastly some RPC routine functions and global variable are declared:
bool IsDeprecatedRPCEnabled(const std::string& method);
extern CRPCTable tableRPC;
void StartRPC();
void InterruptRPC();
void StopRPC();
std::string JSONRPCExecBatch(const JSONRPCRequest& jreq, const UniValue& vReq);
// Retrieves any serialization flags requested in command line argument
int RPCSerializationFlags();The IsDeprecatedRPCEnabled subroutine checks a method name against the -deprecatedrpc configuration option. If a method matches then it returns true.
A global dispatch table is declared in tableRPC – this table will be used to process all RPC commands.
Start, interrupt and stop RPC subroutines do what their names suggest. More about these when we discuss the implementation.
JSONRPCExecBatch executes a batch of vReq against jreq.
Lastly, RPCSerializationFlags checks whether the -rpcserialversion command line argument is equal to 0. If it is, then it sets SERIALIZE_TRANSACTION_NO_WITNESS bits to 1 and returns.
rpc/server.cpp
rpc/server.cpp begins by defining several static variables and a subroutine. Note that these are visible only in rpc/server.cpp
static CCriticalSection cs_rpcWarmup;
static std::atomic<bool> g_rpc_running{false};
static bool fRPCInWarmup GUARDED_BY(cs_rpcWarmup) = true;
static std::string rpcWarmupStatus GUARDED_BY(cs_rpcWarmup) = "RPC server started";
/* Timer-creating functions */
static RPCTimerInterface* timerInterface = nullptr;
/* Map of name to timer. */
static std::map<std::string, std::unique_ptr<RPCTimerBase> > deadlineTimers;
static bool ExecuteCommand(const CRPCCommand& command, const JSONRPCRequest& request, UniValue& result, bool last_handler);cs_rpcWarmup is a critical section (wrapper to a std::recursive_mutex) that will be set when RPC is in warmup. The fRPCInWarmup boolean and the status string rpcWarmupStatus will be guarded by this mutex.
Next we have a section where data types are built on the previous one, sequentially. Note how convenient it is to have these in sequence within the same source file.
struct RPCCommandExecutionInfo
{
std::string method;
int64_t start;
};
struct RPCServerInfo
{
Mutex mutex;
std::list<RPCCommandExecutionInfo> active_commands GUARDED_BY(mutex);
};
static RPCServerInfo g_rpc_server_info;
struct RPCCommandExecution
{
std::list<RPCCommandExecutionInfo>::iterator it;
explicit RPCCommandExecution(const std::string& method)
{
LOCK(g_rpc_server_info.mutex);
it = g_rpc_server_info.active_commands.insert(g_rpc_server_info.active_commands.end(), {method, GetTimeMicros()});
}
~RPCCommandExecution()
{
LOCK(g_rpc_server_info.mutex);
g_rpc_server_info.active_commands.erase(it);
}
};
static struct CRPCSignals
{
boost::signals2::signal<void ()> Started;
boost::signals2::signal<void ()> Stopped;
} g_rpcSignals;
void RPCServer::OnStarted(std::function<void ()> slot)
{
g_rpcSignals.Started.connect(slot);
}
void RPCServer::OnStopped(std::function<void ()> slot)
{
g_rpcSignals.Stopped.connect(slot);
}RPCServerInfoencapsulates a list of RPCCommandExecutionInfo which was defined just 9 lines before.
Next, a static instance of it is constructed:
static RPCServerInfo g_rpc_server_info
Note that g_rpc_server_info is static, meaning only subroutines defined in rpc/server.cpp file will use it.
RPCCommandExecutionencapsulates synchronized execution of a command that’s passed in by name via a std::string. It only has a constructor and destructor and zero methods. Once constructed, it adds the RPC command passed in by name onto g_rpc_server_info‘s active_commands std::list. The destructor does the opposite, removing the command from active_commands. Both the addition and removal are critical sections locked by g_rpc_server_info.mutex.
Next we wire up the OnStarted and OnStopped application signals (using boost::signals2, not to be confused with OS signals):
static struct CRPCSignals
{
boost::signals2::signal<void ()> Started;
boost::signals2::signal<void ()> Stopped;
} g_rpcSignals;
void RPCServer::OnStarted(std::function<void ()> slot)
{
g_rpcSignals.Started.connect(slot);
}
void RPCServer::OnStopped(std::function<void ()> slot)
{
g_rpcSignals.Stopped.connect(slot);
}Again, the class definition and methods that use it are in sequence, making it easy to understand how these work. First struct CRPCSignals encapsulates the two signals and defines a static g_rpcSignalsinstance. Then the RPCServer slots are connected to these two signals using the recently defined g_rpcSignalsinstance.
Next, the std::string CRPCTable::help method simply prints out information about available RPC methods. You may want to read into this method to see how the names and descriptions we discussed in rpc/server.h are used here. Otherwise it’s just iterating through RPC table printing out the requested help item.
Now we begin implementing each RPC command in its own static subroutines. These subroutines are defined static within rpc/server.cpp so they’re not visible anywhere else in the code. This means that all RPC methods are sealed within rpc/server.cpp– keep this in mind as you browse the Bitcoin Core source code.
The first methods handle control/query commands:
UniValue help(const JSONRPCRequest& jsonRequest) subroutine does the same as CRPCTable::help but returns help in JSON UniValue format.
UniValue stop(const JSONRPCRequest& jsonRequest) – Stops the RPC subsystem by calling StartShutdown()
static UniValue uptime(const JSONRPCRequest& jsonRequest) – Returns the number of seconds since the server’s been running.
static UniValue getrpcinfo(const JSONRPCRequest& request) – Returns active commands, method, log path and other RPC server info.
Only these 4 administrative commands are explicitly handled here. All other commands are handled by the dispatch table we discussed earlier when we looked at CRPCTablewhich delegates methods by name.
To handle RPC commands by name, the rpc/server.cpp now implements the methods which manage the dispatch table: appendCommand and removeCommand. Both are intuitive, they simply add and remove items from the mapCommands dispatch table.
Next we have the RPC server control subroutines: StartRPC, InterruptRPC, StopRPC, IsRPCRunning, SetRPCWarmupStatus, SetRPCWarmupFinished, RPCIsInWarmup and IsDeprecatedRPCEnabled. These are trivial getters and setters that simply manipulate the structures we’ve already discussed. For example, InterruptRPC simply sets g_rpc_running = false. Not much commentary is needed for these, they’re all 3 to 4 lines maximum.
Now we reach the JSON execution engine subroutines.
static UniValue JSONRPCExecOne(JSONRPCRequest jreq, const UniValue& req) executes jreq by passing it req parameters. The returned UniValue is not at all related to the one passed as parameter (note it’s a const that can’t be modified within the method).
std::string JSONRPCExecBatch(const JSONRPCRequest& jreq, const UniValue& vReq) interprets vReq as a vector of requests. All it does is iterate through vReq passing each item to JSONRPCExecOne.
static inline JSONRPCRequest transformNamedArguments(const JSONRPCRequest& in, const std::vector<std::string>& argNames) is an interesting subroutine. It iterates through named parameters and then composes those parameters into an ordered list in the order required by the requested command. So if we had command(one,two,three) and we got a json like {"three": 1234, "one": 23423, "two": 234} it would generate command(23423,234,1234) with the correctly ordered argument tuple.
Next, we have the methods and subroutines needed to execute named commands from the CRPCTable.
UniValue CRPCTable::execute(const JSONRPCRequest &request) const – Executes request based on the dispatch table we’ve built.
static bool ExecuteCommand(const CRPCCommand& command, const JSONRPCRequest& request, UniValue& result, bool last_handler) – Executes a CRPCCommand.
std::vector<std::string> CRPCTable::listCommands() const – Lists available commands from dispatch table.
Lastly, we have the timer functions discussed earlier in the rpc/server.h section.
void RPCSetTimerInterfaceIfUnset(RPCTimerInterface *iface)
{
if (!timerInterface)
timerInterface = iface;
}
void RPCSetTimerInterface(RPCTimerInterface *iface)
{
timerInterface = iface;
}
void RPCUnsetTimerInterface(RPCTimerInterface *iface)
{
if (timerInterface == iface)
timerInterface = nullptr;
}
void RPCRunLater(const std::string& name, std::function<void()> func, int64_t nSeconds)
{
if (!timerInterface)
throw JSONRPCError(RPC_INTERNAL_ERROR, "No timer handler registered for RPC");
deadlineTimers.erase(name);
LogPrint(BCLog::RPC, "queue run of timer %s in %i seconds (using %s)n", name, nSeconds, timerInterface->Name());
deadlineTimers.emplace(name, std::unique_ptr<RPCTimerBase>(timerInterface->NewTimer(func, nSeconds*1000)));
}
int RPCSerializationFlags()
{
int flag = 0;
if (gArgs.GetArg("-rpcserialversion", DEFAULT_RPC_SERIALIZE_VERSION) == 0)
flag |= SERIALIZE_TRANSACTION_NO_WITNESS;
return flag;
}
CRPCTable tableRPC;These are pretty much just getters and setters. RPCRunLaterdoes a bit more work because it emplace‘s a command on deadlineTimersto be executed nSecondslater.
On the last statement of rpc/server.cpp a global instance of CRPCTable, tableRPC, is declared which is used by NodeImpl::executeRpc, NodeImpl::listRpcCommands, RpcHandlerImpl constructor, RpcHandlerImpl::disconnect, HTTPReq_JSONRPC in httprpc.cpp and RegisterAllCoreRPCCommands(tableRPC) within AppInitMain on init.cpp.
As you can see that very last line in rpc/server.cpp has enormous reach within the Bitcoin Core source tree. Several major components call into the dispatch table defined in that inconspicuous last line!
A few things to observe in this file is that not all conventions are followed in the Bitcoin Core source code. tableRPC should be prefixed g_ while some of the static variables should not, as they’re not truly global but only global within this compilation unit.
Return to the commented Bitcoin source code introduction.