Bitcoin Core integration/staging tree
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1510 lines
59 KiB

Developer Notes
<!-- markdown-toc start -->
**Table of Contents**
- [Developer Notes](#developer-notes)
- [Coding Style (General)](#coding-style-general)
- [Coding Style (C++)](#coding-style-c)
- [Coding Style (Python)](#coding-style-python)
- [Coding Style (Doxygen-compatible comments)](#coding-style-doxygen-compatible-comments)
- [Generating Documentation](#generating-documentation)
- [Development tips and tricks](#development-tips-and-tricks)
- [Compiling for debugging](#compiling-for-debugging)
- [Show sources in debugging](#show-sources-in-debugging)
- [Compiling for gprof profiling](#compiling-for-gprof-profiling)
- [`debug.log`](#debuglog)
- [Signet, testnet, and regtest modes](#signet-testnet-and-regtest-modes)
- [DEBUG_LOCKORDER](#debug_lockorder)
- [DEBUG_LOCKCONTENTION](#debug_lockcontention)
- [Valgrind suppressions file](#valgrind-suppressions-file)
- [Compiling for test coverage](#compiling-for-test-coverage)
- [Performance profiling with perf](#performance-profiling-with-perf)
- [Sanitizers](#sanitizers)
- [Locking/mutex usage notes](#lockingmutex-usage-notes)
- [Threads](#threads)
- [Ignoring IDE/editor files](#ignoring-ideeditor-files)
- [Development guidelines](#development-guidelines)
- [General Bitcoin Core](#general-bitcoin-core)
- [Wallet](#wallet)
- [General C++](#general-c)
- [C++ data structures](#c-data-structures)
- [Strings and formatting](#strings-and-formatting)
- [Shadowing](#shadowing)
- [Lifetimebound](#lifetimebound)
- [Threads and synchronization](#threads-and-synchronization)
- [Scripts](#scripts)
- [Shebang](#shebang)
- [Source code organization](#source-code-organization)
- [GUI](#gui)
- [Subtrees](#subtrees)
- [Upgrading LevelDB](#upgrading-leveldb)
- [File Descriptor Counts](#file-descriptor-counts)
- [Consensus Compatibility](#consensus-compatibility)
- [Scripted diffs](#scripted-diffs)
- [Suggestions and examples](#suggestions-and-examples)
- [Release notes](#release-notes)
- [RPC interface guidelines](#rpc-interface-guidelines)
- [Internal interface guidelines](#internal-interface-guidelines)
<!-- markdown-toc end -->
Coding Style (General)
Various coding styles have been used during the history of the codebase,
and the result is not very consistent. However, we're now trying to converge to
a single style, which is specified below. When writing patches, favor the new
style over attempting to mimic the surrounding style, except for move-only
Do not submit patches solely to modify the style of existing code.
Coding Style (C++)
- **Indentation and whitespace rules** as specified in
[src/.clang-format](/src/.clang-format). You can use the provided
[clang-format-diff script](/contrib/devtools/
tool to clean up patches automatically before submission.
- Braces on new lines for classes, functions, methods.
- Braces on the same line for everything else.
- 4 space indentation (no tabs) for every block except namespaces.
- No indentation for `public`/`protected`/`private` or for `namespace`.
- No extra spaces inside parenthesis; don't do `( this )`.
- No space after function names; one space after `if`, `for` and `while`.
- If an `if` only has a single-statement `then`-clause, it can appear
on the same line as the `if`, without braces. In every other case,
braces are required, and the `then` and `else` clauses must appear
correctly indented on a new line.
- There's no hard limit on line width, but prefer to keep lines to <100
characters if doing so does not decrease readability. Break up long
function declarations over multiple lines using the Clang Format
style option.
- **Symbol naming conventions**. These are preferred in new code, but are not
required when doing so would need changes to significant pieces of existing
- Variable (including function arguments) and namespace names are all lowercase and may use `_` to
separate words (snake_case).
- Class member variables have a `m_` prefix.
- Global variables have a `g_` prefix.
- Constant names are all uppercase, and use `_` to separate words.
- Enumerator constants may be `snake_case`, `PascalCase` or `ALL_CAPS`.
This is a more tolerant policy than the [C++ Core
which recommend using `snake_case`. Please use what seems appropriate.
- Class names, function names, and method names are UpperCamelCase
(PascalCase). Do not prefix class names with `C`. See [Internal interface
naming style](#internal-interface-naming-style) for an exception to this
- Test suite naming convention: The Boost test suite in file
`src/test/foo_tests.cpp` should be named `foo_tests`. Test suite names
must be unique.
- **Miscellaneous**
- `++i` is preferred over `i++`.
- `nullptr` is preferred over `NULL` or `(void*)0`.
- `static_assert` is preferred over `assert` where possible. Generally; compile-time checking is preferred over run-time checking.
For function calls a namespace should be specified explicitly, unless such functions have been declared within it.
Otherwise, [argument-dependent lookup](, also known as ADL, could be
triggered that makes code harder to maintain and reason about:
#include <filesystem>
namespace fs {
class path : public std::filesystem::path
// The intention is to disallow this function.
bool exists(const fs::path& p) = delete;
} // namespace fs
int main()
//fs::path p; // error
std::filesystem::path p; // compiled
exists(p); // ADL being used for unqualified name lookup
Block style example:
int g_count = 0;
namespace foo {
class Class
std::string m_name;
bool Function(const std::string& s, int n)
// Comment summarising what this section of code does
for (int i = 0; i < n; ++i) {
int total_sum = 0;
// When something fails, return early
if (!Something()) return false;
if (SomethingElse(i)) {
total_sum += ComputeSomething(g_count);
} else {
DoSomething(m_name, total_sum);
// Success return is usually at the end
return true;
} // namespace foo
Coding Style (C++ functions and methods)
- When ordering function parameters, place input parameters first, then any
in-out parameters, followed by any output parameters.
- *Rationale*: API consistency.
- Prefer returning values directly to using in-out or output parameters. Use
`std::optional` where helpful for returning values.
- *Rationale*: Less error-prone (no need for assumptions about what the output
is initialized to on failure), easier to read, and often the same or better
- Generally, use `std::optional` to represent optional by-value inputs (and
instead of a magic default value, if there is no real default). Non-optional
input parameters should usually be values or const references, while
non-optional in-out and output parameters should usually be references, as
they cannot be null.
Coding Style (C++ named arguments)
When passing named arguments, use a format that clang-tidy understands. The
argument names can otherwise not be verified by clang-tidy.
For example:
void function(Addrman& addrman, bool clear);
int main()
function(g_addrman, /*clear=*/false);
### Running clang-tidy
To run clang-tidy on Ubuntu/Debian, install the dependencies:
apt install clang-tidy bear clang
Then, pass clang as compiler to configure, and use bear to produce the `compile_commands.json`:
./ && ./configure CC=clang CXX=clang++
make clean && bear make -j $(nproc) # For bear 2.x
make clean && bear -- make -j $(nproc) # For bear 3.x
To run clang-tidy on all source files:
( cd ./src/ && run-clang-tidy -j $(nproc) )
To run clang-tidy on the changed source lines:
git diff | ( cd ./src/ && clang-tidy-diff -p2 -j $(nproc) )
Coding Style (Python)
Refer to [/test/functional/](/test/functional/
Coding Style (Doxygen-compatible comments)
Bitcoin Core uses [Doxygen]( to generate its official documentation.
Use Doxygen-compatible comment blocks for functions, methods, and fields.
For example, to describe a function use:
* ... Description ...
* @param[in] arg1 input description...
* @param[in] arg2 input description...
* @param[out] arg3 output description...
* @return Return cases...
* @throws Error type and cases...
* @pre Pre-condition for function...
* @post Post-condition for function...
bool function(int arg1, const char *arg2, std::string& arg3)
A complete list of `@xxx` commands can be found at
As Doxygen recognizes the comments by the delimiters (`/**` and `*/` in this case), you don't
*need* to provide any commands for a comment to be valid; just a description text is fine.
To describe a class, use the same construct above the class definition:
* Alerts are for notifying old versions if they become too obsolete and
* need to upgrade. The message is displayed in the status bar.
* @see GetWarnings()
class CAlert
To describe a member or variable use:
//! Description before the member
int var;
int var; //!< Description after the member
Also OK:
/// ... Description ...
bool function2(int arg1, const char *arg2)
Not picked up by Doxygen:
// ... Description ...
Also not picked up by Doxygen:
* ... Description ...
A full list of comment syntaxes picked up by Doxygen can be found at,
but the above styles are favored.
- Avoiding duplicating type and input/output information in function
- Use backticks (&#96;&#96;) to refer to `argument` names in function and
parameter descriptions.
- Backticks aren't required when referring to functions Doxygen already knows
about; it will build hyperlinks for these automatically. See for complete info.
- Avoid linking to external documentation; links can break.
- Javadoc and all valid Doxygen comments are stripped from Doxygen source code
previews (`STRIP_CODE_COMMENTS = YES` in [](doc/ If
you want a comment to be preserved, it must instead use `//` or `/* */`.
### Generating Documentation
The documentation can be generated with `make docs` and cleaned up with `make
clean-docs`. The resulting files are located in `doc/doxygen/html`; open
`index.html` in that directory to view the homepage.
Before running `make docs`, you'll need to install these dependencies:
Linux: `sudo apt install doxygen graphviz`
MacOS: `brew install doxygen graphviz`
Development tips and tricks
### Compiling for debugging
Run configure with `--enable-debug` to add additional compiler flags that
produce better debugging builds.
### Show sources in debugging
If you have ccache enabled, absolute paths are stripped from debug information
with the -fdebug-prefix-map and -fmacro-prefix-map options (if supported by the
compiler). This might break source file detection in case you move binaries
after compilation, debug from the directory other than the project root or use
an IDE that only supports absolute paths for debugging.
There are a few possible fixes:
1. Configure source file mapping.
For `gdb` create or append to `.gdbinit` file:
set substitute-path ./src /path/to/project/root/src
For `lldb` create or append to `.lldbinit` file:
settings set target.source-map ./src /path/to/project/root/src
2. Add a symlink to the `./src` directory:
ln -s /path/to/project/root/src src
3. Use `debugedit` to modify debug information in the binary.
### Compiling for gprof profiling
Run configure with the `--enable-gprof` option, then make.
### `debug.log`
If the code is behaving strangely, take a look in the `debug.log` file in the data directory;
error and debugging messages are written there.
Debug logging can be enabled on startup with the `-debug` and `-loglevel`
configuration options and toggled while bitcoind is running with the `logging`
RPC. For instance, launching bitcoind with `-debug` or `-debug=1` will turn on
all log categories and `-loglevel=trace` will turn on all log severity levels.
The Qt code routes `qDebug()` output to `debug.log` under category "qt": run with `-debug=qt`
to see it.
### Signet, testnet, and regtest modes
If you are testing multi-machine code that needs to operate across the internet,
you can run with either the `-signet` or the `-testnet` config option to test
with "play bitcoins" on a test network.
If you are testing something that can run on one machine, run with the
`-regtest` option. In regression test mode, blocks can be created on demand;
see [test/functional/](/test/functional) for tests that run in `-regtest` mode.
Bitcoin Core is a multi-threaded application, and deadlocks or other
multi-threading bugs can be very difficult to track down. The `--enable-debug`
configure option adds `-DDEBUG_LOCKORDER` to the compiler flags. This inserts
run-time checks to keep track of which locks are held and adds warnings to the
`debug.log` file if inconsistencies are detected.
Defining `DEBUG_LOCKCONTENTION` adds a "lock" logging category to the logging
RPC that, when enabled, logs the location and duration of each lock contention
to the `debug.log` file.
The `--enable-debug` configure option adds `-DDEBUG_LOCKCONTENTION` to the
compiler flags. You may also enable it manually for a non-debug build by running
configure with `-DDEBUG_LOCKCONTENTION` added to your CPPFLAGS,
i.e. `CPPFLAGS="-DDEBUG_LOCKCONTENTION"`, then build and run bitcoind.
You can then use the `-debug=lock` configuration option at bitcoind startup or
`bitcoin-cli logging '["lock"]'` at runtime to turn on lock contention logging.
It can be toggled off again with `bitcoin-cli logging [] '["lock"]'`.
### Assertions and Checks
The util file `src/util/check.h` offers helpers to protect against coding and
internal logic bugs. They must never be used to validate user, network or any
other input.
* `assert` or `Assert` should be used to document assumptions when any
violation would mean that it is not safe to continue program execution. The
code is always compiled with assertions enabled.
- For example, a nullptr dereference or any other logic bug in validation
code means the program code is faulty and must terminate immediately.
* `CHECK_NONFATAL` should be used for recoverable internal logic bugs. On
failure, it will throw an exception, which can be caught to recover from the
- For example, a nullptr dereference or any other logic bug in RPC code
means that the RPC code is faulty and cannot be executed. However, the
logic bug can be shown to the user and the program can continue to run.
* `Assume` should be used to document assumptions when program execution can
safely continue even if the assumption is violated. In debug builds it
behaves like `Assert`/`assert` to notify developers and testers about
nonfatal errors. In production it doesn't warn or log anything, though the
expression is always evaluated.
- For example it can be assumed that a variable is only initialized once,
but a failed assumption does not result in a fatal bug. A failed
assumption may or may not result in a slightly degraded user experience,
but it is safe to continue program execution.
### Valgrind suppressions file
Valgrind is a programming tool for memory debugging, memory leak detection, and
profiling. The repo contains a Valgrind suppressions file
which includes known Valgrind warnings in our dependencies that cannot be fixed
in-tree. Example use:
$ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
$ valgrind --suppressions=contrib/valgrind.supp --leak-check=full \
--show-leak-kinds=all src/test/test_bitcoin --log_level=test_suite
$ valgrind -v --leak-check=full src/bitcoind -printtoconsole
$ ./test/functional/ --valgrind
### Compiling for test coverage
LCOV can be used to generate a test coverage report based upon `make check`
execution. LCOV must be installed on your system (e.g. the `lcov` package
on Debian/Ubuntu).
To enable LCOV report generation during test runs:
./configure --enable-lcov
make cov
# A coverage report will now be accessible at `./test_bitcoin.coverage/index.html`.
### Performance profiling with perf
Profiling is a good way to get a precise idea of where time is being spent in
code. One tool for doing profiling on Linux platforms is called
[`perf`](, and has been integrated into
the functional test framework. Perf can observe a running process and sample
(at some frequency) where its execution is.
Perf installation is contingent on which kernel version you're running; see
[this thread](
for specific instructions.
Certain kernel parameters may need to be set for perf to be able to inspect the
running process's stack.
$ sudo sysctl -w kernel.perf_event_paranoid=-1
$ sudo sysctl -w kernel.kptr_restrict=0
Make sure you [understand the security
trade-offs]( of setting these kernel
To profile a running bitcoind process for 60 seconds, you could use an
invocation of `perf record` like this:
$ perf record \
-g --call-graph dwarf --per-thread -F 140 \
-p `pgrep bitcoind` -- sleep 60
You could then analyze the results by running:
perf report --stdio | c++filt | less
or using a graphical tool like [Hotspot](
See the functional test documentation for how to invoke perf within tests.
### Sanitizers
Bitcoin Core can be compiled with various "sanitizers" enabled, which add
instrumentation for issues regarding things like memory safety, thread race
conditions, or undefined behavior. This is controlled with the
`--with-sanitizers` configure flag, which should be a comma separated list of
sanitizers to enable. The sanitizer list should correspond to supported
`-fsanitize=` options in your compiler. These sanitizers have runtime overhead,
so they are most useful when testing changes or producing debugging builds.
Some examples:
# Enable both the address sanitizer and the undefined behavior sanitizer
./configure --with-sanitizers=address,undefined
# Enable the thread sanitizer
./configure --with-sanitizers=thread
If you are compiling with GCC you will typically need to install corresponding
"san" libraries to actually compile with these flags, e.g. libasan for the
address sanitizer, libtsan for the thread sanitizer, and libubsan for the
undefined sanitizer. If you are missing required libraries, the configure script
will fail with a linker error when testing the sanitizer flags.
The test suite should pass cleanly with the `thread` and `undefined` sanitizers,
but there are a number of known problems when using the `address` sanitizer. The
address sanitizer is known to fail in
[sha256_sse4::Transform](/src/crypto/sha256_sse4.cpp) which makes it unusable
unless you also use `--disable-asm` when running configure. We would like to fix
sanitizer issues, so please send pull requests if you can fix any errors found
by the address sanitizer (or any other sanitizer).
Not all sanitizer options can be enabled at the same time, e.g. trying to build
with `--with-sanitizers=address,thread` will fail in the configure script as
these sanitizers are mutually incompatible. Refer to your compiler manual to
learn more about these options and which sanitizers are supported by your
Additional resources:
* [AddressSanitizer](
* [LeakSanitizer](
* [MemorySanitizer](
* [ThreadSanitizer](
* [UndefinedBehaviorSanitizer](
* [GCC Instrumentation Options](
* [Google Sanitizers Wiki](
* [Issue #12691: Enable -fsanitize flags in Travis](
Locking/mutex usage notes
The code is multi-threaded and uses mutexes and the
`LOCK` and `TRY_LOCK` macros to protect data structures.
Deadlocks due to inconsistent lock ordering (thread 1 locks `cs_main` and then
`cs_wallet`, while thread 2 locks them in the opposite order: result, deadlock
as each waits for the other to release its lock) are a problem. Compile with
`-DDEBUG_LOCKORDER` (or use `--enable-debug`) to get lock order inconsistencies
reported in the `debug.log` file.
Re-architecting the core code so there are better-defined interfaces
between the various components is a goal, with any necessary locking
done by the components (e.g. see the self-contained `FillableSigningProvider` class
and its `cs_KeyStore` lock for example).
- [Main thread (`bitcoind`)](
: Started from `main()` in `bitcoind.cpp`. Responsible for starting up and
shutting down the application.
- [ThreadImport (`b-loadblk`)](
: Loads blocks from `blk*.dat` files or `-loadblock=<file>` on startup.
- [CCheckQueue::Loop (`b-scriptch.x`)](
: Parallel script validation threads for transactions in blocks.
- [ThreadHTTP (`b-http`)](
: Libevent thread to listen for RPC and REST connections.
- [HTTP worker threads(`b-httpworker.x`)](
: Threads to service RPC and REST requests.
- [Indexer threads (`b-txindex`, etc)](
: One thread per indexer.
- [SchedulerThread (`b-scheduler`)](
: Does asynchronous background tasks like dumping wallet contents, dumping
addrman and running asynchronous validationinterface callbacks.
- [TorControlThread (`b-torcontrol`)](
: Libevent thread for tor connections.
- Net threads:
- [ThreadMessageHandler (`b-msghand`)](
: Application level message handling (sending and receiving). Almost
all net_processing and validation logic runs on this thread.
- [ThreadDNSAddressSeed (`b-dnsseed`)](
: Loads addresses of peers from the DNS.
- ThreadMapPort (`b-mapport`)
: Universal plug-and-play startup/shutdown.
- [ThreadSocketHandler (`b-net`)](
: Sends/Receives data from peers on port 8333.
- [ThreadOpenAddedConnections (`b-addcon`)](
: Opens network connections to added nodes.
- [ThreadOpenConnections (`b-opencon`)](
: Initiates new connections to peers.
- [ThreadI2PAcceptIncoming (`b-i2paccept`)](
: Listens for and accepts incoming I2P connections through the I2P SAM proxy.
Ignoring IDE/editor files
In closed-source environments in which everyone uses the same IDE, it is common
to add temporary files it produces to the project-wide `.gitignore` file.
However, in open source software such as Bitcoin Core, where everyone uses
their own editors/IDE/tools, it is less common. Only you know what files your
editor produces and this may change from version to version. The canonical way
to do this is thus to create your local gitignore. Add this to `~/.gitconfig`:
excludesfile = /home/.../.gitignore_global
(alternatively, type the command `git config --global core.excludesfile ~/.gitignore_global`
on a terminal)
Then put your favourite tool's temporary filenames in that file, e.g.
# NetBeans
Another option is to create a per-repository excludes file `.git/info/exclude`.
These are not committed but apply only to one repository.
If a set of tools is used by the build system or scripts the repository (for
example, lcov) it is perfectly acceptable to add its files to `.gitignore`
and commit them.
Development guidelines
A few non-style-related recommendations for developers, as well as points to
pay attention to for reviewers of Bitcoin Core code.
General Bitcoin Core
- New features should be exposed on RPC first, then can be made available in the GUI.
- *Rationale*: RPC allows for better automatic testing. The test suite for
the GUI is very limited.
- Make sure pull requests pass CI before merging.
- *Rationale*: Makes sure that they pass thorough testing, and that the tester will keep passing
on the master branch. Otherwise, all new pull requests will start failing the tests, resulting in
confusion and mayhem.
- *Explanation*: If the test suite is to be updated for a change, this has to
be done first.
- Make sure that no crashes happen with run-time option `-disablewallet`.
General C++
For general C++ guidelines, you may refer to the [C++ Core
Common misconceptions are clarified in those sections:
- Passing (non-)fundamental types in the [C++ Core
- Assertions should not have side-effects.
- *Rationale*: Even though the source code is set to refuse to compile
with assertions disabled, having side-effects in assertions is unexpected and
makes the code harder to understand.
- If you use the `.h`, you must link the `.cpp`.
- *Rationale*: Include files define the interface for the code in implementation files. Including one but
not linking the other is confusing. Please avoid that. Moving functions from
the `.h` to the `.cpp` should not result in build errors.
- Use the RAII (Resource Acquisition Is Initialization) paradigm where possible. For example, by using
`unique_ptr` for allocations in a function.
- *Rationale*: This avoids memory and resource leaks, and ensures exception safety.
C++ data structures
- Never use the `std::map []` syntax when reading from a map, but instead use `.find()`.