v31.1-roots.1 · Commit 8: [PATCH 08/17] chore(roots): integrate v31 identity and maintenance tooling
ci/README.md
Before: ci/README.md · After: ci/README.md
File metadata
index 0b6a34460ca1640f927ff527c68d99824e14fd03..bf73cd69b8ae71792c9fb45665148ecc27fbfd31 100644
Download original patch · Release notes
Diff rows 1–176 of 176. Line numbers refer to the original before/after files.
+ Added− RemovedUnmarked lines are context.
Scroll the code area horizontally to read long lines. The two line-number columns refer to before and after.
ci/README.md
CHANGED
|
@@ -20,30 +20,11 @@ requires `bash`, `docker`, and `python3` to be installed. To run on different ar
|
|
|
20
20
|
sudo apt install bash docker.io python3 qemu-user-static
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
-
For some sanitizer builds, the kernel's address-space layout randomization
|
|
24
|
-
(ASLR) entropy can cause sanitizer shadow memory mappings to fail. When running
|
|
25
|
-
the CI locally you may need to reduce that entropy by running:
|
|
23
|
+
It is recommended to run the ci system in a clean env. To run the test stage
|
|
24
|
+
with a specific configuration,
|
|
26
25
|
|
|
27
26
|
```
|
|
28
|
-
sudo sysctl -w vm.mmap_rnd_bits=28
|
|
29
|
-
```
|
|
30
|
-
|
|
31
|
-
To run a test that requires emulating a CPU architecture different from the
|
|
32
|
-
host, we may rely on the container environment recognizing foreign executables
|
|
33
|
-
and automatically running them using `qemu`. The following sets us up to do so
|
|
34
|
-
(also works for `podman`):
|
|
35
|
-
|
|
36
|
-
```
|
|
37
|
-
docker run --rm --privileged docker.io/multiarch/qemu-user-static --reset -p yes
|
|
38
|
-
```
|
|
39
|
-
|
|
40
|
-
It is recommended to run the CI system in a clean environment. The `env -i`
|
|
41
|
-
command below ensures that *only* specified environment variables are propagated
|
|
42
|
-
into the local CI.
|
|
43
|
-
To run the test stage with a specific configuration:
|
|
44
|
-
|
|
45
|
-
```
|
|
46
|
-
env -i HOME="$HOME" PATH="$PATH" USER="$USER" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh
|
|
27
|
+
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
|
|
47
28
|
```
|
|
48
29
|
|
|
49
30
|
## Configurations
|
|
@@ -62,40 +43,112 @@ It is also possible to force a specific configuration without modifying the
|
|
|
62
43
|
file. For example,
|
|
63
44
|
|
|
64
45
|
```
|
|
65
|
-
env -i HOME="$HOME" PATH="$PATH" USER="$USER" MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh
|
|
46
|
+
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
|
|
66
47
|
```
|
|
67
48
|
|
|
68
49
|
The files starting with `0n` (`n` greater than 0) are the scripts that are run
|
|
69
50
|
in order.
|
|
70
51
|
|
|
71
|
-
## Cache
|
|
72
|
-
|
|
73
|
-
In order to avoid rebuilding all dependencies for each build, the binaries are
|
|
74
|
-
cached and reused when possible. Changes in the dependency-generator will
|
|
75
|
-
trigger cache-invalidation and rebuilds as necessary.
|
|
76
|
-
|
|
77
|
-
## Configuring a repository for CI
|
|
78
|
-
|
|
79
|
-
### Primary repository
|
|
80
|
-
|
|
81
|
-
To configure the primary repository, follow these steps:
|
|
82
|
-
|
|
83
|
-
1. Register with [WarpBuild](https://www.warpbuild.com/) and purchase runners.
|
|
84
|
-
2. Install the WarpBuild GitHub app against the GitHub organization.
|
|
85
|
-
3. Enable organisation-level runners to be used in public repositories:
|
|
86
|
-
1. `Org settings -> Actions -> Runner Groups -> Default -> Allow public repos`
|
|
87
|
-
4. Permit the following actions to run:
|
|
88
|
-
1. actions/cache/restore@\*
|
|
89
|
-
1. actions/cache/save@\*
|
|
90
|
-
1. actions/github-script@\*
|
|
91
|
-
1. docker/setup-buildx-action@\*
|
|
92
|
-
1. warpbuilds/cache/restore@\*
|
|
93
|
-
1. warpbuilds/cache/save@\*
|
|
94
|
-
|
|
95
|
-
### Forked repositories
|
|
96
|
-
|
|
97
|
-
When used in a fork the CI will run on GitHub's free hosted runners by default.
|
|
98
|
-
In this case, GitHub's cache size limitations may cause caches to be frequently evicted and missed, but the workflows will run (slowly).
|
|
52
|
+
## GitHub Actions policy
|
|
53
|
+
|
|
54
|
+
Bitcoin Roots uses a smaller, risk-selected pull-request matrix rather than
|
|
55
|
+
running every Bitcoin Core CI configuration on every change:
|
|
56
|
+
|
|
57
|
+
| Workflow | Events | Purpose |
|
|
58
|
+
| --- | --- | --- |
|
|
59
|
+
| `.github/workflows/ci.yml` | pull request | Required lint plus path-selected build and test coverage. |
|
|
60
|
+
| `.github/workflows/nightly.yml` | schedule, manual dispatch, reusable call | Expensive sanitizer, fuzz, compatibility, and platform assurance. |
|
|
61
|
+
| `.github/workflows/release.yml` | Roots release tags, manual dispatch | Validate a canonical release tag and produce the five release packages and portable patch. |
|
|
62
|
+
|
|
63
|
+
These repository workflows are kept in every canonical Roots branch and release
|
|
64
|
+
tag so a PR tests the code it proposes. They are hosting automation, however,
|
|
65
|
+
so `ci/release/create-patch-series.sh` excludes `.github/**` from the
|
|
66
|
+
downloadable patch applied on top of Bitcoin Core. Source tests and CI support
|
|
67
|
+
under `test/**`, `src/**/test/**`, and `ci/**` remain in that patch.
|
|
68
|
+
|
|
69
|
+
### Fast PR and development coverage
|
|
70
|
+
|
|
71
|
+
`ci/change-classifier.py` retains additive categories and labels. Complete file
|
|
72
|
+
lists select affected groups; malformed, truncated, empty and unknown input
|
|
73
|
+
selects broad **fast** coverage. Lint/classifier checks always run. The stable
|
|
74
|
+
`required result` check rejects failure, cancellation or skipping of a selected
|
|
75
|
+
job. Verified pure promotions may reuse exact successful canonical CI; explicit
|
|
76
|
+
`ci:*` labels still request fresh coverage.
|
|
77
|
+
|
|
78
|
+
| Change | Automatic fast coverage beyond lint |
|
|
79
|
+
| --- | --- |
|
|
80
|
+
| Documentation only | No compiled job. |
|
|
81
|
+
| Branding or GUI | Actual Qt/resource build and GUI unit tests. |
|
|
82
|
+
| Wallet, general C/C++, critical/build/unknown | Headless ASan/UBSan unit tests, affected Roots functional regressions, bounded libFuzzer smoke. |
|
|
83
|
+
|
|
84
|
+
Fast functional selection and fuzz budgets are implemented by
|
|
85
|
+
`ci/test/roots-regressions.py`, invoked through the existing
|
|
86
|
+
`ci/test_run_all.sh` / `ci/test/03_test_script.sh` entry points. Known wallet,
|
|
87
|
+
network, runtime, policy and mining paths select their acceptance groups;
|
|
88
|
+
mixed paths take the union. Unknown input selects all 16 Roots functional
|
|
89
|
+
regressions and 9 fuzz targets. Fast functional execution has a 900-second
|
|
90
|
+
process-group timeout. Each selected libFuzzer target has a 5-second campaign,
|
|
91
|
+
5-second per-input limit and 15-second outer timeout; input counts are reported
|
|
92
|
+
results, never a campaign quota. Fixed seed 1 makes smoke commands reproducible.
|
|
93
|
+
Fast fuzz does not download or replay the full external corpus.
|
|
94
|
+
|
|
95
|
+
Developer commands using an already configured/built profile:
|
|
96
|
+
|
|
97
|
+
```sh
|
|
98
|
+
python3 ci/test/roots-regressions.py plan --cadence fast \
|
|
99
|
+
--files-json '["src/wallet/spend.cpp"]'
|
|
100
|
+
python3 ci/test/roots-regressions.py functional --cadence fast \
|
|
101
|
+
--files-json '["src/wallet/spend.cpp"]' --build-dir build \
|
|
102
|
+
--artifacts /tmp/roots-fast-functional -- -j2 --quiet
|
|
103
|
+
python3 ci/test/roots-regressions.py fuzz --cadence fast \
|
|
104
|
+
--files-json '["src/policy/policy.cpp"]' --build-dir build_fuzz \
|
|
105
|
+
--artifacts /tmp/roots-fast-fuzz
|
|
106
|
+
python3 -m unittest discover -s ci/test
|
|
107
|
+
```
|
|
99
108
|
|
|
100
|
-
It is also possible to use your own WarpBuild Runners in your own fork with an appropriate patch to the `REPO_USE_WARP_RUNNERS` variable in ../.github/workflows/ci.yml
|
|
101
|
-
NB that WarpBuild Runners only work at an organisation level, therefore in order to use your own WarpBuild Runners, *the fork must be within your own organisation*.
|
|
109
|
+
Fuzz execution requires the native libFuzzer build (`--preset libfuzzer` or
|
|
110
|
+
`00_setup_env_native_fuzz.sh`), not a plain standalone corpus-replay binary.
|
|
111
|
+
Selection/result JSON and per-command logs are temporary failure artifacts,
|
|
112
|
+
not a second source of configuration or a release-state ledger. Timed-out owned
|
|
113
|
+
process groups are terminated; unowned node processes are untouched.
|
|
114
|
+
|
|
115
|
+
### Nightly/manual assurance and caches
|
|
116
|
+
|
|
117
|
+
`nightly.yml` keeps cron `23 2 * * *` (UTC), `workflow_dispatch` and explicit
|
|
118
|
+
label-triggered reusable calls. Ordinary PRs do not run the broad platform
|
|
119
|
+
matrix. Nightly/manual coverage includes full ASan/USDT/Qt, TSan, MSan,
|
|
120
|
+
clang-tidy/dependency boundaries, i686 no-IPC, ARM32, native kernel/chainstate,
|
|
121
|
+
previous-release extended compatibility, extended functional tests, Windows and
|
|
122
|
+
both macOS architecture smokes, and native Linux/macOS/Windows full fuzz corpora.
|
|
123
|
+
Selected empty matrices are skipped before strategy expansion. The schedule
|
|
124
|
+
skips only after the same SHA's preceding completed scheduled run succeeded;
|
|
125
|
+
failed or unavailable/malformed history retries conservatively.
|
|
126
|
+
|
|
127
|
+
| Execution | Explicit budget |
|
|
128
|
+
| --- | --- |
|
|
129
|
+
| PR classifier/result and lint | 10 / 20 minutes. |
|
|
130
|
+
| PR native ASan and fuzz builds plus bounded regressions | 60 minutes each. |
|
|
131
|
+
| PR GUI build/unit checks | 120 minutes. |
|
|
132
|
+
| Nightly ASan | 180 minutes; full applicable native suite. |
|
|
133
|
+
| Nightly TSan/MSan/i686/Linux full corpus | 240 minutes each. |
|
|
134
|
+
| Previous-release extended compatibility | 300 minutes. |
|
|
135
|
+
| Extended functional/tidy/native platform fuzz | 360 minutes. |
|
|
136
|
+
| Nightly selected Roots fuzz campaigns | 60 seconds/target, 75-second outer timeout, after full Linux corpus replay. |
|
|
137
|
+
|
|
138
|
+
Direct `--cadence nightly` functional execution has a 7200-second outer budget;
|
|
139
|
+
full/extended hosted suites use their existing native runners and job budgets.
|
|
140
|
+
Labels `ci:sanitizers`, `ci:fuzz`, `ci:platforms` and `ci:full` explicitly request
|
|
141
|
+
reusable heavy coverage; `ci:compat` requests fresh fast compatibility
|
|
142
|
+
regressions. Manual suite selection can request previous-release/extended
|
|
143
|
+
coverage independently. Failure artifacts retain native test logs, CTest
|
|
144
|
+
output and fuzz crashes/results. Job timeouts are bounds, not measured cold
|
|
145
|
+
build predictions. Local selector/fixture validation does not establish a
|
|
146
|
+
hosted job pass or Windows/macOS/32-bit execution.
|
|
147
|
+
|
|
148
|
+
Compiler/dependency caches remain read-only to PR code; trusted default-branch
|
|
149
|
+
nightly/manual compiler cache saves retain the source behavior. Native v31
|
|
150
|
+
cache-action paths, compiler profiles, no-IPC i686 profile and VS2026 static
|
|
151
|
+
preset replace obsolete v30 names. Windows GUI workflows request the optional
|
|
152
|
+
vcpkg `miniupnpc` feature when enabling `WITH_MINIUPNPC`; fuzz builds explicitly
|
|
153
|
+
disable ZMQ to match their wallet-only dependency manifest. Release signing
|
|
154
|
+
secrets never enter these test containers.
|