Contributing¶
What this repository holds in common with the others of the organization — the toolchain, the lint gate, the tool tables behind it, the workflow set and the branch rules — is stated once in the btclib-org repository standard, each rule with the alternative it was decided against. It binds this repository, so a change departing from it is a divergence, and one filed as an issue in that repository rather than here: a difference between two repositories belongs to neither of them.
This file is the same in every repository of the organization up to its last section. What is true of one tree only — the commands that build its environment, the gates it runs, which of its workflows decide a merge — is under that heading, and the comparison stops there.
How the organization decides, and who holds which role, is
GOVERNANCE.md; what it intends to do, and what it
deliberately does not, is ROADMAP.md. Both are the
organization’s, one copy each beside the standard.
The issue tracker¶
Where an issue is filed, and what an alignment finding has to name, is the standard’s What this repository is: an issue spanning repositories, or whose subject is the standard, goes to btclib-org/.github, and one about this tree alone stays here.
A finding noticed while doing something else goes where REVIEWING.md’s
What is filed, and what is not says, for an author as much as for a
reviewer: a pull request answering two questions cannot be accepted for
either.
An issue labelled good first issue is one set aside for a first
contribution: small and self-contained.
One search lists the open ones across the organization, so
a newcomer need not know in advance which repository to look in.
Documentation and comments¶
Section 9 of the standard is the prose style beyond Write plainly below, and it governs the prose this tree ships — comments, docstrings and markdown. It is not restated here: a second wording is the one that goes stale, which is that section’s own One fact in one place.
A commit message is prose this tree ships too, though section 9 does not
say so: the only merge method the rule accepts puts it on main
as the landing commit’s body, so what is written in one is read there
long after the branch is gone.
Working with an LLM¶
Development here is LLM-based. Most issues, code and reviews are written by agentic sessions that the maintainer directs and reads. Humans are welcome, as long as they are comfortable working that way.
A contribution is yours, whoever wrote it. Read every line before you open the pull request, understand it, and be ready to answer a review without the model. Text or code you have not read, or cannot explain, is closed without review.
For a pull request the agent opened, whoever approves it reads every
line and answers for it as their own, the agent being claude[bot],
run on an issue by section 10’s reusable-claude-implement.yml.
The maintainer, landing it through the bypass without an approval, reads
every line and answers for it instead, the landing being their own act;
requiring an approval even where the bypass is kept is the alternative
set aside. The agent signs off nothing, since a bot cannot certify the
Developer Certificate of Origin, so the approval, or that
landing, is the certification.
Write plainly, in the tree and on the forge: the point first, plain
words, one fact per sentence. On the forge, the title and the first
sentence carry the point; a body is at most two short paragraphs before
any code block or template section; a comment is about 250 words, and
never more than 300, with no list and no heading; a review’s summary
takes the shape REVIEWING.md’s The verdict gives it.
Pull requests¶
What main accepts, and what it refuses to everyone, is section 11 of
the standard. Run the gates locally before opening anything —
the last section of this file says which they are — because CI runs
exactly them, so a red run there is a local run that was not done.
Every commit of a pull request carries a Signed-off-by: trailer
naming its author, which certifies the Developer Certificate of
Origin. git commit -s adds it, and git rebase --signoff <base>
adds it to commits already made. The Sign-off check is required, so
only an administrator can merge a pull request with a commit the check
refuses: Landing it says why --admin waits for no required check.
The check’s failure prints the command that adds the trailer. The
standard’s Signatures says why a signature does not replace
it, and which commits the Sign-off job skips.
What a pull request’s title and description have to say about the issues it closes, and why a manual link in the Development panel is a trap neither of them shows, is the standard’s What a pull request says it is. Read it before opening one; it is the rule most often found broken after the fact.
Before it is opened, the branch’s own commit subjects and bodies are read against that same rule. The description does not exist yet to disagree with them, and the standard has the command that scans the branch’s own commit text for a verb in front of a reference.
The two spellings are named here as well as there, against section 9’s
One fact in one place, the paragraph above naming the section
and not the forms, which are the half a citation is got wrong in:
(closes #N) cites an issue the change closes, wherever the citation
sits — the title, the commit subject where Merge method makes
that the thing that lands — and (issue #N)
cites, in those same places, an issue the change advances and does not
close. One token holds one meaning whichever file it sits in, so the
pair is chosen by what is true of the change rather than by which file
is being written, and a tree’s own landed subjects are not what to copy
it from: nothing already landed is rewritten, so what a repository wrote
before the rule stays where it is.
REVIEWING.md is the standard a review is written against, and is this
file’s other half. Read before opening a pull request, it is what the
pull request will be answered against.
A pull request adds no entry to CHANGELOG.md or RELEASE_NOTES.md, a
release’s own pull request excepted (section 9, section 12).
One subject, opened as soon as it is written¶
A pull request answers one question. Issues that share a subject are one pull request, closing each of them; issues that do not are one pull request each, however small either of them is.
It is opened the moment it is written and verified — not held for the previous one to be reviewed or to land, and not batched with the next. A batch arrives as one reviewing job with several subjects, which is the shape that costs the most to read; a finished pull request held back is review that could have started and did not. A security advisory’s fix is the exception, held back as section 12’s A security advisory is worked in this order says.
Working this way stacks branches, which is fine and costs one rule: a child whose base was amended is moved with the old base named,
git rebase --onto <new-base> <old-base-sha> <child>
because a plain rebase replays the base’s old commit inside the child,
and the forge then shows the base’s old text as additions with nothing
red anywhere. Read the child’s diff afterwards rather than trusting the
rebase, and retarget each child onto main as its parent lands.
The review¶
A review is given promptly and on local evidence. It does not wait for CI, does not report a check as a finding, and does not discuss a run at all: whether CI is green is the author’s business, once, at landing time.
The exchange is anchored to a sha rather than to a branch, a branch being free to move under a review:
the author hands off by naming the sha pushed and the evidence run against it, then leaves that head alone;
the reviewer answers with findings — where, what is wrong, how they know it, and whether each is blocking;
the author accepts what is reasonable, declines the rest with a reason in the thread, pushes the answer without waiting for CI, and resolves each thread once answered;
the reviewer reopens a thread whose finding is still blocking, and re-reviews the delta rather than the branch.
What ends the loop is the ack of record, and the author does not
supply their own. A reading that says what it found and delivers no
verdict is a review too and ends nothing; the standard’s Review
has which is which, and REVIEWING.md has how each is written. A
disagreement that survives a second exchange goes to the maintainer
instead of into a third round.
The ack of record is not the approval Landing it asks for: it is a bot’s review of type COMMENT (the standard’s Review says whose).
Landing it¶
CI is read once, and this is where: on the head that lands. main does
not ask that head to be up to date with it, and section 11
says why. So a branch is rebased only where it conflicts with main. A
rebase that moved nothing but the base leaves the ack standing; one that
resolved a conflict does not, that resolution being a change no reviewer
has seen.
When main’s CI is red after a merge, its repair takes priority over
every other pull request, because until it lands main’s runs cannot
tell a new failure from the one already there. A pull request armed with
--auto (below) stays armed, and others keep landing meanwhile. Holding
every landing for a repair costs more than the break it guards against,
so the risk of landing on a red main is accepted
(btclib-org/.github#1777).
A pull request whose required check is red only because main was red
is closed and reopened once the repair lands (gh pr close <n>, then
gh pr reopen <n>). A re-run would test the old merge commit, and the
reopen runs the checks on the merge GitHub then holds. GitHub refreshes
that merge lazily, so the reopen’s runs can still test the old one. A
run’s log is read before the run is relied on: its line HEAD is now at <sha> Merge <head> into <base> names the merge it tested, and where
<base> lacks the repair, the pull request is closed and reopened
again. No push is made, so its approvals and its ack of record stand.
Closing disarms --auto, so it is armed again once the checks are
green. The reopen also runs the review bot again, and changes that
review requests are answered like any other review’s.
Then squash, the only method the rule accepts.
It lands once somebody other than its author has approved that
head, through gh pr merge pinned to it:
gh pr merge <n> --squash --match-head-commit <the head the checks ran on>
With --auto added, GitHub runs the same merge once the approval and
the checks are in.
Where the repository has a bypass, the maintainer may land without
that approval, through it: GOVERNANCE.md’s How a
decision is made says which repositories.
--admin turns off gh’s own refusal, and GitHub applies
the bypass to the merge. enforce_admins is off, so it waits for no
required check either:
gh pr merge <n> --squash --admin \
--match-head-commit <the head the checks ran on>
Without --admin, gh pr merge refuses a head with no approval, saying
it is not mergeable: the base branch policy prohibits the merge.
The pin is not optional, in either command. Reading the ack and
merging are two calls, and the head is free to move between them — the
push that would move it comes out of the same round the verdict does.
Unpinned, the command takes whatever sits at the head when it runs;
pinned, it refuses a head that has moved, and a round lost that way is
cheaper than a tree nobody has read reaching main.
The review above anchors the exchange to a sha and section 11
has an ack name one: the pin is that rule reaching the call that
performs the landing.
Verify what landed rather than trusting the answer, the signature the standard asks for being a valid one rather than a particular signer’s:
gh api repos/{owner}/{repo}/commits/main \
--jq '.commit.verification | {verified, reason}'
What it closed is read again here too, from the landed sha rather than from the pull request: the standard’s What a pull request says it is has the second read, and why the first alone does not reach a squash subject composed after it runs.
The forge deletes the head branch itself, per the setting section 11
names. What is still yours is bringing every checkout sitting on main
up to date,
that being where the next session starts from and a stale one being where
a branch gets built on a base that has moved. REPOSITORY.md carries the
settings and why they are what they are.
Worktrees¶
Run git fetch origin in the primary checkout, then check that
git status --porcelain is empty and git branch --show-current is
main. Run git pull --ff-only only when both checks pass, then verify
git rev-parse HEAD origin/main prints the same sha twice. Create a separate
worktree for each task with
git worktree add <path> origin/main -b <branch>. After committing,
remove it with git worktree remove <path>. If initialized submodules
make that fail,
check for uncommitted, untracked, and ignored files in the worktree and
its initialized submodules. Preserve anything needed before retrying with
git worktree remove --force <path>; it can delete those files. Keep the
primary checkout clean and on main; do not
switch branches or stash there. Hooks can fix files in place, and the local
main and refs/stash are shared by all worktrees.
This repository in particular¶
Everything above is the same file in every repository of the organization; everything below is this one’s, and the comparison stops at this heading.
Worktrees with submodules¶
Initialize submodules as The environment and the gates says. Initialized
submodules make plain git worktree remove fail, so removal takes the
--force path of Worktrees above, after its checks.
--reference to the primary checkout’s submodule modules is declined: it
saves the clone but leaves the worktree’s submodule without objects of its
own, so a git gc in the primary checkout, or moving it, can break the
worktree.
Which repository an issue belongs to¶
These are thin python bindings to libsecp256k1. An issue in the wrong repository is slow to route:
the cryptography is upstream, vendored here as a submodule and only ever read from: a flaw in the library itself is not ours to fix, and it goes to that library’s own tracker. The import line says which library answered — bitcoin-core/secp256k1 outside
btclib_secp256k1.zkp, and BlockstreamResearch/secp256k1-zkp under it..gitmodulesnames every vendored submodule and where it comes fromanything with state of its own — a wallet, a transaction, a signing session — belongs in btclib, which is what these bindings are for. The Design section of the README says where the line is
what belongs here is how the bindings drive what they wrap: the wrapping, the argument validation at the cffi boundary, the packaging, and the wheels
The bug form asks which artifact is installed — a static wheel, a dynamic one, or a build from the sdist — because they differ in how libsecp256k1 is linked and a bug is rarely in all of them. A vulnerability is never an issue: see the security policy.
The environment and the gates¶
uv is the only thing that has to be installed; it fetches interpreters and tools itself. A C toolchain is the second prerequisite, this package compiling the vendored library and an extension against it, and the Build section of the README says what each platform needs.
The vendored libraries are submodules, and a clone gets their directories without the files in them, so a checkout is two commands before it is anything:
$ git submodule init
Submodule 'secp256k1' (https://github.com/bitcoin-core/secp256k1.git) registered for path 'secp256k1'
Submodule 'secp256k1-zkp' (https://github.com/fametrano/secp256k1-zkp.git) registered for path 'secp256k1-zkp'
$ git submodule update
Cloning into '.../secp256k1'...
Cloning into '.../secp256k1-zkp'...
Each Cloning into line carries the absolute path of the directory git
writes to, elided above. A Submodule path ...: checked out ... line
follows per submodule, left out here because the commit it names moves
with the pin.
A git worktree starts with those directories empty however complete
the checkout it was made from, so it needs the same step; --init is
the registering and the cloning above in one command:
git submodule update --init
With no path after it, that reaches every submodule .gitmodules names.
Then the environment. .python-version pins the interpreter and uv
installs it if it is missing, so neither pyenv nor a hand-made virtualenv
is needed; the development dependencies are the PEP 735 groups declared
in pyproject.toml.
uv sync --locked
That also builds and installs the extension in editable mode, which is the minutes rather than seconds part of it.
The test and documentation gates install this package, and installing it
compiles libsecp256k1 out of secp256k1/ before any automodule
directive imports the result. The lint gate installs no project —
--only-group omits it — and needs the vendored clone all the same:
submodule-pin resolves the release README.md names in that clone’s
own refs, which is what lets the check run offline.
That gate fails outright where a submodule .gitmodules names is not
checked out at all: submodules-checked-out asks that on every
invocation, whatever the commit touches.
Every job that installs from uv.lock passes --locked. These install
from somewhere else on purpose:
deps-latest.ymlrunsuv lock --upgrade, to test the newest versions the declared ranges admit.The
dev-versionaction runsuv lockafter writing a rehearsal’s version suffix, and the lock keeps the versions it holds.test.yml’ssuite-sdistinstalls the sdist withpipand no lock, and itscheck-distinstalls a wheel withuv run --isolated --no-project --with: both resolve its dependencies from PyPI, as a user does.pypi-install.ymlinstalls the published package withpip.uv run --no-projectruns a script on a bare interpreter and installs nothing from the lock.deps-latest.yml’suv build,suite-sdist’spip installand the editable builduv syncmakes resolve[build-system] requiresfrom the index, within the rangespyproject.tomlstates. Every build that ships, andwheel-reproducibility’s rebuilds, take them frombuild-constraints.txt, which is exported from the lock.cibuildwheelinstalls itstest-requires, whichpyproject.tomlpins exactly, from the index.
The gates below decide every merge, together with lint / Dependency review, which has no command, lint / Sign-off, whose command
Running what CI runs gives, and sourcery / Sourcery, a pull request’s
check and not a local gate, as section 10 of the organization standard
says. wheel-reproducibility decides a merge that touches what its builds
read, its command being among the sentinels further down. Each command
below is close to the one its workflow runs.
The second is test.yml’s Run the suite against a static build step,
which also sets COVERAGE_FILE:
uv run --locked --only-group lint pre-commit run --all-files \
--show-diff-on-failure
uv run --locked --no-default-groups --group test pytest \
--cov-fail-under=0
uv run --locked --no-default-groups --group docs \
sphinx-build -n -W -b html docs/source docs/build/html
The documentation gate also runs the checks of the built pages that
Running what CI runs gives under docs / Build the documentation,
check_included_links.py among them.
The suite runs under pytest-xdist, -n auto --dist worksteal sitting in
addopts beside --cov. -n 0 runs it in the one process, and -n 2
is the pool to ask for on a shared machine. pyproject.toml’s addopts
comment says what every worker has to collect.
Coverage is measured in branch mode, --cov sitting in addopts so
nothing has to be typed after pytest to see it. The fail_under
ratchet in pyproject.toml gates at 100%, and the command above is not
asked to reach it, so it exits with the tests’ own verdict.
btclib_secp256k1.zkp.musig (#607 onward) is opt-in behind
BTCLIB_LIBSECP256K1_ZKP by decision (#603), and without the flag
nothing calls into it. The module is imported all the
same — tests/all_test.py’s census and tests/secret_test.py’s walk
both descend into the subpackage, and the documentation build imports
it for its own :members: — so what a run against an unflagged build
executes there is its module-level lines and no function body. Nothing
drives it: a test that does carries
pytest.importorskip("_btclib_secp256k1_zkp") and the zkp marker, so
a run against an unflagged build skips it. Nothing in it can be
called there either: every entry point reads ffi, lib or ctx
inside the call rather than at module scope, and that read is what
raises btclib_secp256k1.zkp’s ImportError naming the flag. Against
an unflagged build, which is what uv sync --locked installs, the command
reports short of 100%, and without --cov-fail-under=0 it exits 1 —
that shortfall is not the module alone: [tool.coverage.run] names
tests in source too, so tests/zkp_musig_test.py and
tests/zkp_musig_vectors_test.py are measured the same way, and each
stops at its own importorskip before a line below it runs, reported
missed rather than absent from the table. A flagged module and its test
files appearing short in that report is what a contributor should expect to
see, not a broken tree; which files those are grows with every module #603
still has queued behind it, so nothing here counts them. What decides is
the installed extension and not the command, which sets no flag:
importorskip names the compiled module, and a venv holds it exactly
when the extension in it was built with BTCLIB_LIBSECP256K1_ZKP. The
coverage sequence below ends on such a build, so this command run in that
venv executes the zkp tests, and a reading of 100% there says the
environment moved, not that the shortfall closed. uv sync --locked leaves
that build installed, and
uv sync --locked --reinstall-package btclib-secp256k1 --no-cache puts
an unflagged one back. The command carries neither of these itself, being the
one a contributor repeats, where each repeat would compile the extension
again.
test.yml
runs this same command against three builds: once per linkage, plus
once more against the flagged build restricted to the tests marked
zkp — -m zkp, a selection rather than the whole suite, the same
restriction this paragraph goes on to call out below — and gates the
union of the three at 100% after combine; “Measure coverage, gated at
100%” below is the sequence that reproduces that union, and its own
100.00% is CI’s real number, for whoever builds the extension to check
it locally. A run
that asks for less than the full suite even at that — a path leaving
part of tests out, -k, -m, --deselect, --ignore,
--ignore-glob or --lf — reports its coverage and is gated at
nothing either way, tests/conftest.py being where that is decided and
why.
The group flags are not decoration. uv run syncs the environment
itself, and without --no-default-groups --group test it installs the
whole dev set, wider than what the suite needs.
What they do not do is narrow an environment that is already wide.
uv run takes --exact to remove extraneous packages, so a restriction
without it installs what its groups ask for and leaves everything else
in place: after the uv sync --locked above, each gate runs against the
whole dev set rather than against its own group. A package a group does
not declare passes there, something else on the machine providing it,
and fails in the workflow, whose environment holds only what the group
asks for.
uv sync is exact, so a uv sync --locked --no-default-groups --group <g> before a gate gives it the environment its workflow has, and
uv sync --locked puts the other groups back. That is the line
.readthedocs.yaml and docs/README.rst carry for the documentation
build.
After touching pyproject.toml, uv lock — the uv-lock hook does it
too, and then the gate is a second run.
Check exit codes, not filtered output. pre-commit run ... | grep -v Passed hides a failure, and grep finding nothing exits 1, which is not
the gate’s answer to anything.
The lint gate is not installed as a git hook. pre-commit install
writes into the common git directory, which every worktree of this
repository shares:
git rev-parse --git-path hooks
answers with the primary checkout’s .git/hooks in every worktree, so
one session installing it installs it for every other. Run the gate by
hand before committing.
A wrapped commit subject is checked the same way, by hand. The
squash keeps a subject’s first physical line only, so a subject spread
across two physical lines with no blank line between them lands
truncated on main, its citation left behind in the body —
the standard names the failure and gives the read that does not
conceal it. A commit-msg hook would need installing into the same
shared .git/hooks this section already declines for the lint gate, so
there is none: this loop is the check, run over the branch before it is
opened, at the same moment Pull requests above already reads the
branch’s own commit text for its closing keyword:
for sha in $(git log --format=%H origin/main..); do
a=$(git show -s --format=%s "$sha")
b=$(git show -s --format=%B "$sha" | head -1)
[ "$a" = "$b" ] || echo "wrapped subject: $sha"
done
A clean branch prints nothing; a line named is a subject to rewrap before opening the pull request.
Two hooks are regenerated rather than fixed when they fail. The test data
is private keys, so detect-secrets would report all of it; the known
findings are recorded as reviewed rather than excluded, in two baselines
that differ only in whether the entropy detectors run. The one exception
is a line filter in .secrets.baseline for a rev: line holding a
40-hex sha, which a frozen hook revision would otherwise trip. Adding a hex
constant to a test module, or a vector to a file under tests/ matching
*.csv or *.json, fails the corresponding hook until its baseline is
regenerated.
The first command regenerates .secrets.baseline, over the tree with entropy
detectors on. --slim omits line_number and generated_at, so the
file stops changing when a flagged line moves; the scan is fresh rather
than --baseline, which writes the full form whatever it read —
--slim reaches format_for_output only on the branch that prints to
stdout — so the file exclusion and the rev: line filter are spelled out
rather than read back from the baseline’s own filters:
uvx --from detect-secrets detect-secrets scan --slim \
--exclude-files '^(\.secrets\..*baseline|tests/.*\.(csv|json))$' \
--exclude-lines '^\s*rev: "?[0-9a-f]{40}"?(\s|$)' \
> .secrets.baseline
The second command regenerates .secrets.vectors.baseline, over the
vendored vector data with entropy detectors off: these files are
64-character hex and nothing else, so a new vector would read as a new
secret. The paths are the hook’s files pattern spelled out:
uvx --from detect-secrets detect-secrets scan --slim \
--disable-plugin HexHighEntropyString \
--disable-plugin Base64HighEntropyString \
tests/*.csv tests/*.json \
> .secrets.vectors.baseline
Both are slim, and what that costs is one field of the diff: a new secret
still arrives as a new hashed_secret under its filename, and finding
where it is takes a grep instead of a line number. detect-secrets audit needs the full form and says so, which is a reason to regenerate
without --slim for an audit rather than to keep the churn in the
committed file.
The redirect costs a second thing, which is the one to know before
auditing: --baseline does not only save, it merges, and what it
carries forward is exactly the audit’s product — is_secret and
is_verified per finding, “verification of secrets, both automated and
manual” in its own words. A scan into a file has no old baseline to
merge, so those marks go back to their defaults, and slim output prints
no is_secret at all when it is unset: the loss shows up as a diff of
nothing. jq '[.results[][]] | length' .secrets.baseline (and the same
against .secrets.vectors.baseline) is how to check whether there is
anything to lose — every finding in either file being unverified and none
carrying is_secret means there isn’t, today — but an audit’s marks want
saving elsewhere or re-applying after the next regeneration. Read the
diff before committing it, which is the whole point of a baseline: what
appears there is what nobody has looked at yet.
Two things the gates enforce about the prose, which section 9 of the
standard states as a rule and does not say is checked here. pydoclint
requires an Args entry per parameter and a Returns section, in the
Google style napoleon renders, so a new argument nobody documented fails
the lint gate; Raises is not enforced, and pyproject.toml says why
beside the setting that turns it off. And anything written as a doctest,
in a docstring or in README.md, is run by tests/examples_test.py on
every interpreter and every kind of wheel — which constrains an example
to be deterministic: fixed keys, and a verification rather than a
signature wherever the value depends on randomness that is not pinned.
A docstring under btclib_secp256k1.zkp runs only where the flagged
extension is built: tests/examples_test.py marks that module’s case
zkp and guards it with an importorskip. In CI that is the Build the flagged secp256k1-zkp extension, and run its tests job below and the
flagged step of the coverage job.
To time these bindings against the other python wrappers of
libsecp256k1, clone
btclib-benchmarks and
run scripts/01-libsecp256k1.py there. The comparands are that
project’s dependencies rather than this one’s, which is the point: the
library downstream of these bindings is one of them, and depends on
these bindings itself.
To test against another supported interpreter, bypass the build cache: uv keys it on the sources, which do not tell it that the compiled extension belongs to one ABI version only.
uv run --python 3.11 --no-cache pytest
On a Windows arm64 machine, mind which interpreter that request gets:
uv installs an x86-64 one unless the architecture is named
(--python cpython-3.13-windows-aarch64), reporting that support for
the native architecture is not yet mature. Both work — the build follows
the interpreter — but only the native one exercises what the win_arm64
wheels are.
Beware that this replaces .venv with an environment built on that
interpreter, and leaves it there. Going back is another uv sync, and
--reinstall-package btclib-secp256k1 --no-cache if the extension it
finds in the cache is the one of the ABI just left behind. Requesting a
free-threaded interpreter (--python 3.14t) has a second effect: it
installs it as a managed one, and uv then prefers it to a system 3.14
wherever 3.14 is asked for, so uv python install 3.14 is what makes
such an environment reproducible again.
Naming the environment keeps the default one instead, at the price of a second build of the extension:
UV_PROJECT_ENVIRONMENT=.venv-3.11 uv run --python 3.11 --no-cache pytest
.gitignore matches that name with .venv*/, the comment beside the
pattern saying what ships the environment when nothing matches it.
A git pull that changes scripts/hatch_build.py is not itself enough
to reach a .venv a uv sync --locked already considers satisfied: the
sync reports the package Checked, not rebuilt, and installs nothing
new even where the pulled change moves what the hook decides.
--reinstall-package btclib-secp256k1 is what forces the rebuild;
--refresh-package btclib-secp256k1 alone does not.
To build the distributions:
uv build --sdist
uv build --wheel
The editor¶
.vscode/settings.json and .vscode/extensions.json are tracked, and they
hold no preference: the recommended extensions are the tools
.pre-commit-config.yaml already runs, and the settings put the fixing ones
on save. Installing them is optional and changes nothing about what a local
run enforces — what they buy is learning of a finding while typing rather
than at the local run that trips over it.
Anything machine-local — an interpreter path, a telemetry answer, a theme — belongs in the editor’s own user settings instead, those two files being read by every checkout of this repository.
A Dependabot uv pull request¶
Dependabot’s uv ecosystem rewrites uv.lock, and sometimes
build-constraints.txt, without running the uv-export hook. Where a
bump moves a package that build-constraints.txt pins, the file can then
differ from the hook’s export of the lock, and the hook fails the lint
job. An owner, as GOVERNANCE.md names them, then commits
the hook’s output to the pull request’s branch before it lands, signed
and signed off. On a checkout of that branch:
uv run --locked --only-group lint pre-commit run uv-export --all-files
git commit -S -s -m "Export build-constraints.txt from uv.lock" \
build-constraints.txt
git push origin HEAD:<branch>
The hook exits 1 when it rewrites the file. The push is a plain one, never a force, so a branch that moved meanwhile refuses it.
The commit is made by hand, not by a bot: the ci: block of
.pre-commit-config.yaml says why.
The cost is that Dependabot stops rebasing a pull request once extra
commits have been pushed to it, as
GitHub’s documentation
says. @dependabot recreate in a comment rebuilds the branch without the
commit, which is then made again.
Running what CI runs¶
Each job of the lint, docs and test workflows, and the local command
that reproduces it. Two of them cannot be reproduced on a machine that is
not the runner, and that is worth knowing before trying; codeql has no
command at all, for the reason below, and nothing requires its result.
lint / Lint and type-checkuv run --locked --only-group lint pre-commit run --all-files \ --show-diff-on-failure
--show-diff-on-failurerunsgit diffafter a failing run and changes nothing else, the exit status included. A local checkout still holds what the fixers wrote, so the flag is in the block because the job carries it rather than because a local run needs the diff printedlint / Dependency reviewreproduces as nothing: it asks the forge’s dependency graph what a pull request adds, and has no local commandlint / Sign-off, run from a checkout ofbtclib-org/.githubuv run --no-project --python 3.15 \ <that checkout>/.github/scripts/check_sign_off.py origin/main..HEAD
Ask which files the pull request touches, whose answer decides whether the rest oftest.ymlruns at all – a step ofbtclib-org/.github’sreusable-changes.ymlnow,test.yml’s ownchangesjob being a bareuses:call to it with nosteps:of its own, so this reproduces what that step does rather than what this repository runsgh api "repos/btclib-org/btclib-secp256k1/pulls/<number>/files" \ --paginate --jq '.[].filename' > files.txt prose=$(/usr/bin/grep -A1 'prose-pattern: |-' .github/workflows/test.yml \ | tail -1 | sed 's/^ *//') if [ ! -s files.txt ] || /usr/bin/grep -qvE "$prose" files.txt; then echo "everything runs" else echo "prose only" fi
The second line lifts the pattern out of
test.yml’s ownwith:block rather than restating it here, so the two cannot drift; the rest of the step isbtclib-org/.github’s own, and a change there is not one this block would notice./usr/bin/greprather thangrep, because where the shell’s is ugrep the third line answers the opposite question: ugrep takes the status from whether the pattern matched anywhere and inverts that, rather than from whether-vselected a line. A list of only prose and a list of only code come out right either way; one mixing the two exits 1 and reads as “prose only”, which is the direction that skips a matrix that should have run, and nothing says it has.-qis what asks the question of the file rather than of the lines, and ugrep takes the same path when the output is discarded, so-cvEand a numeric test agree with both greps. The runner’sgrepis GNU’s, so what ugrep can get wrong here is the reproduction and not the workflow (https://github.com/btclib-org/btclib-secp256k1/issues/242)Measure coverage, gated at 100%, which is the suite against each linkage, the tests markedzkpagainst the flagged build, and the union of the three gated at the ratchetCOVERAGE_FILE=coverage-data-static \ uv run --locked --no-default-groups --group test \ --reinstall-package btclib-secp256k1 --no-cache pytest \ --cov-fail-under=0 BTCLIB_LIBSECP256K1_DYNAMIC=true COVERAGE_FILE=coverage-data-dynamic \ uv run --locked --no-default-groups --group test \ --reinstall-package btclib-secp256k1 --no-cache pytest \ --cov-fail-under=0 BTCLIB_LIBSECP256K1_ZKP=true COVERAGE_FILE=coverage-data-zkp \ uv run --locked --no-default-groups --group test \ --reinstall-package btclib-secp256k1 --no-cache pytest \ -m zkp --cov-fail-under=0 uv run --locked --no-default-groups --group test \ coverage combine coverage-data-static coverage-data-dynamic \ coverage-data-zkp uv run --locked --no-default-groups --group test coverage report
the static run carries
--reinstall-package btclib-secp256k1 --no-cachetoo, for the same reason the other two do:uv run --locked’s build cache is keyed on the source tree, not onBTCLIB_LIBSECP256K1_DYNAMICorBTCLIB_LIBSECP256K1_ZKP, so without it a venv already holding either flagged build – left there by an earlier pass through this sequence in the same environment – serves that build back under the namecoverage-data-staticinstead of rebuilding for the run at hand.none of the three is gated at 100 on its own any more, each carrying its own
--cov-fail-under=0for a different line it cannot execute: the linked-in branch of_load_libfor the dynamic run, andbtclib_secp256k1.zkp.musig(#607 onward) for both the static and the dynamic run, since neither setsBTCLIB_LIBSECP256K1_ZKPand the zkp run’s own-m zkpselection is what reaches it. What answers for each missing line is one of the other two runs, and the combined report – the first and only place 100 is actually asked for – is what answers for a line none of the three reaches, taking its threshold from[tool.coverage.report]. The data files are written where the command runs and.gitignorenames them,combineconsuming all three and leaving.coverage, which it names tooBuild the flagged secp256k1-zkp extension, and run its testsBTCLIB_LIBSECP256K1_ZKP=true uv run --locked --no-default-groups \ --group test --reinstall-package btclib-secp256k1 --no-cache \ python -c "import _btclib_secp256k1_zkp as m; print(m.lib)" BTCLIB_LIBSECP256K1_ZKP=true uv run --locked --no-default-groups \ --group test pytest -m zkp --no-cov
--reinstall-packageand--no-cacheon the first command for the same reason the coverage job’s own dynamic run needs them: uv’s build cache is not keyed on this variable, only on the source tree, so a local venv already holding an unflagged build serves it back unless told not to – a fresh venv does not need either flag, and the job carries them regardless of which a runner happens to be. The second command selects the tests markedzkp, overbtclib_secp256k1.zkp.musig(#607) and the modules #608 and #609 add; an exit of 5, pytest’s own “no tests ran”, is a real failure here – a selection that collected nothing, from a renamed marker, a moved file, or a-mmatching no testBuild wheels on <os>, for this platform onlyuv run --locked --only-group build cibuildwheel
the job pins the build timestamp first, and reproducing the wheel means pinning the same one:
export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
every member of the archive carries it:
hatchlingstamps the members it writes at a constant of its own without it, andauditwheelordelocategives the clock of the moment to the members the repair rewrites, which is what makes two builds of one commit differ where nothing else does.[tool.cibuildwheel.linux]passes the variable into the container; the reasoning is intest.yml, next to the step that exports it.The Linux wheels of that job are built in a manylinux container, so reproducing them needs a container runtime (
colimaon macOS)Build dynamic wheel on <os>PIP_CONSTRAINT=build-constraints.txt BTCLIB_LIBSECP256K1_DYNAMIC=true \ uv run --locked --only-group build python -m build -w
on macOS the job exports a deployment target first, 11.0, or 10.13 on x86_64, and reproducing it means exporting the same one:
export MACOSX_DEPLOYMENT_TARGET=11.0
a dynamic wheel compiles no extension, so nothing derives its platform tag from a toolchain: without that variable
hatch_build.pyfalls back toplatform.mac_ver(), and the wheel comes outmacosx_26_0_arm64on a macOS 26 machine — a tagpiprefuses on every older macOS the file would in fact have loaded on. CMake reads the same variable, so the vendored library is built for the floor the tag then claims; the two values and the reasoning behind them are intest.yml, next to the step that exports them.On every platform of that job the build timestamp is pinned as well, as in
Build wheels on <os>above –auditwheelanddelocaterun here as steps of the job rather than insidecibuildwheel, and read it the same way:export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
Build sdist, andInstall from the sdist and run the suite on <os>after it. The build timestamp is pinned as inBuild wheels on <os>above, and the normalizer that follows the build reads that same variable to rewrite every member’smtime, refusing to run without it – its own docstring has the reasoning. The bill of materials reads it too, as the document’s own timestamp, and is written after the normalizer because the rewrite moves the digest it records;sbom/rather thandist/, an index taking distribution files. On a releaseBuild sdistbuilds nothing:release.yml’sbuildjob builds the sdist and its bill of materials the same way through btclib-org/.github’sreusable-build.yml, which signs them, and the install job reads that upload. The install job uses pip rather than uv, its subject being pip resolving the published artifact rather than uv reading the lock, so reproducing the install wants a fresh venv rather than the project’s own:export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) PIP_CONSTRAINT=build-constraints.txt \ uv run --locked --only-group build python -m build -s uv run --no-project --python 3.15 \ .github/scripts/normalize_sdist.py dist/ uv run --no-project --python 3.15 \ .github/scripts/generate_sbom.py --sdist-only dist/ sbom/ python -m pip install --verbose dist/*.tar.gz
Inspect the distribution files and install oneuv run --locked --only-group check twine check --strict dist/* uv run --locked --only-group check check-wheel-contents dist/*.whl uv run --locked --only-group check pyroma --min 10 dist/*.tar.gz
Build on Linux for Windowsneedsmingw-w64, and a Linux host to be faithful: the cross-compilation CI does is from ubuntu, not from macOS. That job pins the build timestamp too, and nothing repairs its wheel, sohatchlingalone writes the members and this is what decides their timestamp:export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct)
test: every job passedreproduces as nothing, and is here because it is the required check and therefore the one a contributor sees red: it reads the conclusions of the jobs above it and runs no command of its own. What turned it red is one of them, in the rundocs / Build the documentation, the same command.readthedocs.yamlruns anddocs/README.rstdocumentsuv run --locked --no-default-groups --group docs \ sphinx-build -n -W -b html docs/source docs/build/html if grep -rn 'href="#\.\.\?/' docs/build/html --include='*.html'; then echo "::error::the links above resolve to no page (unresolved relative path)" exit 1 fi served=$(mktemp -d) git clone --quiet --filter=blob:none --sparse \ https://github.com/btclib-org/.github "$served" git -C "$served" sparse-checkout set .github/scripts git -C "$served" checkout --quiet \ "$(sed -n 's/.*reusable-docs\.yml@\([0-9a-f]*\).*/\1/p' .github/workflows/docs.yml)" found=0 uv run --no-project --python 3.15 \ "$served"/.github/scripts/check_included_links.py docs/build/html || found=$? rm -rf "$served" test "$found" -eq 0
A link myst cannot resolve renders as an anchor on the page it is already on — an id nothing defines — and
-n -Walready fails the build on it, for as long as myst warns unconditionally andconf.pysuppresses nothing (verified: appending a dead relative link to this file takes the build above from exit 0 to exit 1 onmyst.xref_missing). Thatgrepasks the same question of the pages sphinx wrote instead, and is what still answers for the artifact if either of those changescheck_included_links.pyrefuses a built page that links a file the documentation renders at its GitHub URL instead of its page. It is btclib-org/.github’s: the commands above check it out at the commitdocs.ymlpins, as the job does, and end non-zero where it fails
The pypi-install workflow has no local equivalent by design: what it
installs is what PyPI serves.
The sentinels beside it gate nothing, so a red one is read in the Actions
tab rather than fixed on a branch. wheel-reproducibility is the exception:
its aggregate is a required check on a pull request, and What gates a
merge, and what only reports below says why. Each is dispatchable, and all
but links run locally.
os-ubuntu,os-macosandos-windows, the suite on the images of a platform (both Linux ones, both Windows ones and the arm64 macOS one) and on every interpreter, so only the row matching the machine at hand reproduces. A cell is these two commands in this order — the suite run once against each linkage, with--pythonstanding for the interpreter the row names:uv run --locked --no-default-groups --group test --python 3.11 \ pytest --no-cov BTCLIB_LIBSECP256K1_DYNAMIC=true uv run --locked --no-default-groups \ --group test --python 3.11 --reinstall-package btclib-secp256k1 \ --no-cache pytest --no-cov
the second re-installs rather than reusing what the first built, because neither the environment nor the build cache is keyed on the variable that chooses the linkage.
--no-covis in both, against the--covaddoptscarries: what a cell asks is whether an (image, interpreter) pair passes, and coverage is measured and gated once, on the gate’s celldeps-latest, which resolves every dependency at the newest release its bounds admit and runs four jobs against it: the suite over a matrix of its own, narrower thantest.yml’s; the coverage union in a job of its own;Lint and type-check, dependencies at latest; andBuild the distributions and inspect them, dependencies at latest. The upgrade rewritesuv.lock, so restore it afterwards withgit checkout uv.lock.The suite matrix’s own block is one cell of it, so only the row matching the machine and the interpreter at hand reproduces:
uv lock --upgrade uv run --locked --no-default-groups --group test pytest \ --cov-fail-under=0
--cov-fail-under=0is the cell’s own flag, not a local convenience: a barepytestcannot reach the ratchet, so the cells report the number andMeasure coverage against the latest dependencies, gated at 100%is what holds it. That job isMeasure coverage, gated at 100%’s block above run against an upgraded lock, so reproducing it isuv lock --upgradein front of that block rather than a recipe of its own.Lint and type-check, dependencies at latestis the same shape:uv lock --upgradein front oflint / Lint and type-check’s own block above, the identicalpre-commit run --all-files --show-diff-on-failurerun against the upgraded lock rather than the pinned one, so it is not repeated here.Build the distributions and inspect them, dependencies at latestbuilds withuv builddirectly, unlikeBuild sdist’spython -m build -s:uv lock --upgrade uv build --sdist uv build --wheel uv run --locked --only-group check twine check --strict dist/* uv run --locked --only-group check check-wheel-contents dist/*.whl uv run --locked --only-group check pyroma --min 10 dist/*.tar.gz
Its last three lines are
Inspect the distribution files and install one’s own block, run here against the upgraded lock rather than the pinned one.linksneeds a tool uv does not provide, lychee being a rust binary, so the workflow uses the action..lycheeignoreholds the URLs a checker cannot judge, each with the reason it cannot be checked rather than the reason checking it is inconvenientmutation, scoped by.github/mutation/bindings.tomland.github/mutation/zkp.toml, which are what the workflow reads too, as the two sessions of one profile, in that order:( export BTCLIB_LIBSECP256K1_ZKP=true uv run --locked --no-default-groups --group test --group mutation \ --reinstall-package btclib-secp256k1 --no-cache \ python -c "import _btclib_secp256k1_zkp as m; print(m.lib)" git clone --depth 1 https://github.com/btclib-org/.github standard for session in bindings zkp; do uv run --locked --no-default-groups --group test --group mutation \ cosmic-ray baseline .github/mutation/$session.toml uv run --locked --no-default-groups --group test --group mutation \ cosmic-ray init .github/mutation/$session.toml $session.sqlite uv run --locked --no-default-groups --group test --group mutation \ cr-filter-operators $session.sqlite .github/mutation/$session.toml uv run --locked --no-default-groups --group test --group mutation \ cosmic-ray exec .github/mutation/$session.toml $session.sqlite uv run --locked --no-default-groups --group test --group mutation \ cr-report --surviving-only --show-diff $session.sqlite uv run --locked --no-default-groups \ python standard/.github/scripts/mutation_counts.py \ $session.sqlite $session done )
The counter is
standard/’s: cloningbtclib-org/.githubthere is a local stand-in for the second checkoutreusable-mutation.ymlruns beside this tree’s own.Both sessions need the build that has the
zkpextension:zkp.toml’s mutants are judged by tests that skip without it, and so are thezkptests of the whole suite thatbindings.tomlruns. The workflow exports the variable through theextra-envinput of the workflow it calls, where a local run exports it in a subshell, which everyuv runof the block then inherits – one that rebuilt the package without it would drop the extension – and which leaves the shell it was pasted into as it was; the first command above is the build line ofBuild the flagged secp256k1-zkp extension, and run its testswith the proof that the build took –--reinstall-packageand--no-cachefor the reason that entry gives. The variable’s value istrueand nothing else. A session against a build without the extension does not fail: the tests markedzkpskip, and thezkpmutants are judged by the tests that run without the library.baselinefirst, always: it runs the configured test command against the unmutated tree, and without it a stale command fails every mutant identically and the session reports a perfect kill rate — the one failure mode of a mutation run that looks like good news. The session mutates the source in place and restores it, so nothing else may read the tree while it runs: no other session, which is why the loop above is one at a time, nopytestin another shell, and agit statusin the middle is a working tree with a mutant in it.execis resumable, so interrupting one costs only the mutant it was on, and the.sqliteis what the workflow uploads beside the reports —cr-report,cr-htmland the counter all read one.cr-filter-operatorsmarks as skipped what a configuration excludes by operator, andexcluded-modulesdrops a module’s mutants atinit, before any is enumerated: it is howbindings.tomlhands thezkpsubpackage tozkp.toml. What is excluded, and why, is in the two files, each exclusion with the command that keeps its claim honest: a mutant excluded is one no session reports, so the claim is what to re-check and not the comment beside it.--surviving-onlyis the whole of what anybody acts on, a killed mutant being the suite doing its job. A survivor is a test nobody has written, except for the kindsbindings.toml’s header records as ones no test worth writing can kill: the diff the report prints tells them from the rest, without another session, and the header says how.The counter last, and not
cr-rate: that tool reads anything that is not SURVIVED as a kill, so it counts the skipped mutants among them and divides by the whole session.mutation_counts.pyprints killed, survived and skipped with the rate over what actually ran, and exits non-zero on an outcome that is no verdict at all — an INCOMPETENT mutant, or a worker that raised, which is Cosmic Ray not having measured rather than a test that is missing. What its rate means, and what to read it against, is inbindings.toml’s header.vendored-vectors, whose jobs ask unrelated questions and reproduce separately.checkre-reads every pin intests/README.mdagainst upstream, and hashes each vendored file against the blobtests/README.mdrecords for it: a mismatch fails the run and opens no issue.--dry-runis what every trigger but the schedule passes, through thereusable-vendored-vectors.ymlit calls, so that the run edits no tracking issue — which is what a run by hand wants too:uv run --no-project python \ .github/scripts/check_vendored_vectors.py \ tests/README.md "Vendored vectors behind upstream" --dry-run
--no-projectbecause there is no environment to build for it: the script imports only the standard library and shells out togh, which has to be authenticated. The called workflow’s own line is a barepython, which is setup-python’s on the runner and need not be anything on a machine that has uv and no interpreter of its own onPATH. Without the flag it is the scheduled run, and it opens, edits or closes the issue it finds.pinasks the two halvessubmodule-pincannot ask offline — whether the tagREADME.mdnames is the one upstream publishes, and whether a libsecp256k1 maintainer signed it. The last two lines below are the comparison itself: the first prints what upstream’s tag resolves to and the second what this tree’s gitlink pins, and the pin is correct where the two commits agree:gpg --keyserver hkps://keys.openpgp.org --recv-keys \ $(sed -n '/FINGERPRINTS:/,/run:/p' \ .github/workflows/vendored-vectors.yml \ | grep -oE '\b[0-9A-F]{40}\b') named=$(sed -n 's|.*secp256k1/releases/tag/\(v[0-9][0-9.]*\).*|\1|p' \ README.md | head -1) git -C secp256k1 fetch --force origin "refs/tags/${named}:refs/tags/${named}" git -C secp256k1 tag -v "${named}" git -C secp256k1 rev-parse "${named}^{commit}" git ls-tree HEAD secp256k1
the fingerprints are lifted out of the workflow rather than written again here, a second list being one that drifts. The
zkp-pinkeys come from the recipe below, not from a keyserver. What the runner never has to care about and a developer does is where the two writes land —--recv-keysputs each of the pin job’s own third-party public keys in whichever keyring it is pointed at, the default one unlessGNUPGHOMEsays otherwise, and thefetch --forcemoves the tagREADME.mdnames in the vendored clone to what upstream serves, which is the question being asked and is also whatsubmodule-pinresolves against afterwards.${named}is braced against this shell rather than against the runner’s. zsh reads the:rof an unbraced$named:refs/tags/...as a history modifier, strips the last dotted component and fetchesrefs/tags/v0.8efs/tags/v0.8.0, which upstream does not have; bash, which is what the workflow runs, reads the same line as written. The same asymmetry as/usr/bin/grepabove, and in the same direction: what a local shell gets wrong here is the reproduction and not the workflowzkp-pinasks the narrower question its own block comment states: whether the pin’s delta againstBlockstreamResearch/secp256k1-zkp’s own master does not exceed what README.md documents as reviewed, and whether every commit in that delta is one a recognized signer wrote — not whether the pin itself is a tagged release, secp256k1-zkp cutting none. Two keys are recognized while the submodule points at the fork (issue #855): Andrew Poelstra’s, from his own published bundle rather than fromkeys.openpgp.org, which serves this one stripped of the user IDs GnuPG needs before it accepts a key (#690); and the maintainer’s own, fromhttps://github.com/<user>.gpg, whichkeys.openpgp.orgalso serves, user IDs included. Both fingerprint/URL pairs are lifted from the two import steps’ own env blocks inzkp-pin.yml, keyed offFINGERPRINT:(singular), which the last step’sFINGERPRINTS:(plural) does not match:grep -A1 'FINGERPRINT:' .github/workflows/zkp-pin.yml curl --fail --silent --show-error --location \ --output andrew.gpg https://www.wpsoftware.net/andrew/andrew.gpg curl --fail --silent --show-error --location \ --output maintainer.gpg https://github.com/<user>.gpg gpg --import andrew.gpg maintainer.gpg gpg --list-keys <fingerprint 1> <fingerprint 2>
gpg --list-keyson both is the job’s own import-step assertion, kept for the same reason it is there: GnuPG accepts a key stripped of every user ID and exits 0 having imported nothing (#690), so acurlthat silently failed to fetch a usable key is caught here rather than misread, one step later, as a bad pin.sed -n 's|.*secp256k1-zkp/commit/\([0-9a-f]\{40\}\).*|\1|p' README.md pinned=$(git ls-tree HEAD secp256k1-zkp | awk '{print $3}') git ls-tree HEAD secp256k1-zkp git -C secp256k1-zkp fetch --unshallow origin "$pinned" 2>/dev/null || \ git -C secp256k1-zkp fetch --force origin "$pinned" git -C secp256k1-zkp fetch --force \ https://github.com/BlockstreamResearch/secp256k1-zkp.git \ master:refs/remotes/upstream/master delta=$(git -C secp256k1-zkp rev-list --count \ refs/remotes/upstream/master.."$pinned") echo "delta=$delta"
the
sedline and thegit ls-treeline beside it are the comparison thepinrecipe’s own two closing lines make for the other submodule: the first prints whatREADME.mdlinks to and the second what this tree’s gitlink pins, and the pin is correct where the two commits agree.--unshallowfalls back to a plain--forcefetch on a clone this recipe already ran once and so is no longer shallow — unlike the job, which checks out fresh every run and always needs it — anddeltais what the workflow step gates atEXPECTED_DELTA, read from that same step’s own env block since a hand recipe is exactly the case the block comment above says to update it for at the next re-pin.if [ "$delta" -eq 0 ]; then subjects="$pinned" else git -C secp256k1-zkp rev-list refs/remotes/upstream/master.."$pinned" subjects=$(git -C secp256k1-zkp rev-list \ refs/remotes/upstream/master.."$pinned") fi for commit in $subjects; do git -C secp256k1-zkp verify-commit --raw "$commit" 2>&1 done
What the last loop prints is the whole answer for each commit in the delta, or for the pin alone where the delta is empty, and it is read directly rather than filtered: an unanchored search for a fingerprint in that output also matches
ERRSIG’s own trailing field, which carries the expected signer’s fingerprint even when the key never arrived, so a missing key can print the very string the reader was told to look for — measured against the real pin, with the keys above never imported. Read the status lines the way the job’s own comment reads them instead: a line starting[GNUPG:] VALIDSIGfollowed by one of the two fingerprints is that commit verifying.NO_PUBKEYnaming a different key id is, past the assertion above, a commit signed by somebody this recipe holds no fingerprint for.BADSIGis a signature that does not verify at all.REVKEYSIG— which can sit beside aVALIDSIGfor the same fingerprint — is the one status that means stop trusting the key regardless, its own owner’s act rather than a calendar date;EXPKEYSIGbeside aVALIDSIGinstead is the opposite: the key’s stated expiry lapsing, which does not weaken a signature it already made.wheel-reproducibility, which builds this commit’s wheel twice, from two clones ofHEAD, and diffs the two archives member by member. The job pins the build timestamp first, as inBuild wheels on <os>above;hatchling’s own fallback constant would make the two local builds agree either way, so what pinning it buys here is that this measurement and the wheel a release actually builds do not differ in an exported variable:export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) uv run --no-project python \ .github/scripts/check_wheel_reproducibility.py
--no-projectfor the same reason ascheck_vendored_vectors.pyabove: the script imports only the standard library and shells out touvitself to run each build. Only the environment the command runs in is answered locally; the workflow’s own matrix is what asks the rest, the second image it builds each platform on includedThe workflow’s
repairedjob asks the same question of the file PyPI actually receives:uv buildstops at the archivehatchlingwrites, wherecibuildwheelrepairs it withauditwheel,delocateordelvewheel, and it is that rewritten archive the job builds twice and diffs. It pins the same timestamp first, and here the export is not optional:auditwheelanddelocatetake each member’smtimefrom the clock of the moment where the variable is unset, so two sequential builds of one unchanged commit disagree and the check reports a wheel that does reproduce as one that does not. It reproduces the same way otherwise, for this platform only, and needscibuildwheelonPATH:export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) uv run --locked --only-group build python \ .github/scripts/check_wheel_reproducibility.py --repaired
as with
Build wheels on <os>above, the Linux build runs inside a manylinux container, so reproducing it needs a container runtime (colimaon macOS).CIBW_BUILDnarrows the workflow’s own run to one interpreter; left unset, this builds every interpretercibuildwheelselects for the platform at handThe
dynamicandcross-windowsjobs ask the same question of thepy3-none-*wheels, whichbuild-dynamicandbuild-windowsbuild withpython -m buildrather than with either frontend above and whichpublish-pypiuploads with the rest.--dynamicbuilds the wheel of the platform it runs on and repairs each build the way that job does, withauditwheelordelocate;--cross-windowsbuilds thewin_amd64one and repairs nothing, wanting amingw-w64toolchain where the job above wants a container. Both refuse an unsetSOURCE_DATE_EPOCH: where a repair runs the reason is the one above, and where none does it is that the archive would carryhatchling’s fallback constant rather than the instant the rest of the release carries:export SOURCE_DATE_EPOCH=$(git log -1 --pretty=%ct) uv run --locked --only-group build python \ .github/scripts/check_wheel_reproducibility.py --dynamic
What a change here has to satisfy¶
Past the gates above, and past what section 9 of the standard asks of any tree’s prose:
new wrapped functionality is validated against external vectors. A test that compares these bindings against themselves proves nothing.
tests/vectors_test.pydocuments where each vendored vector file comes from — BIP340, RFC6979, trezor-firmware — and a new wrapper should reach for something published elsewhere in the same way. A test’s docstring says which side of the assertion is the independent one, which is why a docstring is required of a test at all: the name says which call is under test, and not what is being claimed about ita new wrapper checks sizes and delegates the rest. What the boundary is for is stopping a short buffer from reaching a bare pointer; whether the bytes are a valid key, point or signature is libsecp256k1’s answer to give, but for the zero r or s the README refuses, and an argument of the wrong size or form raises rather than being padded or reinterpreted into a valid one. The reasoning is in the README, under What the boundary checks
warnings are errors (
filterwarnings), because the spread of interpreters this package claims turns a deprecation into a breakagea script under
.github/scriptsruns on Python 3.11.9, which is older than any 3.11 a machine here is likely to have. cibuildwheel pins the macOScp311to python.org’spython-3.11.9-macos11.pkgand the Windows one to that same patch — 3.11.9 is the last 3.11 python.org publishes an installer for — where a manylinux image carries a current patch instead, so a standard library keyword backported after it is fatal on the macOS and Windows wheel jobs and fine on the Linux ones. No static check reaches it: mypy’spython_versionis a minor version and it refuses a patch, so nothing it can be aimed at tells one patch from the next. What does is the suite the wheel jobs run undercp311, which a pull request builds on every image but the x86_64 macOS one, built at a releasethe version is declared once, in
pyproject.toml, and__version__reads the installed metadata. Never bump it in an ordinary change: releases are cut by a maintainer following RELEASING.md, andrelease.ymlchecks the tag against ita vendored submodule moves on purpose. Bumping one is a change of what this package wraps and belongs in its own pull request, with what README.md’s Versioning section says about that submodule’s pin moved with it. Where the upstream cuts releases, as
secp256k1does, the pin is the commit a release tag names, the version number tracks that release, andRELEASE_NOTES.mdnames it in the release the bump lands in; where the upstream cuts none, assecp256k1-zkp, the pin is a commit no tag names, reviewed against the one it replaces, and there is no version for either file to carry. Dependabot signals upstream movement but tracks the upstream default branch, so its pull request says that a bump is available and not which commit to pin
What gates a merge, and what only reports¶
.pre-commit-config.yaml is the single definition of what “clean” means,
and the lint workflow runs that very file, so what CI enforces is what
the local gate enforces. Never add a check that exists only in a
workflow, and never leave a hook weaker locally than on a runner: a hook
that needs a tool the developer may not have carries it in
additional_dependencies, as typos does, or takes it from its own
repository: actionlint-shellcheck is the fork’s id that installs
shellcheck beside actionlint, and zizmor reaches the same end at a
pinned rev, which pre-commit.ci moves where an additional_dependencies
pin is moved by nothing, rather than from a local hook naming the tool by
hand.
pyroma is a local hook that names no version at all: it runs out of the
check group through uv run --locked, so the version is uv.lock’s. A
check discovered by CI after a push is a check in the wrong place.
The aggregate of test, the lint workflow’s jobs, the
documentation build, the aggregate of wheel-reproducibility and the job of
sourcery.yml are the required checks, and REPOSITORY.md reads that rule
back from the endpoint rather than restating it. release reuses the first
three. wheel-reproducibility is among them because a pull request can turn
it red by changing the build, and REPOSITORY.md says why its trigger lets
it be. Section 10 of the organization standard says when a tree requires
sourcery.
Everything
else reports: a sentinel opens no issue when it fails, vendored-vectors
excepted for the reason the header of the workflow it calls gives,
because each is expected to go red for something no pull request
introduced and a red check nobody can act on from a branch is noise.
verified-signatures.yml reports after each push to main whether every
commit pushed is verified; the standard’s Signatures has the rule.
upstream-tests reports for another reason. A pull request touching what its
path filter names can turn it red, which is what it is for, but that filter
would leave a required check Pending, as REPOSITORY.md says of
wheel-reproducibility.
workflow |
when |
what it varies |
|---|---|---|
|
pull request, push |
the wheels, those of x86_64 macOS at a release and a dispatch only, the sdist, one suite cell |
|
pull request, push |
— |
|
pull request |
— |
|
pull request, and |
— |
|
pull request |
— |
|
push to main |
— |
|
weekly, a pull request touching what it reads |
— |
|
a pull request touching what the build of either pin reads or |
x86_64 and aarch64, each in its pinned |
|
pull request, push to main, weekly |
the two scanned languages |
|
push to main, weekly |
— |
|
weekly, a release |
both ubuntu images × every interpreter |
|
weekly, a release |
the arm64 macOS image × every interpreter |
|
weekly, a release |
both Windows images × every interpreter |
|
weekly |
the dependencies, at the newest their bounds admit |
|
weekly |
the dependencies, at their floors |
|
weekly, a pull request touching its own configuration |
— |
|
weekly |
— |
|
weekly |
— |
|
weekly, pull request, building only where it touches what it builds |
every wheel platform, on two images, built twice on each, and the repaired, dynamic and cross-compiled wheels, built twice on one image per platform; x86_64 macOS weekly and on a dispatch only |
|
weekly, a pull request touching its own file |
— |
|
weekly, a release |
what PyPI serves |
|
a tag |
calls the gates and the rows marked a release |
The first three rows and wheel-reproducibility are what a merge waits for,
by the rule REPOSITORY.md reads back from the endpoint, and the suite cell
among them is one: ubuntu-latest on the interpreter
.python-version pins, measured for coverage. Which day each of the rest
runs, and at which minute, is section 10 of the organization standard in
btclib-org/.github and not this file’s to restate — one calendar covering
the organization is one thing to remember, and a copy of it per repository is
one more thing to keep true in each.
So few checks gate a merge because the plan caps the jobs the whole organization
may run at once, and REPOSITORY.md’s Plan-gated settings records that
ceiling. A commit here, in bitcoin-core-rpc and in one more repository each ask
for more jobs than that ceiling alone allows, so a pull request in any of the
three spent its wall clock waiting for a slot. At that ceiling a cell before a
review buys a rarer answer at the price of every review: macOS runners queue for
tens of minutes rather than for two, and the twenty-one Windows suite cells were
27.3 of a run’s 112.9 runner-minutes, the largest family of jobs in it and ahead
of every wheel build. The numbers are in test.yml’s header.
What the sentinels vary, they vary whole. os-ubuntu runs both images
on every interpreter, the cell the gate already covers included. A matrix with
the gate’s cell cut out of it is one nobody can read the shape of, and
whoever asked what ran would have to re-derive the hole from the gate.
Every image but x86_64 macOS still builds wheels on every pull request:
cibuildwheel runs the suite against each wheel as it builds it, and the
release publishes the artifacts of a run that built every one. What narrows
on a branch is which interpreters per image — the oldest one and the
free-threaded one rather than every interpreter, ubuntu included — because
what a pull request asks of an image is whether this tree still builds
there, and the toolchain, the CMake build of the vendored library and the
cffi extension are what differ per image rather than per interpreter.
The free-threaded one is kept for a reason of its own: pyproject.toml
declares the Free Threading classifier on the ground that the gate refuses
a landing that breaks the cp315t wheel, so the required check builds that
wheel and runs the suite against it. Nothing on a branch reads past those
wheels: check-dist installs one wheel by path and takes it from
build-dynamic, which builds whole. test.yml carries the reasoning beside
the step.
What a pull request no longer asks is pip’s selection among a directory of
wheels tagged for several interpreters, which now runs nowhere; the wheel
each interpreter would get is still tested by the job that builds it, and
os-ubuntu, os-macos and os-windows still compile both linkages from
the tree on every interpreter. What no run asks at all is the suite
against the dynamic wheel as a package — the sentinels compile that
linkage from the tree instead. os-ubuntu.yml’s header records both
costs beside each other.
x86_64 macOS builds only at a release, on a dispatch and on the calendar,
never on a pull request or a push to main. test builds its wheels at a
release and a dispatch, and check-dist fails those runs unless they are
there; wheel-reproducibility builds its images weekly and on a dispatch.
Section 10 of the organization standard in btclib-org/.github has the
reason and when it ends.
Every workflow here takes workflow_dispatch, the gates included, except
claude-review, sourcery and scorecard, and for the three platform
workflows it is the only way to ask about a branch at all. Both of the jobs
claude-review.yml calls into read the pull request or the comment that
triggered them, and sourcery.yml’s reads the pull request’s files, so a
manual run would start with nothing to read.
scorecard’s triggers are its action’s rather than this organization’s:
ossf/scorecard-action’s own README names
push and schedule on the default branch as what it supports, and calls
workflow_dispatch experimental.
codeql runs on a pull request as well as on main and its weekly
schedule, and none of the three makes it a gate: nothing requires the
result, and REPOSITORY.md is where the rule that could is read back from
the endpoint. What the pull-request trigger buys is that such a rule has a
name to hold — one aggregate over the matrix rather than one context per
language — at the price of those cells on every push to a branch.
There is no local command for it either: reproducing it means the CodeQL
CLI, a bundle GitHub distributes rather than a dependency uv.lock can
pin, so what answers a finding is the run itself and the Security tab
beside it.
scorecard gates nothing and could not: it runs on main and on its
weekly schedule, so there is no pull request run for a branch rule to
name. It has no local command either — the score is the action’s, and
README.md’s badge reads it back from the API that serves it — and what
the run finds arrives as code scanning alerts beside CodeQL’s. Which
repositories of the organization run it at all is section 10 of the
standard, this one among them.
Traps¶
A hand-applied mutation of the same length can outlive its restore through a
.pycwhose mtime and size still match. UsePYTHONDONTWRITEBYTECODE=1, with a passing baseline run before and a control run after. cosmic-ray sets that variable itself (#229) but still reads a stale.pyc.To check that a
.pycholds its source’s bytecode, compare code objects, not bytes:stored == compile(src, stored.co_filename, "exec"), whererawis the.pyc’s bytes,storedismarshal.loads(raw[16:])andsrcis the source. On a clean tree the marshalled bytes can differ, round-tripped or not, anddisoutput differs by object addresses. Either reads as a live mutant inside a green gate.check-sdistfails with “SDist does not match git” on an untracked file that neither.gitignorenor the sdistexcludeinpyproject.tomlcovers.docs/_build/is one:.gitignorehasbuild/, andgit check-ignore -v docs/_build/xprints nothing. Build the docs intodocs/build/htmlas above, and anything else outside the worktree.pre-commit’s log names that hookcheck sdist, with a space, though theid:ischeck-sdist.grep -c check-sdistover a passing run’s log answers0; grep the display name.