Skip to main content

Module breaker

Expand description

The circuit breaker the study-registry phases share within one worker process.

BREAKERS is a process-local static: credit-registrar and suotar-syncer are separate OS processes (see programs/credit_registrar.rs and programs/suotar_syncer.rs), each with its own map, so an outage tripping the breaker in one does not pause the study-registry phases of the other. Only the phases within the same process actually share a breaker per scope key.

Keyed by scope rather than global: a test driving a deliberate outage for its own course must not silence the pipeline for every other test running at the same moment. Production only ever uses the global key.

Structs§

BreakerSnapshot
What one breaker holds right now, in this process.
BreakerState 🔒

Enums§

ScopeKey
What one breaker counts failures for.

Constants§

FAILURE_RUN_MEMORY 🔒
How long a run of failures that never tripped the breaker is remembered, so a scope never run again leaves the map. Much longer than a worker tick, so a real outage never loses its count.
MAX_CONSECUTIVE_SUOTAR_FAILURES
SUOTAR_COOLDOWN_SECS
TEST_SUOTAR_COOLDOWN_SECS
Playwright’s per-test budget is 100 s, which the production cooldown does not fit inside: a test that trips the breaker deliberately has to be able to watch it recover.

Statics§

BREAKERS 🔒

Functions§

cooldown
is_open
Whether the phases that call the study registry should skip this iteration.
lock 🔒
record_failure
Returns whether this failure opened the breaker.
record_success
snapshot
Reads a breaker without touching it, for the dashboard. Not is_open, which clears an elapsed cooldown as a side effect.