Module breaker 

The circuit breakers the study-registry phases share within one worker process: one every such phase stops for, and one only the phase that submits to Sisu stops for, since Sisu timing out on submissions says nothing about the rest of Suotar.

BREAKERS is a process-local static: credit-registrar and suotar-syncer are separate OS processes (see crate::WorkerProcess), each with its own map, so an outage tripping the breaker in one does not pause the study-registry phases of the other. Only the phases within the same process actually share a breaker per scope key.

Keyed by ScopeKey, so a test driving a deliberate outage for its own course does not silence the pipeline for every other test running at the same moment.

Structs

BreakerSnapshot 🔒

What one breaker holds right now, in this process.

BreakerState 🔒

BreakerTrip 🔒

What one failure that tripped or re-tripped a breaker did, for the caller’s log line.

Enums

BreakerTarget 🔒

Which phases one circuit breaker pauses.

Constants

FAILURE_RUN_MEMORY 🔒

How long a run of failures that never tripped the breaker is remembered, so a scope never run again leaves the map. Failures an outage spreads between hour-long timed-out calls must still add up.

MAX_CONSECUTIVE_SUOTAR_FAILURES 🔒

MAX_COOLDOWN_TRIPS 🔒

SUOTAR_COOLDOWN 🔒

The first cooldown; each trip without a success between adds another, up to MAX_COOLDOWN_TRIPS of them.

TEST_SUOTAR_COOLDOWN 🔒

Playwright’s per-test budget is 100 s, which the production cooldown does not fit inside: a test that trips the breaker deliberately has to be able to watch it recover.

Statics

BREAKERS 🔒

REPORTED 🔒

The global breakers as this process last wrote them for the dashboard.

Functions

cooldown 🔒

The first cooldown, which later trips multiply.

is_half_open 🔒

Whether the breaker’s cooldown has ended with no success since: the next iteration is a probe, and sends one item only.

is_open 🔒

Whether the phases target covers should skip this iteration.

is_waiting_to_probe

Whether a breaker with this run of failures and cooldown is past the cooldown without the success that closes it: what is_half_open asks of a live breaker, for one read back from the database.

record_failure 🔒

Returns what this failure did to the breaker, if it tripped or re-tripped it. base_cooldown is the first trip’s; a failure while half-open trips again at once, for longer.

record_success 🔒

Returns the number of times this breaker had tripped, if this success closed it.

snapshot 🔒

Reads a breaker without touching it, for the dashboard. Not is_open, which clears an elapsed cooldown as a side effect.

Type Aliases

BreakerKey 🔒