[tkjaer]

Important

Proposal only. This draft makes a client-side circuit-breaker direction concrete for owner review. It does not enable any consumer or commit the owning team to this API, policy, or rollout.

Summary

Adds an opt-in transport circuit-breaker seam to Freno::Client. This can stop repeated Freno connection attempts locally during dependency outages.

The proposal follows the github/traffic-alerts#20837 reconnect-storm investigation, where unavailable Freno backends caused tens of millions of short-lived connection attempts and high GLB CPU.

GitHub Copilot produced this implementation draft from that investigation. The owning team has not reviewed or endorsed it.

Proposed direction

• enable_transport_circuit_breaker(circuit_breaker:) accepts an existing object implementing allow_request?, success, and failure; behavior remains unchanged without it.
• The adapter records exactly one outcome after Faraday retries and response validators finish. Transport errors, invalid responses, HTTP 408, and HTTP 5xx count as failures; other valid HTTP responses prove endpoint reachability.
• An open breaker returns the existing timeout result when raise_on_timeout: false and raises a typed Freno::Error when timeout errors are enabled.
• Applications share one Freno::Client per endpoint. Configure all other Faraday middleware first, then enable the breaker before the connection's first request.

The caller owns breaker construction, policy, instrumentation, force modes, and state scope. The gem adds no breaker dependency or Freno-specific threshold configuration.

Ownership and scope

The submitter is offering this as a concrete direction from the incident investigation, not volunteering to own the final API, rollout, consumer adoption, or ongoing maintenance. Repository owners may take it over, reshape it, reimplement it, or close it.

No consumer is enabled by this draft. If owners pursue it, they would still need to select the breaker implementation and policy, decide whether to start in observe-only mode, choose initial consumers, and own rollout and rollback.

Integration constraints

• Freno::FailOpenClient intentionally converts failures into an allowed result. It can convert both a raised circuit rejection and the failed timeout result returned with raise_on_timeout: false.
• The transport breaker and existing Freno::Throttler breaker represent different operations. Applications must use separate stateful breaker instances and policies, or enable only one seam.
• Sharing a non-thread-safe breaker across multiple clients remains the caller's responsibility.

Validation

The default and Faraday 0/1/2 suites each pass with 69 tests and 239 assertions, including retries, validators, nested requests, invalid statuses, disabled behavior, and concurrent mixed outcomes. Full RuboCop and diff checks pass.

CI pins Ubuntu 24.04 so the supported Ruby 3.0 and 3.1 matrix remains available.

Related

• GLB reconnect-storm analysis
• Require stable Freno recovery #37