Skip to content

fix(api): query routes report a failed settings read as the retryable warehouse outage - #1833

Merged
Makisuo merged 1 commit into
mainfrom
fix/query-settings-outage-copy
Oct 25, 2026
Merged

Makisuo merged 1 commit into
mainfrom
fix/query-settings-outage-copy

Conversation

@Makisuo

@Makisuo Makisuo commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Data pages (Services, Traces, Logs, Replays and others) said "Data connection settings temporarily unavailable. Maple couldn't load or save this organization's data connection settings." whenever the org's saved ClickHouse settings could not be read from Postgres. That message belongs to the Settings page. On a read-only data page the user can do nothing with it, and the design-pass judges flagged it repeatedly (#1826 to #1832).

Services showing the settings error

Change

Every warehouse query resolves the org's saved settings first (WarehouseQueryService.resolveRoute). A failed settings read there is now remapped with catchTag to the existing WarehouseUpstreamError: 503, warehouse_unavailable, retryable, "Database is temporarily unavailable". The error is redacted, so the API returns its public message ("The query backend is unreachable. Retry in a few seconds."); the internal detail stays in the cause and the logs.

  • read and raw queries both go through this path. ingest never reads settings.
  • OrgClickHouseSettingsEncryptionError and OrgClickHouseSettingsStoredConfigInvalidError still pass through: they are actionable (reconnect, contact support), not transient.
  • The settings endpoints (/internal/org-clickhouse-settings) keep clickhouse_settings_unavailable and their existing copy.

v2 contract

  • 33 query routes stop declaring OrgClickHouseSettingsPersistenceError on 503. All of them already declared WarehouseUpstreamError, and no route declares the same error twice.
  • The affected route groups are traces, logs, metrics, services, session replays, error issues, sources usage, alert rule test/preview, anomaly timeseries, service map, environments, widget summary and AWS connection ingest.
  • apps/ios/.../openapi.json is regenerated with bun scripts/generate-ios-openapi.ts. Its diff only drops the persistence error from 18 response descriptions.
  • Clients that matched on clickhouse_settings_unavailable from query routes now get warehouse_unavailable. That is the retryable outage code they already handle.

Verification

  • Typecheck passes for domain, query-engine, backend, api, web and ai.
  • Backend: WarehouseQueryService.test.ts gains a test that both compiledQuery and rawSqlQuery turn a settings persistence failure into WarehouseUpstreamError, with the original error as cause. Encryption and invalid-config errors still pass through as before. Together with OrgClickHouseSettingsService.test.ts, that is 96 tests.
  • Settings endpoints: OrgClickHouseSettingsService.test.ts gains a test that get and delete still fail with OrgClickHouseSettingsPersistenceError (the settings page path).
  • API: phase1-resources.http.test.ts (34 tests) checks that GET /v2/error_issues answers 503 warehouse_unavailable, retryable, without leaking the internal message.
  • Domain: openapi.test.ts (274 tests in src/http) checks that the read routes declare WarehouseUpstreamError and not the persistence error.
  • No after screenshot: the local stack has since moved to a different warehouse failure, so it cannot reproduce the settings-read failure. The tests above pin the remap.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Devin Review

Summary by CodeRabbit

  • Bug Fixes
    • Warehouse queries now report organization settings read failures as retryable warehouse-unavailable errors, making them consistent with other temporary warehouse issues.
    • API error responses no longer expose the underlying settings-persistence error for these failures; other warehouse error responses remain unchanged.

… warehouse outage

Every warehouse query resolves the org's saved ClickHouse settings first. When that Postgres read
failed, data pages showed "couldn't load or save this organization's data connection settings",
a settings-page message users cannot act on there. resolveRoute now remaps the persistence error
to WarehouseUpstreamError (503 warehouse_unavailable, retryable); encryption and invalid-config
errors still pass through, and the settings endpoints keep clickhouse_settings_unavailable.

v2 contract: 33 query routes stop declaring OrgClickHouseSettingsPersistenceError (all already
declared WarehouseUpstreamError). The iOS openapi.json is regenerated.
@maple-review-bot

maple-review-bot Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 9/10 · safe to merge
Contained error remap with the cause preserved, pinned by backend, HTTP and OpenAPI tests; only the client-visible error code change raises risk.
no findings · tests covered · risk high −1

A failed org-settings read during route resolution now surfaces as the retryable WarehouseUpstreamError (503 warehouse_unavailable) instead of the settings-page error, and the 33 query routes drop that declaration. The remap preserves the original error as cause and is safe to merge.

  • resolveRoute remaps a failed settings read to WarehouseUpstreamError, original as cause
  • warehouseSettingsRouteHttpErrors drops OrgClickHouseSettingsPersistenceError; read/raw routes declare WarehouseUpstreamError
  • Settings endpoints keep clickhouse_settings_unavailable on their own error lists
What was checked
  • WarehouseUpstreamError is exposure: "redacted" (warehouse-errors.ts:65); the HTTP test asserts the internal message never reaches the body
  • resolveRuntimeConfig has one production caller (WarehouseQueryService.ts:464); settings routes keep explicit persistence-error lists (org-clickhouse-settings.ts:225-290)
  • Neither invalidateRuntimeConfig (Effect.Effect<boolean>, no error) nor isWarehouseWriteReady (wrapped at CloudflareAnalyticsService.ts:3003) can leak the persistence error into a query route

4b34cfd · Updated on every push. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7218582b-73c4-4dca-a042-64f4d8027441
Commits

Reviewing files that changed from the base of the PR and between 8ce7b07 and 4b34cfd.

Files selected for processing: 7
  • apps/api/src/routes/v2/phase1-resources.http.test.ts
  • apps/ios/Packages/MapleAPI/Sources/MapleAPI/openapi.json
  • packages/backend/src/services/org/OrgClickHouseSettingsService.test.ts
  • packages/backend/src/services/warehouse/WarehouseQueryService.test.ts
  • packages/backend/src/services/warehouse/WarehouseQueryService.ts
  • packages/domain/src/http/v2/openapi.test.ts
  • packages/domain/src/http/warehouse-errors.ts

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

Walkthrough

Organization settings persistence failures during warehouse query resolution now surface as WarehouseUpstreamError. The HTTP error contract, tests, and OpenAPI 503 descriptions were updated to reflect this mapping.

Changes: Warehouse settings error handling

Layer / File(s) Summary
Map settings persistence failures
packages/backend/src/services/org/OrgClickHouseSettingsService.test.ts, packages/backend/src/services/warehouse/WarehouseQueryService.ts, packages/backend/src/services/warehouse/WarehouseQueryService.test.ts
WarehouseQueryService wraps settings persistence failures in WarehouseUpstreamError for raw and compiled queries, retaining the original error as the cause. Tests also cover persistence errors from settings get and delete.
Update HTTP error contracts
packages/domain/src/http/warehouse-errors.ts, apps/api/src/routes/v2/phase1-resources.http.test.ts, packages/domain/src/http/v2/openapi.test.ts, apps/ios/Packages/MapleAPI/Sources/MapleAPI/openapi.json
The settings-route error type and HTTP test expectations use WarehouseUpstreamError. The OpenAPI test checks for that 503 error, and 18 endpoint descriptions no longer list OrgClickHouseSettingsPersistenceError.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Possibly related PRs

  • MapleTechLabs/maple#1814: Updates the presentation of OrgClickHouseSettingsPersistenceError, which this change remaps for warehouse query routes.

Suggested reviewers: jeremyfunk

Merge Risk: Minimal · up to 4b34c

The settings-read failure is reported as a retryable warehouse outage without exposing its internal detail. No identified issue prevents merging after normal checks.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: query routes now report failed settings reads as retryable warehouse outages.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 6…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 6 files. (1 skipped: 1 unsupported.)

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@Makisuo
Makisuo merged commit bf88f5f into main Oct 25, 2026
36 checks passed
@Makisuo
Makisuo deleted the fix/query-settings-outage-copy branch October 25, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant