From 1c4456848d935b0dae7df4a282645885d9d5a760 Mon Sep 17 00:00:00 2001 From: Buildkite Date: Sat, 17 Oct 2026 02:54:44 +0000 Subject: [PATCH] =?UTF-8?q?=F0=9F=A4=96=20sync'ing=20generated=20docs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/ai/models.mdx | 4 +- docs/self-hosted/observability/alerts.mdx | 41 +- docs/self-hosted/observability/dashboards.mdx | 474 ++++++++++-------- 3 files changed, 315 insertions(+), 204 deletions(-) diff --git a/docs/ai/models.mdx b/docs/ai/models.mdx index 2db4b2a80..352af168c 100644 --- a/docs/ai/models.mdx +++ b/docs/ai/models.mdx @@ -8,9 +8,9 @@ This reference lists the models currently used by [Sourcegraph AI features](/ai) | Model | Used by | | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | GPT-5.6 Luna | Change request file grouping, Deep Search compaction, [Code Finder](/api/mcp#code-finder), [Diff Tour](/diff-tour), Agentic Batch Changes workspace analysis, Agentic Batch Changes schema help | -| Claude Sonnet 5 | [Deep Search](/deep-search) | +| Claude Sonnet 5.5 | [Deep Search](/deep-search) | | GPT-6 Luna | Deep Search categorization, Deep Search answer summaries, Deep Search titles, Agentic Batch Changes titles, Agentic Batch Changes categorization, Agentic Batch Changes diagnostics, Coding agent low Finder, Coding agent medium Finder, Coding agent high Finder | -| GPT-5.4 Nano | [Smart Hover](/code-navigation/smart-hover) | +| Claude Haiku 5.5 | [Smart Hover](/code-navigation/smart-hover) | | Gemini 3.5 Flash-Lite | [Query Assist](/code-search/query-assist), Agentic Batch Changes repository search | | Claude Opus 5.5 | [Agentic Batch Changes](/agentic-batch-changes), Agentic Batch Changes compaction | | GPT-6.1 Sol | Coding agent low, Coding agent medium | diff --git a/docs/self-hosted/observability/alerts.mdx b/docs/self-hosted/observability/alerts.mdx index 0d6b41e12..6d742994d 100644 --- a/docs/self-hosted/observability/alerts.mdx +++ b/docs/self-hosted/observability/alerts.mdx @@ -3225,6 +3225,43 @@ Generated query for critical alert: `max((max(src_codeintel_commit_graph_fast_pa
+## worker: codeintel_uploads_writing_duration + +

maximum time a processed SCIP upload has been waiting for the destination writer

+ +**Descriptions** + +- warning worker: 3600s+ maximum time a processed SCIP upload has been waiting for the destination writer for 15m0s +- critical worker: 86400s+ maximum time a processed SCIP upload has been waiting for the destination writer for 15m0s + +**Next steps** + +- Check that the worker runs the codeintel-destination-writer job, and its logs for write or promotion errors. +- Check that the worker can reach the codeintel destination bundle store and the code intelligence database. +- Check the codeintel-db dashboard for resource saturation. +- More help interpreting this metric is available in the [dashboards reference](dashboards#worker-codeintel_uploads_writing_duration). +- **Silence this alert:** If you are aware of this alert and want to silence notifications for it, add the following to your site configuration and set a reminder to re-evaluate the alert: + +```json +"observability.silenceAlerts": [ + "warning_worker_codeintel_uploads_writing_duration", + "critical_worker_codeintel_uploads_writing_duration" +] +``` + +*Managed by the Sourcegraph Code Understanding team.* + +
+Technical details + +Generated query for warning alert: `max((max(src_codeintel_uploads_writing_max_age_seconds)) >= 3600)` + +Generated query for critical alert: `max((max(src_codeintel_uploads_writing_max_age_seconds)) >= 86400)` + +
+ +
+ ## worker: src_repoupdater_max_sync_backoff

time since oldest sync

@@ -7535,9 +7572,9 @@ Generated query for critical alert: `max((max(src_workerutil_queue_duration_seco
Technical details -Generated query for warning alert: `max((sum(increase(src_agent_runs_total{stop_reason="max_input_tokens",workload="batch_change_agent"}[1h]))) > 0)` +Generated query for warning alert: `max((sum(increase(src_agent_runs_total{agent="main",stop_reason="max_input_tokens",workload="batch_change_agent"}[1h]))) > 0)` -Generated query for critical alert: `max((sum(increase(src_agent_runs_total{stop_reason="max_input_tokens",workload="batch_change_agent"}[1h]))) > 3)` +Generated query for critical alert: `max((sum(increase(src_agent_runs_total{agent="main",stop_reason="max_input_tokens",workload="batch_change_agent"}[1h]))) > 3)`
diff --git a/docs/self-hosted/observability/dashboards.mdx b/docs/self-hosted/observability/dashboards.mdx index 07c44a2ff..7b0830bf2 100644 --- a/docs/self-hosted/observability/dashboards.mdx +++ b/docs/self-hosted/observability/dashboards.mdx @@ -14620,6 +14620,56 @@ max(src_codeintel_commit_graph_fast_path_newest_missing_upload_age_seconds{job=~
+### Worker: Code intelligence uploads waiting for the destination writer + +#### worker: codeintel_uploads_writing + +

Processed SCIP uploads waiting for the destination writer

+ +Uploads in the writing state, which are hidden from code navigation until the destination writer has written them into the new code intelligence store. Only write mode new, and dual with CODEINTEL_DESTINATION_READS_ENABLED, leave uploads writing. + +This panel has no related alerts. + +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100500` on your Sourcegraph instance. + +*Managed by the Sourcegraph Code Understanding team.* + +
+Technical details + +Query: + +``` +max(src_codeintel_uploads_writing_total) +``` +
+ +
+ +#### worker: codeintel_uploads_writing_duration + +

Maximum time a processed SCIP upload has been waiting for the destination writer

+ +Measured from when processing of the upload started, so it includes processing time. + +Refer to the [alerts reference](alerts#worker-codeintel_uploads_writing_duration) for 2 alerts related to this panel. + +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100501` on your Sourcegraph instance. + +*Managed by the Sourcegraph Code Understanding team.* + +
+Technical details + +Query: + +``` +max(src_codeintel_uploads_writing_max_age_seconds) +``` +
+ +
+ ### Worker: Codeintel: Repository commit graph updates #### worker: codeintel_commit_graph_processor_total @@ -14628,7 +14678,7 @@ max(src_codeintel_commit_graph_fast_path_newest_missing_upload_age_seconds{job=~ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100500` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100600` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14650,7 +14700,7 @@ sum(increase(src_codeintel_commit_graph_processor_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100501` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100601` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14672,7 +14722,7 @@ sum by (le)(rate(src_codeintel_commit_graph_processor_duration_seconds_bucket{j This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100502` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100602` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14694,7 +14744,7 @@ sum(increase(src_codeintel_commit_graph_processor_errors_total{job=~"^worker.*"} This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100503` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100603` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14718,7 +14768,7 @@ sum(increase(src_codeintel_commit_graph_processor_errors_total{job=~"^worker.*"} This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100600` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100700` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14740,7 +14790,7 @@ sum(increase(src_codeintel_autoindexing_total{op='HandleIndexSchedule',job=~"^wo This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100601` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100701` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14762,7 +14812,7 @@ sum by (le)(rate(src_codeintel_autoindexing_duration_seconds_bucket{op='HandleI This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100602` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100702` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14784,7 +14834,7 @@ sum(increase(src_codeintel_autoindexing_errors_total{op='HandleIndexSchedule',jo This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100603` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100703` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14808,7 +14858,7 @@ sum(increase(src_codeintel_autoindexing_errors_total{op='HandleIndexSchedule',jo This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100700` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100800` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14830,7 +14880,7 @@ sum(increase(src_codeintel_uploads_store_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100701` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100801` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14852,7 +14902,7 @@ sum by (le)(rate(src_codeintel_uploads_store_duration_seconds_bucket{job=~"^wor This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100702` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100802` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14874,7 +14924,7 @@ sum(increase(src_codeintel_uploads_store_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100703` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100803` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14896,7 +14946,7 @@ sum(increase(src_codeintel_uploads_store_errors_total{job=~"^worker.*"}[5m])) / This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100710` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100810` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14918,7 +14968,7 @@ sum by (op)(increase(src_codeintel_uploads_store_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100711` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100811` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14940,7 +14990,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_codeintel_uploads_store_durati This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100712` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100812` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14962,7 +15012,7 @@ sum by (op)(increase(src_codeintel_uploads_store_errors_total{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100713` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100813` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -14986,7 +15036,7 @@ sum by (op)(increase(src_codeintel_uploads_store_errors_total{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100800` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100900` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15008,7 +15058,7 @@ sum(increase(src_codeintel_uploads_lsifstore_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100801` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100901` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15030,7 +15080,7 @@ sum by (le)(rate(src_codeintel_uploads_lsifstore_duration_seconds_bucket{job=~" This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100802` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100902` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15052,7 +15102,7 @@ sum(increase(src_codeintel_uploads_lsifstore_errors_total{job=~"^worker.*"}[5m]) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100803` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100903` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15074,7 +15124,7 @@ sum(increase(src_codeintel_uploads_lsifstore_errors_total{job=~"^worker.*"}[5m]) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100810` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100910` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15096,7 +15146,7 @@ sum by (op)(increase(src_codeintel_uploads_lsifstore_total{job=~"^worker.*"}[5m] This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100811` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100911` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15118,7 +15168,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_codeintel_uploads_lsifstore_du This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100812` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100912` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15140,7 +15190,7 @@ sum by (op)(increase(src_codeintel_uploads_lsifstore_errors_total{job=~"^worker. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100813` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100913` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15164,7 +15214,7 @@ sum by (op)(increase(src_codeintel_uploads_lsifstore_errors_total{job=~"^worker. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100900` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101000` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15186,7 +15236,7 @@ sum(increase(src_gitserver_client_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100901` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101001` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15208,7 +15258,7 @@ sum by (le)(rate(src_gitserver_client_duration_seconds_bucket{job=~"^worker.*"} This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100902` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101002` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15230,7 +15280,7 @@ sum(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100903` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101003` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15252,7 +15302,7 @@ sum(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5m])) / (sum(in This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100910` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101010` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15274,7 +15324,7 @@ sum by (op)(increase(src_gitserver_client_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100911` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101011` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15296,7 +15346,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_gitserver_client_duration_seco This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100912` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101012` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15318,7 +15368,7 @@ sum by (op)(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=100913` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101013` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15342,7 +15392,7 @@ sum by (op)(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5m])) / This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101000` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101100` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15364,7 +15414,7 @@ sum(increase(src_sourcegraphobjects_symbol_indexes_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101001` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101101` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15386,7 +15436,7 @@ sum by (le)(rate(src_sourcegraphobjects_symbol_indexes_duration_seconds_bucket{ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101002` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101102` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15408,7 +15458,7 @@ sum(increase(src_sourcegraphobjects_symbol_indexes_errors_total{job=~"^worker.*" This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101003` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101103` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15430,7 +15480,7 @@ sum(increase(src_sourcegraphobjects_symbol_indexes_errors_total{job=~"^worker.*" This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101010` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101110` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15452,7 +15502,7 @@ sum by (op)(increase(src_sourcegraphobjects_symbol_indexes_total{job=~"^worker.* This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101011` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101111` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15474,7 +15524,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_sourcegraphobjects_symbol_inde This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101012` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101112` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15496,7 +15546,7 @@ sum by (op)(increase(src_sourcegraphobjects_symbol_indexes_errors_total{job=~"^w This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101013` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101113` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -15523,7 +15573,7 @@ If the value is persistently high, make sure all external services have valid to This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101100` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101200` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15545,7 +15595,7 @@ max(timestamp(vector(time()))) - max(src_repoupdater_syncer_sync_last_time) Refer to the [alerts reference](alerts#worker-src_repoupdater_max_sync_backoff) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101101` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101201` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15567,7 +15617,7 @@ max(src_repoupdater_max_sync_backoff) Refer to the [alerts reference](alerts#worker-src_repoupdater_syncer_sync_errors_total) for 2 alerts related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101102` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101202` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15589,7 +15639,7 @@ max by (family) (rate(src_repoupdater_syncer_sync_errors_total{owner!="user",rea Refer to the [alerts reference](alerts#worker-syncer_sync_start) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101110` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101210` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15611,7 +15661,7 @@ max by (family) (rate(src_repoupdater_syncer_start_sync{family="Syncer.SyncExter Refer to the [alerts reference](alerts#worker-syncer_sync_duration) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101111` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101211` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15633,7 +15683,7 @@ histogram_quantile(0.95, max by (le, family, success) (rate(src_repoupdater_sync Refer to the [alerts reference](alerts#worker-source_duration) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101112` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101212` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15655,7 +15705,7 @@ histogram_quantile(0.95, max by (le) (rate(src_repoupdater_source_duration_secon Refer to the [alerts reference](alerts#worker-syncer_synced_repos) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101120` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101220` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15677,7 +15727,7 @@ max(rate(src_repoupdater_syncer_synced_repos_total[1m])) Refer to the [alerts reference](alerts#worker-sourced_repos) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101121` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101221` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15699,7 +15749,7 @@ max(rate(src_repoupdater_source_repos_total[1m])) Refer to the [alerts reference](alerts#worker-sched_auto_fetch) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101130` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101230` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15724,7 +15774,7 @@ This does not indicate anything if there are no user added code hosts. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101131` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101231` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15746,7 +15796,7 @@ max(rate(src_repoupdater_sched_manual_fetch[1m])) Refer to the [alerts reference](alerts#worker-sched_loops) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101140` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101240` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15768,7 +15818,7 @@ max(rate(src_repoupdater_sched_loops[1m])) Refer to the [alerts reference](alerts#worker-src_repoupdater_stale_repos) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101141` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101241` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15790,7 +15840,7 @@ max(src_repoupdater_stale_repos) Refer to the [alerts reference](alerts#worker-sched_error) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101142` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101242` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15814,7 +15864,7 @@ Repository fetches that failed due to an externally classified code-host access This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101143` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101243` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15836,7 +15886,7 @@ max(rate(src_repoupdater_sched_external_error[1m])) Refer to the [alerts reference](alerts#worker-src_repoupdater_cleanup_failed_repos) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101150` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101250` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15862,7 +15912,7 @@ max(src_repoupdater_cleanup_failed_repos) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101200` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101300` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15890,7 +15940,7 @@ were correct. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101201` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101301` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15915,7 +15965,7 @@ Check the reasons for why they were deleted. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101202` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101302` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15940,7 +15990,7 @@ Check the reasons for why they were deleted. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101203` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101303` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15966,7 +16016,7 @@ The number of entitlement usage history events exported as telemetry over the la This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101300` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101400` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -15991,7 +16041,7 @@ If the distribution trends to the maximum bucket, export throughput is at or app This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101301` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101401` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16015,7 +16065,7 @@ The number of already-exported entitlement usage history events pruned over the This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101310` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101410` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16039,7 +16089,7 @@ The number of entitlement usage history events waiting to be exported as telemet This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101311` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101411` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16077,7 +16127,7 @@ The age in seconds of the oldest entitlement usage history event waiting to be e Refer to the [alerts reference](alerts#worker-entitlements_usage_history_telemetry_queue_oldest_age) for 2 alerts related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101320` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101420` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16115,7 +16165,7 @@ Query: Refer to the [alerts reference](alerts#worker-src_repoupdater_external_services_total) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101400` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101500` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16137,7 +16187,7 @@ max(src_repoupdater_external_services_total) Refer to the [alerts reference](alerts#worker-repoupdater_queued_sync_jobs_total) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101410` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101510` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16159,7 +16209,7 @@ max(src_repoupdater_queued_sync_jobs_total) Refer to the [alerts reference](alerts#worker-repoupdater_completed_sync_jobs_total) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101411` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101511` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16181,7 +16231,7 @@ max(src_repoupdater_completed_sync_jobs_total) Refer to the [alerts reference](alerts#worker-repoupdater_errored_sync_jobs_percentage) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101412` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101512` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16203,7 +16253,7 @@ max(src_repoupdater_errored_sync_jobs_percentage) Refer to the [alerts reference](alerts#worker-github_graphql_rate_limit_remaining) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101420` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101520` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16225,7 +16275,7 @@ max by (name) (src_github_rate_limit_remaining_v2{resource="graphql"}) Refer to the [alerts reference](alerts#worker-github_rest_rate_limit_remaining) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101421` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101521` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16247,7 +16297,7 @@ max by (name) (src_github_rate_limit_remaining_v2{resource="rest"}) Refer to the [alerts reference](alerts#worker-github_search_rate_limit_remaining) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101422` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101522` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16271,7 +16321,7 @@ Indicates how long we`re waiting on the rate limit once it has been exceeded This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101430` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101530` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16295,7 +16345,7 @@ Indicates how long we`re waiting on the rate limit once it has been exceeded This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101431` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101531` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16319,7 +16369,7 @@ Indicates how long we`re waiting on the rate limit once it has been exceeded This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101432` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101532` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16341,7 +16391,7 @@ max by(name) (rate(src_github_rate_limit_wait_duration_seconds{resource="search" Refer to the [alerts reference](alerts#worker-gitlab_rest_rate_limit_remaining) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101440` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101540` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16365,7 +16415,7 @@ Indicates how long we`re waiting on the rate limit once it has been exceeded This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101441` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101541` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16389,7 +16439,7 @@ Indicates how long we`re waiting on our internal rate limiter when communicating This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101450` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101550` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16413,7 +16463,7 @@ The rate at which we fail our internal rate limiter. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101451` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101551` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16439,7 +16489,7 @@ Indicates the total number of user permissions sync completed. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101500` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101600` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16463,7 +16513,7 @@ Indicates the number of users permissions syncs completed. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101501` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101601` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16487,7 +16537,7 @@ Indicates the number of permissions syncs done for the first time for the user. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101502` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101602` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16511,7 +16561,7 @@ Indicates the total number of repo permissions sync completed. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101510` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101610` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16535,7 +16585,7 @@ Indicates the number of repos permissions syncs completed. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101511` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101611` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16559,7 +16609,7 @@ Indicates the number of permissions syncs done for the first time for the repo. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101512` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101612` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16583,7 +16633,7 @@ Indicates the max delay between two consecutive permissions sync for a user duri This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101520` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101620` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16607,7 +16657,7 @@ Indicates the max delay between two consecutive permissions sync for a repo duri This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101521` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101621` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16631,7 +16681,7 @@ Indicates the max delay between user creation and their permissions sync This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101530` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101630` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16655,7 +16705,7 @@ Indicates the max delay between repo creation and their permissions sync This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101531` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101631` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16679,7 +16729,7 @@ Indicates the number permissions found during users/repos permissions sync. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101540` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101640` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16703,7 +16753,7 @@ Indicates the average number permissions found during permissions sync per user/ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101541` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101641` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16725,7 +16775,7 @@ avg by (type) (src_repo_perms_syncer_perms_found) Refer to the [alerts reference](alerts#worker-perms_syncer_outdated_perms) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101550` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101650` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16747,7 +16797,7 @@ max by (type) (src_repo_perms_syncer_outdated_perms) Refer to the [alerts reference](alerts#worker-perms_syncer_sync_duration) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101560` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101660` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16773,7 +16823,7 @@ Permissions sync errors are often transient and rarely actionable. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101570` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101670` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16798,7 +16848,7 @@ More about repository permissions synchronization [here](https://sourcegraph.com This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101571` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101671` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16822,7 +16872,7 @@ sum by (type) (increase(src_repo_perms_syncer_schedule_total[1m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101600` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101700` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16844,7 +16894,7 @@ sum(increase(src_gitserver_client_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101601` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101701` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16866,7 +16916,7 @@ sum by (le)(rate(src_gitserver_client_duration_seconds_bucket{job=~"^worker.*"} This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101602` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101702` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16888,7 +16938,7 @@ sum(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101603` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101703` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16910,7 +16960,7 @@ sum(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5m])) / (sum(in This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101610` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101710` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16932,7 +16982,7 @@ sum by (op,scope)(increase(src_gitserver_client_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101611` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101711` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16954,7 +17004,7 @@ histogram_quantile(0.99, sum by (le,op,scope)(rate(src_gitserver_client_duratio This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101612` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101712` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -16976,7 +17026,7 @@ sum by (op,scope)(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5 This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101613` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101713` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17000,7 +17050,7 @@ sum by (op,scope)(increase(src_gitserver_client_errors_total{job=~"^worker.*"}[5 This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101700` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101800` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17022,7 +17072,7 @@ sum(increase(src_gitserver_repositoryservice_client_total{job=~"^worker.*"}[5m]) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101701` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101801` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17044,7 +17094,7 @@ sum by (le)(rate(src_gitserver_repositoryservice_client_duration_seconds_bucket This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101702` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101802` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17066,7 +17116,7 @@ sum(increase(src_gitserver_repositoryservice_client_errors_total{job=~"^worker.* This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101703` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101803` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17088,7 +17138,7 @@ sum(increase(src_gitserver_repositoryservice_client_errors_total{job=~"^worker.* This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101710` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101810` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17110,7 +17160,7 @@ sum by (op,scope)(increase(src_gitserver_repositoryservice_client_total{job=~"^w This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101711` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101811` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17132,7 +17182,7 @@ histogram_quantile(0.99, sum by (le,op,scope)(rate(src_gitserver_repositoryserv This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101712` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101812` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17154,7 +17204,7 @@ sum by (op,scope)(increase(src_gitserver_repositoryservice_client_errors_total{j This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101713` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101813` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17178,7 +17228,7 @@ sum by (op,scope)(increase(src_gitserver_repositoryservice_client_errors_total{j This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101800` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101900` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17200,7 +17250,7 @@ sum(increase(src_batches_dbstore_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101801` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101901` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17222,7 +17272,7 @@ sum by (le)(rate(src_batches_dbstore_duration_seconds_bucket{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101802` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101902` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17244,7 +17294,7 @@ sum(increase(src_batches_dbstore_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101803` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101903` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17266,7 +17316,7 @@ sum(increase(src_batches_dbstore_errors_total{job=~"^worker.*"}[5m])) / (sum(inc This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101810` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101910` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17288,7 +17338,7 @@ sum by (op)(increase(src_batches_dbstore_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101811` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101911` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17310,7 +17360,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_batches_dbstore_duration_secon This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101812` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101912` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17332,7 +17382,7 @@ sum by (op)(increase(src_batches_dbstore_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101813` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101913` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17356,7 +17406,7 @@ sum by (op)(increase(src_batches_dbstore_errors_total{job=~"^worker.*"}[5m])) / This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101900` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102000` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17378,7 +17428,7 @@ sum(increase(src_batches_service_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101901` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102001` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17400,7 +17450,7 @@ sum by (le)(rate(src_batches_service_duration_seconds_bucket{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101902` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102002` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17422,7 +17472,7 @@ sum(increase(src_batches_service_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101903` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102003` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17444,7 +17494,7 @@ sum(increase(src_batches_service_errors_total{job=~"^worker.*"}[5m])) / (sum(inc This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101910` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102010` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17466,7 +17516,7 @@ sum by (op)(increase(src_batches_service_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101911` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102011` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17488,7 +17538,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_batches_service_duration_secon This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101912` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102012` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17510,7 +17560,7 @@ sum by (op)(increase(src_batches_service_errors_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=101913` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102013` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17534,7 +17584,7 @@ sum by (op)(increase(src_batches_service_errors_total{job=~"^worker.*"}[5m])) / This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102000` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102100` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17556,7 +17606,7 @@ sum(src_query_runner_worker_processor_handlers{job=~"^worker.*"}) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102010` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102110` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17578,7 +17628,7 @@ sum(increase(src_query_runner_worker_processor_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102011` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102111` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17600,7 +17650,7 @@ sum by (le)(rate(src_query_runner_worker_processor_duration_seconds_bucket{job= This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102012` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102112` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17622,7 +17672,7 @@ sum(increase(src_query_runner_worker_processor_errors_total{job=~"^worker.*"}[5m This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102013` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102113` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17646,7 +17696,7 @@ sum(increase(src_query_runner_worker_processor_errors_total{job=~"^worker.*"}[5m This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102100` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102200` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17668,7 +17718,7 @@ sum(increase(src_workerutil_dbworker_store_total{domain='insights_query_runner_j This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102101` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102201` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17690,7 +17740,7 @@ sum by (le)(rate(src_workerutil_dbworker_store_duration_seconds_bucket{domain=' This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102102` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102202` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17712,7 +17762,7 @@ sum(increase(src_workerutil_dbworker_store_errors_total{domain='insights_query_r This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102103` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102203` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17734,7 +17784,7 @@ sum(increase(src_workerutil_dbworker_store_errors_total{domain='insights_query_r This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102110` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102210` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17756,7 +17806,7 @@ sum by (op)(increase(src_workerutil_dbworker_store_total{domain='insights_query_ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102111` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102211` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17778,7 +17828,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_workerutil_dbworker_store_dura This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102112` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102212` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17800,7 +17850,7 @@ sum by (op)(increase(src_workerutil_dbworker_store_errors_total{domain='insights This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102113` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102213` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -17827,7 +17877,7 @@ A value of 0 indicates the routine isn`t running currently, it awaits it`s next This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102200` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102300` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -17852,7 +17902,7 @@ A low or zero value could indicate that a routine is stalled or encountering err This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102201` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102301` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -17877,7 +17927,7 @@ A sustained high error rate may indicate a problem with the routine`s configurat Refer to the [alerts reference](alerts#worker-goroutine_error_rate) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102210` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102310` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17902,7 +17952,7 @@ A value above 5% indicates that a significant portion of routine executions are Refer to the [alerts reference](alerts#worker-goroutine_error_percentage) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102211` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102311` on your Sourcegraph instance. *Managed by the Sourcegraph Services team.* @@ -17927,7 +17977,7 @@ Longer durations might indicate increased load or processing time. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102220` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102320` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -17952,7 +18002,7 @@ This represents how long a complete loop iteration takes before sleeping for the This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102221` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102321` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -17977,7 +18027,7 @@ Higher values indicate that tenant processing is taking longer and may affect ov This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102230` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102330` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18002,7 +18052,7 @@ Consistently high values might indicate problematic tenants or inefficient proce This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102231` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102331` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18027,7 +18077,7 @@ Unexpected changes can indicate tenant configuration issues or scaling events. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102240` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102340` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18052,7 +18102,7 @@ A healthy routine should maintain a consistent processing rate. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102241` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102341` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18077,7 +18127,7 @@ Consistent errors indicate problems with specific tenants. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102250` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102350` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18102,7 +18152,7 @@ Values above 5% indicate significant tenant processing problems. This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102251` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102351` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18126,7 +18176,7 @@ Query: This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102300` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102400` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18148,7 +18198,7 @@ sum by (app_name, db_name) (src_pgsql_conns_max_open{app_name="worker"}) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102301` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102401` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18170,7 +18220,7 @@ sum by (app_name, db_name) (src_pgsql_conns_open{app_name="worker"}) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102310` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102410` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18192,7 +18242,7 @@ sum by (app_name, db_name) (src_pgsql_conns_in_use{app_name="worker"}) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102311` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102411` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18214,7 +18264,7 @@ sum by (app_name, db_name) (src_pgsql_conns_idle{app_name="worker"}) Refer to the [alerts reference](alerts#worker-mean_blocked_seconds_per_conn_request) for 2 alerts related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102320` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102420` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18236,7 +18286,7 @@ sum by (app_name, db_name) (increase(src_pgsql_conns_blocked_seconds{app_name="w This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102330` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102430` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18258,7 +18308,7 @@ sum by (app_name, db_name) (increase(src_pgsql_conns_closed_max_idle{app_name="w This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102331` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102431` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18280,7 +18330,7 @@ sum by (app_name, db_name) (increase(src_pgsql_conns_closed_max_lifetime{app_nam This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102332` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102432` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -18304,7 +18354,7 @@ sum by (app_name, db_name) (increase(src_pgsql_conns_closed_max_idle_time{app_na Refer to the [alerts reference](alerts#worker-cpu_usage_percentage) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102400` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102500` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18328,7 +18378,7 @@ An estimate for the active memory in use, which includes anonymous memory, file This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102401` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102501` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18352,7 +18402,7 @@ An estimate for the active memory in use in bytes, which includes anonymous memo This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102402` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102502` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18376,7 +18426,7 @@ The total anonymous memory in use by the application, which includes Go stack an Refer to the [alerts reference](alerts#worker-memory_rss) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102410` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102510` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18400,7 +18450,7 @@ This metric shows the total active file-backed memory currently in use by the ap This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102411` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102511` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18424,7 +18474,7 @@ The kernel usage metric shows the amount of memory used by the kernel on behalf This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102412` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102512` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18458,7 +18508,7 @@ value change independent of deployment events (such as an upgrade), it could ind This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102500` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102600` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18480,7 +18530,7 @@ count by(name) ((time() - container_last_seen{name=~"^worker.*"}) > 60) Refer to the [alerts reference](alerts#worker-container_cpu_usage) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102501` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102601` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18502,7 +18552,7 @@ cadvisor_container_cpu_usage_percentage_total{name=~"^worker.*"} Refer to the [alerts reference](alerts#worker-container_memory_usage) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102502` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102602` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18527,7 +18577,7 @@ When extremely high, this can indicate a resource usage problem, or can cause pr This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102503` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102603` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18551,7 +18601,7 @@ sum by(name) (rate(container_fs_reads_total{name=~"^worker.*"}[1h]) + rate(conta Refer to the [alerts reference](alerts#worker-provisioning_container_cpu_usage_long_term) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102600` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102700` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18573,7 +18623,7 @@ quantile_over_time(0.9, cadvisor_container_cpu_usage_percentage_total{name=~"^wo Refer to the [alerts reference](alerts#worker-provisioning_container_memory_usage_long_term) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102601` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102701` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18597,7 +18647,7 @@ Short CPU spikes are expected while the worker picks up a burst of jobs. Saturat This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102610` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102710` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18621,7 +18671,7 @@ Short memory spikes are expected while the worker processes large jobs. Saturati This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102611` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102711` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18646,7 +18696,7 @@ When it occurs frequently, it is an indicator of underprovisioning. Refer to the [alerts reference](alerts#worker-container_oomkill_events_total) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102612` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102712` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18672,7 +18722,7 @@ A high value here indicates a possible goroutine leak. Refer to the [alerts reference](alerts#worker-go_goroutines) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102700` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102800` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18696,7 +18746,7 @@ Long GC pauses usually mean the worker is under memory pressure. Read this with This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102701` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102801` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18720,7 +18770,7 @@ max by(instance) (go_gc_duration_seconds{job=~".*worker"}) Refer to the [alerts reference](alerts#worker-pods_available_percentage) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102800` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102900` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18744,7 +18794,7 @@ sum by(app) (up{app=~".*worker"}) / count by (app) (up{app=~".*worker"}) * 100 This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102900` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103000` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18766,7 +18816,7 @@ sum(increase(src_workerutil_dbworker_store_total{domain='own_background_worker_s This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102901` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103001` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18788,7 +18838,7 @@ sum by (le)(rate(src_workerutil_dbworker_store_duration_seconds_bucket{domain=' This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102902` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103002` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18810,7 +18860,7 @@ sum(increase(src_workerutil_dbworker_store_errors_total{domain='own_background_w This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102903` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103003` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18832,7 +18882,7 @@ sum(increase(src_workerutil_dbworker_store_errors_total{domain='own_background_w This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102910` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103010` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18854,7 +18904,7 @@ sum by (op)(increase(src_workerutil_dbworker_store_total{domain='own_background_ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102911` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103011` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18876,7 +18926,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_workerutil_dbworker_store_dura This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102912` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103012` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18898,7 +18948,7 @@ sum by (op)(increase(src_workerutil_dbworker_store_errors_total{domain='own_back This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=102913` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103013` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -18922,7 +18972,7 @@ sum by (op)(increase(src_workerutil_dbworker_store_errors_total{domain='own_back This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103000` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103100` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18944,7 +18994,7 @@ sum(src_own_background_worker_processor_handlers{job=~"^worker.*"}) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103010` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103110` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18966,7 +19016,7 @@ sum(increase(src_own_background_worker_processor_total{job=~"^worker.*"}[5m])) This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103011` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103111` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -18988,7 +19038,7 @@ sum by (le)(rate(src_own_background_worker_processor_duration_seconds_bucket{jo This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103012` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103112` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -19010,7 +19060,7 @@ sum(increase(src_own_background_worker_processor_errors_total{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103013` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103113` on your Sourcegraph instance. *Managed by the Sourcegraph Code Plane team.* @@ -19034,7 +19084,7 @@ sum(increase(src_own_background_worker_processor_errors_total{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103100` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103200` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -19056,7 +19106,7 @@ sum by (op)(increase(src_own_background_index_scheduler_total{job=~"^worker.*"}[ This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103101` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103201` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -19078,7 +19128,7 @@ histogram_quantile(0.99, sum by (le,op)(rate(src_own_background_index_scheduler This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103102` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103202` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -19100,7 +19150,7 @@ sum by (op)(increase(src_own_background_index_scheduler_errors_total{job=~"^work This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103103` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103203` on your Sourcegraph instance. *Managed by the Sourcegraph Code Understanding team.* @@ -19126,7 +19176,7 @@ The duration since the configuration client used by the "worker" service last su This panel has no related alerts. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103200` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103300` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -19148,7 +19198,7 @@ src_conf_client_time_since_last_successful_update_seconds{job=~`^worker.*`,insta Refer to the [alerts reference](alerts#worker-worker_site_configuration_duration_since_last_successful_update_by_instance) for 1 alert related to this panel. -To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103201` on your Sourcegraph instance. +To see this panel, visit `/-/debug/grafana/d/worker/worker?viewPanel=103301` on your Sourcegraph instance. *Managed by the Sourcegraph Platform team.* @@ -37132,7 +37182,7 @@ histogram_quantile(0.50, sum(rate(src_deepsearch_turn_duration_seconds_bucket[5m

Agent runs per minute by agent and stop reason

-Why agent loop runs ended. no_tool_calls is a normal answer, awaiting_tool_result is a pause on an external tool result such as ask_user, max_input_tokens means the prompt did not fit even after compaction, and error or context_done are runs that failed or were canceled. Agent is main for the top-level loop or the name of the subagent tool whose loop ran. +Why agent loop runs ended. no_tool_calls is a normal answer, awaiting_tool_result is a pause on an external tool result such as ask_user, max_input_tokens means the prompt did not fit even after compaction, input_budget_summary means a subagent ran low on context and returned a partial summary, and error or context_done are runs that failed or were canceled. Agent is main for the top-level loop or the name of the subagent tool whose loop ran. This panel has no related alerts. @@ -37944,7 +37994,7 @@ sum by (workload, agent, tool) (rate(src_agent_tool_calls_total{workload=~"batch

Agent runs per minute by agent and stop reason

-Why agent loop runs ended. no_tool_calls is a normal answer, awaiting_tool_result is a pause on an external tool result such as ask_user, max_input_tokens means the prompt did not fit even after compaction, and error or context_done are runs that failed or were canceled. Agent is main for the top-level loop or the name of the subagent tool whose loop ran. +Why agent loop runs ended. no_tool_calls is a normal answer, awaiting_tool_result is a pause on an external tool result such as ask_user, max_input_tokens means the prompt did not fit even after compaction, input_budget_summary means a subagent ran low on context and returned a partial summary, and error or context_done are runs that failed or were canceled. Agent is main for the top-level loop or the name of the subagent tool whose loop ran. This panel has no related alerts. @@ -38044,7 +38094,7 @@ histogram_quantile(0.95, sum by (le, workload) (rate(src_agent_compaction_durati

Runs stopped because the conversation no longer fits the context budget over 1h

-Agent runs that ended with stop reason max_input_tokens: the prompt exceeded the budget even after compaction, or the provider rejected an already compacted prompt. Expected to be zero. +Top-level agent runs that ended with stop reason max_input_tokens: the prompt exceeded the budget even after compaction, or the provider rejected an already compacted prompt. Expected to be zero. Subagent runs are excluded because the main agent receives their failure as a tool error and can recover; see the subagent context limits panel. Refer to the [alerts reference](alerts#batch-change-agents-batch_change_agent_runs_over_context_budget) for 2 alerts related to this panel. @@ -38058,7 +38108,7 @@ To see this panel, visit `/-/debug/grafana/d/batch-change-agents/batch-change-ag Query: ``` -sum(increase(src_agent_runs_total{workload="batch_change_agent",stop_reason="max_input_tokens"}[1h])) +sum(increase(src_agent_runs_total{workload="batch_change_agent",agent="main",stop_reason="max_input_tokens"}[1h])) ``` @@ -38088,6 +38138,30 @@ sum(increase(src_agent_compactions_total{workload="batch_change_agent",status!="
+#### batch-change-agents: batch_change_agent_subagent_context_limits + +

Subagent runs that ran low on or out of context over 1h

+ +Subagents do not compact. input_budget_summary means the subagent reached its summarize threshold and returned partial findings; a steady rate means its tasks are too broad for its budget, so consider its prompt or tool page sizes. max_input_tokens means it went past its budget without fitting a final summary, and the main agent received an error asking it to retry with a narrower task. Neither fails the user message on its own. + +This panel has no related alerts. + +To see this panel, visit `/-/debug/grafana/d/batch-change-agents/batch-change-agents?viewPanel=100510` on your Sourcegraph instance. + +*Managed by the Sourcegraph Code Plane team.* + +
+Technical details + +Query: + +``` +sum by (agent, stop_reason) (increase(src_agent_runs_total{workload="batch_change_agent",agent!="main",stop_reason=~"input_budget_summary|max_input_tokens"}[1h])) +``` +
+ +
+ ### Batch Change Agents: Compaction summarizer #### batch-change-agents: batch_change_agent_compaction_p95_summary_input_tokens