Skip to content

fix(accessibility): read any report link directly: scanner links by their run's build id, Workflow Analyzer links - #527

Open
Aaryan430 wants to merge 1 commit into
mainfrom
fix/a11y-report-links
Open

Aaryan430 wants to merge 1 commit into
mainfrom
fix/a11y-report-links

Conversation

@Aaryan430

Copy link
Copy Markdown
Collaborator

Needs browserstack/accessibility#5689 deployed (it is on preprod now). Until it reaches an environment, scanner links there fall back to the scan-list search, so nothing regresses.

Why

A user usually has only a report's dashboard link. Two kinds of link failed:

  • Website Scanner links. A scanner link carries the run's build id (thBuildId), not the scan's ids. The tool therefore searched the newest 15 pages of the scan list (375 scans) for a scan whose link matched.
    • A scan run longer ago than that was "not found". A real link on page 36 of a 3,000-scan account failed after ~17 s.
    • Agents then tried the id as a buildId (rejected), then Ask AI (step limit), then asked the user for ids the dashboard doesn't show.
  • Workflow Analyzer links (/workflow-analyzer/report?ids=…) were not recognised at all.

What

Scanner links are read by the run's build id

  • The link resolves to a new source, scanBuild, with no API call.
  • Its issues come through the scanner route that takes the build id (GET /api/website-scanner/v1/scan_runs/issues?thBuildId=…, from browserstack/accessibility#5689). It uses the same CSV export and polling as the other sources, so the link works however old the scan is.
  • The next-page hint, and the hint for continuing a still-generating export, name the run by a clean copy of its link (reportUrl: "…/site-scanner/projects/<name>/builds?thBuildId=…"). Later pages need no scan ids.
  • Fallback. If the API doesn't know the run by that id, the scan list is searched as before, and the answer says which case it was:
    • an API without the route yet answers a 404 page: "This API does not read a run by its build id yet";
    • an API with the route answers a JSON 404: "not found, or not visible to this account".
  • Processing status. It is read by the scan's ids, which a build id doesn't give, so it isn't checked for these reads. The first page says so in one line.
  • Comparing runs. compareWith from a scanner link still needs the scan's ids, so it still searches the list.

Workflow Analyzer links are read by the report id they carry

  • Both /workflow-analyzer/report?ids=N and the older /reports/report?ids=N work.
  • A link opening several reports asks for one as workflowReportId.

Assisted Test links (/assisted-test/report?ids=…) now say plainly that this tool doesn't read Assisted Test reports, instead of the generic "this link does not name a report".

A buildId no build API knows (when the Automated Tests and App Accessibility APIs both answer 404) is tried as a Website Scanner run's build id before it is refused. The refusal then names all three APIs, or, where the API has no route yet, points at the report link.

The "where ids come from" line in the tool description is left to #524, which changes it (it drops the Ask AI pointer) and would otherwise conflict. docs/TOOLS.md is updated.

Verified

  • npm run build: lint, prettier, 1,616 tests, tsc. New tests in accessibilityReportLinks.test.ts, plus flow tests in accessibilityScanOutput.test.ts and accessibilityFetchPagesAndCompare.test.ts. They cover the route and its URLs, both kinds of 404, the fallback, both Workflow Analyzer link forms, several ids, the Assisted Test message, the buildId fallthrough, and comparing from a link.
  • Production (the new route isn't deployed; it answers a 404 page), through the built code:
Case Result
Recent scanner link "This API does not read a run by its build id yet, so it was found in the scan list instead." Then 125 issues read.
Scanner link on page 36 Fails after ~17 s, saying the build-id read wasn't available and the scan-list search found nothing. Expected until #5689 is on production.
Workflow Analyzer link (?ids=) Reads report 275165 (47 issues)
Automated Tests link Reads the build (92 issues)
Assisted Test link Clear message
Workflow Analyzer link with two ids Clear message
  • Preprod: the route is live. An unauthenticated call gets the API's JSON 401, where production gives a 404 page. The end-to-end preprod run (MCP read by link, next page, same total as the ids route, CSV and JSON modes, unknown → 404, malformed → 422, Automated Tests build id → 404) is scripted and is waiting on a preprod account.
  • Tests use made-up ids.

🤖 Generated with Claude Code

…heir run's build id, Workflow Analyzer links

A user usually has only a report's dashboard link, and two kinds of link failed:

- A Website Scanner link carries the run's build id (thBuildId), not the scan's ids, so
  the tool searched the 15 newest pages of the scan list for it. A scan run more than a
  few hundred scans ago (page 36 of a 3,000-scan account) was "not found", and agents fell
  back to Ask AI or asked the user for ids the dashboard does not show. The link is now
  read by its build id through the scanner issues route that takes one
  (browserstack/accessibility#5689), however old the scan; later pages and a
  still-generating export are named by the same link. If the API does not know the run by
  that id (an API without the route answers a 404 page; one that does, a 404), the scan
  list is searched as before, and the answer says which happened. Comparing runs needs
  the scan's ids, so compareWith from a link still searches the list.
- A Workflow Analyzer link (workflow-analyzer/report?ids=…, or the older reports/report)
  was not recognised. It is now read by the report id it carries; a link opening several
  reports asks for one, and an Assisted Test link says this tool does not read those.

A build id that neither the Automated Tests nor the App Accessibility API knows is tried
as a Website Scanner run's build id before it is refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 27, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Central YAML (base), Organization UI (inherited), Workspace UI (inherited); Enforced overrides: Organization
  • Review profile: ASSERTIVE
  • Plan: Enterprise
  • Run ID: d220e920-5dd7-4dd0-99a5-bf9373a04e5f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant