[mcp-analysis] MCP Structural Analysis - 2026-10-08

[github-actions[bot]]

Tested 10 representative GitHub MCP tool calls across 9 toolsets plus the injected workflow-context block. Today's calls totaled 3,794 tokens at an average usefulness of 3.0/5 — best was github_context (5/5, zero MCP overhead) and worst was search_code (1/5, hit a secondary rate limit on the very first attempt, the third such hit in four runs).

Key Findings

• Integrity-policy suppression is now the dominant quality issue, not raw size: list_issues and list_code_scanning_alerts both returned a clean empty array while silently withholding real items (1 issue, 5 alerts) behind an "approved integrity" gate. An agent cannot tell "nothing found" from "results hidden" — this is worse for agentic reliability than any token-bloat problem.
• Base64 server-icon metadata in _meta is a persistent, avoidable tax on every single-item/search call (get_label, search_users, search_code, list_discussions) — consistently 60-85% of response bytes across all 4 days, with zero payload value.
• The large-file guard works as designed: a fields-filtered root directory listing (770 tokens) correctly surfaced that README.md is 49.7KB before any full-file read was attempted, avoiding the overflow failures seen on 2026-10-05 and 2026-10-07.
• The mcpscripts list_workflows wrapper remains the cleanest pagination example — honors perPage, returns a tight envelope with total_count, and carries no icon-metadata bloat (plain CLI JSON passthrough).

Full Structural Analysis Report

Executive Summary

Metric	Value
Tools Analyzed	10
Total Tokens (Today)	3,794
Average Usefulness Rating	3.0/5
Best Rated Tool	github_context: 5/5
Worst Rated Tool	search_code: 1/5

Usefulness Ratings for Agentic Work

Tool	Toolset	Rating	Assessment
github_context	workflow_context	⭐⭐⭐⭐⭐	Excellent — zero-cost identity metadata, immediately actionable
list_pull_requests	pull_requests	⭐⭐⭐⭐⭐	Excellent — flat, compact, nested bot identity, no follow-up needed
get_file_contents	repos	⭐⭐⭐⭐	Good — directory listing correctly gated the large-file read
list_workflows	actions	⭐⭐⭐⭐	Good — clean pagination envelope via mcpscripts wrapper
list_discussions	discussions	⭐⭐⭐	Adequate — usable items array, but totalCount is repo-wide and icon bloat is heavy
list_issues	issues	⭐⭐	Limited — real issue silently withheld by integrity policy
list_code_scanning_alerts	code_security	⭐⭐	Limited — 5 alerts silently withheld, indistinguishable from "none open"
get_label	labels	⭐⭐	Limited — ~140 chars of payload buried in ~4x icon metadata
search_users	users	⭐⭐	Limited — thin fields, needs follow-up get_user call
search_code	search	⭐	Poor — secondary rate limit (429) on first call, no retry-after field

Schema Analysis

Tool	Type	Depth	Key Fields	Notes
github_context	markdown	1	actor, repository, workspace, workflow-run-id	Prompt-injected, no MCP call
get_file_contents	array	1	name, path, size, type	Field-filtered directory listing
list_issues	array	1	(none visible)	Empty + 1 filtered item
list_pull_requests	array	2	number, title, state, user, html_url	Nested bot identity
list_code_scanning_alerts	array	1	(none visible)	Empty + 5 filtered items
list_discussions	object	3	discussions, pageInfo, totalCount, category, user	Repo-wide totalCount
get_label	object	1	name, color, description, id	Small payload, heavy _meta
search_users	object	2	total_count, incomplete_results, items	Thin item fields
search_code	error	1	message	429 rate limit
list_workflows	object	3	total_count, per_page, page, workflows	Clean CLI passthrough

Response Size Analysis (all-time toolset averages, 4 days)

Toolset	Avg Tokens	Tools Tested
code_security	8,481	4
repos	7,000	4
users	902	4
discussions	796	4
labels	712	4
search	707	4
actions	172	4
workflow_context	84	4
pull_requests	69	4
issues	38	4

Note: code_security and repos averages are skewed by two outlier full-text failures (33.7k-token alert dump on 2026-10-06, 13.2k-token README overflow on 2026-10-07) — the large-file guard applied today kept both toolsets well-behaved.

Tool-by-Tool Analysis (today, 2026-10-08)

Tool	Toolset	Tokens	Schema	Rating	Notes
github_context	workflow_context	61	markdown	⭐⭐⭐⭐⭐	Zero-cost identity block
list_pull_requests	pull_requests	75	array	⭐⭐⭐⭐⭐	Clean single dependabot-style PR
get_file_contents	repos	770	array	⭐⭐⭐⭐	Correctly gated the 49.7KB README from a full read
list_workflows	actions	172	object	⭐⭐⭐⭐	total_count=406, perPage honored
list_discussions	discussions	675	object	⭐⭐⭐	totalCount=11,266 repo-wide; icon bloat ~70%
list_issues	issues	70	array	⭐⭐	1 issue withheld by integrity policy
list_code_scanning_alerts	code_security	190	array	⭐⭐	5 alerts withheld by integrity policy
get_label	labels	585	object	⭐⭐	~140 char payload, ~4x icon overhead
search_users	users	608	object	⭐⭐	Thin fields only
search_code	search	588	error	⭐	429 secondary rate limit

30-Day Trend Summary

Metric	Value
Data Points	40 (4 days)
Average Daily Tokens	18,960
Daily Totals	10-05: 17,627 · 10-06: 38,132 · 10-07: 16,285 · 10-08: 3,794
Daily Avg Rating	10-05: 2.9 · 10-06: 3.2 · 10-07: 2.6 · 10-08: 3.0
Average Rating Trend	Stable, fluctuating 2.6-3.2 with no sustained direction

Recommendations

• High-value tools (rating 4-5): github_context, list_pull_requests, get_file_contents (when paired with field-filtered directory pre-check), list_workflows (via mcpscripts wrapper).
• Tools needing improvement: list_issues and list_code_scanning_alerts — both now routinely mask real data behind integrity-policy filtering with no visible count of what was withheld; search_code — needs backoff/retry handling at the caller or server level given 3 rate-limit hits in 4 runs.
• Context-efficient tools (low tokens, high rating): github_context (61 tok, 5★), list_pull_requests (75 tok, 5★), list_workflows (172 tok, 4★).
• Context-heavy tools: historically code_security and repos when the large-file guard isn't applied — both stayed small today specifically because the directory-listing-first pattern was used before any full-content read.

Visualizations

Response Size by Toolset

Usefulness Ratings

Daily Token Trend

Size vs Usefulness

Generated by 🔍 GitHub MCP Structural Analysis · claude · agent · 105.3 AIC · ⊞ 7.9K · ◷

• expires on Oct 9, 2026, 4:02 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by GitHub MCP Structural Analysis.

A newer discussion is available at Discussion #67192.