diff --git a/openspec/changes/feedback-channels/tasks.md b/openspec/changes/feedback-channels/tasks.md index 7b9e7bc0..2bb25511 100644 --- a/openspec/changes/feedback-channels/tasks.md +++ b/openspec/changes/feedback-channels/tasks.md @@ -12,13 +12,13 @@ ## 3. Recipes and the agent index -- [ ] 3.1 `git mv packages/cli/src/agent/feedback.md packages/cli/src/agent/rule-feedback.md`, update its header/topic name and its payload instructions to include `kind: "rule"`, and repoint `feedback-invite.md` at `agent rule-feedback`; verify `pnpm cli agent rule-feedback` (after `pnpm build`) opens with `# Topic: rule-feedback` and embeds the rule schema -- [ ] 3.2 Write the new `packages/cli/src/agent/feedback.md` (general channel: user's words as `verbatim`, agent-drafted `context`, redaction, show-payload-and-wait-for-yes, telemetry-off relay with the issues URL); verify `pnpm cli agent feedback` embeds only the `general` schema -- [ ] 3.3 Write `packages/cli/src/agent/bug-report.md` (agent drafts from the session, asks only for gaps, redaction, show-payload-and-wait-for-yes, no version-information ask, telemetry-off relay); verify `pnpm cli agent bug-report` embeds only the `bug` schema and contains no version key -- [ ] 3.4 Wire `TOPIC_INPUT_SCHEMAS` in `prompts/recipes.ts` (`rule-feedback` → rule branch, `feedback` → general, `bug-report` → bug) and add `rule-feedback` and `bug-report` to `INTERNAL_TOPICS` in `prompts/index.ts`; verify `prompts.test.ts` and `feedback-recipes.test.ts` pass after updating them for the new topic names -- [ ] 3.5 Add a `FEEDBACK_TOPICS` section to the `taskless agent` index in `commands/agent.ts` listing `feedback` and `bug-report`, keep the `feedback` command in `UNLISTED_COMMANDS` with its comment rewritten per design.md; verify a test that the index lists both and does not list `rule-feedback` -- [ ] 3.6 Update the `feedback` command's `meta.description` and the comment above `feedbackCommand` to describe three channels; verify `pnpm cli feedback --help` reads correctly -- [ ] 3.7 Run Vale over the three recipes and fix findings; verify `pnpm lint` reports no recipe prose errors +- [x] 3.1 `git mv packages/cli/src/agent/feedback.md packages/cli/src/agent/rule-feedback.md`, update its header/topic name and its payload instructions to include `kind: "rule"`, and repoint `feedback-invite.md` at `agent rule-feedback`; verify `pnpm cli agent rule-feedback` (after `pnpm build`) opens with `# Topic: rule-feedback` and embeds the rule schema +- [x] 3.2 Write the new `packages/cli/src/agent/feedback.md` (general channel: user's words as `verbatim`, agent-drafted `context`, redaction, show-payload-and-wait-for-yes, telemetry-off relay with the issues URL); verify `pnpm cli agent feedback` embeds only the `general` schema +- [x] 3.3 Write `packages/cli/src/agent/bug-report.md` (agent drafts from the session, asks only for gaps, redaction, show-payload-and-wait-for-yes, no version-information ask, telemetry-off relay); verify `pnpm cli agent bug-report` embeds only the `bug` schema and contains no version key +- [x] 3.4 Wire `TOPIC_INPUT_SCHEMAS` in `prompts/recipes.ts` (`rule-feedback` → rule branch, `feedback` → general, `bug-report` → bug) and add `rule-feedback` and `bug-report` to `INTERNAL_TOPICS` in `prompts/index.ts`; verify `prompts.test.ts` and `feedback-recipes.test.ts` pass after updating them for the new topic names +- [x] 3.5 Add a `FEEDBACK_TOPICS` section to the `taskless agent` index in `commands/agent.ts` listing `feedback` and `bug-report`, keep the `feedback` command in `UNLISTED_COMMANDS` with its comment rewritten per design.md; verify a test that the index lists both and does not list `rule-feedback` +- [x] 3.6 Update the `feedback` command's `meta.description` and the comment above `feedbackCommand` to describe three channels; verify `pnpm cli feedback --help` reads correctly +- [x] 3.7 Run Vale over the three recipes and fix findings; verify `pnpm lint` reports no recipe prose errors ## 4. Skill, spec purpose, and changeset diff --git a/packages/cli/src/agent/bug-report.md b/packages/cli/src/agent/bug-report.md new file mode 100644 index 00000000..596534ca --- /dev/null +++ b/packages/cli/src/agent/bug-report.md @@ -0,0 +1,114 @@ +# Topic: bug-report (CLI v%(CLI_VERSION)s / topic v1) + +## You are here +This is `bug-report`. It helps you report a bug in Taskless to the +Taskless team on the user's behalf, without a GitHub account: the CLI +or a recipe did something wrong, failed, or produced a result that does +not match what it promised. The user started this; no invite did. +For an opinion or a wish rather than a defect, `%(TASKLESS_CLI)s agent feedback` +is the better recipe. + +## Goal +Produce one JSON payload that a maintainer could act on without asking +a follow-up question: what was being done, what should have happened, +what did happen. Show it to the user, send it only on their yes, and +delete the file. + +## Preconditions +- The user asked to report a Taskless bug, in this conversation. +- The agent can write a file and run a shell command. +- No auth required. + +## Steps + +1. **Draft every answer from the session.** You usually saw the bug + happen, so you already hold most of the report: + - `summary`: one line naming the command or recipe and the defect. + `check exits 0 when a rule file fails to parse`, not `check is + broken`. + - `trying`: what the user was trying to do, and the exact command + you ran, if there was one. + - `expected`: what should have happened, and where that expectation + came from (a recipe, the docs, `--help`) when you know. + - `actual`: what happened instead. Quote the error message or the + output that shows it, trimmed to the lines that matter. + - `context`: anything else that would help fix it, such as the steps + to reproduce, whether it happens every time, or a workaround you + found. Omit the key when there is nothing to add. + +2. **Ask the user only for what the session does not show.** If you + did not see the bug yourself, ask what they ran and what happened. + One question, covering everything missing, not one per field. + +3. **Leave version information out.** The CLI adds its version, the + installed scaffold version, the platform, and the Node.js version to + the report itself. There is no key for it. + +4. **Keep it shareable.** The payload leaves this machine. Leave out + secrets, tokens, credentials, absolute paths, and any source code the + user has not chosen to share. Replace a path with its project-relative + form, and a snippet of their code with a description of its shape. + An error message or CLI output is fine once it is clean of those. + +5. **Write the payload** to `.taskless/.tmp-feedback.json`, matching the + input schema below, with `"kind": "bug"`. Use the keys exactly as + given; the CLI maps them to the survey's own question identifiers. + +6. **Show it, and wait for a yes.** Put every key in the chat, labelled + and written out in full, exactly as it will be sent, and say that + the CLI will add version information. Ask whether to send it. + - **Yes.** Go to step 7. + - **Corrections.** Apply them, rewrite the file, show the payload in + full again, and ask again. Send only on the user's go-ahead. + - **No.** Delete the file and carry on with the user's task. Nothing + is sent. + +7. **Send.** Run: + ``` + %(TASKLESS_CLI)s feedback send --from .taskless/.tmp-feedback.json --json + ``` + On success the command prints a thank-you. If it says telemetry is + disabled, nothing was sent: tell the user that, and that they can + file the bug at https://github.com/taskless/cli/issues instead, using + the payload you showed them as the issue body. + +8. **Clean up.** Delete `.taskless/.tmp-feedback.json` whether the call + succeeded or failed. + +9. **Return to the user's task.** Thank them in one line. If you found a + workaround, offer it, then carry on. + +## Input schema + +The `--from` JSON file conforms to: + +```json +%(INPUT_SCHEMA)s +``` + +`kind` is always `bug`. `summary`, `trying`, `expected`, and `actual` +are required. `context` is optional, and an omitted key is how it is +left out, not an empty string. + +## Important Notes + +- Do NOT send before the user has seen the payload and said yes. +- Do NOT guess at the cause in `actual`. Say what happened; a theory of + why belongs in `context`, labelled as one. +- Do NOT run `%(TASKLESS_CLI)s feedback dismiss` here. It answers the + survey invite, which this is not. +- The event carries the same anonymous or logged-in identity as the + CLI's other telemetry. If the user asks, say so plainly. + +## Errors + +With `--json`, failures emit `{ ok: false, code, message }`: + +| code | meaning | fix | +|-----------------|---------------------------------------------|-------------------------------------------------------| +| `INVALID_INPUT` | `--from` missing, unreadable, or failed validation | the message names the field; fix the payload and retry | + +## See Also + +- `%(TASKLESS_CLI)s agent feedback`: for an opinion or a wish rather than a defect +- `%(TASKLESS_CLI)s agent`: the topic index, if you arrived here by mistake diff --git a/packages/cli/src/agent/feedback-invite.md b/packages/cli/src/agent/feedback-invite.md index 7728aa8f..3b8b37b0 100644 --- a/packages/cli/src/agent/feedback-invite.md +++ b/packages/cli/src/agent/feedback-invite.md @@ -1,4 +1,4 @@ -# Topic: feedback-invite (CLI v%(CLI_VERSION)s / topic v3) +# Topic: feedback-invite (CLI v%(CLI_VERSION)s / topic v4) ## Before you finish: one question for the user @@ -12,16 +12,16 @@ and only then, put this to the user exactly once, in these words: Then act on the reply: - **They said `review`**, alone or alongside their own words. Fetch - `%(TASKLESS_CLI)s agent feedback` and follow it in review mode: the + `%(TASKLESS_CLI)s agent rule-feedback` and follow it in review mode: the words they wrote besides `review`, if any, are `verbatim`, and you show them every answer in the chat, and take their corrections, before anything is sent. Check for this first: a reply that asks for a review and also gives feedback is a review, not plain feedback. - **They gave feedback** without asking for a review. Fetch - `%(TASKLESS_CLI)s agent feedback` and follow it. It takes their words as + `%(TASKLESS_CLI)s agent rule-feedback` and follow it. It takes their words as they are and asks you, not them, for the rest. - **They said `skip`, said nothing, or replied about something else.** - Fetch `%(TASKLESS_CLI)s agent feedback` all the same and follow it with + Fetch `%(TASKLESS_CLI)s agent rule-feedback` all the same and follow it with no `verbatim`. The rest of the survey is your account of the session, and it is worth sending on its own. An unrelated reply is not feedback, and it is not a reason to ask again. diff --git a/packages/cli/src/agent/feedback.md b/packages/cli/src/agent/feedback.md index 13b5b819..5474fcaa 100644 --- a/packages/cli/src/agent/feedback.md +++ b/packages/cli/src/agent/feedback.md @@ -1,123 +1,71 @@ -# Topic: feedback (CLI v%(CLI_VERSION)s / topic v3) +# Topic: feedback (CLI v%(CLI_VERSION)s / topic v1) ## You are here -This is `feedback`. It helps you turn what a user just said about -Taskless into a survey response the CLI can send, and send it. -You reach it from the invite at the end of an authoring or onboarding -recipe, whether the user gave you their words, said `skip`, or said -`review`. -If that is not why you are reading this, re-run `%(TASKLESS_CLI)s agent` and -find the topic you meant. +This is `feedback`. It helps you send the Taskless team feedback the user +asked to give: something that works well, something that does not, or +something they wish Taskless did. The user started this; no invite did. +If the user is reporting something broken, `%(TASKLESS_CLI)s agent bug-report` +asks the questions a fix needs, and is the better recipe. +If you arrived here from the survey invite at the end of an authoring or +onboarding recipe, you want `%(TASKLESS_CLI)s agent rule-feedback` instead. ## Goal -Produce one JSON payload that answers the survey, write it to -`.taskless/.tmp-feedback.json`, send it with `feedback send`, and delete -the file. The whole thing is one short exchange with the user and a few -sentences from you; it is not an interview. The one exception is a user -who replied `review`: they get to see the answers, and correct them, -before they are sent. +Produce one JSON payload carrying the user's feedback in their own words +and, if it helps, your account of what led to it. Show it to the user, +send it only on their yes, and delete the file. No GitHub account is +needed. ## Preconditions -- The invite was put to the user and they replied, or did not. Their - words, if any, are the only input you take from them. -- The user did not ask you to send nothing. If they did, this is the - wrong recipe: run `%(TASKLESS_CLI)s feedback dismiss` and continue with what - they asked for. +- The user asked to give Taskless feedback, in this conversation. - The agent can write a file and run a shell command. - No auth required. -## You are the respondent - -The survey is addressed to you, the agent, not to the user. One answer is -the user's words, if they gave any, and you record them verbatim. The -other six are your own account of the session you just ran: what kind -of rule it was for, whether they got it, what went well, what did not, -what you are running in, and which rule has earned its keep. You -already know all of that. -Do not put the survey's questions to the user one by one. - ## Steps -1. **Take the user's reply as it is, if there is one.** Whatever they - wrote after the invite is `verbatim`. If the reply asked for a - `review`, that word is the request, not their feedback: leave it out - of `verbatim`, keep everything else they wrote, and remember to show - the answers at step 4. Do not paraphrase, shorten, or +1. **Get the feedback in the user's words.** If they already said what + they want to tell Taskless, that is `verbatim`. If they only said they + have feedback, ask once what it is. Do not paraphrase, shorten, or tidy it. If they wrote several messages, join them in order with a - blank line between. If they said `skip`, said nothing, or replied - about something else, omit `verbatim` and go on: the rest of the - survey is yours to answer. - -2. **Fill the rest from the session.** Ask the user nothing further; - the invite already asked for their time once. A `review` does not - change that: you still answer every key yourself, and the user sees - your answers rather than being asked for them. - - `ruleKind`: the engine and what the rule was for, in a phrase - (`ast-grep, forbid eval in TypeScript`; `vale, no hedging in - docs`; `runtime, env var must be set`). If the recipe was - `onboard`, write `none (onboarding)`. This is the one required - answer. - - `completed`: `Yes`, `No`, or `Unknown`. Success is binary here. A - rule that verifies and the user accepted is `Yes`; a rule the user - abandoned or that never verified is `No`; if the session ended - before you could tell, `Unknown`. There is no partial. - - `workedWell`: the steps of the interaction with Taskless that went - smoothly. Omit the key if nothing stands out. - - `needsImprovement`: the steps that cost time, needed a retry, or - that you had to work around. Be specific: name the command, the - field, or the message. Omit the key if nothing stands out. - - `agents`: the agent you are, and any agent framework you can see - the project using, by product name. Only open-source, publicly - available software belongs here; omit the key rather than name an - internal or proprietary tool. - - `mostValuableRule`: of the rules under `.taskless/rules/`, the one - doing the most for this team and why, if the session gave you a - view of that. Most sessions will not have; omit the key then, and - do not ask the user for it. - - Keep your own answers to a few sentences each. The people reading - them want the shape of the friction, not a transcript. - -3. **Write the payload** to `.taskless/.tmp-feedback.json`, matching the - input schema below. Use the human keys exactly as given; the CLI maps - them to the survey's own question identifiers, and a payload carrying - a `$survey_` key is not what it expects. - -4. **Show it first, if the user asked for a `review`.** Otherwise go - straight to step 5. Put every answer in the payload in the chat, - one per line, labelled with its key and written out in full, exactly - as it will be sent; name the keys you omitted, so the user can see - what is not being said too. Then ask whether they would like - anything corrected before you send it. - - **Nothing to correct, or a go-ahead.** Send the payload as shown. - - **Corrections.** Apply them as the user gives them and rewrite the - file. A correction to your own answer replaces it with what the - user said; a correction to `verbatim` is theirs to make. A request - to drop an answer omits its key, except `ruleKind`, which is - required: say so in one line and keep the user's preferred wording - for it. Then show the corrected payload in full, the same way, and - ask again. Repeat until the user is satisfied; send only on their - go-ahead, never on your own judgement that the corrections are - done. - - **They decide not to send it.** That is a refusal, and it is - honoured: delete the file, run `%(TASKLESS_CLI)s feedback dismiss`, and - carry on with the user's task. - -5. **Send.** Run: + blank line between. + +2. **Add context, if the session has any.** `context` is your account + of what led to the feedback: the command or recipe involved, what + happened, what the user was trying to do. A few sentences. Omit the + key when the feedback stands on its own. + +3. **Keep it shareable.** The payload leaves this machine. Leave out + secrets, tokens, credentials, absolute paths, and any source code the + user has not chosen to share. Describe instead of quoting: "a + TypeScript file in the API layer", not its contents. This applies to + your `context`; the user's own words are theirs to choose, so if + `verbatim` contains something that looks like a secret, point it out + at step 5 rather than removing it yourself. + +4. **Write the payload** to `.taskless/.tmp-feedback.json`, matching the + input schema below, with `"kind": "general"`. Use the keys exactly as + given; the CLI maps them to the survey's own question identifiers. + +5. **Show it, and wait for a yes.** Put every key in the chat, labelled + and written out in full, exactly as it will be sent. Ask whether to + send it. + - **Yes.** Go to step 6. + - **Corrections.** Apply them, rewrite the file, show the payload in + full again, and ask again. Send only on the user's go-ahead. + - **No.** Delete the file and carry on with the user's task. Nothing + is sent, and there is nothing to dismiss. + +6. **Send.** Run: ``` %(TASKLESS_CLI)s feedback send --from .taskless/.tmp-feedback.json --json ``` - Under `--json`, a failure is `{ ok: false, code, message }`; see the - table below. On success the command prints a thank-you. + On success the command prints a thank-you. If it says telemetry is + disabled, nothing was sent: tell the user that, and that they can + reach the team at https://github.com/taskless/cli/issues instead. -6. **Clean up.** Delete `.taskless/.tmp-feedback.json` whether the call - succeeded or failed. `.taskless/.gitignore` already ignores it, so a - forgotten file is a stray rather than a commit, but leave nothing - behind. +7. **Clean up.** Delete `.taskless/.tmp-feedback.json` whether the call + succeeded or failed. -7. **Return to the user's task.** Thank them in one line and carry on. - Do not ask for more, and do not run this recipe a second time in the - same session. +8. **Return to the user's task.** Thank them in one line and carry on. ## Input schema @@ -127,24 +75,19 @@ The `--from` JSON file conforms to: %(INPUT_SCHEMA)s ``` -`ruleKind` is required. Every other key is optional, and an optional -answer you have nothing for is an omitted key rather than an empty -string. +`kind` is always `general`, and `verbatim` is required. `context` is +optional, and an omitted key is how it is left out, not an empty string. ## Important Notes +- Do NOT send before the user has seen the payload and said yes. They + asked to give feedback, not for you to decide what it says. - Do NOT edit the user's words. `verbatim` is the one answer that is - theirs, and its value to the people reading it is that it is theirs. -- Do NOT invent a follow-up interview. The invite asked once, and this - recipe asks nothing beyond the correction rounds a `review` earns. - An answer you cannot give is an omitted key, or `Unknown` for - `completed`. -- Do NOT send before the user has seen the answers when they asked for - a `review`. The point of the review is that nothing leaves without - their look at it. -- If telemetry is disabled in this environment the command says so and - exits 0 with nothing sent. That is the expected outcome there, not an - error to retry. + theirs. +- Do NOT run `%(TASKLESS_CLI)s feedback dismiss` here. It answers the + survey invite, which this is not. +- The event carries the same anonymous or logged-in identity as the + CLI's other telemetry. If the user asks, say so plainly. ## Errors @@ -156,5 +99,5 @@ With `--json`, failures emit `{ ok: false, code, message }`: ## See Also -- `%(TASKLESS_CLI)s feedback dismiss`: what to run when the user asked for nothing to be sent +- `%(TASKLESS_CLI)s agent bug-report`: for something that is broken - `%(TASKLESS_CLI)s agent`: the topic index, if you arrived here by mistake diff --git a/packages/cli/src/agent/rule-feedback.md b/packages/cli/src/agent/rule-feedback.md new file mode 100644 index 00000000..50808d23 --- /dev/null +++ b/packages/cli/src/agent/rule-feedback.md @@ -0,0 +1,165 @@ +# Topic: rule-feedback (CLI v%(CLI_VERSION)s / topic v4) + +## You are here +This is `rule-feedback`. It helps you turn what a user just said about +Taskless into a survey response the CLI can send, and send it. +You reach it from the invite at the end of an authoring or onboarding +recipe, whether the user gave you their words, said `skip`, or said +`review`. +If that is not why you are reading this, re-run `%(TASKLESS_CLI)s agent` and +find the topic you meant. A user who asked, on their own, to send +feedback wants `%(TASKLESS_CLI)s agent feedback`; one reporting a bug +wants `%(TASKLESS_CLI)s agent bug-report`. + +## Goal +Produce one JSON payload that answers the survey, write it to +`.taskless/.tmp-feedback.json`, send it with `feedback send`, and delete +the file. The whole thing is one short exchange with the user and a few +sentences from you; it is not an interview. The one exception is a user +who replied `review`: they get to see the answers, and correct them, +before they are sent. + +## Preconditions +- The invite was put to the user and they replied, or did not. Their + words, if any, are the only input you take from them. +- The user did not ask you to send nothing. If they did, this is the + wrong recipe: run `%(TASKLESS_CLI)s feedback dismiss` and continue with what + they asked for. +- The agent can write a file and run a shell command. +- No auth required. + +## You are the respondent + +The survey is addressed to you, the agent, not to the user. One answer is +the user's words, if they gave any, and you record them verbatim. The +other six are your own account of the session you just ran: what kind +of rule it was for, whether they got it, what went well, what did not, +what you are running in, and which rule has earned its keep. You +already know all of that. +Do not put the survey's questions to the user one by one. + +## Steps + +1. **Take the user's reply as it is, if there is one.** Whatever they + wrote after the invite is `verbatim`. If the reply asked for a + `review`, that word is the request, not their feedback: leave it out + of `verbatim`, keep everything else they wrote, and remember to show + the answers at step 4. Do not paraphrase, shorten, or + tidy it. If they wrote several messages, join them in order with a + blank line between. If they said `skip`, said nothing, or replied + about something else, omit `verbatim` and go on: the rest of the + survey is yours to answer. + +2. **Fill the rest from the session.** Ask the user nothing further; + the invite already asked for their time once. A `review` does not + change that: you still answer every key yourself, and the user sees + your answers rather than being asked for them. + - `ruleKind`: the engine and what the rule was for, in a phrase + (`ast-grep, forbid eval in TypeScript`; `vale, no hedging in + docs`; `runtime, env var must be set`). If the recipe was + `onboard`, write `none (onboarding)`. This is the one required + answer. + - `completed`: `Yes`, `No`, or `Unknown`. Success is binary here. A + rule that verifies and the user accepted is `Yes`; a rule the user + abandoned or that never verified is `No`; if the session ended + before you could tell, `Unknown`. There is no partial. + - `workedWell`: the steps of the interaction with Taskless that went + smoothly. Omit the key if nothing stands out. + - `needsImprovement`: the steps that cost time, needed a retry, or + that you had to work around. Be specific: name the command, the + field, or the message. Omit the key if nothing stands out. + - `agents`: the agent you are, and any agent framework you can see + the project using, by product name. Only open-source, publicly + available software belongs here; omit the key rather than name an + internal or proprietary tool. + - `mostValuableRule`: of the rules under `.taskless/rules/`, the one + doing the most for this team and why, if the session gave you a + view of that. Most sessions will not have; omit the key then, and + do not ask the user for it. + + Keep your own answers to a few sentences each. The people reading + them want the shape of the friction, not a transcript. + +3. **Write the payload** to `.taskless/.tmp-feedback.json`, matching the + input schema below, with `"kind": "rule"`. Use the human keys exactly + as given; the CLI maps + them to the survey's own question identifiers, and a payload carrying + a `$survey_` key is not what it expects. + +4. **Show it first, if the user asked for a `review`.** Otherwise go + straight to step 5. Put every answer in the payload in the chat, + one per line, labelled with its key and written out in full, exactly + as it will be sent; name the keys you omitted, so the user can see + what is not being said too. Then ask whether they would like + anything corrected before you send it. + - **Nothing to correct, or a go-ahead.** Send the payload as shown. + - **Corrections.** Apply them as the user gives them and rewrite the + file. A correction to your own answer replaces it with what the + user said; a correction to `verbatim` is theirs to make. A request + to drop an answer omits its key, except `ruleKind`, which is + required: say so in one line and keep the user's preferred wording + for it. Then show the corrected payload in full, the same way, and + ask again. Repeat until the user is satisfied; send only on their + go-ahead, never on your own judgement that the corrections are + done. + - **They decide not to send it.** That is a refusal, and it is + honoured: delete the file, run `%(TASKLESS_CLI)s feedback dismiss`, and + carry on with the user's task. + +5. **Send.** Run: + ``` + %(TASKLESS_CLI)s feedback send --from .taskless/.tmp-feedback.json --json + ``` + Under `--json`, a failure is `{ ok: false, code, message }`; see the + table below. On success the command prints a thank-you. + +6. **Clean up.** Delete `.taskless/.tmp-feedback.json` whether the call + succeeded or failed. `.taskless/.gitignore` already ignores it, so a + forgotten file is a stray rather than a commit, but leave nothing + behind. + +7. **Return to the user's task.** Thank them in one line and carry on. + Do not ask for more, and do not run this recipe a second time in the + same session. + +## Input schema + +The `--from` JSON file conforms to: + +```json +%(INPUT_SCHEMA)s +``` + +`kind` is always `rule`, and `ruleKind` is required. Every other key is +optional, and an optional +answer you have nothing for is an omitted key rather than an empty +string. + +## Important Notes + +- Do NOT edit the user's words. `verbatim` is the one answer that is + theirs, and its value to the people reading it is that it is theirs. +- Do NOT invent a follow-up interview. The invite asked once, and this + recipe asks nothing beyond the correction rounds a `review` earns. + An answer you cannot give is an omitted key, or `Unknown` for + `completed`. +- Do NOT send before the user has seen the answers when they asked for + a `review`. The point of the review is that nothing leaves without + their look at it. +- If telemetry is disabled in this environment the command says so and + exits 0 with nothing sent. That is the expected outcome there, not an + error to retry. Tell the user nothing was sent, and that they can + still reach the team at https://github.com/taskless/cli/issues. + +## Errors + +With `--json`, failures emit `{ ok: false, code, message }`: + +| code | meaning | fix | +|-----------------|---------------------------------------------|-------------------------------------------------------| +| `INVALID_INPUT` | `--from` missing, unreadable, or failed validation | the message names the field; fix the payload and retry | + +## See Also + +- `%(TASKLESS_CLI)s feedback dismiss`: what to run when the user asked for nothing to be sent +- `%(TASKLESS_CLI)s agent`: the topic index, if you arrived here by mistake diff --git a/packages/cli/src/commands/agent.ts b/packages/cli/src/commands/agent.ts index 32f99075..8cc012e3 100644 --- a/packages/cli/src/commands/agent.ts +++ b/packages/cli/src/commands/agent.ts @@ -35,6 +35,20 @@ const RECIPE_TOPICS: ReadonlyArray<[string, string]> = [ ["create-remote-rule", "Generate a rule via the Taskless service (login)"], ]; +/** + * The feedback channels a user asks for, listed under their own heading: + * neither is an authoring recipe, and `bug-report` is not a command. The + * invited survey's `rule-feedback` is absent on purpose; see the note on + * `feedback` in `UNLISTED_COMMANDS`. + */ +const FEEDBACK_TOPICS: ReadonlyArray<[string, string]> = [ + [ + "feedback", + "Send the Taskless team feedback (user asks; no GitHub account)", + ], + ["bug-report", "Report a Taskless bug (user asks; no GitHub account)"], +]; + async function unwrap(resolvable: Resolvable): Promise { if (typeof resolvable === "function") { return (resolvable as () => T | Promise)(); @@ -67,11 +81,13 @@ async function resolveDescription( * verb gave it discoverability it does not want yet, and this is the cost of * that choice, paid here. * - * - `feedback` is reached only through the survey invite a served recipe - * carries. Listed, it would invite an agent to run it unprompted, and a - * `survey sent` with no invite behind it is noise in the funnel. When a - * general feedback channel exists this surface folds into it, and that is - * the point to reconsider listing. + * - `feedback` the COMMAND is reached through a recipe, never directly. The + * index lists the `feedback` and `bug-report` recipes instead (see + * `FEEDBACK_TOPICS`), because they are what show the user the payload and + * wait for a yes; listing the command beside them would offer a path that + * skips the consent. The invited survey's `rule-feedback` recipe is listed + * nowhere: reached unprompted, it would put a `survey sent` with no invite + * behind it into the funnel. * * Absence from this list is what puts a command in the index, so adding one is * a decision someone made rather than a step they forgot. @@ -148,7 +164,8 @@ export function createAgentCommand(subCommands: SubCommandsDef) { // sections line up. const maxLength = Math.max( ...entries.map(([name]) => name.length), - ...RECIPE_TOPICS.map(([name]) => name.length) + ...RECIPE_TOPICS.map(([name]) => name.length), + ...FEEDBACK_TOPICS.map(([name]) => name.length) ); for (const [name, description] of entries) { console.log(` ${name.padEnd(maxLength + 2)}${description}`); @@ -159,6 +176,11 @@ export function createAgentCommand(subCommands: SubCommandsDef) { console.log(` ${name.padEnd(maxLength + 2)}${description}`); } + console.log("\nFeedback recipes:"); + for (const [name, description] of FEEDBACK_TOPICS) { + console.log(` ${name.padEnd(maxLength + 2)}${description}`); + } + console.log( "\nAppend `--anonymous` to any rule/check command to skip the Taskless API" ); diff --git a/packages/cli/src/commands/feedback.ts b/packages/cli/src/commands/feedback.ts index 3fbeed76..3e88dc0f 100644 --- a/packages/cli/src/commands/feedback.ts +++ b/packages/cli/src/commands/feedback.ts @@ -122,7 +122,7 @@ const sendCommand = defineCommand({ meta: { name: "send", description: - "Send a completed feedback survey (use --from to specify the input file)", + "Send feedback, a bug report, or a rule survey response (use --from to specify the input file)", }, args: { dir: { @@ -197,15 +197,18 @@ const sendCommand = defineCommand({ }); /** - * Reached only through the survey invite. Deliberately absent from the - * `taskless agent` index (see `UNLISTED_COMMANDS` there): listing it would - * invite an agent to run it unprompted. When a general feedback channel - * exists, this surface folds into it. + * Three channels behind one command, chosen by the payload's `kind`: the + * invited rule-authoring survey (`agent rule-feedback`, the only one with + * `dismiss` and a cadence), general feedback (`agent feedback`), and bug + * reports (`agent bug-report`). The command itself stays out of the + * `taskless agent` index (see `UNLISTED_COMMANDS` there): the recipes are what + * an agent should reach, because they show the user the payload first. */ export const feedbackCommand = defineCommand({ meta: { name: "feedback", - description: "Send or dismiss the Taskless feedback survey", + description: + "Send Taskless feedback, a bug report, or the rule survey; or dismiss the survey", }, subCommands: { dismiss: dismissCommand, diff --git a/packages/cli/src/prompts/index.ts b/packages/cli/src/prompts/index.ts index a0858730..e1d01c5b 100644 --- a/packages/cli/src/prompts/index.ts +++ b/packages/cli/src/prompts/index.ts @@ -69,7 +69,7 @@ export const TOPICS = [ /** * Recipes deliberately withheld from the export, recorded so they stay visible - * decisions rather than oversights. Two groups: + * decisions rather than oversights. Three groups: * * - Command recipes (`auth` … `update`) walk an agent through running a CLI * subcommand on a developer's machine. There is no caller for them outside @@ -78,15 +78,18 @@ export const TOPICS = [ * the boundary from the client's side, `detect` documents a CLI subprocess a * Worker cannot spawn, `create-legacy-rule` targets a local toolchain, and * `rule-meta` describes a local sidecar file the CLI never writes. - * - `feedback` and `feedback-invite` belong to the survey the CLI appends to - * a served recipe. The invite is a fragment the `agent` command renders - * header-less and attaches after a recipe's last section; it lives here as - * a recipe so Vale and the cross-reference tests cover its prose, and it is - * servable by name only as a side effect of that. Neither has a reader - * outside the CLI that sends the response. + * - The feedback recipes send a response through the CLI on a developer's + * machine. `rule-feedback` and `feedback-invite` belong to the survey the + * CLI appends to a served recipe: the invite is a fragment the `agent` + * command renders header-less and attaches after a recipe's last section; + * it lives here as a recipe so Vale and the cross-reference tests cover its + * prose, and it is servable by name only as a side effect of that. + * `feedback` and `bug-report` are the channels a user asks for. None has a + * reader outside the CLI that sends the response. */ export const INTERNAL_TOPICS = [ "auth", + "bug-report", "check", "ci", "create-legacy-rule", @@ -102,6 +105,7 @@ export const INTERNAL_TOPICS = [ "onboard", "recover-rule", "rule", + "rule-feedback", "rule-meta", "update", "verify-rule", diff --git a/packages/cli/src/prompts/recipes.ts b/packages/cli/src/prompts/recipes.ts index 16e4be44..c8496bb2 100644 --- a/packages/cli/src/prompts/recipes.ts +++ b/packages/cli/src/prompts/recipes.ts @@ -8,7 +8,11 @@ import { } from "../util/invocation"; import { inputSchema as ruleCreateInputSchema } from "../schemas/rules-create"; import { inputSchema as ruleImproveInputSchema } from "../schemas/rules-improve"; -import { ruleInputSchema } from "../schemas/feedback"; +import { + bugInputSchema, + generalInputSchema, + ruleInputSchema, +} from "../schemas/feedback"; import { AST_GREP_VERSION, VALE_VERSION, @@ -80,7 +84,11 @@ export function canonicalRecipeTopics(): string[] { const TOPIC_INPUT_SCHEMAS: Record = { "create-remote-rule": ruleCreateInputSchema, "improve-rule": ruleImproveInputSchema, - feedback: ruleInputSchema, + // Each feedback recipe embeds only its own branch of the payload union, so + // an agent reading one never sees the keys of another survey. + "rule-feedback": ruleInputSchema, + feedback: generalInputSchema, + "bug-report": bugInputSchema, }; /** Agent-fill marker used when the caller does not supply a real value. */ diff --git a/packages/cli/test/feedback-command.test.ts b/packages/cli/test/feedback-command.test.ts index 0c5cd197..87285b0c 100644 --- a/packages/cli/test/feedback-command.test.ts +++ b/packages/cli/test/feedback-command.test.ts @@ -403,10 +403,17 @@ describe("feedback in the built CLI", () => { const execFileAsync = promisify(execFile); const binPath = builtCli(); - it("is absent from the agent index", async () => { + it("lists the user-initiated recipes, and only them, in the agent index", async () => { const { stdout } = await execFileAsync("node", [binPath, "agent"]); - expect(stdout).toContain("Topics:"); - expect(stdout).not.toMatch(/^\s*feedback\b/m); + const topics = stdout.slice(0, stdout.indexOf("Authoring recipes:")); + const feedback = stdout.slice(stdout.indexOf("Feedback recipes:")); + // The command is reached through a recipe, never listed as a command. + expect(topics).toContain("Topics:"); + expect(topics).not.toMatch(/^\s*feedback\b/m); + expect(feedback).toMatch(/^\s*feedback\s/m); + expect(feedback).toMatch(/^\s*bug-report\s/m); + // The invited survey is reached through the invite alone. + expect(stdout).not.toContain("rule-feedback"); }); it("still serves --help with both verbs", async () => { diff --git a/packages/cli/test/feedback-recipes.test.ts b/packages/cli/test/feedback-recipes.test.ts index 7876ed2c..d6d83b4a 100644 --- a/packages/cli/test/feedback-recipes.test.ts +++ b/packages/cli/test/feedback-recipes.test.ts @@ -4,7 +4,11 @@ import { promisify } from "node:util"; import { describe, expect, it } from "vitest"; import { getRecipe } from "../src/prompts/recipes"; -import { ruleInputSchema } from "../src/schemas/feedback"; +import { + bugInputSchema, + generalInputSchema, + ruleInputSchema, +} from "../src/schemas/feedback"; import { COMPLETED_CHOICES } from "../src/survey/constants"; import { builtCli } from "./support/built-cli"; @@ -25,14 +29,14 @@ function unwrapQuote(text: string): string { .join(" "); } -describe("the feedback recipe", () => { +describe("the rule-feedback recipe", () => { it("opens with its header and embeds the payload schema", async () => { const { stdout } = await execFileAsync("node", [ binPath, "agent", - "feedback", + "rule-feedback", ]); - expect(stdout.startsWith("# Topic: feedback ")).toBe(true); + expect(stdout.startsWith("# Topic: rule-feedback ")).toBe(true); // The schema is rendered from the Zod source, so the choices the agent // reads are the ones `feedback send` accepts. for (const choice of COMPLETED_CHOICES) { @@ -43,8 +47,14 @@ describe("the feedback recipe", () => { } }); + it("relays the telemetry-off outcome with the issues page", () => { + const rendered = getRecipe("rule-feedback", { invocation }) ?? ""; + expect(rendered).toMatch(/If telemetry is disabled/); + expect(rendered).toContain("https://github.com/taskless/cli/issues"); + }); + it("names the send and dismiss commands by the rendered invocation", () => { - const rendered = getRecipe("feedback", { invocation }); + const rendered = getRecipe("rule-feedback", { invocation }); expect(rendered).toContain( `${invocation} feedback send --from .taskless/.tmp-feedback.json --json` ); @@ -52,7 +62,7 @@ describe("the feedback recipe", () => { }); it("tells the agent it is the respondent and hands the user's words through verbatim", () => { - const rendered = getRecipe("feedback", { invocation }) ?? ""; + const rendered = getRecipe("rule-feedback", { invocation }) ?? ""; expect(rendered).toContain("You are the respondent"); expect(rendered).toContain( "Do not put the survey's questions to the user one by one" @@ -61,7 +71,7 @@ describe("the feedback recipe", () => { }); it("shows every answer before sending when the user asked for a review", () => { - const rendered = getRecipe("feedback", { invocation }) ?? ""; + const rendered = getRecipe("rule-feedback", { invocation }) ?? ""; const review = rendered.slice( rendered.indexOf("**Show it first, if the user asked for a `review`.**"), rendered.indexOf("**Send.** Run:") @@ -107,7 +117,7 @@ describe("the feedback invite", () => { invocation, header: false, }) ?? ""; - expect(fragment).toContain(`${invocation} agent feedback`); + expect(fragment).toContain(`${invocation} agent rule-feedback`); expect(fragment).toContain(`${invocation} feedback dismiss`); }); @@ -118,7 +128,7 @@ describe("the feedback invite", () => { fragment.indexOf("They said `review`"), fragment.indexOf("`skip`, said nothing, or replied about something else") ); - expect(reviewDoor).toContain(`${invocation} agent feedback`); + expect(reviewDoor).toContain(`${invocation} agent rule-feedback`); expect(reviewDoor).toContain("review mode"); expect(reviewDoor).not.toContain("feedback dismiss"); }); @@ -141,7 +151,7 @@ describe("the feedback invite", () => { fragment.indexOf("`skip`, said nothing, or replied about something else"), fragment.indexOf("They asked you not to send anything") ); - expect(skipDoor).toContain(`${invocation} agent feedback`); + expect(skipDoor).toContain(`${invocation} agent rule-feedback`); expect(skipDoor).toContain("no `verbatim`"); expect(skipDoor).not.toContain("feedback dismiss"); }); @@ -157,3 +167,64 @@ describe("the feedback invite", () => { expect(refusal).toContain("TASKLESS_TELEMETRY_DISABLED=1"); }); }); + +/** Every key in a branch's JSON Schema, as the recipe embeds it. */ +function schemaKeys(schema: { shape: Record }): string[] { + return Object.keys(schema.shape); +} + +describe.each([ + ["feedback", "general", generalInputSchema, ruleInputSchema], + ["bug-report", "bug", bugInputSchema, ruleInputSchema], +] as const)("the %s recipe", (topic, kind, ownSchema, otherSchema) => { + it("opens with its header and embeds only its own payload schema", async () => { + const { stdout } = await execFileAsync("node", [binPath, "agent", topic]); + expect(stdout.startsWith(`# Topic: ${topic} `)).toBe(true); + for (const key of schemaKeys(ownSchema)) { + expect(stdout).toContain(`"${key}"`); + } + expect(stdout).toContain(`"const": "${kind}"`); + for (const key of schemaKeys(otherSchema)) { + if (key in ownSchema.shape) continue; + expect(stdout).not.toContain(`"${key}"`); + } + }); + + it("shows the payload and waits for a yes before sending", () => { + const rendered = getRecipe(topic, { invocation }) ?? ""; + const show = rendered.indexOf("**Show it, and wait for a yes.**"); + expect(show).toBeGreaterThan(-1); + expect(show).toBeLessThan(rendered.indexOf(`${invocation} feedback send`)); + expect(rendered).toContain("Send only on the user's go-ahead"); + expect(rendered).toContain( + "Do NOT send before the user has seen the payload and said yes" + ); + }); + + it("keeps secrets and unshared code out of the payload", () => { + const rendered = getRecipe(topic, { invocation }) ?? ""; + expect(rendered).toContain("**Keep it shareable.**"); + expect(rendered).toContain("secrets, tokens, credentials, absolute paths"); + }); + + it("relays the telemetry-off outcome with the issues page", () => { + const rendered = getRecipe(topic, { invocation }) ?? ""; + expect(rendered).toMatch(/If it says telemetry is\s+disabled/); + expect(rendered).toContain("https://github.com/taskless/cli/issues"); + }); + + it("never dismisses the invited survey", () => { + const rendered = getRecipe(topic, { invocation }) ?? ""; + expect(rendered).toContain( + `Do NOT run \`${invocation} feedback dismiss\` here` + ); + }); +}); + +describe("the bug-report recipe's version information", () => { + it("leaves version information to the CLI", () => { + const rendered = getRecipe("bug-report", { invocation }) ?? ""; + expect(rendered).toContain("**Leave version information out.**"); + expect(Object.keys(bugInputSchema.shape)).not.toContain("version"); + }); +}); diff --git a/packages/cli/test/survey-invite.test.ts b/packages/cli/test/survey-invite.test.ts index 71917d2a..3ec87fb2 100644 --- a/packages/cli/test/survey-invite.test.ts +++ b/packages/cli/test/survey-invite.test.ts @@ -116,6 +116,16 @@ describe("the survey gate", () => { expect(await surveyGateIsOpen({ topic, now: () => NOW })).toBe(true); }); + // Telemetry is on and the cadence is open here, so only the topic set + // decides: the opt-in channels and the survey's own recipe never carry + // the invite. + it.each(["feedback", "bug-report", "rule-feedback"])( + "does not survey %s, even with the gate otherwise open", + async (topic) => { + expect(await surveyGateIsOpen({ topic, now: () => NOW })).toBe(false); + } + ); + it("serves the bare recipe within the window, touching nothing", async () => { await writeNextAsk(RULE_SURVEY_ID, NOW + 1); const recipe =