Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,11 @@ on:
permissions:
contents: read

# Release and nightly publishing require green CI on the exact main commit, so a
# main push never cancels or replaces another commit's run.
concurrency:
group: ci-${{ github.workflow }}-${{ github.ref }}
group: >-
ci-${{ github.workflow }}-${{ github.event_name == 'pull_request' && github.ref || github.sha }}
cancel-in-progress: true

jobs:
Expand Down
29 changes: 8 additions & 21 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ name: Release
# (npm trusted publishing pins the workflow filename and the npm-publish
# environment). scripts/release-plan.mjs maps the triggering event to a mode:
# dry-run (PR), nightly (schedule or dispatch on main), or stable (a vX.Y.Z tag on
# main, which only a repository admin can create, normally by publishing a GitHub
# release).
# main). The tag ruleset, which only admins can bypass, is the stable release's
# approval: a tag publishes without another review.
on:
pull_request:
paths:
Expand All @@ -18,9 +18,9 @@ on:
tags:
- 'v[0-9]+.[0-9]+.[0-9]+'
schedule:
# Retried every few hours: a nightly waits for green CI on main's HEAD and
# publishes at most once per UTC day.
- cron: '23 */3 * * *'
# A nightly ships main's HEAD once its CI passes. CI that fails, is cancelled,
# or outlasts the Plan job's wait skips that day.
- cron: '23 1 * * *'
# On main: publish a nightly now. On a vX.Y.Z tag: retry that stable release.
workflow_dispatch:

Expand All @@ -37,7 +37,7 @@ jobs:
plan:
name: Plan
runs-on: ubuntu-latest
# A stable tag waits up to 20 minutes for its commit's CI to finish.
# Waits up to 20 minutes for the commit's CI to finish.
timeout-minutes: 25
permissions:
actions: read
Expand Down Expand Up @@ -65,17 +65,6 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}
run: node scripts/release-plan.mjs

approve:
name: Approve ${{ needs.plan.outputs.version }}
needs: plan
if: needs.plan.outputs.mode == 'stable'
runs-on: ubuntu-latest
timeout-minutes: 5
environment: release
permissions: {}
steps:
- run: echo "Approved the agent-device ${{ needs.plan.outputs.version }} release."

build:
name: Build ${{ needs.plan.outputs.version }}
needs: plan
Expand Down Expand Up @@ -124,10 +113,8 @@ jobs:

publish:
name: Publish ${{ needs.plan.outputs.version }} to npm
needs: [plan, approve, build]
if: >-
!cancelled() && needs.build.result == 'success'
&& (needs.plan.outputs.mode == 'nightly' || needs.approve.result == 'success')
needs: [plan, build]
if: contains(fromJSON('["nightly", "stable"]'), needs.plan.outputs.mode)
runs-on: ubuntu-latest
timeout-minutes: 15
environment: npm-publish
Expand Down
35 changes: 17 additions & 18 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,18 +93,20 @@ described below. A local `npm publish` from the repository root refuses to run.
The core and every public workspace package release together at one version. Private packages
stay unpublished. There are three channels:

- **Nightly** (`npm install -g agent-device@nightly`): every few hours the workflow checks `main`.
It publishes `X.Y.Z-nightly.<YYYYMMDD>.<run>` under the `nightly` dist-tag when `main`'s HEAD has
changed since the last nightly, its CI passed, and no nightly shipped that UTC day. It then tags
the commit `nightly/v<nightly version>`. To publish one now, run **Release** on `main`.
- **Nightly** (`npm install -g agent-device@nightly`): once a day, at 01:23 UTC, the workflow
checks `main`. It publishes `X.Y.Z-nightly.<YYYYMMDD>.<run>` under the `nightly` dist-tag when
Comment thread
thymikee marked this conversation as resolved.
`main`'s HEAD has changed since the last nightly and its CI passed, waiting up to 20 minutes for
CI that is still running. If that CI fails, is cancelled, or is still running after the wait, no
nightly ships that day. A published nightly is tagged `nightly/v<nightly version>`. To publish one now, run **Release** on `main`.
- **Stable** (`latest`): a repository admin publishes a GitHub release with a new `vX.Y.Z` tag
targeting `main`. Use the version `main` builds towards (its `X.Y.Z-dev`), or a later one, and
write the notes or generate them in the release form. The tag starts a **Release** run. It waits
up to 20 minutes for that commit's CI to pass, then waits for approval in the `release`
environment. Once approved, it publishes the packages, attaches the iOS runner and Android helper
assets to the release, publishes the MCP Registry entry, and commits `main`'s next `-dev` version.
If you reject the run, delete the release and its tag. Pushing the tag instead (`git push origin
main:refs/tags/vX.Y.Z`) starts the same run, which then drafts the release itself.
up to 20 minutes for that commit's CI to pass, then publishes the packages without asking for
approval, attaches the iOS runner and Android helper assets to the release, publishes the MCP
Registry entry, and commits `main`'s next `-dev` version. Creating the tag is the approval. To
stop a release, cancel the run before its publish job starts, then delete the release and its
tag. Pushing the tag instead (`git push origin main:refs/tags/vX.Y.Z`) starts the same run, which
then drafts the release itself.
- **Dry run**: a pull request that changes release tooling builds and verifies every package with a
nightly version, and publishes nothing.

Expand Down Expand Up @@ -137,20 +139,19 @@ npm access set mfa=publish @agent-device/<name>
### One-time repository setup

npm trusts every run of `release.yml` that reaches the `npm-publish` environment, so GitHub
settings, not the workflow, decide who can publish. The plan job refuses to publish until the first
three are in place. Apply them before granting npm trust:
settings, not the workflow, decide who can publish. The tag ruleset approves a stable release, so
the plan job refuses to publish until the first two settings are in place. Apply them before
granting npm trust:

1. **Environment `npm-publish`** (Settings → Environments): deployment branches and tags limited to
the branch `main` and the tag pattern `v*`. No reviewers and no secrets, so nightlies run
unattended.
2. **Environment `release`**: required reviewers. Turn on "Prevent self-review" only when another
maintainer can approve, because the admin who creates the tag cannot approve their own run.
3. **Tag ruleset** (Settings → Rules → Rulesets → New tag ruleset): active, targeting
2. **Tag ruleset** (Settings → Rules → Rulesets → New tag ruleset): active, targeting
`refs/tags/v*` with no exclusions, restricting creations, updates, and deletions, with only the
Repository admin role allowed to bypass. Nightly tags live under `nightly/` and stay outside it.
4. **Actions** (Settings → Actions → General): workflow permissions set to read repository contents,
3. **Actions** (Settings → Actions → General): workflow permissions set to read repository contents,
and "Allow GitHub Actions to create and approve pull requests" turned off.
5. **Releases** (Settings → General): leave release immutability off, because the run attaches its
4. **Releases** (Settings → General): leave release immutability off, because the run attaches its
assets after the release is published.

Confirm the result, then grant npm trust to each public package and, once a nightly has published,
Expand All @@ -159,8 +160,6 @@ refuse token-based publishing:
```bash
gh api repos/callstack/agent-device/environments/npm-publish/deployment-branch-policies \
--jq '.branch_policies[] | "\(.type) \(.name)"' # branch main, tag v*
gh api repos/callstack/agent-device/environments/release \
--jq '.protection_rules[] | select(.type == "required_reviewers")'
gh api repos/callstack/agent-device/rulesets --jq '.[] | select(.target == "tag") | .name'
npm trust github <package> --file release.yml --repository callstack/agent-device \
--environment npm-publish --allow-publish --yes
Expand Down
19 changes: 5 additions & 14 deletions scripts/__tests__/release-plan.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,16 +40,13 @@ test('a scheduled nightly publishes main once its CI passed', () => {
});
});

test('a scheduled nightly waits for new commits, a new day, and green CI', () => {
test('a scheduled nightly waits for new commits and green CI', () => {
assert.match(planRelease(run({ sha: NIGHTLY_COMMIT })).reason, /already ships/);
assert.match(planRelease(run({ today: '20261010' })).reason, /shipped today/);
assert.match(planRelease(run({ ciConclusion: 'in_progress' })).reason, /is in_progress/);
const published = run({ publishedNightly: '0.21.25-nightly.20261011.40' });
assert.match(planRelease(published).reason, /20261011\.40 shipped today/);
assert.match(planRelease(run({ ciConclusion: 'failure' })).reason, /is failure/);
});

test('a dispatched nightly skips the cadence checks but still requires green CI', () => {
const dispatched = run({ event: 'workflow_dispatch', sha: NIGHTLY_COMMIT, today: '20261010' });
test('a dispatched nightly republishes the shipped HEAD but still requires green CI', () => {
const dispatched = run({ event: 'workflow_dispatch', sha: NIGHTLY_COMMIT });
assert.equal(planRelease(dispatched).mode, 'nightly');
assert.throws(() => planRelease({ ...dispatched, ciConclusion: 'failure' }), /CI on .* failure/);
});
Expand Down Expand Up @@ -123,9 +120,6 @@ function protectedSetup() {
{ name: 'main', type: 'branch' },
{ name: 'v*', type: 'tag' },
],
release: {
protection_rules: [{ type: 'required_reviewers', reviewers: [{ type: 'User' }] }],
},
tagRulesets: [
{
enforcement: 'active',
Expand All @@ -142,10 +136,9 @@ test('publishing needs the protections GitHub enforces, not this script', () =>
repositorySetupGaps({
npmPublish: null,
npmPublishPolicies: [],
release: null,
tagRulesets: [],
}).length,
3,
2,
);
});

Expand All @@ -156,8 +149,6 @@ test('each missing protection is named on its own', () => {
npmPublishPolicies: [...setup.npmPublishPolicies, { name: '*', type: 'branch' }],
};
assert.match(repositorySetupGaps(anyBranch).join(), /npm-publish environment/);
const unreviewed = { ...setup, release: { protection_rules: [] } };
assert.match(repositorySetupGaps(unreviewed).join(), /release environment/);
const evaluating = {
...setup,
tagRulesets: [{ ...setup.tagRulesets[0], enforcement: 'evaluate' }],
Expand Down
25 changes: 6 additions & 19 deletions scripts/release-plan.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,6 @@ function planNightly(run) {
if (scheduled && latest && run.tags.get(latest) === run.sha) {
return skip(`${latest} already ships ${run.sha}.`);
}
// The npm dist-tag is the durable record: a nightly can publish and then fail to tag.
const shippedToday = [latest, run.publishedNightly].find((shipped) =>
shipped?.includes(`-nightly.${run.today}.`),
);
if (scheduled && shippedToday) return skip(`${shippedToday} shipped today.`);
if (run.ciConclusion !== 'success') {
const reason = `CI on ${run.sha} is ${run.ciConclusion}; a nightly ships only a commit whose CI passed.`;
if (scheduled) return skip(reason);
Expand All @@ -61,7 +56,8 @@ function planStable(run, tag) {
if (!run.onMain) throw new Error(`${tag} is not on main.`);
if (run.ciConclusion !== 'success') {
throw new Error(
`CI on ${run.sha} is ${run.ciConclusion}. Once it passes, run Release on ${tag} to publish it.`,
`CI on ${run.sha} is ${run.ciConclusion}. Re-running Release does not re-run CI: once CI ` +
`passes on that commit, or ${tag} moves to a commit whose CI passed, run Release on ${tag}.`,
);
}
const previousTag = previousReleaseTag([...run.tags.keys()], version) ?? '';
Expand Down Expand Up @@ -98,15 +94,13 @@ function skip(reason) {
* What GitHub must enforce before any publish, since npm trusts every run of release.yml that
* reaches the npm-publish environment and a run's own copy of this script proves nothing.
*/
export function repositorySetupGaps({ npmPublish, npmPublishPolicies, release, tagRulesets }) {
export function repositorySetupGaps({ npmPublish, npmPublishPolicies, tagRulesets }) {
const gaps = [];
const allowed = new Set(npmPublishPolicies.map((policy) => `${policy.type}:${policy.name}`));
const onlyMainAndTags = allowed.size === 2 && allowed.has('branch:main') && allowed.has('tag:v*');
if (!npmPublish?.deployment_branch_policy?.custom_branch_policies || !onlyMainAndTags) {
gaps.push('the npm-publish environment must deploy only from branch main and tags v*');
}
const reviewers = release?.protection_rules?.find((rule) => rule.type === 'required_reviewers');
if (!reviewers?.reviewers?.length) gaps.push('the release environment must require a reviewer');
if (!tagRulesets.some(guardsReleaseTags)) {
gaps.push(
'an active tag ruleset must restrict creating, updating, and deleting every refs/tags/v* tag, ' +
Expand Down Expand Up @@ -184,7 +178,7 @@ const UNSETTLED_CI = new Set([
'in_progress',
]);

/** A tag pushed right after a merge waits, within the Plan job's budget, for that merge's CI. */
/** A commit pushed moments ago waits, within the Plan job's budget, for its CI to finish. */
async function settledCiConclusion(env, sha) {
for (let attempt = 0; ; attempt += 1) {
const conclusion = await ciConclusion(env, sha);
Expand Down Expand Up @@ -231,14 +225,12 @@ function absentOn404(fallback) {
}

async function readRepositorySetup(env) {
const environment = (name) => github(env, `environments/${name}`).catch(absentOn404(null));
const summaries = await github(env, 'rulesets?targets=tag');
return {
npmPublish: await environment('npm-publish'),
npmPublish: await github(env, 'environments/npm-publish').catch(absentOn404(null)),
npmPublishPolicies: await github(env, 'environments/npm-publish/deployment-branch-policies')
.then((body) => body.branch_policies)
.catch(absentOn404([])),
release: await environment('release'),
tagRulesets: await Promise.all(
summaries.map((summary) => github(env, `rulesets/${summary.id}`)),
),
Expand All @@ -259,12 +251,7 @@ async function readRun(env) {
runNumber: env.GITHUB_RUN_NUMBER,
tags: event === 'pull_request' ? new Map() : remoteTags(),
latestStable: onBranch || onTag ? await publishedVersion('latest') : null,
publishedNightly: onBranch ? await publishedVersion('nightly') : null,
ciConclusion: onTag
? await settledCiConclusion(env, env.GITHUB_SHA)
: onBranch
? await ciConclusion(env, env.GITHUB_SHA)
: undefined,
ciConclusion: onBranch || onTag ? await settledCiConclusion(env, env.GITHUB_SHA) : undefined,
onMain: onTag ? await isOnMain(env) : undefined,
};
}
Expand Down
13 changes: 0 additions & 13 deletions test/ci/release-workflow.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ const source = fs.readFileSync(path.join(repoRoot, '.github/workflows/release.ym

type Job = {
environment?: string;
if?: string;
needs?: string[];
permissions?: Record<string, string>;
steps?: Array<{ uses?: string; with?: Record<string, string> }>;
uses?: string;
Expand Down Expand Up @@ -42,14 +40,3 @@ test('the published bytes are built without a restored dependency cache', () =>
);
expect(setup?.with?.['cache-store']).toBe('false');
});

// The npm-publish environment cannot require a reviewer (nightlies run unattended), so a stable
// publish is held only by the approve job and its `release` environment.
test('a stable publish waits for the release environment approval', () => {
expect(workflow.jobs.approve?.environment).toBe('release');
expect(workflow.jobs.approve?.if).toBe("needs.plan.outputs.mode == 'stable'");
expect(workflow.jobs.publish?.needs).toContain('approve');
expect(workflow.jobs.publish?.if?.replaceAll(/\s+/g, ' ').trim()).toBe(
"!cancelled() && needs.build.result == 'success' && (needs.plan.outputs.mode == 'nightly' || needs.approve.result == 'success')",
);
});
Loading