Merge pull request #6 from anysphere/cursor/byo-ecr-microvm-image-5528

Document BYO ECR images as the MicroVM container base

Author: pertschuk

.github/workflows/ci.yml

@@ -18,6 +18,7 @@ jobs:
           test -f controller/Dockerfile
           test -f controller/handler.py
           test -f microvm-image/Dockerfile
+          test -f microvm-image/Dockerfile.ecr
           test -f microvm-image/entrypoint.sh
           test -f microvm-image/hook.py
           test ! -f handler.mjs
@@ -39,3 +40,10 @@ jobs:
           grep -q 'endswith("/validate")' microvm-image/hook.py
           grep -q 'endswith("/run")' microvm-image/hook.py
           grep -q '"validate":"ENABLED"' README.md
+          grep -q "Bring your own ECR image" README.md
+          grep -q "ecr:GetAuthorizationToken" cloudformation.yaml
+          grep -q "ecr:BatchCheckLayerAvailability" cloudformation.yaml
+          grep -q "ecr:GetDownloadUrlForLayer" cloudformation.yaml
+          grep -q "ecr:BatchGetImage" cloudformation.yaml
+          grep -q "COPY entrypoint.sh hook.py" microvm-image/Dockerfile.ecr
+          grep -q 'ENTRYPOINT \["python3", "/opt/cursor/hook.py"\]' microvm-image/Dockerfile.ecr

README.md

@@ -96,7 +96,7 @@ Users pick `octocat/Hello-World` in the dashboard (the pool appears under that r
 
    `https://github.com/octocat/Hello-World` (`octocat/Hello-World`) is a public sample so you can clone without configuring git credentials. Replace it with your real repository before you run real work. Private repos need git auth (HTTPS token or SSH) on the worker.
 
-3. Build the worker image from [`microvm-image/`](microvm-image/) (needs the stack outputs). Enable both `ready` and `validate` image hooks. After this template change, **rebuild the MicroVM image** so those hooks are in the snapshot path:
+3. Build the worker image from [`microvm-image/`](microvm-image/) (needs the stack outputs). Enable both `ready` and `validate` image hooks. After this template change, **rebuild the MicroVM image** so those hooks are in the snapshot path. The stock Dockerfile starts from `public.ecr.aws/lambda/microvms:al2023-minimal`. To start from an application image you already publish to ECR, see [Bring your own ECR image](#bring-your-own-ecr-image).
 
    ```bash
    BUCKET=$(aws cloudformation describe-stacks --stack-name cursor-lambda-workers \
@@ -117,6 +117,65 @@ Users pick `octocat/Hello-World` in the dashboard (the pool appears under that r
 
 4. Start an agent from [cursor.com/agents](https://cursor.com/agents) against the pool, or against `octocat/Hello-World` for the repo-bound walkthrough. That GitHub repo is a public sample so you can clone without configuring git credentials. Replace it with your real repository before you run real work.
 
+## Bring your own ECR image
+
+`create-microvm-image` still takes `--code-artifact uri=s3://.../app.zip` (a zip whose root contains a `Dockerfile` plus app artifacts) and `--base-image-arn` (a Lambda-managed MicroVM OS from `list-managed-microvm-images`). Your ECR image is the **container** base via `FROM` in that Dockerfile, not an argument to `--code-artifact` or `--base-image-arn`. Lambda builds the Dockerfile inside the managed OS, then snapshots the result. Official docs: [Container base images / Using a private ECR image](https://docs.aws.amazon.com/lambda/latest/dg/microvms-images.html).
+
+Keep publishing the application image from CI/CD as you already do (deps, toolchain, repo-specific packages). The MicroVM zip is thin: a Dockerfile that `FROM`s that image (tag or digest) plus this repo’s Cursor worker files. Rebuilding the MicroVM image is what picks up a new CI image. `run-microvm` still uses the MicroVM image name or ARN (`cursor-pool-worker` here), not the ECR URI.
+
+### Dockerfile
+
+Use [`microvm-image/Dockerfile.ecr`](microvm-image/Dockerfile.ecr). Set `FROM` to your image, then layer the same worker bits the stock image installs. Do not drop them: snapshot and smoke-test still POST `/ready` and `/validate` on port 9000.
+
+```dockerfile
+FROM 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-ci-image:tag
+# linux/arm64 (this template’s guests are aarch64). Linux, snapshot-compatible.
+# Image must be reachable from Lambda build (public internet or ECR in this account).
+# Ensure git, python3, curl, tar, and awscli if the CI image does not already
+# have them (package manager depends on FROM).
+
+COPY cursor-agent-version /tmp/cursor-agent-version
+# install cursor-agent for linux/arm64 (same RUN as the stock Dockerfile)
+
+COPY entrypoint.sh hook.py /opt/cursor/
+RUN chmod +x /opt/cursor/entrypoint.sh /opt/cursor/hook.py && mkdir -p /opt/cursor/workspaces
+ENV HOOK_PORT=9000
+ENTRYPOINT ["python3", "/opt/cursor/hook.py"]
+```
+
+The guest still needs:
+
+- Cursor agent CLI (`cursor-agent` / `agent worker … start`)
+- git (repo-bound clone in [`entrypoint.sh`](microvm-image/entrypoint.sh))
+- `/run`, `/ready`, `/validate` on port 9000 ([`hook.py`](microvm-image/hook.py))
+- entrypoint that starts the worker with `CURSOR_*` on `/run`
+
+Zip `Dockerfile.ecr` as `Dockerfile` so you do not overwrite the stock quickstart:
+
+```bash
+rm -f /tmp/app.zip
+TMP=$(mktemp -d)
+cp microvm-image/entrypoint.sh microvm-image/hook.py microvm-image/cursor-agent-version "$TMP/"
+cp microvm-image/Dockerfile.ecr "$TMP/Dockerfile"
+( cd "$TMP" && zip -r /tmp/app.zip . )
+aws s3 cp /tmp/app.zip "s3://${BUCKET}/app.zip"
+```
+
+Then the same `create-microvm-image` as Deploy step 3: same `--base-image-arn` from `list-managed-microvm-images`, `--build-role-arn`, and `--hooks` with `ready` and `validate` **ENABLED**. Keep those hooks; BYO ECR does not change the snapshot path.
+
+### IAM (private ECR)
+
+The MicroVM **build role** must pull `FROM`. [`cloudformation.yaml`](cloudformation.yaml) `BuildRole` includes:
+
+- `ecr:GetAuthorizationToken` (`Resource: *` — that action does not support resource-level IAM)
+- `ecr:BatchCheckLayerAvailability`, `ecr:GetDownloadUrlForLayer`, `ecr:BatchGetImage` on this account’s ECR repositories
+
+Redeploy the stack so those statements exist before the first BYO build. Cross-account ECR needs extra policy on the role and a repository policy on the other account; this template does not add that.
+
+### Architecture
+
+This template’s MicroVM guests are **aarch64**. The ECR image must be `linux/arm64`. An amd64-only CI image fails the MicroVM build or fails at runtime (`Exec format error` on `node`). Publish an arm64 or multi-arch tag from CI.
+
 ## Run a cloud agent
 
 Open [cursor.com/agents](https://cursor.com/agents). Choose **Self-hosted**.
@@ -155,6 +214,7 @@ aws lambda-microvms list-microvms --image-identifier cursor-pool-worker
 | Symptom | What to check |
 | --- | --- |
 | Image build fails (S3 or IAM) | Confirm stack outputs `ArtifactBucketName` and `BuildRoleArn`. The zip must land in that bucket, and the build role must be able to read it. |
+| Image build fails pulling `FROM` (ECR) | Do not pass an ECR URI to `--code-artifact` or `--base-image-arn`. Put the URI in the zip’s `Dockerfile` `FROM`. Redeploy so `BuildRole` can pull private ECR. Image must be Linux `linux/arm64`, snapshot-compatible, and in this account (or public). |
 | Image built before `/ready`+`/validate` | Rebuild the MicroVM image so those hooks are in the snapshot path. Existing snapshots were taken without them. |
 | No MicroVM | Confirm the controller is running and can call `run-microvm`. For a local controller, assume `SpawnRoleArn`. Confirm image `cursor-pool-worker` exists. |
 | Worker dies immediately | The guest needs `CURSOR_API_KEY` (SSM `/cursor-lambda-workers/cursor-api-key`). Confirm the `/run` hook started `cursor-agent worker --pool … start`. If logs show `Exec format error` on `node`, the image installed the wrong CLI arch (MicroVMs here are aarch64). If auth says the API key is invalid, do not set `CURSOR_API_ENDPOINT` to `https://api.cursor.com`. |
@@ -163,6 +223,7 @@ aws lambda-microvms list-microvms --image-identifier cursor-pool-worker
 ## Related resources
 
 - [AWS Lambda MicroVMs](https://docs.aws.amazon.com/lambda/latest/dg/lambda-microvms-guide.html)
+- [MicroVM images (container base / private ECR)](https://docs.aws.amazon.com/lambda/latest/dg/microvms-images.html)
 - [Cursor self-hosted pools](https://cursor.com/docs/cloud-agent/self-hosted-guides/pool.md) ([Any repo / repo-less](https://cursor.com/docs/cloud-agent/self-hosted-guides/pool.md#repo-less-pools), [pool names](https://cursor.com/docs/cloud-agent/self-hosted-guides/pool.md#pool-names), [multiple repo roots](https://cursor.com/docs/cloud-agent/self-hosted-guides/pool.md#register-multiple-repo-roots))
 - This repo: [`spawn.sh`](spawn.sh), [`cloudformation.yaml`](cloudformation.yaml), [`microvm-image/`](microvm-image/)
 

cloudformation.yaml

@@ -105,6 +105,17 @@ Resources:
               - Effect: Allow
                 Action: ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
                 Resource: !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:/aws/lambda/microvms/*"
+              # Private ECR FROM during create-microvm-image. GetAuthorizationToken
+              # does not support resource-level IAM (same as AWS's sample).
+              - Effect: Allow
+                Action: ecr:GetAuthorizationToken
+                Resource: "*"
+              - Effect: Allow
+                Action:
+                  - ecr:BatchCheckLayerAvailability
+                  - ecr:GetDownloadUrlForLayer
+                  - ecr:BatchGetImage
+                Resource: !Sub "arn:${AWS::Partition}:ecr:*:${AWS::AccountId}:repository/*"
 
   SpawnRole:
     Type: AWS::IAM::Role

microvm-image/Dockerfile

@@ -1,5 +1,8 @@
 ARG BASE_IMAGE=public.ecr.aws/lambda/microvms:al2023-minimal
 FROM ${BASE_IMAGE}
+# To FROM a private ECR application image instead, see Dockerfile.ecr and the
+# README section "Bring your own ECR image". --base-image-arn stays a
+# Lambda-managed OS from list-managed-microvm-images.
 
 RUN dnf install -y --setopt=install_weak_deps=0 \
       bash tar gzip git ca-certificates findutils awscli python3 \

microvm-image/Dockerfile.ecr

@@ -0,0 +1,76 @@
+# Example: layer the Cursor worker runtime on an application image you already
+# publish to ECR from CI/CD. This is not the default; the stock Dockerfile
+# remains the quickstart.
+#
+# create-microvm-image does not take an ECR URI as the MicroVM image.
+# --code-artifact is a zip with a Dockerfile (this file, named Dockerfile in
+# the zip) plus Cursor files. --base-image-arn is a Lambda-managed MicroVM OS
+# from `list-managed-microvm-images`. Your ECR image is only the container
+# FROM below. See README "Bring your own ECR image".
+#
+# Edit FROM to your image tag or digest. create-microvm-image may not pass
+# --build-arg; do not rely on ARG for the image URI.
+#
+# Keep this file in sync with Dockerfile for CLI install, COPY, and ENTRYPOINT.
+
+FROM 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-ci-image:tag
+
+# Guest MicroVMs in this template are aarch64. The FROM image must be linux/arm64.
+# AWS also requires Linux, snapshot-compatible, and reachable from Lambda build
+# (public internet or ECR in the same account).
+#
+# Required on the guest (skip any your CI image already has). Package manager
+# depends on FROM (dnf / yum / apt-get / apk). Amazon Linux 2023 example:
+# RUN dnf install -y --setopt=install_weak_deps=0 \
+#       bash tar gzip git ca-certificates findutils awscli python3 \
+#       $(command -v curl >/dev/null 2>&1 || echo curl) && \
+#     dnf clean all
+#
+# If the CI image sets USER to non-root, switch back so CLI install and
+# /opt/cursor land where hook.py and entrypoint.sh expect:
+# USER root
+
+RUN missing=""; \
+    command -v python3 >/dev/null || missing="${missing} python3"; \
+    command -v git >/dev/null || missing="${missing} git"; \
+    command -v curl >/dev/null || missing="${missing} curl"; \
+    command -v tar >/dev/null || missing="${missing} tar"; \
+    command -v aws >/dev/null || missing="${missing} awscli"; \
+    if [ -n "${missing}" ]; then \
+      echo "FROM image is missing:${missing}" >&2; \
+      echo "Install them with this image's package manager, or uncomment the Amazon Linux dnf block above." >&2; \
+      exit 1; \
+    fi
+
+# CLI install matches Dockerfile. Empty cursor-agent-version = prod installer.
+# create-microvm-image zips this directory and may not pass --build-arg.
+COPY cursor-agent-version /tmp/cursor-agent-version
+ARG CURSOR_AGENT_VERSION=
+# Guest MicroVMs here are aarch64. The previous x64 default installed a node
+# binary that failed at runtime with "cannot execute binary file".
+ARG CURSOR_AGENT_ARCH=arm64
+RUN VERSION="$(sed -e 's/#.*//' -e 's/[[:space:]]//g' /tmp/cursor-agent-version | sed -e '/^$/d' | head -n 1)" && \
+    VERSION="${VERSION:-${CURSOR_AGENT_VERSION:-}}" && \
+    ARCH="${CURSOR_AGENT_ARCH:-arm64}" && \
+    echo "installing cursor-agent ${VERSION:-prod} linux/${ARCH} (build uname=$(uname -m))" && \
+    if [ -n "${VERSION}" ]; then \
+      mkdir -p "/root/.local/share/cursor-agent/versions/${VERSION}" /root/.local/bin && \
+      curl -fsSL "https://downloads.cursor.com/lab/${VERSION}/linux/${ARCH}/agent-cli-package.tar.gz" \
+        | tar --strip-components=1 -xzf - -C "/root/.local/share/cursor-agent/versions/${VERSION}" && \
+      ln -sf "/root/.local/share/cursor-agent/versions/${VERSION}/cursor-agent" /root/.local/bin/agent && \
+      ln -sf "/root/.local/share/cursor-agent/versions/${VERSION}/cursor-agent" /root/.local/bin/cursor-agent; \
+    else \
+      curl -fsSL https://cursor.com/install | bash; \
+    fi && \
+    ln -sf /root/.local/bin/agent /usr/local/bin/agent && \
+    ln -sf /root/.local/bin/agent /usr/local/bin/cursor-agent && \
+    rm -f /tmp/cursor-agent-version
+
+ENV PATH="/root/.cursor/bin:/root/.local/bin:/usr/local/bin:${PATH}"
+ENV HOME=/root
+ENV NODE_COMPILE_CACHE=/tmp/cursor-compile-cache
+WORKDIR /opt/cursor
+COPY entrypoint.sh hook.py /opt/cursor/
+RUN chmod +x /opt/cursor/entrypoint.sh /opt/cursor/hook.py && mkdir -p /opt/cursor/workspaces
+ENV HOOK_PORT=9000
+ENTRYPOINT ["python3", "/opt/cursor/hook.py"]