Skip to content

Add Deploy-to-Azure Bicep template - #1477

Open
Josh Duffney (duffney) wants to merge 9 commits into
mainfrom
duffney-deploy-azure-button-infra
Open

Add Deploy-to-Azure Bicep template#1477
Josh Duffney (duffney) wants to merge 9 commits into
mainfrom
duffney-deploy-azure-button-infra

Conversation

@duffney

@duffney Josh Duffney (duffney) commented Oct 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Adds a "Deploy to Azure" portal button that provisions the Azure infrastructure needed to run Scope: AKS, networking, Key Vault, Cosmos DB, Redis, Storage, and a container registry. This gives anyone a one-click way to stand up a ready-to-use environment, as a sanitized, minimal, resource-group-scoped Bicep template under a new deploy/azure/ folder.

The existing infra/ folder (azd dev-CosmosDB setup) and root azure.yaml are untouched; this is a separate, independent deployment path intended for new environments, not local/dev workflows.

Approach

deploy/azure/main.bicep orchestrates 7 modules under deploy/azure/modules/ (networking, aks, keyvault, cosmosdb, redis, storage, acr), built against a shared parameter/output contract and using Azure Verified Modules (AVM) where available:

  • AKS: one System + one User node pool (Linux only), Cilium network policy, OIDC issuer + workload identity enabled, native Azure Key Vault Secrets Provider add-on for pulling secrets into the cluster.
  • Networking: VNet + subnets, with private endpoints + private DNS zones for Key Vault, Cosmos DB, Redis, and Storage.
  • Key Vault: RBAC authorization mode, workload identity granted read access via role assignment, no hardcoded secret names.
  • Cosmos DB: Mongo API only, serverless by default.
  • Redis: Azure Cache for Redis with Entra ID auth preferred over access keys where supported.
  • Storage: blob + queue only, shared-key access disabled.
  • ACR: admin user disabled, AcrPull granted to the AKS kubelet identity (used to host the worker image).
  • No observability/monitoring resources in this v1 (no Log Analytics, no Grafana, no alerting) — kept deliberately minimal for a first pass.

main.bicep is compiled to deploy/azure/azuredeploy.json (the file the Deploy-to-Azure button URL points at), alongside main.parameters.json, createUiDefinition.json (Basics + Advanced portal UI flow), and metadata.json for the template gallery. A new CI workflow (.github/workflows/deploy-azure-drift.yml) rebuilds main.bicep and fails if azuredeploy.json has drifted, plus lints all module files. deploy/azure/README.md documents prerequisites, the CLI deploy path, and explicitly what this does not automate: the Helm-based app install itself, and BYO secrets the user must populate in Key Vault before/after deploying. The root README.md links to it with the Deploy-to-Azure button.

Notable assumptions (documented in deploy/azure/README.md)

  • AKS API server stays publicly reachable (not a private cluster) and local accounts remain enabled, so az aks get-credentials + kubectl/helm work without a jumpbox; only data-plane services (Key Vault, Cosmos, Redis, Storage) are private.
  • Cosmos DB uses serverless throughput by default to minimize idle cost for a quickstart template.
  • Classic Azure Cache for Redis is used rather than Azure Managed Redis (Enterprise) for v1.
  • Redis per-principal access-policy assignment isn't exposed by the AVM module version used, so it's a documented manual post-deployment step; the primary key is retained as a fallback.

Testing

  • az bicep build on main.bicep and each of the 7 modules individually: all compile with zero errors and zero warnings.
  • Verified azuredeploy.json matches a fresh rebuild of main.bicep exactly (no drift), matching the new CI check's logic.
  • Validated createUiDefinition.json and main.parameters.json as syntactically valid JSON.
  • Not run: live az deployment group validate/what-if against a real Azure subscription (no Azure credentials available in this environment).

Documentation and compatibility

  • New deploy/azure/README.md added; root README.md updated with a "Deploy to Azure" section and button.
  • No breaking changes; this is a new, additive deployment path alongside the existing infra//azd setup.

Checklist

  • If Portal features changed, keep CLI capabilities in sync. (N/A - infra-only change)
  • If Portal components changed, update their Storybook stories. (N/A)
  • If database changes require a migration, include up() / down() and keep it CosmosDB-compatible. (N/A)
  • If dependencies changed, update the lockfile and regenerate NOTICE / NOTICE-REVIEW.txt with pnpm notice as needed. (N/A - no dependency changes)
  • Video showing the behavior before the suggested change (N/A - infra/deployment template, not a UI change)
  • Video showing the behavior after the suggested change (N/A - infra/deployment template, not a UI change)

@duffney
Add sanitized Deploy-to-Azure Bicep template under deploy/azure/
00602cc 
Provisions AKS (OIDC issuer, workload identity, native Key Vault Secrets
Provider add-on, Cilium network policy), a VNet with private endpoints for
Key Vault/Cosmos DB/Redis/Storage, Cosmos DB (MongoDB API only), Azure Cache
for Redis (Entra ID auth preferred), a blob+queue storage account, and an ACR
with AcrPull granted to the AKS kubelet identity.

Explicitly excludes azd, FluxCD/GitOps, ASO, ESO, KEDA-via-Flux, CorpNet IP
allowlisting, GitHub App/Entra bootstrap scripts, and Windows node pools.
Does not touch the existing infra/ (azd dev CosmosDB) or root azure.yaml.

Includes:
- main.bicep orchestrator + 7 modules (networking, aks, keyvault, cosmosdb,
  redis, storage, acr), using Azure Verified Modules where available
- main.parameters.json with sensible defaults
- Compiled azuredeploy.json for the Deploy-to-Azure button, plus a CI
  workflow (.github/workflows/deploy-azure-drift.yml) that fails if it
  drifts from main.bicep
- createUiDefinition.json with a Basics/Advanced portal UI flow
- metadata.json for the template gallery
- deploy/azure/README.md documenting prerequisites, the CLI deploy path,
  and what this does NOT automate (the Helm app install, BYO secrets)
- Deploy-to-Azure button + link added to the root README

All Bicep files compile cleanly via 'az bicep build' with zero errors and
zero warnings.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@github-actions github-actions Bot added type: documentation Documentation additions, corrections, and improvements. area: cicd Build, test, release, and deployment pipelines. labels Oct 8, 2026
@duffney Josh Duffney (duffney) changed the title Add sanitized Deploy-to-Azure Bicep template under deploy/azure/ Add Deploy-to-Azure Bicep template Oct 8, 2026
Josh Duffney (duffney) and others added 2 commits October 8, 2026 12:30
@duffney
Fix AKS module: retired preview API version causing deployment failure
becc7e4 
Upgrade avm/res/container-service/managed-cluster from 0.9.0 to 0.14.0.
The pinned 0.9.0/0.13.0 module versions generated a managedClusters
resource at API version 2024-09-02-preview, which Azure has since
retired (not present in the supported api-versions list), causing
deployments to fail with:

  NoRegisteredProviderFound: No registered resource provider found for
  location 'westus3' and API version '2024-09-02-preview' for type
  'managedClusters'.

0.14.0 generates the resource at 2025-10-01 (current, non-preview).
Updated param usage to match this version's schema: enablePrivateCluster
moved under apiServerAccessProfile, and enableWorkloadIdentity moved
under securityProfile.workloadIdentity.enabled. All module outputs
(resourceId, name, oidcIssuerUrl, kubeletIdentityObjectId,
systemAssignedMIPrincipalId) are unchanged between versions.

Verified: deploy/azure/modules/aks.bicep and deploy/azure/main.bicep
both compile with zero errors/warnings via 'az bicep build'; confirmed
2024-09-02-preview no longer appears anywhere in the compiled
azuredeploy.json.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney
Fix Cosmos DB module: MongoDB private endpoint rejected due to accoun…
5c50901 
…t kind

The AVM database-account module (br/public:avm/res/document-db/database-account:0.10.0)
derives the account's 'kind' property from whether mongodbDatabases is
non-empty, independent of capabilitiesToAdd. Passing an empty
mongodbDatabases array left the account at kind=GlobalDocumentDB (SQL
API) even with EnableMongo set, which then rejected the MongoDB-groupId
private endpoint with:

  BadRequest: Call to Microsoft.DocumentDB/databaseAccounts failed.
  Error message: GroupId MongoDB is not supported

Fixed by provisioning a default database (name configurable via the new
'databaseName' param, default 'scope') so the module correctly resolves
kind=MongoDB. The child mongodb-database module already skips setting
throughput when EnableServerless is present, so this has no cost impact
under the default serverless configuration.

Verified: deploy/azure/modules/cosmosdb.bicep and the full
deploy/azure/main.bicep both compile with zero errors/warnings via
'az bicep build'; confirmed the compiled azuredeploy.json's kind
expression now resolves to 'MongoDB' when mongodbDatabases is populated.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
manekinekko
Wassim Chegham (manekinekko) reviewed Oct 9, 2026
View reviewed changes

@manekinekko Wassim Chegham (manekinekko) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, Josh! The module separation, private-endpoint wiring, and secure connection-string outputs are a good foundation. I've left three initial comments on resource naming, the portal wizard, and the post-deployment instructions.

Comment thread deploy/azure/main.bicep

var vnetName = 'vnet-${environmentName}-${resourceToken}'
var aksName = 'aks-${environmentName}-${resourceToken}'
var keyVaultName = take('kv-${environmentName}-${resourceToken}', 24)

@manekinekko Wassim Chegham (manekinekko) Oct 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Preserve a valid name and the uniqueness suffix when truncating.

A 20-character environmentName is accepted by the template, but this expression produces a Key Vault name ending in a hyphen. For example, abcdefghijklmnopqrst becomes kv-abcdefghijklmnopqrst-, which Azure rejects. With a 19-character name, only one character of the uniqueness suffix survives.

Could we truncate the environment portion before appending the suffix, rather than truncating the completed resource name? The Storage account naming expression has the same suffix-truncation problem and should preserve its uniqueness suffix too.

Comment thread deploy/azure/createUiDefinition.json
}
]
},
"outputs": {

@manekinekko Wassim Chegham (manekinekko) Oct 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Put the output mappings inside parameters and connect this wizard to the buttons.

outputs is currently at the document root, but the CreateUiDefinition schema requires parameters.outputs to bind the controls to the ARM template parameters.

Also, both Deploy to Azure buttons reference only azuredeploy.json; neither references this UI definition, so users won't see the custom wizard or its validation. Could we move these mappings into parameters and wire the UI definition into both deployment links?

Comment thread deploy/azure/README.md

- **Installing Scope itself.** This template provisions infrastructure only. The
Scope application (API, workers, Judge, Portal, Token Manager) is installed
separately via Helm charts against the AKS cluster this template creates. See the

@manekinekko Wassim Chegham (manekinekko) Oct 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Point to an available application-install procedure.

This directs users to install Scope via Helm charts and sends them to the documentation site for instructions, but I couldn't find a Helm chart or Helm installation instructions. Is this part of a separate PR?

Josh Duffney (duffney) and others added 6 commits October 9, 2026 10:03
@duffney
Fix redis.bicep: missing dependency causes redis-primary-key secret w…
79de318 
…rite to race cache creation

The primaryKey output called listKeys() against a resourceId() built purely
from the redisName parameter, with no symbolic reference to the redis AVM
module. Bicep only infers an implicit dependsOn from symbolic-name
references, so this output had no dependency edge on the cache's actual
provisioning - ARM could evaluate listKeys() before the long-running Redis
create finished, which explains reports of the redis-primary-key secret
being missing from Key Vault after otherwise-successful deployments (the
cache itself provisions fine; only the secret write silently failed).

Fixed by declaring an existing resource reference for the cache (a
compile-time-known ID, valid as a listKeys() target) with an explicit
dependsOn: [redis] to force correct ordering.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney
Fix cosmosdb.bicep: wrong TLD on Mongo private DNS zone breaks privat…
3577230 
…e connectivity

The private DNS zone for the Cosmos DB MongoDB API private endpoint was
named 'privatelink.mongo.cosmos.azure.net' - wrong TLD, likely a copy/paste
mix-up with the Key Vault zone (privatelink.vaultcore.azure.net). The
canonical zone name, per the public DNS CNAME chain
(*.mongo.cosmos.azure.com -> *.privatelink.mongo.cosmos.azure.com), is
'privatelink.mongo.cosmos.azure.com' (.com).

With the wrong zone name, pods resolved the Cosmos Mongo hostname to its
public IP (no matching records in the misnamed zone), and Cosmos's
firewall rejected the connection since publicNetworkAccess is Disabled -
MongoServerSelectionError connecting to the public IP on port 10255.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney
Grant Key Vault Secrets Officer (not just Secrets User) to the worklo…
b3847c6 
…ad identity

The shared workload identity only had Key Vault Secrets User (read-only),
which is correct for the Helm chart's Secrets Store CSI mounts but blocks
the Token Manager service (apps/token-manager), which writes BYO
credentials (GitHub/Anthropic API keys, etc.) into the vault at runtime via
SecretClient.setSecret() using this same shared identity. Read-only access
403s that write path, leaving the portal's built-in credential
registration UI (POST/DELETE /api/v1/keys) non-functional out of the box
on a freshly provisioned environment.

Key Vault Secrets Officer is a superset of Secrets User (adds set/delete
alongside get/list on secrets, still scoped to secrets only - no key/cert
or permission-management access), so this replaces rather than stacks
alongside the previous role assignment.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney
Upgrade workload identity storage role: Blob Data Contributor -> Blob…
45a799c 
… Data Owner

Storage Blob Data Contributor's data actions do not include
Microsoft.Storage/storageAccounts/blobServices/containers/blobs/tags/write,
so the app's practice of setting Blob Index Tags (requestId/runId/iteration)
via the x-ms-tags header on per-iteration workspace snapshot uploads
403s with AuthorizationPermissionMismatch under Contributor alone.
100% reproducible, confirmed via raw PUT with/without x-ms-tags using a
storage-scoped AAD token exchanged from the workload identity's federated
token.

Storage Blob Data Owner includes the tags/write data action alongside
everything Contributor already grants, so this keeps tagged snapshot
uploads working without the app needing to drop tags.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney
Migrate redis.bicep from classic Azure Cache for Redis to Azure Manag…
38a2300 
…ed Redis

Classic Azure Cache for Redis (Microsoft.Cache/redis) is being retired for
new deployments, causing DeploymentFailed errors on the redis sub-deployment
("Azure Cache for Redis is retiring, create Azure Managed Redis instance
instead").

Rewrote redis.bicep to use the AVM module avm/res/cache/redis-enterprise:0.5.0
(Microsoft.Cache/redisEnterprise), targeting the "Balanced" Azure Managed
Redis SKU tier:

- Replaced the Basic/Standard/Premium skuName + C/P skuFamily + integer
  skuCapacity parameter trio with a single skuName enum of AMR SKU strings
  (Balanced_B0/B1/B3/B5/B10; capacity is now encoded in the SKU name itself).
  Default is Balanced_B1.
- Private endpoint service/groupId changed from redisCache to redisEnterprise;
  private DNS zone changed from privatelink.redis.cache.windows.net to
  privatelink.redisenterprise.cache.azure.net.
- Entra ID (Azure AD) data-plane auth is now granted via an access policy
  assignment on the database submodule (accessPolicyAssignments), replacing
  the classic module's redisConfiguration 'aad-enabled' flag. The shared
  workload identity's principal ID is threaded through from main.bicep (same
  pattern already used for storage.bicep).
- Access-key auth (accessKeysAuthentication: 'Enabled') is kept as a
  compatibility fallback; the primary access key is still surfaced as a
  @secure() output and written to the redis-primary-key Key Vault secret, now
  sourced directly from the AVM module's own output instead of a manual
  listKeys() call — the AMR database submodule's output is correctly ordered,
  so the existing-resource + explicit dependsOn workaround from the prior
  dependsOn fix is no longer needed.
- Updated main.bicep (params, redisName max length 63->60 per
  Microsoft.Cache/redisEnterprise naming rules, module wiring, outputs),
  main.parameters.json, createUiDefinition.json (single SKU dropdown, capacity
  slider removed), metadata.json, and README.md to match.

Verified with `az bicep build` on the module and full orchestrator (clean,
zero errors/warnings), `az bicep lint`, and a rebuild-and-diff drift check
against the committed azuredeploy.json (no drift).

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney
Fix InvalidTemplate failure in redis access policy assignment
7d3bd71 
Live deployment hit:

  Unable to process template language expressions for resource
  '.../redis-apa-0'... 'The language expression property name doesnt
  exist, available properties are userObjectId.' (Code: InvalidTemplate)

Root cause: the AVM redis-enterprise module's nested access-policy-assignment
deployment reads parameters('accessPolicyAssignments')[i].name directly
(not via tryGet), even though name is documented as optional/nullable on
accessPolicyAssignmentType. ARM strips omitted optional properties from the
JSON object entirely (there is no null placeholder), so direct property
access on a missing key throws InvalidTemplate. Our accessPolicyAssignments
array item only set userObjectId, omitting name as the (incorrect) docs
imply is fine.

Fix: explicitly set name: 'workload-identity-access' on the access policy
assignment entry in redis.bicep's database block, working around the
module's property-access bug.

Verified with az bicep build (module + full orchestrator, clean), az bicep
lint (clean), and a rebuild-and-diff drift check against the committed
azuredeploy.json (no drift) - confirmed the compiled ARM template now
includes the explicit name property on the access policy assignment.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2250d38f-729e-4dd7-b6e1-41b525e44a72
@duffney Josh Duffney (duffney) mentioned this pull request Oct 9, 2026
Open
6 tasks done

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@manekinekko Wassim Chegham (manekinekko) manekinekko left review comments
@cedricvidal Cedric Vidal (cedricvidal) Awaiting requested review from cedricvidal cedricvidal is a code owner

Assignees

No one assigned

Labels

area: cicd Build, test, release, and deployment pipelines. type: documentation Documentation additions, corrections, and improvements.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@duffney @manekinekko