Skip to content

Register existing file state into best-effort deployment history - #7171

Draft
shreyas-goenka wants to merge 2 commits into
shreyas-goenka_data/shreyas/history-best-effort-defaultfrom
shreyas-goenka_data/shreyas/history-best-effort-migration
Draft

Register existing file state into best-effort deployment history#7171
shreyas-goenka wants to merge 2 commits into
shreyas-goenka_data/shreyas/history-best-effort-defaultfrom
shreyas-goenka_data/shreyas/history-best-effort-migration

Conversation

@shreyas-goenka

@shreyas-goenka shreyas-goenka commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Enabling best-effort history through YAML or the environment flag now attempts INITIAL_REGISTER for existing file state before normal deployment planning. IDs, payloads, and dependencies remain in resources.json. Empty snapshots skip registration versions.

  • Acquire the deployment lock and recheck state before registration, preventing an older snapshot from overwriting a newer deploy.
  • Back up remote state, register at max(file serial, latest DMS version) + 1, and persist the enforced-history marker as a separate phase.
  • Warn on DMS recording errors; file persistence errors remain fatal. Migration stays committed if later planning fails or deployment is declined.

Saved plans cannot enable initial registration; Terraform state must first migrate to direct with history disabled. Existing DMS-only state remains supported.

Stack: #7151 → #7169 → #7171 (this PR). Adapts #6989.

Validation: affected Go/race tests and lint pass. Acceptance coverage checks backups, IDs/dependencies, repeat registration, planning failure/decline, environment-only opt-in, and fail-open uploads. Regression tests cover state changing before migration and independent file/service counters.

@shreyas-goenka shreyas-goenka mentioned this pull request Oct 23, 2026
Draft
@github-actions

github-actions Bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Commit: a0f283ddb | Workflow run

Regression test report

Selected 20 changed tests (limit=20, 8 not selected)

21 tests: regression=19 unreleased=0 coverage=2 failing=0

test branch main (4d88efe) latest (v1.22.1)
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS= ✅ ✅ ✅
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/declined-deploy/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/depends-on/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/deployment-metadata-change/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/existing-state/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/failed-delete/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/failed-recreate/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/migration-before-deploy/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/migration-fails-open/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/migration/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/mismatch-before-read/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/no-drift/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/operation-upload-fails-once/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/plan-without-dms/DMS= ✅ ✅ ✅
TestAccept/bundle/dms/record-failure/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/repeated-migration/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/stale-local-state/DMS= ✅ ❌ ❌
TestAccept/bundle/dms/state-features/DMS=true ✅ ❌ ❌
TestAccept/bundle/dms/version-never-created/DMS=true ✅ ❌ ❌
TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1311
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1081
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n>>> musterr [CLI] bundle deploy --plan plan.json\nWarn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]\nError: plan lineage \"different-lineage\" does not match state lineage \"\"; the state may have been modified by another process\n\n"
        	            	actual  : "\n>>> musterr [CLI] bundle deploy --plan plan.json\nWarn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]\nError: this plan was created for a different set of state features than the target now has; run 'bundle plan' again\n\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,3 +3,3 @@
        	            	 Warn: Plan was created with CLI version 0.0.0-test but current version is [CLI_VERSION]
        	            	-Error: plan lineage "different-lineage" does not match state lineage ""; the state may have been modified by another process
        	            	+Error: this plan was created for a different set of state features than the target now has; run 'bundle plan' again
        	            	 
        	Test:       	TestAccept/bundle/deploy/readplan/lineage-mismatch/DMS=true
        	Messages:   	bundle/deploy/readplan/lineage-mismatch/output.txt vs /tmp/TestAcceptbundledeployreadplanlineage-mismatchDMS=true3400071297/002/output.txt

latest (v1.22.1): same output as main (4d88efe)

TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1311
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1081
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:659
        	Error:      	Not equal: 
        	            	expected: "\n=== bind is refused while recording deployment history\n>>> musterr [CLI] bundle deployment bind foo [JOB_ID]\nError: bind is not supported for a bundle target that records deployment history\n\n\n=== Unbind is allowed for file-backed history state\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-bind-unbind-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> [CLI] bundle deployment unbind foo\n\n>>> jq {features, state} .databricks/bundle/default/resources.json\n{\n  \"features\": null,\n  \"state\": {}\n}\n"
        	            	actual  : "\n=== bind is refused while recording deployment history\n>>> musterr [CLI] bundle deployment bind foo [JOB_ID]\nError: bind is not supported for a bundle target that records deployment history\n\n\n=== Unbind is allowed for file-backed history state\n>>> [CLI] bundle deploy\nUploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-bind-unbind-[UNIQUE_NAME]/default/files...\nCreated jobs.foo\nFiles: 2 uploaded, 0 deleted\nResources: 1 created, 0 changed, 0 deleted, 0 unchanged\n\n>>> [CLI] bundle deployment unbind foo\nError: unbind is not supported for a bundle target that records deployment history\n\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -14,8 +14,6 @@
        	            	 >>> [CLI] bundle deployment unbind foo
        	            	+Error: unbind is not supported for a bundle target that records deployment history
        	            	 
        	            	->>> jq {features, state} .databricks/bundle/default/resources.json
        	            	-{
        	            	-  "features": null,
        	            	-  "state": {}
        	            	-}
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/bundle/dms/bind-unbind-unsupported/DMS=true
        	Messages:   	bundle/dms/bind-unbind-unsupported/output.txt vs /tmp/TestAcceptbundledmsbind-unbind-unsupportedDMS=true1245855442/002/output.txt

latest (v1.22.1): same output as main (4d88efe)

TestAccept/bundle/dms/declined-deploy/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/declined-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsdeclined-deployDMS=true4100836320/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -81,8 +81,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         >>> print_state.py --no-dms
         {
        @@ -91,17 +97,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_declined_deploy_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_declined_deploy_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A destructive change without --auto-approve is declined: this console cannot prompt
        @@ -124,6 +121,10 @@
           "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The recorded resources are unchanged by the decline: the schema is still at version 1
         >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        @@ -160,6 +161,10 @@
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
           "method": "POST",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
           "q": {
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793015103469,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_a3e3cqx4krdibiwo2tsttwi3t4",
        LOG.catalog:   "metastore_id": "7a0710a6-32ed-5872-7a07-10a632ed5962",
        LOG.catalog:   "name": "dms_other_a3e3cqx4krdibiwo2tsttwi3t4",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793015103469,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }

latest (v1.22.1):

acceptance_test.go:1311: Diff:
        --- bundle/dms/declined-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsdeclined-deployDMS=true542532682/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -81,8 +81,16 @@
         }
         {
           "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
        +  "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         >>> print_state.py --no-dms
         {
        @@ -91,17 +97,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_declined_deploy_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "name": "dms_declined_deploy_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A destructive change without --auto-approve is declined: this console cannot prompt
        @@ -124,6 +121,10 @@
           "method": "GET",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
        +{
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
         
         === The recorded resources are unchanged by the decline: the schema is still at version 1
         >>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-resources deployments/[DEPLOYMENT_ID] -o json
        @@ -160,6 +161,10 @@
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"
         }
         {
        +  "method": "GET",
        +  "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/resources"
        +}
        +{
           "method": "POST",
           "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions",
           "q": {
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793015115449,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_mhbvx4gsvzhhzclqby6ur57e2e",
        LOG.catalog:   "metastore_id": "7a0710a8-fd06-8a26-7a07-10a8fd068b17",
        LOG.catalog:   "name": "dms_other_mhbvx4gsvzhhzclqby6ur57e2e",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793015115449,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
FAIL
TestAccept/bundle/dms/depends-on/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/depends-on/output.txt
        +++ /tmp/TestAcceptbundledmsdepends-onDMS=true3931880826/002/output.txt
        @@ -5,7 +5,7 @@
           "plan_version": [PLAN_VERSION],
           "cli_version": "[CLI_VERSION]",
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -61,7 +61,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -91,53 +91,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.child": {
        -   "__id__": "[CHILD_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "description": "depends on [PARENT_ID]",
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "child",
        -    "queue": {
        -     "enabled": true
        -    }
        -   },
        -   "depends_on": [
        -    {
        -     "node": "resources.jobs.parent",
        -     "label": "${resources.jobs.parent.id}"
        -    }
        -   ]
        -  },
        -  "resources.jobs.parent": {
        -   "__id__": "[PARENT_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-depends-on-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "parent",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         >>> [CLI] bundle destroy --auto-approve

latest (v1.22.1): same output as main (4d88efe)

TestAccept/bundle/dms/deployment-metadata-change/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/deployment-metadata-change/output.txt
        +++ /tmp/TestAcceptbundledmsdeployment-metadata-changeDMS=true2451752711/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -128,7 +128,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }

latest (v1.22.1): same output as main (4d88efe)

TestAccept/bundle/dms/existing-state/DMS=: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/existing-state/output.txt
        +++ /tmp/TestAcceptbundledmsexisting-stateDMS=1226068654/002/output.txt
        @@ -27,64 +27,9 @@
         
         === The environment flag enables best-effort history for the existing deployment
         >>> DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=true [CLI] bundle deploy
        -Enabling deployment history...
        -Deployment history enabled: [DATABRICKS_URL]/deployments/[DEPLOYMENT_ID]?version=2
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files...
        -Updated jobs.one
        -Files: 0 uploaded, 0 deleted
        -Resources: 0 created, 1 changed, 0 deleted, 0 unchanged
        +Error: deployment history setting (true) does not match the existing state (false)
         
        ->>> print_requests.py --dms //api/2.0/bundle --oneline
        -{"method": "POST", "path": "/api/2.0/bundle/deployments", "body": {"display_name": "dms-existing-state-[UNIQUE_NAME]", "initial_parent_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state", "target_name": "default", "workspace_info": {"file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files", "root_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default"}}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "2"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_INITIAL_REGISTER", "resource_key": "jobs.one"}], "version_type": "VERSION_TYPE_DEPLOY"}}
        -{"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/operations/jobs.one", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[ONE_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state/metadata.json\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"one\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "3"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_UPDATE", "resource_key": "jobs.one"}], "previous_version_id": "2", "version_type": "VERSION_TYPE_DEPLOY"}}
        -{"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/operations/jobs.one", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[ONE_ID]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"deployment_id\":\"[DEPLOYMENT_ID]\",\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state/metadata.json\",\"version_id\":\"3\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"one\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/3/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        +Update experimental.deployment_history to match the existing deployment, or run "databricks bundle destroy" to start over
         
        -=== Wiping the local cache pulls history-enabled file state without repeating registration
        ->>> DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=true [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files...
        -Files: 2 uploaded, 0 deleted
        -Resources: 0 created, 0 changed, 0 deleted, 1 unchanged
         
        -=== A leftover local WAL with the wrong lineage is still refused
        ->>> DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=true [CLI] bundle deploy
        -Error: reading state from [TEST_TMP_DIR]/.databricks/bundle/default/resources.json: WAL recovery failed: WAL lineage ("stale") does not match state lineage ("[UUID]")
        -
        -
        -=== Destroy clears the deployment. With nothing recorded remotely the bundle is treated as new, so recording can be enabled once the stale WAL is gone
        ->>> DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=false [CLI] bundle destroy --auto-approve
        -The following resources will be deleted:
        -  delete resources.jobs.one
        -
        -All files and directories at the following location will be deleted: /Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default
        -
        -Destroy: 1 deleted
        -
        ->>> DATABRICKS_BUNDLE_DEPLOYMENT_HISTORY=true [CLI] bundle deploy
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files...
        -Created jobs.one
        -Files: 2 uploaded, 0 deleted
        -Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
        -
        ->>> print_requests.py --dms //api/2.0/bundle --oneline
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions", "q": {"version_id": "4"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_DELETE", "resource_key": "jobs.one"}], "previous_version_id": "3", "version_type": "VERSION_TYPE_DESTROY"}}
        -{"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/4/operations/jobs.one", "q": {"update_mask": "state,error_message,status"}, "body": {"error_message": "", "sequence_id": 0, "status": "OPERATION_STATUS_SUCCEEDED"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/4/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        -{"method": "DELETE", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID]"}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments", "body": {"display_name": "dms-existing-state-[UNIQUE_NAME]", "initial_parent_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state", "target_name": "default", "workspace_info": {"file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/files", "root_path": "/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default"}}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID_2]/versions", "q": {"version_id": "1"}, "body": {"cli_version": "[CLI_VERSION]", "operations": [{"action_type": "OPERATION_ACTION_TYPE_CREATE", "resource_key": "jobs.one"}], "version_type": "VERSION_TYPE_DEPLOY"}}
        -{"method": "PATCH", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID_2]/versions/1/operations/jobs.one", "q": {"update_mask": "state,error_message,resource_id,status"}, "body": {"error_message": "", "resource_id": "[ONE_ID_2]", "sequence_id": 0, "state": "{\"state\":{\"deployment\":{\"deployment_id\":\"[DEPLOYMENT_ID_2]\",\"kind\":\"BUNDLE\",\"metadata_file_path\":\"/Workspace/Users/[USERNAME]/.bundle/dms-existing-state-[UNIQUE_NAME]/default/state/metadata.json\",\"version_id\":\"1\"},\"edit_mode\":\"UI_LOCKED\",\"format\":\"MULTI_TASK\",\"max_concurrent_runs\":1,\"name\":\"one\",\"queue\":{\"enabled\":true}}}", "status": "OPERATION_STATUS_SUCCEEDED"}}
        -{"method": "POST", "path": "/api/2.0/bundle/deployments/[DEPLOYMENT_ID_2]/versions/1/complete", "body": {"completion_reason": "VERSION_COMPLETE_SUCCESS"}}
        -
        ->>> find.py resources[.]json[.]wal --expect 0
        -
        ->>> jq {state_version, features} .databricks/bundle/default/resources.json
        -{
        -  "state_version": 3,
        -  "features": {
        -    "deployment_history_best_effort": {}
        -  }
        -}
        +Exit code: 1

latest (v1.22.1): same output as main (4d88efe)

TestAccept/bundle/dms/failed-delete/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-delete/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-deleteDMS=true385399027/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,28 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A resource that fails to delete is recorded as a failed operation carrying the error, so the history says why rather than showing the resource gone
        @@ -201,30 +181,11 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
        + "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === The remote checkpoint stays at serial 1 while DMS advances to version 2
        
FAIL

latest (v1.22.1):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-delete/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-deleteDMS=true2761838469/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,28 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A resource that fails to delete is recorded as a failed operation carrying the error, so the history says why rather than showing the resource gone
        @@ -201,30 +181,11 @@
          "state_version": 3,
          "cli_version": "[CLI_VERSION]",
          "lineage": "[UUID]",
        - "serial": 1,
        + "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.jobs.stuck": {
        -   "__id__": "[STUCK_ID]",
        -   "state": {
        -    "deployment": {
        -     "deployment_id": "[DEPLOYMENT_ID]",
        -     "kind": "BUNDLE",
        -     "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/dms-failed-delete-[UNIQUE_NAME]/default/state/metadata.json",
        -     "version_id": "1"
        -    },
        -    "edit_mode": "UI_LOCKED",
        -    "format": "MULTI_TASK",
        -    "max_concurrent_runs": 1,
        -    "name": "stuck",
        -    "queue": {
        -     "enabled": true
        -    }
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === The remote checkpoint stays at serial 1 while DMS advances to version 2
TestAccept/bundle/dms/failed-recreate/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-recreate/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-recreateDMS=true2413774046/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,18 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_failed_recreate_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_failed_recreate_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate deletes and then fails to create. The delete went through, so the resource really is gone: the failed operation clears its state (self-contained, not assuming the delete's own write reached the service), rather than describing the resource as it was before the deploy
        @@ -188,7 +178,7 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793015102208,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_u6uoh6kchnaarh3wakxz363rcy",
        LOG.catalog:   "metastore_id": "7a0710a5-e7cf-5ed0-7a07-10a5e7cf5fac",
        LOG.catalog:   "name": "dms_other_u6uoh6kchnaarh3wakxz363rcy",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793015102208,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
    acceptance_test.go:1118: 
        LOG.plan_final: create schemas.foo
        LOG.plan_final: 
        LOG.plan_final: Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged

latest (v1.22.1):

acceptance_test.go:1311: Diff:
        --- bundle/dms/failed-recreate/output.txt
        +++ /tmp/TestAcceptbundledmsfailed-recreateDMS=true3060603710/002/output.txt
        @@ -67,7 +67,7 @@
           "lineage": "[UUID]",
           "serial": 1,
           "features": {
        -    "deployment_history_best_effort": {}
        +    "deployment_history": {}
           }
         }
         
        @@ -92,18 +92,9 @@
          "lineage": "[UUID]",
          "serial": 1,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
        - "state": {
        -  "resources.schemas.foo": {
        -   "__id__": "main.dms_failed_recreate_schema_[UNIQUE_NAME]",
        -   "state": {
        -    "catalog_name": "main",
        -    "comment": "v1",
        -    "name": "dms_failed_recreate_schema_[UNIQUE_NAME]"
        -   }
        -  }
        - }
        + "state": {}
         }
         
         === A recreate deletes and then fails to create. The delete went through, so the resource really is gone: the failed operation clears its state (self-contained, not assuming the delete's own write reached the service), rather than describing the resource as it was before the deploy
        @@ -188,7 +178,7 @@
          "lineage": "[UUID]",
          "serial": 2,
          "features": {
        -  "deployment_history_best_effort": {}
        +  "deployment_history": {}
          },
          "state": {}
         }
        
    acceptance_test.go:1118: 
        LOG.catalog: {
        LOG.catalog:   "catalog_type": "MANAGED_CATALOG",
        LOG.catalog:   "created_at": 1793015112788,
        LOG.catalog:   "created_by": "tester@databricks.com",
        LOG.catalog:   "full_name": "dms_other_afzyip7uezgsfhv26u34ayk4ji",
        LOG.catalog:   "metastore_id": "7a0710a8-5e6c-ec14-7a07-10a85e6cecf1",
        LOG.catalog:   "name": "dms_other_afzyip7uezgsfhv26u34ayk4ji",
        LOG.catalog:   "owner": "tester@databricks.com",
        LOG.catalog:   "updated_at": 1793015112788,
        LOG.catalog:   "updated_by": "tester@databricks.com"
        LOG.catalog: }
    acceptance_test.go:1118: 
        LOG.plan_final: create schemas.foo
        LOG.plan_final: 
        LOG.plan_final: Plan: 1 to add, 0 to change, 0 to delete, 0 unchanged
TestAccept/bundle/dms/migration-before-deploy/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/migration-before-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsmigration-before-deployDMS=true2497996233/002/output.txt
        @@ -6,74 +6,10 @@
         >>> fault.py GET /api/2.2/jobs/get* 403 0 1
         
         >>> musterr [CLI] bundle deploy -qq
        -Enabling deployment history...
        -Deployment history enabled: [DATABRICKS_URL]/deployments/[DEPLOYMENT_ID]?version=2
        -Error: cannot plan resources.jobs.job: reading id="[JOB_ID]": Fault injected by test. (403 INJECTED)
        +Error: deployment history setting (true) does not match the existing state (false)
         
        -Endpoint: GET [DATABRICKS_URL]/api/2.2/jobs/get?job_id=[JOB_ID]
        -HTTP Status: 403 Forbidden
        -API error_code: INJECTED
        -API message: Fault injected by test.
        +Update experimental.deployment_history to match the existing deployment, or run "databricks bundle destroy" to start over
         
        -Error: planning failed
        +Error: Path (/Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.deployment.json) doesn't exist.
         
        -
        -=== Request order: complete migration -> upload full file state -> plan jobs
        ->>> print_requests.py --dms --get //api/2.0/bundle //jobs/get //workspace/import //workspace-files/import-file --oneline
        -{"method":"POST","path":"/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/complete"}
        -{"method":"POST","path":"/Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json"}
        -{"method":"GET","path":"/api/2.2/jobs/get"}
        -
        -=== Remote state: version 2 file state with resources; original snapshot preserved in the backup
        ->>> MSYS_NO_PATHCONV=1 [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json --format AUTO --file .databricks/resources.marker.json
        -
        ->>> jq -c {serial, features, state} .databricks/resources.marker.json
        -{"serial":2,"features":{"deployment_history_best_effort":{}},"state":{"resources.jobs.job":{"__id__":"[JOB_ID]","state":{"deployment":{"kind":"BUNDLE","metadata_file_path":"/Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/metadata.json"},"edit_mode":"UI_LOCKED","format":"MULTI_TASK","max_concurrent_runs":1,"name":"migration-before-deploy","queue":{"enabled":true}}},"resources.schemas.schema":{"__id__":"[SCHEMA_ID]","state":{"catalog_name":"main","name":"dms_migration_before_deploy_[UNIQUE_NAME]"}}}}
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json.backup --format AUTO --file .databricks/resources.backup.json
        -
        ->>> cmp .databricks/resources.before-migration.json .databricks/resources.backup.json
        -
        -=== Migration result: only completed version 2, with INITIAL_REGISTER operations
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-operations deployments/[DEPLOYMENT_ID]/versions/2 -o json
        -[{"resource_key":"jobs.job","action_type":"OPERATION_ACTION_TYPE_INITIAL_REGISTER","status":"OPERATION_STATUS_SUCCEEDED"},{"resource_key":"schemas.schema","action_type":"OPERATION_ACTION_TYPE_INITIAL_REGISTER","status":"OPERATION_STATUS_SUCCEEDED"}]
        -
        -=== 2. Retry without approval
        -Expect: deployment declined; migration stays at version 2 and is not repeated.
        ->>> musterr [CLI] bundle deploy -qq
        -
        -This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
        -  recreate resources.schemas.schema
        -Error: the deployment requires destructive actions, but the current console does not support prompting.
        -Deleting data assets such as schemas, pipelines, or volumes may cause permanent data loss and should be carefully reviewed.
        -To proceed, use --auto-approve after reviewing the plan above.
        -
        -
        -=== Local and remote markers must stay byte-for-byte unchanged
        ->>> cmp .databricks/resources.marker.json .databricks/bundle/default/resources.json
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json --format AUTO --file .databricks/resources.after-decline.json
        -
        ->>> cmp .databricks/resources.marker.json .databricks/resources.after-decline.json
        -
        -=== No DMS writes and no extra version on the declined retry
        ->>> print_requests.py --dms //api/2.0/bundle
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        -
        -=== 3. Approve deployment
        -Expect: ordinary deployment version 3; version 2 remains the only migration.
        ->>> [CLI] bundle deploy --auto-approve -qq
        -
        -This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
        -  recreate resources.schemas.schema
        -
        ->>> [CLI] bundle summary -o json
        -{"latest_version_id":3}
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"},{"version_id":"3","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        +Exit code: 1
        
    acceptance_test.go:1118: 
        LOG.cleanup: Warn: planning resources.jobs.job: reading resources.jobs.job id="8793015099603535746": Fault injected by test.
        LOG.cleanup: The following resources will be deleted:
        LOG.cleanup:   delete resources.jobs.job
        LOG.cleanup:   delete resources.schemas.schema
        LOG.cleanup: 
        LOG.cleanup: This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
        LOG.cleanup:   delete resources.schemas.schema
        LOG.cleanup: 
        LOG.cleanup: All files and directories at the following location will be deleted: /Workspace/Users/tester@databricks.com/.bundle/dms-migration-before-deploy-jya5k6gedfhnpnhleeh6e2sdfy/default
        LOG.cleanup: 
        LOG.cleanup: Destroy: 2 deleted
    acceptance_test.go:1118: 
        LOG.setup: {
        LOG.setup:   "catalog_type": "MANAGED_CATALOG",
        LOG.setup:   "created_at": 1793015099769,
        LOG.setup:   "created_by": "tester@databricks.com",
        LOG.setup:   "full_name": "dms_other_jya5k6gedfhnpnhleeh6e2sdfy",
        LOG.setup:   "metastore_id": "7a0710a5-566f-5bbf-7a07-10a5566f5c9b",
        LOG.setup:   "name": "dms_other_jya5k6gedfhnpnhleeh6e2sdfy",
        LOG.setup:   "owner": "tester@databricks.com",
        LOG.setup:   "updated_at": 1793015099769,
        LOG.setup:   "updated_by": "tester@databricks.com"
        LOG.setup: }

latest (v1.22.1):

acceptance_test.go:1311: Diff:
        --- bundle/dms/migration-before-deploy/output.txt
        +++ /tmp/TestAcceptbundledmsmigration-before-deployDMS=true4111869773/002/output.txt
        @@ -6,74 +6,10 @@
         >>> fault.py GET /api/2.2/jobs/get* 403 0 1
         
         >>> musterr [CLI] bundle deploy -qq
        -Enabling deployment history...
        -Deployment history enabled: [DATABRICKS_URL]/deployments/[DEPLOYMENT_ID]?version=2
        -Error: cannot plan resources.jobs.job: reading id="[JOB_ID]": Fault injected by test. (403 INJECTED)
        +Error: deployment history setting (true) does not match the existing state (false)
         
        -Endpoint: GET [DATABRICKS_URL]/api/2.2/jobs/get?job_id=[JOB_ID]
        -HTTP Status: 403 Forbidden
        -API error_code: INJECTED
        -API message: Fault injected by test.
        +Update experimental.deployment_history to match the existing deployment, or run "databricks bundle destroy" to start over
         
        -Error: planning failed
        +Error: Path (/Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.deployment.json) doesn't exist.
         
        -
        -=== Request order: complete migration -> upload full file state -> plan jobs
        ->>> print_requests.py --dms --get //api/2.0/bundle //jobs/get //workspace/import //workspace-files/import-file --oneline
        -{"method":"POST","path":"/api/2.0/bundle/deployments/[DEPLOYMENT_ID]/versions/2/complete"}
        -{"method":"POST","path":"/Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json"}
        -{"method":"GET","path":"/api/2.2/jobs/get"}
        -
        -=== Remote state: version 2 file state with resources; original snapshot preserved in the backup
        ->>> MSYS_NO_PATHCONV=1 [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json --format AUTO --file .databricks/resources.marker.json
        -
        ->>> jq -c {serial, features, state} .databricks/resources.marker.json
        -{"serial":2,"features":{"deployment_history_best_effort":{}},"state":{"resources.jobs.job":{"__id__":"[JOB_ID]","state":{"deployment":{"kind":"BUNDLE","metadata_file_path":"/Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/metadata.json"},"edit_mode":"UI_LOCKED","format":"MULTI_TASK","max_concurrent_runs":1,"name":"migration-before-deploy","queue":{"enabled":true}}},"resources.schemas.schema":{"__id__":"[SCHEMA_ID]","state":{"catalog_name":"main","name":"dms_migration_before_deploy_[UNIQUE_NAME]"}}}}
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json.backup --format AUTO --file .databricks/resources.backup.json
        -
        ->>> cmp .databricks/resources.before-migration.json .databricks/resources.backup.json
        -
        -=== Migration result: only completed version 2, with INITIAL_REGISTER operations
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-operations deployments/[DEPLOYMENT_ID]/versions/2 -o json
        -[{"resource_key":"jobs.job","action_type":"OPERATION_ACTION_TYPE_INITIAL_REGISTER","status":"OPERATION_STATUS_SUCCEEDED"},{"resource_key":"schemas.schema","action_type":"OPERATION_ACTION_TYPE_INITIAL_REGISTER","status":"OPERATION_STATUS_SUCCEEDED"}]
        -
        -=== 2. Retry without approval
        -Expect: deployment declined; migration stays at version 2 and is not repeated.
        ->>> musterr [CLI] bundle deploy -qq
        -
        -This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
        -  recreate resources.schemas.schema
        -Error: the deployment requires destructive actions, but the current console does not support prompting.
        -Deleting data assets such as schemas, pipelines, or volumes may cause permanent data loss and should be carefully reviewed.
        -To proceed, use --auto-approve after reviewing the plan above.
        -
        -
        -=== Local and remote markers must stay byte-for-byte unchanged
        ->>> cmp .databricks/resources.marker.json .databricks/bundle/default/resources.json
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] workspace export /Workspace/Users/[USERNAME]/.bundle/dms-migration-before-deploy-[UNIQUE_NAME]/default/state/resources.json --format AUTO --file .databricks/resources.after-decline.json
        -
        ->>> cmp .databricks/resources.marker.json .databricks/resources.after-decline.json
        -
        -=== No DMS writes and no extra version on the declined retry
        ->>> print_requests.py --dms //api/2.0/bundle
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        -
        -=== 3. Approve deployment
        -Expect: ordinary deployment version 3; version 2 remains the only migration.
        ->>> [CLI] bundle deploy --auto-approve -qq
        -
        -This action will result in the deletion or recreation of the following UC schemas. Any underlying data may be lost:
        -  recreate resources.schemas.schema
        -
        ->>> [CLI] bundle summary -o json
        -{"latest_version_id":3}
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"},{"version_id":"3","status":"VERSION_STATUS_COMPLETED","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        +Exit code: 1
        
    acceptance_test.go:1118: 
        LOG.cleanup: Warn: planning resources.jobs.job: reading resources.jobs.job id="8793015111855135264": Fault injected by test.
        LOG.cleanup: The following resources will be deleted:
        LOG.cleanup:   delete resources.jobs.job
        LOG.cleanup:   delete resources.schemas.schema
        LOG.cleanup: 
        LOG.cleanup: This action will result in the deletion of the following UC schemas. Any underlying data may be lost:
        LOG.cleanup:   delete resources.schemas.schema
        LOG.cleanup: 
        LOG.cleanup: All files and directories at the following location will be deleted: /Workspace/Users/tester@databricks.com/.bundle/dms-migration-before-deploy-qudvxzbbmjgaled454th2cgoba/default
        LOG.cleanup: 
        LOG.cleanup: Destroy: 2 deleted
    acceptance_test.go:1118: 
        LOG.setup: {
        LOG.setup:   "catalog_type": "MANAGED_CATALOG",
        LOG.setup:   "created_at": 1793015111966,
        LOG.setup:   "created_by": "tester@databricks.com",
        LOG.setup:   "full_name": "dms_other_qudvxzbbmjgaled454th2cgoba",
        LOG.setup:   "metastore_id": "7a0710a8-2d6e-5bcd-7a07-10a82d6e5caa",
        LOG.setup:   "name": "dms_other_qudvxzbbmjgaled454th2cgoba",
        LOG.setup:   "owner": "tester@databricks.com",
        LOG.setup:   "updated_at": 1793015111966,
        LOG.setup:   "updated_by": "tester@databricks.com"
        LOG.setup: }
TestAccept/bundle/dms/migration-fails-open/DMS=true: branch ✅ | main (4d88efe) ❌ | latest (v1.22.1) ❌

main (4d88efe):

acceptance_test.go:1311: Diff:
        --- bundle/dms/migration-fails-open/output.txt
        +++ /tmp/TestAcceptbundledmsmigration-fails-openDMS=true110686942/002/output.txt
        @@ -15,37 +15,9 @@
         >>> update_file.py databricks.yml name: one name: updated
         
         >>> [CLI] bundle deploy
        -Enabling deployment history...
        -Warn: Initial history registration was incomplete; continuing with file state: recording migration version: recording operation for resources.jobs.one: Fault injected by test.
        -Deployment history enabled: [DATABRICKS_URL]/deployments/[DEPLOYMENT_ID]?version=2
        -Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/dms-migration-fails-open-[UNIQUE_NAME]/default/files...
        -Updated jobs.one
        -Updated jobs.two
        -Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 2 changed, 0 deleted, 0 unchanged
        +Error: deployment history setting (true) does not match the existing state (false)
         
        -=== The file still owns both resources, with the original IDs
        ->>> jq {serial, features, resources: (.state | keys), names: (.state | with_entries(.value = .value.state.name))} .databricks/bundle/default/resources.json
        -{
        -  "serial": 3,
        -  "features": {
        -    "deployment_history_best_effort": {}
        -  },
        -  "resources": [
        -    "resources.jobs.one",
        -    "resources.jobs.two"
        -  ],
        -  "names": {
        -    "resources.jobs.one": "updated",
        -    "resources.jobs.two": "two"
        -  }
        -}
        +Update experimental.deployment_history to match the existing deployment, or run "databricks bundle destroy" to start over
         
        ->>> cmp .databricks/ids.before.json .databricks/ids.after.json
         
        -=== Migration failed, but the following deployment completed successfully
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-versions deployments/[DEPLOYMENT_ID] -o json
        -[{"version_id":"2","completion_reason":"VERSION_COMPLETE_FAILURE"},{"version_id":"3","completion_reason":"VERSION_COMPLETE_SUCCESS"}]
        -
        ->>> MSYS_NO_PATHCONV=1 [CLI] bundle-deployments list-operations deployments/[DEPLOYMENT_ID]/versions/2 -o json
        -[{"resource_key":"jobs.one","status":"OPERATION_STATUS_PENDING"},{"resource_key":"jobs.two","status":"OPERATION_STATUS_SUCCEEDED"}]
        +Exit code: 1

latest (v1.22.1): same output as main (4d88efe)

9 more failure details omitted, see the full report in the run.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: a0f283d

Run: 40316689865

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux-2core-8gb 312 28 8:02
✅​ aws-windows-latest-4core-16gb 314 26 8:17
✅​ azure linux-2core-8gb 311 28 15:09
🔄​ azure-windows-latest-4core-16gb 3 310 26 12:04
🔄​ gcp linux-2core-8gb 4 308 28 17:07
✅​ gcp-windows-latest-4core-16gb 314 26 10:13
7 interesting tests: 7 flaky
Test Name azure-windows-latest-4core-16gb gcp linux-2core-8gb
🔄​ TestAccept 🔄​f ✅​p
🔄​ TestAccept/bundle/dms/declined-deploy ✅​p 🔄​f
🔄​ TestAccept/bundle/dms/declined-deploy/DMS=true ✅​p 🔄​f
🔄​ TestAccept/bundle/dms/stale-local-state 🔄​f ✅​p
🔄​ TestAccept/bundle/dms/stale-local-state/DMS= 🔄​f ✅​p
🔄​ TestAccept/bundle/dms/state-features ✅​p 🔄​f
🔄​ TestAccept/bundle/dms/state-features/DMS=true ✅​p 🔄​f
Top 23 slowest tests (at least 2 minutes):
duration env testname
4:21 aws-windows-latest-4core-16gb TestAccept
4:00 gcp-windows-latest-4core-16gb TestFilerReadDir/workspace_files_extensions
3:47 aws linux-2core-8gb TestAccept
3:42 azure linux-2core-8gb TestAccept
3:36 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/repeated-migration/DMS=true
3:25 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/existing-state/DMS=
3:19 azure-windows-latest-4core-16gb TestAccept/bundle/dms/existing-state/DMS=
3:06 azure linux-2core-8gb TestAccept/bundle/dms/repeated-migration/DMS=true
3:00 gcp-windows-latest-4core-16gb TestAccept
2:43 azure-windows-latest-4core-16gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=
2:43 gcp linux-2core-8gb TestAccept/bundle/dms/existing-state/DMS=
2:21 azure-windows-latest-4core-16gb TestAccept/bundle/dms/migration/DMS=
2:20 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/deployment-metadata-change/DMS=true
2:18 azure linux-2core-8gb TestFilerReadWrite/workspace_files
2:13 gcp linux-2core-8gb TestImportFileUsingContentFormatAuto
2:13 gcp-windows-latest-4core-16gb TestAccept/bundle/dms/successful-recreate/DMS=true
2:11 gcp linux-2core-8gb TestAccept/bundle/dms/deployment-metadata-change/DMS=true
2:09 gcp linux-2core-8gb TestFilerWorkspaceFilesExtensionsDelete
2:08 azure linux-2core-8gb TestAccept/bundle/dms/stale-plan/DMS=true/READPLAN=1
2:04 azure linux-2core-8gb TestAccept/bundle/dms/migration/DMS=
2:03 gcp linux-2core-8gb TestAccept/bundle/dms/repeated-migration/DMS=true
2:02 azure-windows-latest-4core-16gb TestAccept/bundle/dms/state-features/DMS=true
2:01 azure linux-2core-8gb TestAccept/bundle/dms/existing-state/DMS=

@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch 3 times, most recently from f9da0d8 to 10da8f6 Compare October 26, 2026 05:16
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 4017609 to 0b70621 Compare October 26, 2026 05:25
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 10da8f6 to e4f88b3 Compare October 26, 2026 05:25
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 0b70621 to d2f84b5 Compare October 26, 2026 05:35
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from e4f88b3 to 06e09d2 Compare October 26, 2026 05:35
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from d2f84b5 to 7767812 Compare October 26, 2026 09:41
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 06e09d2 to a682521 Compare October 26, 2026 09:41
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 7767812 to fd7ab56 Compare October 26, 2026 10:00
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from a682521 to 4661cf7 Compare October 26, 2026 10:00
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from fd7ab56 to 9ca0c3e Compare October 26, 2026 10:04
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 4661cf7 to 24b5917 Compare October 26, 2026 10:04
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 9ca0c3e to 30e3aef Compare October 26, 2026 10:19
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 24b5917 to 6a7fbc8 Compare October 26, 2026 10:19
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 30e3aef to e977af6 Compare October 26, 2026 10:34
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 6a7fbc8 to ee4757c Compare October 26, 2026 10:34
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from e977af6 to 0471b06 Compare October 26, 2026 10:36
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from ee4757c to 10360b9 Compare October 26, 2026 10:36
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 0471b06 to 8542de3 Compare October 26, 2026 10:47
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 10360b9 to eeb8a3f Compare October 26, 2026 10:47
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 8542de3 to f1e49cf Compare October 26, 2026 10:58
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from eeb8a3f to 98b7afd Compare October 26, 2026 10:58
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from f1e49cf to ed3ba2e Compare October 26, 2026 11:09
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 98b7afd to 97c8e12 Compare October 26, 2026 11:09
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from ed3ba2e to 8692c4f Compare October 26, 2026 11:21
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from 97c8e12 to fa7bad4 Compare October 26, 2026 11:21
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 8692c4f to 8ca44ba Compare October 26, 2026 11:37
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-migration branch from fa7bad4 to a0f283d Compare October 26, 2026 11:37
@shreyas-goenka
shreyas-goenka force-pushed the shreyas-goenka_data/shreyas/history-best-effort-default branch from 8ca44ba to 49f895d Compare October 26, 2026 12:34

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@shreyas-goenka @eng-dev-ecosystem-bot