feat(git): git commit, git merge and git doctor — work with Mendix history the way Studio Pro does

.claude/skills/mendix/teamserver-git/SKILL.md

@@ -12,11 +12,11 @@ description: "Commit, branch and push a Mendix project in git the way Studio Pro
 |---|---|
 | 35-41 | The rule: let Studio Pro do version control when it can |
 | 42-55 | Branches: create them on the server |
-| 56-78 | Commit from the agent |
-| 79-93 | Push the branch and the notes together, and soon |
-| 94-106 | Never rewrite or merge model history with git |
-| 107-117 | Reading what Studio Pro recorded |
-| 118-126 | Checklist |
+| 56-97 | Commit from the agent |
+| 98-112 | Push the branch and the notes together, and soon |
+| 113-125 | Never rewrite or merge model history with git |
+| 126-136 | Reading what Studio Pro recorded |
+| 137-145 | Checklist |
 
 <!-- /skill-index -->
 # Working with Git on a Mendix Team Server Project
@@ -55,6 +55,21 @@ Suggest that first. Everything below is for committing from the agent.
 
 ## Commit from the agent
 
+The one command does it all — hashes, staging, commit, the note:
+
+```bash
+mxcli git commit -m "<what changed for the user>" -p app.mpr
+```
+
+It repairs ContentsHash mismatches from the files first (MISSING/ORPHAN refuse
+the commit), stages everything and unstages the files Studio Pro never commits
+(`project-settings.user.json`, `*.mpr.lock`, `*.mpr.bak`, `.mendix-cache/`,
+`deployment/`, `theme-cache/`, `.mxcli/`), commits, and attaches the note. Add
+`--push` to push the branch and the notes ref together; `--dry-run` stages,
+reports and unstages without committing. Refused while the `.mpr.lock` exists.
+
+The manual equivalent, one step at a time:
+
 ```bash
 mxcli fix hashes -p app.mpr                     # unit index matches the files (exit 0)
 git add App.mpr mprcontents <other source you changed>
@@ -75,6 +90,10 @@ mxcli git note -p app.mpr --write               # attach the mx_metadata note
   so run it after committing. It never overwrites a note Studio Pro wrote
   (`--force` does) and it does replace Studio Pro's placeholders. Without
   `--write` it only previews.
+- A repository that has known only plain git may drift further: missing
+  `mendix.*`/`core.autocrlf` local config, an SSH remote Studio Pro cannot
+  use, notes missing or malformed across history. `mxcli git doctor` reports
+  all of it in one pass and `--fix` repairs it.
 
 ## Push the branch and the notes together, and soon
 

CHANGELOG.md

@@ -6,6 +6,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Added
+
+- **`mxcli git commit -m … -p app.mpr` — commit the way Studio Pro does** (mendixlabs/mxcli#1669) — Studio Pro's *Commit* is more than `git commit`: it keeps the ContentsHash index in step, stages exactly the files a Mendix repository ships, and attaches the mx_metadata note. The new command is that sequence in one line: mismatched hashes are repaired from the files on disk first (`MISSING` and `ORPHAN` entries have no automatic repair and refuse the commit), everything is staged and the files Studio Pro never commits (`project-settings.user.json`, `*.mpr.lock`, `*.mpr.bak`, `.mendix-cache/`, `deployment/`, `theme-cache/`, `.mxcli/`) are unstaged again, `git commit` runs with the caller's message and identity, and the mx_metadata note is computed from the new commit and attached, so Studio Pro can version-check the revision and show what changed. `--push` pushes the branch and the notes ref together (`git push` alone loses the note the next time Studio Pro fetches); `--dry-run` stages, reports and unstages without committing — and without repairing anything, hash drift included, because a dry run writes nothing. The commit is refused while Studio Pro has the project open (`.mpr.lock`) — honouring the same `MXCLI_ALLOW_STUDIO_PRO_OPEN=1` override as every write — and during an unresolved merge, which `git add -A` would otherwise commit with its conflict markers. Docs: "Working Outside Studio Pro" › *One commit, all the steps*.
+- **`mxcli git doctor [-p app.mpr] [--fix]` — audit a repository for everything a plain-git history misses, and repair it** (mendixlabs/mxcli#1669) — a repository committed only with git drifts from what Studio Pro expects in four ways, each invisible on its own: the local git config Studio Pro writes is absent (`core.autocrlf=False` keeps a checkout from rewriting line endings inside `.mxunit` files on Windows; `mendix.commits-since-gc` and `mendix.lineEndingResetDone` are its own bookkeeping), the remote is SSH which Studio Pro cannot use (HTTPS only), commits have no note under `refs/notes/mx_metadata` (history shows `(unknown)`, the version check fails), and a malformed note Studio Pro cannot read at all. `doctor` reports all four in one read-only pass and exits 1 while an issue remains, so it can gate CI. `--fix` sets the config keys, rewrites SSH remote URLs (`git@host:org/app.git`, `ssh://…`) to HTTPS, and backfills notes: computed from the model with `-p` (the same engine `git note` uses), or a template note with a `--version` ModelerVersion where the model at a commit cannot be read (MPR v1 history, a revision without the `.mpr`); with neither, the newest valid note in history supplies the version. Notes Studio Pro wrote are never touched, with or without `--fix`; `--fix` does not push. Docs: "Working Outside Studio Pro" › *Auditing a repository*.
+
 ### Fixed
 
 - **`check -p --references` types the result of `call microflow`, so a Hidden attribute read on it is reported (CE1406)** (ako/mxcli#1701, mendixlabs/mxcli#1654) — `$Request = call microflow AddOn.Request_GetFirst(); declare $Secret String = $Request/Secret;` in an app module passed `check` and `exec` while `Secret` was a Hidden attribute of a Protected (add-on) module, and mxbuild refused it: `[CE1406] "Create variable activity 'Create String variable' is not allowed to use attribute 'AddOn.Request.Secret' because app modules cannot reference hidden elements in add-on modules."` The same read through a parameter was reported. The variable-typing walk the `--references` member checks share typed parameters, `create`, `retrieve` and loop iterators but not a call's output; it now takes the callee's object or list return type — the script's own definition of the callee first (a `create or modify` changes what exec writes), else the stored microflow. The member-existence check gains the same reach: a CHANGE on a call result naming a member the returned entity does not have now reports CE1613 instead of passing. Measured on Mendix 11.13.0: a declare, an end event and a change on the result, and a change on a loop over a returned list, each refused on the Hidden attribute; the Usable one beside it is 0 errors.

cmd/mxcli/cmd_git_commit.go

@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package main
+
+import (
+	"errors"
+	"fmt"
+	"io"
+	"os"
+	"path"
+	"path/filepath"
+	"strings"
+
+	"github.com/mendixlabs/mxcli/modelsdk/mpr"
+	"github.com/spf13/cobra"
+)
+
+// cmd_git_commit.go: `mxcli git commit` is the commit Studio Pro would have
+// made, for when the agent (or CI) commits instead: it repairs the
+// ContentsHash index, stages everything except the files Studio Pro never
+// commits, commits, and attaches the mx_metadata note — the one sequence that
+// leaves the repository the way Studio Pro expects to find it. The single
+// steps are `mxcli fix hashes`, plain git, and `mxcli git note`.
+
+var gitCommitCmd = &cobra.Command{
+	Use:   "commit -m <message> -p <project.mpr>",
+	Short: "Commit the working tree the way Studio Pro does: hashes, staging, note",
+	Long: `Commit every change with the steps Studio Pro's own commit wraps:
+
+  1. the .mpr's ContentsHash index is repaired against the unit files first —
+     a hash that drifted (typically from a git checkout of a single file) is
+     rewritten from the file, which is the source of truth; MISSING or ORPHAN
+     entries have no automatic repair and refuse the commit;
+  2. everything is staged, then the files Studio Pro never commits are
+     unstaged again: project-settings.user.json, *.mpr.lock, *.mpr.bak,
+     .mendix-cache/, deployment/, theme-cache/, .mxcli/;
+  3. git commit runs with your message and your git identity;
+  4. the mx_metadata note is computed from the new commit and attached, so
+     Studio Pro can version-check it and show what changed.
+
+--push pushes the branch and the notes ref together (git push alone loses the
+note the next time Studio Pro fetches). The commit is refused while Studio Pro
+has the project open (<project>.mpr.lock).
+
+Exits non-zero when the commit did not happen, or when its note could not be
+attached afterwards (the commit itself stays; re-run 'mxcli git note').
+MPR v2 projects only (the note needs the unit index; Mendix 10.18 and later).`,
+	Example: `  mxcli git commit -m "Add order history page" -p app.mpr
+  mxcli git commit -m "Fix stock filter" -p app.mpr --push
+  mxcli git commit -m "WIP" -p app.mpr --dry-run`,
+	Args:          cobra.NoArgs,
+	RunE:          runGitCommit,
+	SilenceUsage:  true,
+	SilenceErrors: true,
+}
+
+func init() {
+	gitCommitCmd.Flags().StringP("message", "m", "", "commit message")
+	_ = gitCommitCmd.MarkFlagRequired("message")
+	gitCommitCmd.Flags().StringP("project", "p", "", "path to the Mendix project (.mpr)")
+	_ = gitCommitCmd.MarkFlagRequired("project")
+	gitCommitCmd.Flags().Bool("push", false, "push the branch and the notes ref to their remote afterwards")
+	gitCommitCmd.Flags().Bool("dry-run", false, "stage and report, then unstage; commit nothing")
+	gitCmd.AddCommand(gitCommitCmd)
+}
+
+func runGitCommit(cmd *cobra.Command, _ []string) error {
+	message, _ := cmd.Flags().GetString("message")
+	push, _ := cmd.Flags().GetBool("push")
+	dryRun, _ := cmd.Flags().GetBool("dry-run")
+	project, _ := cmd.Flags().GetString("project")
+	return commitRun(project, message, push, dryRun, cmd.OutOrStdout())
+}
+
+// commitRun is the command without its wiring.
+func commitRun(mprPath, message string, push, dryRun bool, out io.Writer) error {
+	if dryRun && push {
+		return errors.New("--dry-run and --push cannot be combined: a dry run commits nothing to push")
+	}
+	abs, err := filepath.Abs(mprPath)
+	if err != nil {
+		return err
+	}
+	// The lock guard, honouring the same overrides every mxcli write honours:
+	// a lock Studio Pro left behind on a crash must not deadlock the user.
+	if !mpr.AllowWritesWhileStudioProOpen && os.Getenv(mpr.AllowStudioProOpenEnv) != "1" {
+		if lock, mod := mpr.StudioProLockFile(abs); lock != "" {
+			return &mpr.StudioProOpenError{MprPath: abs, LockPath: lock, LockTime: mod}
+		}
+	}
+	repo, err := openGitNoteRepo(mprPath)
+	if err != nil {
+		return err
+	}
+	defer repo.Close()
+	root := repo.Root
+
+	// An unresolved merge must stop here: `git add -A` below would stage the
+	// conflict markers and commit them, which plain git refuses.
+	if unmerged := strings.TrimSpace(gitOutOr(root, "ls-files", "--unmerged")); unmerged != "" {
+		return errors.New("refusing to commit: the merge has unresolved conflicts (git ls-files --unmerged lists the files); resolve them first, or commit the merge from Studio Pro")
+	}
+
+	// 1. Hashes first: the repair rewrites the .mpr, which is staged below.
+	//    A dry run writes nothing, so it verifies only and reports the drift.
+	repair := !dryRun
+	if err := fixHashes(mprPath, repair, out); err != nil {
+		switch {
+		case errors.Is(err, errHashIssuesRemain) && dryRun:
+			fmt.Fprintln(out, "ContentsHash drift found; a real commit would repair it from the files on disk.")
+		case errors.Is(err, errHashIssuesRemain):
+			return errors.New("refusing to commit: the ContentsHash index has MISSING or ORPHAN entries, which have no automatic repair (see above)")
+		default:
+			return err
+		}
+	}
+
+	// 2. Stage everything, then unstage what Studio Pro never commits.
+	if _, err := gitOutIn(root, nil, "add", "-A"); err != nil {
+		return fmt.Errorf("stage changes: %w", err)
+	}
+	// -z: paths come back NUL-separated, so spaces and quotes stay inside one path.
+	staged := nulFields(gitOutOr(root, "diff", "--cached", "--name-only", "-z"))
+	var junk []string
+	for _, f := range staged {
+		if isJunkPath(f) {
+			junk = append(junk, f)
+		}
+	}
+	if len(junk) > 0 {
+		args := append([]string{"reset", "-q", "--"}, junk...)
+		if _, err := gitOutIn(root, nil, args...); err != nil {
+			return fmt.Errorf("unstage files Studio Pro never commits: %w", err)
+		}
+		fmt.Fprintf(out, "Left out %d file(s) Studio Pro never commits (project-settings.user.json, *.mpr.lock, caches).\n", len(junk))
+	}
+	if len(staged) == len(junk) {
+		fmt.Fprintln(out, "nothing to commit, working tree clean")
+		return nil
+	}
+	fmt.Fprintf(out, "Staged %d file(s):\n", len(staged)-len(junk))
+	kept := staged
+	if len(kept) > 10 {
+		kept = kept[:10]
+	}
+	for _, f := range kept {
+		fmt.Fprintf(out, "  %s\n", f)
+	}
+	if len(staged)-len(junk) > 10 {
+		fmt.Fprintf(out, "  … and %d more\n", len(staged)-len(junk)-10)
+	}
+
+	if dryRun {
+		if _, err := gitOutIn(root, nil, "reset", "-q"); err != nil {
+			return fmt.Errorf("unstage: %w", err)
+		}
+		fmt.Fprintln(out, "Dry run: staged nothing permanently, committed nothing.")
+		return nil
+	}
+
+	// 3. The commit itself, with the user's message and identity.
+	if _, err := gitOutIn(root, nil, "commit", "-m", message); err != nil {
+		return fmt.Errorf("git commit: %w", err)
+	}
+	head := strings.TrimSpace(gitOutOr(root, "rev-parse", "HEAD"))
+	branch := strings.TrimSpace(gitOutOr(root, "symbolic-ref", "--quiet", "--short", "HEAD"))
+	fmt.Fprintf(out, "Committed %s %q.\n", shortCommit(head), message)
+
+	// 4. The note, computed from the commit that now exists. The commit is
+	// already made, so a failure here must say so — the note is attachable
+	// later, and a CI retry must not make a second commit.
+	if err := annotateCommits(out, repo, []string{head}, branch, true, false); err != nil {
+		return fmt.Errorf("committed %s, but attaching the note failed (%v) — run 'mxcli git note -p %s --write' to attach it", shortCommit(head), err, mprPath)
+	}
+
+	if push {
+		if branch == "" {
+			return errors.New("detached HEAD; cannot push — check out a branch and push manually")
+		}
+		if _, err := gitOutIn(root, nil, "push", "origin", branch, mxMetadataRef); err != nil {
+			return fmt.Errorf("push %s and %s: %w", branch, mxMetadataRef, err)
+		}
+		fmt.Fprintf(out, "Pushed %s and %s to origin.\n", branch, mxMetadataRef)
+	}
+	return nil
+}
+
+// nulFields splits git's -z output (NUL-separated, no quoting) into paths.
+func nulFields(s string) []string {
+	var out []string
+	for _, f := range strings.Split(s, "\x00") {
+		if f != "" {
+			out = append(out, f)
+		}
+	}
+	return out
+}
+
+// isJunkPath reports whether a repo-relative path is one of the files Studio
+// Pro never commits. Matched on any path level and without case, the way
+// Studio Pro matches its own lock file (whose .gitignore entry is lowercase).
+func isJunkPath(rel string) bool {
+	f := filepath.ToSlash(rel)
+	base := path.Base(f)
+	switch {
+	case strings.EqualFold(base, "project-settings.user.json"):
+		return true
+	case strings.HasSuffix(strings.ToLower(base), ".mpr.lock"), strings.HasSuffix(strings.ToLower(base), ".mpr.bak"):
+		return true
+	}
+	for _, seg := range strings.Split(f, "/") {
+		switch strings.ToLower(seg) {
+		case ".mendix-cache", "deployment", "theme-cache", ".mxcli":
+			return true
+		}
+	}
+	return false
+}

cmd/mxcli/cmd_git_doctor.go

@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: Apache-2.0
+
+package main
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/spf13/cobra"
+)
+
+// cmd_git_doctor.go: `mxcli git doctor` audits a repository for everything a
+// commit made outside Studio Pro gets wrong, and with --fix repairs it. The
+// checks and their fixes live in gitdoctor.go.
+
+var gitDoctorCmd = &cobra.Command{
+	Use:   "doctor [-p <project.mpr>]",
+	Short: "Audit (and with --fix, repair) a repository for Studio Pro",
+	Long: `Report what in this repository breaks Studio Pro, and with --fix repair it.
+
+Checked:
+  1. Local git config: core.autocrlf=False (a checkout must not rewrite line
+     endings inside .mxunit files), mendix.commits-since-gc and
+     mendix.lineEndingResetDone (Studio Pro's own bookkeeping). --fix sets them.
+  2. The remote URL. Studio Pro can only use HTTPS; an SSH remote
+     (git@host:org/app.git or ssh://…) is rewritten to HTTPS by --fix. Other
+     non-HTTPS URLs are reported without being changed.
+  3. The remote's refs/notes/mx_metadata. Absent or unreachable is a hint:
+     push the notes ref together with the branch.
+  4. Every commit's mx_metadata note. Missing notes make Studio Pro show
+     "(unknown)"; a malformed note it cannot read at all. --fix computes real
+     notes from the model (with -p), and falls back to a template note with a
+     --version ModelerVersion where the model at a commit cannot be read.
+     Notes Studio Pro wrote are never touched.
+
+Exits 1 while an issue remains, so the audit can gate CI. --fix does not push:
+push the branch and the notes ref together yourself.
+
+  git push <remote> <branch> refs/notes/mx_metadata`,
+	Example: `  mxcli git doctor
+  mxcli git doctor -p app.mpr --fix
+  mxcli git doctor --fix --version 10.21.0`,
+	Args:          cobra.NoArgs,
+	RunE:          runGitDoctor,
+	SilenceUsage:  true,
+	SilenceErrors: true,
+}
+
+func init() {
+	gitDoctorCmd.Flags().StringP("project", "p", "", "path to the Mendix project (.mpr); lets --fix compute real notes")
+	gitDoctorCmd.Flags().Bool("fix", false, "apply the repairs (default: report only)")
+	gitDoctorCmd.Flags().String("remote", "", "remote to check (default: tracking remote, then origin)")
+	gitDoctorCmd.Flags().String("version", "", "ModelerVersion for template notes (default: the newest valid note in history)")
+	gitCmd.AddCommand(gitDoctorCmd)
+}
+
+func runGitDoctor(cmd *cobra.Command, _ []string) error {
+	project, _ := cmd.Flags().GetString("project")
+	fix, _ := cmd.Flags().GetBool("fix")
+	remote, _ := cmd.Flags().GetString("remote")
+	version, _ := cmd.Flags().GetString("version")
+	out := cmd.OutOrStdout()
+
+	base := project
+	dir := ""
+	if base == "" {
+		var err error
+		if base, err = os.Getwd(); err != nil {
+			return err
+		}
+		dir = base // a working directory is already a directory
+	} else if _, err := os.Stat(project); err != nil {
+		return fmt.Errorf("project not found: %s", project)
+	}
+	abs, err := filepath.Abs(base)
+	if err != nil {
+		return err
+	}
+	if dir == "" {
+		dir = filepath.Dir(abs) // a .mpr path: git must run beside the file
+	}
+	root, err := gitOut(dir, "rev-parse", "--show-toplevel")
+	if err != nil {
+		return fmt.Errorf("%s is not inside a git repository: %w", base, err)
+	}
+	if err := doctorRun(strings.TrimSpace(root), project, remote, version, fix, out); err != nil {
+		if errors.Is(err, errDoctorIssuesRemain) {
+			cmd.SilenceErrors = true
+			os.Exit(1)
+		}
+		return err
+	}
+	return nil
+}