diff --git a/.claude/skills/mendix/teamserver-git/SKILL.md b/.claude/skills/mendix/teamserver-git/SKILL.md index 27fec27f20..3b2af2afdb 100644 --- a/.claude/skills/mendix/teamserver-git/SKILL.md +++ b/.claude/skills/mendix/teamserver-git/SKILL.md @@ -12,11 +12,11 @@ description: "Commit, branch and push a Mendix project in git the way Studio Pro |---|---| | 35-41 | The rule: let Studio Pro do version control when it can | | 42-55 | Branches: create them on the server | -| 56-78 | Commit from the agent | -| 79-93 | Push the branch and the notes together, and soon | -| 94-106 | Never rewrite or merge model history with git | -| 107-117 | Reading what Studio Pro recorded | -| 118-126 | Checklist | +| 56-97 | Commit from the agent | +| 98-112 | Push the branch and the notes together, and soon | +| 113-125 | Never rewrite or merge model history with git | +| 126-136 | Reading what Studio Pro recorded | +| 137-145 | Checklist | # Working with Git on a Mendix Team Server Project @@ -55,6 +55,21 @@ Suggest that first. Everything below is for committing from the agent. ## Commit from the agent +The one command does it all — hashes, staging, commit, the note: + +```bash +mxcli git commit -m "" -p app.mpr +``` + +It repairs ContentsHash mismatches from the files first (MISSING/ORPHAN refuse +the commit), stages everything and unstages the files Studio Pro never commits +(`project-settings.user.json`, `*.mpr.lock`, `*.mpr.bak`, `.mendix-cache/`, +`deployment/`, `theme-cache/`, `.mxcli/`), commits, and attaches the note. Add +`--push` to push the branch and the notes ref together; `--dry-run` stages, +reports and unstages without committing. Refused while the `.mpr.lock` exists. + +The manual equivalent, one step at a time: + ```bash mxcli fix hashes -p app.mpr # unit index matches the files (exit 0) git add App.mpr mprcontents @@ -75,6 +90,10 @@ mxcli git note -p app.mpr --write # attach the mx_metadata note so run it after committing. It never overwrites a note Studio Pro wrote (`--force` does) and it does replace Studio Pro's placeholders. Without `--write` it only previews. +- A repository that has known only plain git may drift further: missing + `mendix.*`/`core.autocrlf` local config, an SSH remote Studio Pro cannot + use, notes missing or malformed across history. `mxcli git doctor` reports + all of it in one pass and `--fix` repairs it. ## Push the branch and the notes together, and soon diff --git a/CHANGELOG.md b/CHANGELOG.md index d635f89cbf..653b52ad19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Added + +- **`mxcli git commit -m … -p app.mpr` — commit the way Studio Pro does** (mendixlabs/mxcli#1669) — Studio Pro's *Commit* is more than `git commit`: it keeps the ContentsHash index in step, stages exactly the files a Mendix repository ships, and attaches the mx_metadata note. The new command is that sequence in one line: mismatched hashes are repaired from the files on disk first (`MISSING` and `ORPHAN` entries have no automatic repair and refuse the commit), everything is staged and the files Studio Pro never commits (`project-settings.user.json`, `*.mpr.lock`, `*.mpr.bak`, `.mendix-cache/`, `deployment/`, `theme-cache/`, `.mxcli/`) are unstaged again, `git commit` runs with the caller's message and identity, and the mx_metadata note is computed from the new commit and attached, so Studio Pro can version-check the revision and show what changed. `--push` pushes the branch and the notes ref together (`git push` alone loses the note the next time Studio Pro fetches); `--dry-run` stages, reports and unstages without committing — and without repairing anything, hash drift included, because a dry run writes nothing. The commit is refused while Studio Pro has the project open (`.mpr.lock`) — honouring the same `MXCLI_ALLOW_STUDIO_PRO_OPEN=1` override as every write — and during an unresolved merge, which `git add -A` would otherwise commit with its conflict markers. Docs: "Working Outside Studio Pro" › *One commit, all the steps*. +- **`mxcli git doctor [-p app.mpr] [--fix]` — audit a repository for everything a plain-git history misses, and repair it** (mendixlabs/mxcli#1669) — a repository committed only with git drifts from what Studio Pro expects in four ways, each invisible on its own: the local git config Studio Pro writes is absent (`core.autocrlf=False` keeps a checkout from rewriting line endings inside `.mxunit` files on Windows; `mendix.commits-since-gc` and `mendix.lineEndingResetDone` are its own bookkeeping), the remote is SSH which Studio Pro cannot use (HTTPS only), commits have no note under `refs/notes/mx_metadata` (history shows `(unknown)`, the version check fails), and a malformed note Studio Pro cannot read at all. `doctor` reports all four in one read-only pass and exits 1 while an issue remains, so it can gate CI. `--fix` sets the config keys, rewrites SSH remote URLs (`git@host:org/app.git`, `ssh://…`) to HTTPS, and backfills notes: computed from the model with `-p` (the same engine `git note` uses), or a template note with a `--version` ModelerVersion where the model at a commit cannot be read (MPR v1 history, a revision without the `.mpr`); with neither, the newest valid note in history supplies the version. Notes Studio Pro wrote are never touched, with or without `--fix`; `--fix` does not push. Docs: "Working Outside Studio Pro" › *Auditing a repository*. + ### Fixed - **`check -p --references` types the result of `call microflow`, so a Hidden attribute read on it is reported (CE1406)** (ako/mxcli#1701, mendixlabs/mxcli#1654) — `$Request = call microflow AddOn.Request_GetFirst(); declare $Secret String = $Request/Secret;` in an app module passed `check` and `exec` while `Secret` was a Hidden attribute of a Protected (add-on) module, and mxbuild refused it: `[CE1406] "Create variable activity 'Create String variable' is not allowed to use attribute 'AddOn.Request.Secret' because app modules cannot reference hidden elements in add-on modules."` The same read through a parameter was reported. The variable-typing walk the `--references` member checks share typed parameters, `create`, `retrieve` and loop iterators but not a call's output; it now takes the callee's object or list return type — the script's own definition of the callee first (a `create or modify` changes what exec writes), else the stored microflow. The member-existence check gains the same reach: a CHANGE on a call result naming a member the returned entity does not have now reports CE1613 instead of passing. Measured on Mendix 11.13.0: a declare, an end event and a change on the result, and a change on a loop over a returned list, each refused on the Hidden attribute; the Usable one beside it is 0 errors. diff --git a/cmd/mxcli/cmd_git_commit.go b/cmd/mxcli/cmd_git_commit.go new file mode 100644 index 0000000000..cbdb52cbea --- /dev/null +++ b/cmd/mxcli/cmd_git_commit.go @@ -0,0 +1,218 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "errors" + "fmt" + "io" + "os" + "path" + "path/filepath" + "strings" + + "github.com/mendixlabs/mxcli/modelsdk/mpr" + "github.com/spf13/cobra" +) + +// cmd_git_commit.go: `mxcli git commit` is the commit Studio Pro would have +// made, for when the agent (or CI) commits instead: it repairs the +// ContentsHash index, stages everything except the files Studio Pro never +// commits, commits, and attaches the mx_metadata note — the one sequence that +// leaves the repository the way Studio Pro expects to find it. The single +// steps are `mxcli fix hashes`, plain git, and `mxcli git note`. + +var gitCommitCmd = &cobra.Command{ + Use: "commit -m -p ", + Short: "Commit the working tree the way Studio Pro does: hashes, staging, note", + Long: `Commit every change with the steps Studio Pro's own commit wraps: + + 1. the .mpr's ContentsHash index is repaired against the unit files first — + a hash that drifted (typically from a git checkout of a single file) is + rewritten from the file, which is the source of truth; MISSING or ORPHAN + entries have no automatic repair and refuse the commit; + 2. everything is staged, then the files Studio Pro never commits are + unstaged again: project-settings.user.json, *.mpr.lock, *.mpr.bak, + .mendix-cache/, deployment/, theme-cache/, .mxcli/; + 3. git commit runs with your message and your git identity; + 4. the mx_metadata note is computed from the new commit and attached, so + Studio Pro can version-check it and show what changed. + +--push pushes the branch and the notes ref together (git push alone loses the +note the next time Studio Pro fetches). The commit is refused while Studio Pro +has the project open (.mpr.lock). + +Exits non-zero when the commit did not happen, or when its note could not be +attached afterwards (the commit itself stays; re-run 'mxcli git note'). +MPR v2 projects only (the note needs the unit index; Mendix 10.18 and later).`, + Example: ` mxcli git commit -m "Add order history page" -p app.mpr + mxcli git commit -m "Fix stock filter" -p app.mpr --push + mxcli git commit -m "WIP" -p app.mpr --dry-run`, + Args: cobra.NoArgs, + RunE: runGitCommit, + SilenceUsage: true, + SilenceErrors: true, +} + +func init() { + gitCommitCmd.Flags().StringP("message", "m", "", "commit message") + _ = gitCommitCmd.MarkFlagRequired("message") + gitCommitCmd.Flags().StringP("project", "p", "", "path to the Mendix project (.mpr)") + _ = gitCommitCmd.MarkFlagRequired("project") + gitCommitCmd.Flags().Bool("push", false, "push the branch and the notes ref to their remote afterwards") + gitCommitCmd.Flags().Bool("dry-run", false, "stage and report, then unstage; commit nothing") + gitCmd.AddCommand(gitCommitCmd) +} + +func runGitCommit(cmd *cobra.Command, _ []string) error { + message, _ := cmd.Flags().GetString("message") + push, _ := cmd.Flags().GetBool("push") + dryRun, _ := cmd.Flags().GetBool("dry-run") + project, _ := cmd.Flags().GetString("project") + return commitRun(project, message, push, dryRun, cmd.OutOrStdout()) +} + +// commitRun is the command without its wiring. +func commitRun(mprPath, message string, push, dryRun bool, out io.Writer) error { + if dryRun && push { + return errors.New("--dry-run and --push cannot be combined: a dry run commits nothing to push") + } + abs, err := filepath.Abs(mprPath) + if err != nil { + return err + } + // The lock guard, honouring the same overrides every mxcli write honours: + // a lock Studio Pro left behind on a crash must not deadlock the user. + if !mpr.AllowWritesWhileStudioProOpen && os.Getenv(mpr.AllowStudioProOpenEnv) != "1" { + if lock, mod := mpr.StudioProLockFile(abs); lock != "" { + return &mpr.StudioProOpenError{MprPath: abs, LockPath: lock, LockTime: mod} + } + } + repo, err := openGitNoteRepo(mprPath) + if err != nil { + return err + } + defer repo.Close() + root := repo.Root + + // An unresolved merge must stop here: `git add -A` below would stage the + // conflict markers and commit them, which plain git refuses. + if unmerged := strings.TrimSpace(gitOutOr(root, "ls-files", "--unmerged")); unmerged != "" { + return errors.New("refusing to commit: the merge has unresolved conflicts (git ls-files --unmerged lists the files); resolve them first, or commit the merge from Studio Pro") + } + + // 1. Hashes first: the repair rewrites the .mpr, which is staged below. + // A dry run writes nothing, so it verifies only and reports the drift. + repair := !dryRun + if err := fixHashes(mprPath, repair, out); err != nil { + switch { + case errors.Is(err, errHashIssuesRemain) && dryRun: + fmt.Fprintln(out, "ContentsHash drift found; a real commit would repair it from the files on disk.") + case errors.Is(err, errHashIssuesRemain): + return errors.New("refusing to commit: the ContentsHash index has MISSING or ORPHAN entries, which have no automatic repair (see above)") + default: + return err + } + } + + // 2. Stage everything, then unstage what Studio Pro never commits. + if _, err := gitOutIn(root, nil, "add", "-A"); err != nil { + return fmt.Errorf("stage changes: %w", err) + } + // -z: paths come back NUL-separated, so spaces and quotes stay inside one path. + staged := nulFields(gitOutOr(root, "diff", "--cached", "--name-only", "-z")) + var junk []string + for _, f := range staged { + if isJunkPath(f) { + junk = append(junk, f) + } + } + if len(junk) > 0 { + args := append([]string{"reset", "-q", "--"}, junk...) + if _, err := gitOutIn(root, nil, args...); err != nil { + return fmt.Errorf("unstage files Studio Pro never commits: %w", err) + } + fmt.Fprintf(out, "Left out %d file(s) Studio Pro never commits (project-settings.user.json, *.mpr.lock, caches).\n", len(junk)) + } + if len(staged) == len(junk) { + fmt.Fprintln(out, "nothing to commit, working tree clean") + return nil + } + fmt.Fprintf(out, "Staged %d file(s):\n", len(staged)-len(junk)) + kept := staged + if len(kept) > 10 { + kept = kept[:10] + } + for _, f := range kept { + fmt.Fprintf(out, " %s\n", f) + } + if len(staged)-len(junk) > 10 { + fmt.Fprintf(out, " … and %d more\n", len(staged)-len(junk)-10) + } + + if dryRun { + if _, err := gitOutIn(root, nil, "reset", "-q"); err != nil { + return fmt.Errorf("unstage: %w", err) + } + fmt.Fprintln(out, "Dry run: staged nothing permanently, committed nothing.") + return nil + } + + // 3. The commit itself, with the user's message and identity. + if _, err := gitOutIn(root, nil, "commit", "-m", message); err != nil { + return fmt.Errorf("git commit: %w", err) + } + head := strings.TrimSpace(gitOutOr(root, "rev-parse", "HEAD")) + branch := strings.TrimSpace(gitOutOr(root, "symbolic-ref", "--quiet", "--short", "HEAD")) + fmt.Fprintf(out, "Committed %s %q.\n", shortCommit(head), message) + + // 4. The note, computed from the commit that now exists. The commit is + // already made, so a failure here must say so — the note is attachable + // later, and a CI retry must not make a second commit. + if err := annotateCommits(out, repo, []string{head}, branch, true, false); err != nil { + return fmt.Errorf("committed %s, but attaching the note failed (%v) — run 'mxcli git note -p %s --write' to attach it", shortCommit(head), err, mprPath) + } + + if push { + if branch == "" { + return errors.New("detached HEAD; cannot push — check out a branch and push manually") + } + if _, err := gitOutIn(root, nil, "push", "origin", branch, mxMetadataRef); err != nil { + return fmt.Errorf("push %s and %s: %w", branch, mxMetadataRef, err) + } + fmt.Fprintf(out, "Pushed %s and %s to origin.\n", branch, mxMetadataRef) + } + return nil +} + +// nulFields splits git's -z output (NUL-separated, no quoting) into paths. +func nulFields(s string) []string { + var out []string + for _, f := range strings.Split(s, "\x00") { + if f != "" { + out = append(out, f) + } + } + return out +} + +// isJunkPath reports whether a repo-relative path is one of the files Studio +// Pro never commits. Matched on any path level and without case, the way +// Studio Pro matches its own lock file (whose .gitignore entry is lowercase). +func isJunkPath(rel string) bool { + f := filepath.ToSlash(rel) + base := path.Base(f) + switch { + case strings.EqualFold(base, "project-settings.user.json"): + return true + case strings.HasSuffix(strings.ToLower(base), ".mpr.lock"), strings.HasSuffix(strings.ToLower(base), ".mpr.bak"): + return true + } + for _, seg := range strings.Split(f, "/") { + switch strings.ToLower(seg) { + case ".mendix-cache", "deployment", "theme-cache", ".mxcli": + return true + } + } + return false +} diff --git a/cmd/mxcli/cmd_git_doctor.go b/cmd/mxcli/cmd_git_doctor.go new file mode 100644 index 0000000000..1e17d23f3c --- /dev/null +++ b/cmd/mxcli/cmd_git_doctor.go @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "errors" + "fmt" + "os" + "path/filepath" + "strings" + + "github.com/spf13/cobra" +) + +// cmd_git_doctor.go: `mxcli git doctor` audits a repository for everything a +// commit made outside Studio Pro gets wrong, and with --fix repairs it. The +// checks and their fixes live in gitdoctor.go. + +var gitDoctorCmd = &cobra.Command{ + Use: "doctor [-p ]", + Short: "Audit (and with --fix, repair) a repository for Studio Pro", + Long: `Report what in this repository breaks Studio Pro, and with --fix repair it. + +Checked: + 1. Local git config: core.autocrlf=False (a checkout must not rewrite line + endings inside .mxunit files), mendix.commits-since-gc and + mendix.lineEndingResetDone (Studio Pro's own bookkeeping). --fix sets them. + 2. The remote URL. Studio Pro can only use HTTPS; an SSH remote + (git@host:org/app.git or ssh://…) is rewritten to HTTPS by --fix. Other + non-HTTPS URLs are reported without being changed. + 3. The remote's refs/notes/mx_metadata. Absent or unreachable is a hint: + push the notes ref together with the branch. + 4. Every commit's mx_metadata note. Missing notes make Studio Pro show + "(unknown)"; a malformed note it cannot read at all. --fix computes real + notes from the model (with -p), and falls back to a template note with a + --version ModelerVersion where the model at a commit cannot be read. + Notes Studio Pro wrote are never touched. + +Exits 1 while an issue remains, so the audit can gate CI. --fix does not push: +push the branch and the notes ref together yourself. + + git push refs/notes/mx_metadata`, + Example: ` mxcli git doctor + mxcli git doctor -p app.mpr --fix + mxcli git doctor --fix --version 10.21.0`, + Args: cobra.NoArgs, + RunE: runGitDoctor, + SilenceUsage: true, + SilenceErrors: true, +} + +func init() { + gitDoctorCmd.Flags().StringP("project", "p", "", "path to the Mendix project (.mpr); lets --fix compute real notes") + gitDoctorCmd.Flags().Bool("fix", false, "apply the repairs (default: report only)") + gitDoctorCmd.Flags().String("remote", "", "remote to check (default: tracking remote, then origin)") + gitDoctorCmd.Flags().String("version", "", "ModelerVersion for template notes (default: the newest valid note in history)") + gitCmd.AddCommand(gitDoctorCmd) +} + +func runGitDoctor(cmd *cobra.Command, _ []string) error { + project, _ := cmd.Flags().GetString("project") + fix, _ := cmd.Flags().GetBool("fix") + remote, _ := cmd.Flags().GetString("remote") + version, _ := cmd.Flags().GetString("version") + out := cmd.OutOrStdout() + + base := project + dir := "" + if base == "" { + var err error + if base, err = os.Getwd(); err != nil { + return err + } + dir = base // a working directory is already a directory + } else if _, err := os.Stat(project); err != nil { + return fmt.Errorf("project not found: %s", project) + } + abs, err := filepath.Abs(base) + if err != nil { + return err + } + if dir == "" { + dir = filepath.Dir(abs) // a .mpr path: git must run beside the file + } + root, err := gitOut(dir, "rev-parse", "--show-toplevel") + if err != nil { + return fmt.Errorf("%s is not inside a git repository: %w", base, err) + } + if err := doctorRun(strings.TrimSpace(root), project, remote, version, fix, out); err != nil { + if errors.Is(err, errDoctorIssuesRemain) { + cmd.SilenceErrors = true + os.Exit(1) + } + return err + } + return nil +} diff --git a/cmd/mxcli/gitdoctor.go b/cmd/mxcli/gitdoctor.go new file mode 100644 index 0000000000..48f0457286 --- /dev/null +++ b/cmd/mxcli/gitdoctor.go @@ -0,0 +1,386 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "os" + "regexp" + "strings" +) + +// gitdoctor.go audits a repository for the state Studio Pro expects, and +// repairs it with --fix. Four things a commit made with plain git gets wrong: +// +// - the local git config Studio Pro writes is missing (core.autocrlf=False +// keeps a checkout from rewriting line endings inside .mxunit files on +// Windows; mendix.commits-since-gc and mendix.lineEndingResetDone are +// Studio Pro's own bookkeeping), +// - the remote is SSH, which Studio Pro cannot use (HTTPS only), +// - commits have no note under refs/notes/mx_metadata, so Studio Pro shows +// "(unknown)" and cannot version-check the revision, +// - a note is malformed, which Studio Pro cannot read at all. +// +// Real notes are never touched: a note is only written where none exists, or +// over one that is unreadable. Where the model at a commit cannot be read +// (MPR v1 history, a revision without the .mpr), a template note with a +// detected or --version ModelerVersion is written instead. + +var errDoctorIssuesRemain = errors.New("issues remain (re-run with --fix to repair)") + +// mxConfigKeys is the local git config Studio Pro writes and expects. +var mxConfigKeys = [][2]string{ + {"core.autocrlf", "False"}, + {"mendix.commits-since-gc", "0"}, + {"mendix.lineEndingResetDone", "True"}, +} + +var ( + sshScpURLRe = regexp.MustCompile(`^git@([^:/]+):(.+?)(?:\.git)?$`) + sshURLRe = regexp.MustCompile(`^ssh://(?:[^@/]+@)?([^/:]+?)(:\d+)?/(.+?)(?:\.git)?$`) + mxVersionRe = regexp.MustCompile(`^[0-9]+(\.[0-9]+)+$`) +) + +// doctorRun audits the repository at root and reports one line per finding. +// mprPath ("" when not given) lets --fix compute real notes from the model. +// With fix, repairs are applied. Returns errDoctorIssuesRemain when something +// is still wrong. +func doctorRun(root, mprPath, remoteOverride, version string, fix bool, out io.Writer) error { + found, fixed := 0, 0 + + fmt.Fprintln(out, "Git local config") + for _, kv := range mxConfigKeys { + key, want := kv[0], kv[1] + current := gitConfigLocal(root, key) + if strings.EqualFold(current, want) { + fmt.Fprintf(out, " [x] %s = %s\n", key, current) + continue + } + found++ + if current == "" { + fmt.Fprintf(out, " [ ] %s — missing (expected %s)\n", key, want) + } else { + fmt.Fprintf(out, " [ ] %s = %s (expected %s)\n", key, current, want) + } + if fix { + if _, err := gitOut(root, "config", "--local", key, want); err != nil { + return fmt.Errorf("set %s: %w", key, err) + } + fixed++ + fmt.Fprintf(out, " set to %s\n", want) + } + } + + fmt.Fprintln(out, "Remote") + remote := detectDoctorRemote(root, remoteOverride) + switch { + case remote == "": + fmt.Fprintln(out, " [i] no remote configured; skipping remote checks") + default: + urlRaw, urlErr := gitOut(root, "remote", "get-url", remote) + if urlErr != nil { + found++ + fmt.Fprintf(out, " [ ] remote %s does not exist (git remote -v lists the names)\n", remote) + break + } + url := strings.TrimSpace(urlRaw) + if host, p, ok := sshToHTTPS(url); ok { + found++ + fmt.Fprintf(out, " [ ] %s: %s — SSH; Studio Pro needs HTTPS\n", remote, url) + if fix { + https := "https://" + host + "/" + p + if _, err := gitOut(root, "remote", "set-url", remote, https); err != nil { + return fmt.Errorf("rewrite remote URL: %w", err) + } + fixed++ + fmt.Fprintf(out, " rewritten to %s\n", https) + } + break + } + if strings.HasPrefix(url, "https://") { + fmt.Fprintf(out, " [x] %s: %s (HTTPS)\n", remote, url) + } else { + fmt.Fprintf(out, " [i] %s: %s (not HTTPS; Studio Pro can only use HTTPS remotes)\n", remote, url) + } + // Absent or unreachable notes ref is a hint, not an issue. + ls, err := gitOut(root, "ls-remote", remote, mxMetadataRef) + switch { + case err != nil: + fmt.Fprintf(out, " [i] could not reach %s to check %s\n", remote, mxMetadataRef) + case strings.Contains(ls, mxMetadataRef): + fmt.Fprintf(out, " [x] %s has %s\n", remote, mxMetadataRef) + default: + fmt.Fprintf(out, " [i] %s has no %s yet; push it together with the branch\n", remote, mxMetadataRef) + } + } + + fn, fx, err := doctorNotes(root, mprPath, version, fix, out) + if err != nil { + return err + } + found, fixed = found+fn, fixed+fx + + remaining := found - fixed + if remaining < 0 { + remaining = 0 // placeholder replacements were never counted as issues + } + switch { + case remaining > 0: + fmt.Fprintf(out, "\n%d issue(s) remain.\n", remaining) + return errDoctorIssuesRemain + case fixed > 0 && found > 0: + fmt.Fprintf(out, "\nFixed %d issue(s).\n", fixed) + case fixed > 0: + fmt.Fprintf(out, "\nReplaced %d placeholder note(s).\n", fixed) + default: + fmt.Fprintln(out, "\nNothing to fix.") + } + return nil +} + +// doctorNotes reports (and with fix, repairs) missing and malformed notes over +// the whole current history. It returns how many it found and how many it fixed. +func doctorNotes(root, mprPath, version string, fix bool, out io.Writer) (found, fixed int, err error) { + fmt.Fprintln(out, "mx_metadata notes") + if _, err := gitOut(root, "rev-parse", "--verify", "--quiet", "HEAD"); err != nil { + fmt.Fprintln(out, " [i] no commits yet") + return 0, 0, nil + } + revs := strings.Fields(gitOutOr(root, "rev-list", "--reverse", "HEAD")) + noted := map[string]bool{} + for _, line := range strings.Split(strings.TrimRight(gitOutOr(root, "notes", "--ref="+mxMetadataRef, "list"), "\n"), "\n") { + if f := strings.Fields(line); len(f) == 2 { + noted[f[1]] = true + } + } + + var missing, malformed, placeholders []string + for _, c := range revs { + if !noted[c] { + missing = append(missing, c) + continue + } + text := gitNoteText(root, c) + var n struct { + HasModelerVersion *bool `json:"HasModelerVersion"` + } + if json.Unmarshal([]byte(text), &n) != nil { + malformed = append(malformed, c) + continue + } + if n.HasModelerVersion != nil && !*n.HasModelerVersion { + placeholders = append(placeholders, c) + } + } + for _, c := range missing { + fmt.Fprintf(out, " [ ] %s without a note\n", shortCommit(c)) + } + for _, c := range malformed { + fmt.Fprintf(out, " [ ] malformed note on %s\n", shortCommit(c)) + } + if len(placeholders) > 0 { + fmt.Fprintf(out, " [i] %d placeholder note(s) Studio Pro back-filled\n", len(placeholders)) + } + found = len(missing) + len(malformed) + if !fix || found+len(placeholders) == 0 { + return found, 0, nil + } + + // An explicit --version must look like one; a typo would be stamped into + // every template note in history. + if version != "" && !mxVersionRe.MatchString(version) { + return found, 0, fmt.Errorf("--version %q is not a Mendix version (digits and dots, e.g. 10.21.0 or 11.13.0.12345) — pass --version or fix it", version) + } + var repo *gitNoteRepo + if mprPath != "" { + // Refuse a path that does not exist before anything else: a snapshot + // of a revision without the .mpr reads as an empty model, and a + // computed note from it would carry no version. + if _, err := os.Stat(mprPath); err != nil { + return found, 0, fmt.Errorf("project not found: %s", mprPath) + } + r, err := openGitNoteRepo(mprPath) + if err != nil { + return found, 0, err + } + defer r.Close() + repo = r + } + tmplVersion := version + if tmplVersion == "" { + tmplVersion, _ = newestNotedVersion(root) + } + branch := strings.TrimSpace(gitOutOr(root, "symbolic-ref", "--quiet", "--short", "HEAD")) + + // Oldest first, every commit once: a missing note is computed, a malformed + // or placeholder note is replaced — from the model where it can be read, + // from the template otherwise. + unfixable := 0 + for _, c := range revs { + replace := false + for _, l := range malformed { + replace = replace || l == c + } + for _, l := range placeholders { + replace = replace || l == c + } + if noted[c] && !replace { + continue // a real note: never touched + } + text, computed, err := noteForCommit(repo, tmplVersion, c, branch) + if err != nil { + unfixable++ + fmt.Fprintf(out, " [ ] %s: %v\n", shortCommit(c), err) + continue + } + if err := writeNoteRaw(root, c, text); err != nil { + unfixable++ + fmt.Fprintf(out, " [ ] %s: write note: %v\n", shortCommit(c), err) + continue + } + kind := "template note" + if computed { + kind = "computed note" + } + fmt.Fprintf(out, " [x] %s: wrote %s\n", shortCommit(c), kind) + fixed++ + } + if unfixable > 0 && tmplVersion == "" { + return found, fixed, fmt.Errorf("%d commit(s) still without a note: no ModelerVersion is known — pass --version", unfixable) + } + return found, fixed, nil +} + +// --- helpers ----------------------------------------------------------------- + +func gitConfigLocal(root, key string) string { + out, err := gitOut(root, "config", "--local", key) + if err != nil { + return "" + } + return strings.TrimSpace(out) +} + +func gitOutOr(root string, args ...string) string { + out, _ := gitOut(root, args...) + return out +} + +func shortCommit(c string) string { + if len(c) > 10 { + return c[:10] + } + return c +} + +// detectDoctorRemote picks the remote the way Studio Pro would use it: the +// override, else the current branch's tracking remote, else origin, else the +// first remote. "" when there is none. +func detectDoctorRemote(root, override string) string { + if override != "" { + return override + } + if up, err := gitOut(root, "rev-parse", "--abbrev-ref", "--symbolic-full-name", "@{upstream}"); err == nil { + if i := strings.Index(strings.TrimSpace(up), "/"); i > 0 { + return strings.TrimSpace(up)[:i] + } + } + remotes := strings.Fields(gitOutOr(root, "remote")) + for _, r := range remotes { + if r == "origin" { + return r + } + } + if len(remotes) > 0 { + return remotes[0] + } + return "" +} + +// sshToHTTPS rewrites the two SSH spellings git uses to an HTTPS URL. ok is +// false for everything else, including ssh:// URLs with a port (an HTTPS port +// cannot be derived). +func sshToHTTPS(url string) (host, p string, ok bool) { + if m := sshScpURLRe.FindStringSubmatch(url); m != nil { + return m[1], m[2], true + } + if m := sshURLRe.FindStringSubmatch(url); m != nil { + if m[2] != "" { + return "", "", false + } + return m[1], m[3], true + } + return "", "", false +} + +func gitNoteText(root, c string) string { + out, err := gitOut(root, "notes", "--ref="+mxMetadataRef, "show", c) + if err != nil { + return "" + } + return out +} + +// newestNotedVersion scans the history newest first and returns the +// ModelerVersion of the first readable note, the way Studio Pro's own +// version column would read it. +func newestNotedVersion(root string) (string, error) { + for _, c := range strings.Fields(gitOutOr(root, "rev-list", "HEAD")) { + text := gitNoteText(root, c) + if text == "" { + continue + } + var n struct { + ModelerVersion string `json:"ModelerVersion"` + HasModelerVersion bool `json:"HasModelerVersion"` + } + if json.Unmarshal([]byte(text), &n) != nil { + continue + } + if n.HasModelerVersion && n.ModelerVersion != "" && n.ModelerVersion != "(unknown)" { + return n.ModelerVersion, nil + } + } + return "", errors.New("no readable note in history") +} + +// noteForCommit builds the note text for a commit: computed from the model +// when repo is set and the revision reads, the template otherwise. A computed +// note without a ModelerVersion is a snapshot of a revision without a readable +// .mpr — treated as unreadable, because a "real" note with an empty version is +// one no later run would ever touch again. +func noteForCommit(repo *gitNoteRepo, tmplVersion, c, branch string) (text string, computed bool, err error) { + if repo != nil { + if n, err := repo.buildNote(c, branch); err == nil && n.ModelerVersion != "" { + return n.marshal(), true, nil + } + } + if tmplVersion == "" { + return "", false, errors.New("model unreadable and no template version known") + } + return templateNote(tmplVersion), false, nil +} + +// templateNote is the note of last resort: structurally exactly what Studio +// Pro writes, with the model changes it cannot know left empty. +func templateNote(v string) string { + return mxNote{ + BranchName: "", + ModelerVersion: v, + ModelChanges: []mxModelChange{}, + RelatedStories: []string{}, + SolutionVersion: "", + MPRFormatVersion: "Version2", + HasModelerVersion: true, + }.marshal() +} + +// writeNoteRaw attaches text to c, replacing whatever is there (-f is what +// makes a malformed note repairable; missing notes have nothing to lose). +func writeNoteRaw(root, c, text string) error { + _, err := gitOutIn(root, []byte(text), "notes", "--ref="+mxMetadataRef, "add", "-f", "-F", "-", c) + return err +} diff --git a/cmd/mxcli/gitdoctor_test.go b/cmd/mxcli/gitdoctor_test.go new file mode 100644 index 0000000000..c0e0393f4f --- /dev/null +++ b/cmd/mxcli/gitdoctor_test.go @@ -0,0 +1,580 @@ +// SPDX-License-Identifier: Apache-2.0 + +package main + +import ( + "bytes" + "encoding/json" + "errors" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + + "go.mongodb.org/mongo-driver/v2/bson" +) + +// `mxcli git doctor` audits a repository the way Studio Pro needs it, and +// `mxcli git commit` does what Studio Pro's commit does: hashes, staging, +// the mx_metadata note. Both are factored as doctorRun / commitRun so the +// tests drive them without the cobra wiring. + +func doctorEnv(t *testing.T, dir string) { + t.Helper() + requireGit(t) + t.Setenv("GIT_CEILING_DIRECTORIES", filepath.Dir(dir)) + t.Setenv("GIT_AUTHOR_NAME", "t") + t.Setenv("GIT_AUTHOR_EMAIL", "t@example.com") + t.Setenv("GIT_COMMITTER_NAME", "t") + t.Setenv("GIT_COMMITTER_EMAIL", "t@example.com") +} + +func configLocal(t *testing.T, dir, key string) string { + t.Helper() + out, err := gitOut(dir, "config", "--local", key) + if err != nil { + return "" + } + return strings.TrimSpace(out) +} + +// --- doctor: git local config ---------------------------------------------- + +func TestGitDoctor_ConfigMissingReported(t *testing.T) { + dir := t.TempDir() + doctorEnv(t, dir) + gitIn(t, dir, "init", "-q", "-b", "main") + + var out bytes.Buffer + err := doctorRun(dir, "", "", "", false, &out) + if !errors.Is(err, errDoctorIssuesRemain) { + t.Fatalf("want errDoctorIssuesRemain, got %v", err) + } + for _, key := range []string{"core.autocrlf", "mendix.commits-since-gc", "mendix.lineEndingResetDone"} { + if !strings.Contains(out.String(), key) { + t.Errorf("report must name %s:\n%s", key, out.String()) + } + } +} + +func TestGitDoctor_FixWritesConfigAndPasses(t *testing.T) { + dir := t.TempDir() + doctorEnv(t, dir) + gitIn(t, dir, "init", "-q", "-b", "main") + + var out bytes.Buffer + if err := doctorRun(dir, "", "", "", true, &out); err != nil { + t.Fatalf("--fix must leave no issue: %v\n%s", err, out.String()) + } + if got := configLocal(t, dir, "core.autocrlf"); !strings.EqualFold(got, "False") { + t.Errorf("core.autocrlf = %q, want False", got) + } + if got := configLocal(t, dir, "mendix.commits-since-gc"); got != "0" { + t.Errorf("mendix.commits-since-gc = %q, want 0", got) + } + if got := configLocal(t, dir, "mendix.lineEndingResetDone"); !strings.EqualFold(got, "True") { + t.Errorf("mendix.lineEndingResetDone = %q, want True", got) + } + // A second run with nothing to fix stays clean. + out.Reset() + if err := doctorRun(dir, "", "", "", false, &out); err != nil { + t.Fatalf("clean repo must pass: %v\n%s", err, out.String()) + } + // A repo without a remote is reported, but not broken. + if !strings.Contains(out.String(), "remote") { + t.Errorf("report must mention the remote check:\n%s", out.String()) + } +} + +// --- doctor: remote URL scheme ---------------------------------------------- + +func TestGitDoctor_SSHRemoteRewrittenByFix(t *testing.T) { + dir := t.TempDir() + doctorEnv(t, dir) + gitIn(t, dir, "init", "-q", "-b", "main") + gitIn(t, dir, "remote", "add", "origin", "git@github.com:org/app.git") + + var out bytes.Buffer + if err := doctorRun(dir, "", "", "", false, &out); !errors.Is(err, errDoctorIssuesRemain) { + t.Fatalf("SSH remote must be an issue: %v", err) + } + if !strings.Contains(out.String(), "git@github.com:org/app.git") { + t.Errorf("report must name the URL:\n%s", out.String()) + } + + out.Reset() + if err := doctorRun(dir, "", "", "", true, &out); err != nil { + t.Fatalf("--fix must repair the remote: %v\n%s", err, out.String()) + } + got, err := gitOut(dir, "remote", "get-url", "origin") + if err != nil { + t.Fatal(err) + } + if strings.TrimSpace(got) != "https://github.com/org/app" { + t.Errorf("remote url = %q, want https://github.com/org/app", got) + } +} + +func TestGitDoctor_SSHURLFormRewrittenByFix(t *testing.T) { + dir := t.TempDir() + doctorEnv(t, dir) + gitIn(t, dir, "init", "-q", "-b", "main") + gitIn(t, dir, "remote", "add", "origin", "ssh://git@github.com/org/app.git") + + var out bytes.Buffer + if err := doctorRun(dir, "", "", "", true, &out); err != nil { + t.Fatalf("--fix must repair ssh:// remotes: %v\n%s", err, out.String()) + } + got, err := gitOut(dir, "remote", "get-url", "origin") + if err != nil { + t.Fatal(err) + } + if strings.TrimSpace(got) != "https://github.com/org/app" { + t.Errorf("remote url = %q, want https://github.com/org/app", got) + } +} + +// --- doctor: mx_metadata notes ---------------------------------------------- + +// doctorNoteRepo makes a two-commit repository holding an MPR v2 project: +// the base commit, then a page retitle. Returns dir and the .mpr path. +func doctorNoteRepo(t *testing.T) (string, string) { + t.Helper() + dir := t.TempDir() + gitIn(t, dir, "init", "-q", "-b", "feature") + units := baseUnits(t) + writeProject(t, dir, "11.13.0", units) + gitIn(t, dir, "add", "-A") + gitIn(t, dir, "commit", "-q", "-m", "base") + units[tPage] = tUnit{tFolder, unitDoc(t, "Forms$Page", "Home", bson.E{Key: "Title", Value: "Welcome"})} + writeProject(t, dir, "11.13.0", units) + gitIn(t, dir, "add", "-A") + gitIn(t, dir, "commit", "-q", "-m", "retitle home") + return dir, filepath.Join(dir, "App.mpr") +} + +func noteOn(t *testing.T, dir, rev string) string { + t.Helper() + out, err := gitOut(dir, "notes", "--ref="+mxMetadataRef, "show", rev) + if err != nil { + return "" + } + return out +} + +func TestGitDoctor_MissingAndMalformedNotesReportedAndFixed(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + gitIn(t, dir, "notes", "--ref="+mxMetadataRef, "add", "-f", "-m", "not json", "HEAD~1") + + var out bytes.Buffer + if err := doctorRun(dir, mprPath, "", "", false, &out); !errors.Is(err, errDoctorIssuesRemain) { + t.Fatalf("missing and malformed notes must be issues: %v\n%s", err, out.String()) + } + if !strings.Contains(out.String(), "malformed") || !strings.Contains(out.String(), "without a note") { + t.Errorf("report must name both findings:\n%s", out.String()) + } + + out.Reset() + if err := doctorRun(dir, mprPath, "", "", true, &out); err != nil { + t.Fatalf("--fix must repair the notes: %v\n%s", err, out.String()) + } + var head, base mxNote + if err := json.Unmarshal([]byte(noteOn(t, dir, "HEAD")), &head); err != nil { + t.Fatalf("HEAD note is not valid JSON: %v", err) + } + if err := json.Unmarshal([]byte(noteOn(t, dir, "HEAD~1")), &base); err != nil { + t.Fatalf("HEAD~1 note (was malformed) is not valid JSON: %v", err) + } + if !head.HasModelerVersion || head.ModelerVersion != "11.13.0" { + t.Errorf("HEAD note header: %+v", head) + } + if len(head.ModelChanges) != 1 || head.ModelChanges[0].UnitID != tPage { + t.Errorf("HEAD note must list the retitled page: %+v", head.ModelChanges) + } + if len(base.ModelChanges) == 0 { + t.Errorf("base commit note must list the added units: %+v", base) + } +} + +func TestGitDoctor_FixKeepsRealNotes(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + gitIn(t, dir, "notes", "--ref="+mxMetadataRef, "add", "-f", "-m", realStudioProNote, "HEAD") + + var out bytes.Buffer + doctorRun(dir, mprPath, "", "", true, &out) // config findings remain; the note must not + if got := strings.TrimSpace(noteOn(t, dir, "HEAD")); got != realStudioProNote { + t.Fatalf("real note was changed:\n got %s\nwant %s", got, realStudioProNote) + } +} + +func TestGitDoctor_TemplateWithoutProjectUsesVersionFlag(t *testing.T) { + dir := t.TempDir() + doctorEnv(t, dir) + gitIn(t, dir, "init", "-q", "-b", "main") + gitIn(t, dir, "commit", "-q", "--allow-empty", "-m", "docs") + + // Without --version there is nothing to derive a ModelerVersion from. + var out bytes.Buffer + err := doctorRun(dir, "", "", "", true, &out) + if err == nil || !strings.Contains(err.Error(), "--version") { + t.Fatalf("want an error naming --version, got %v", err) + } + + out.Reset() + if err := doctorRun(dir, "", "", "10.6.0", true, &out); err != nil { + t.Fatalf("--fix --version must fill the notes: %v\n%s", err, out.String()) + } + var n mxNote + if err := json.Unmarshal([]byte(noteOn(t, dir, "HEAD")), &n); err != nil { + t.Fatal(err) + } + if n.ModelerVersion != "10.6.0" || n.BranchName != "" || !n.HasModelerVersion || len(n.ModelChanges) != 0 { + t.Errorf("template note: %+v", n) + } +} + +func TestGitDoctor_TemplateDetectsVersionFromHistory(t *testing.T) { + dir, _ := doctorNoteRepo(t) + doctorEnv(t, dir) + gitIn(t, dir, "notes", "--ref="+mxMetadataRef, "add", "-f", "-m", realStudioProNote, "HEAD~1") + + // No -p: the note cannot be computed from the model, so the template is + // written with the version detected from the newest valid note. + var out bytes.Buffer + if err := doctorRun(dir, "", "", "", true, &out); err != nil { + t.Fatalf("--fix: %v\n%s", err, out.String()) + } + var n mxNote + if err := json.Unmarshal([]byte(noteOn(t, dir, "HEAD")), &n); err != nil { + t.Fatal(err) + } + if n.ModelerVersion != "11.13.0" { + t.Errorf("template ModelerVersion = %q, want 11.13.0 from the newest valid note", n.ModelerVersion) + } +} + +// --- git commit --------------------------------------------------------------- + +func commitSubject(t *testing.T, dir string) string { + t.Helper() + out, err := gitOut(dir, "show", "-s", "--format=%s", "HEAD") + if err != nil { + t.Fatal(err) + } + return strings.TrimSpace(out) +} + +func committedFiles(t *testing.T, dir string) string { + t.Helper() + out, err := gitOut(dir, "show", "--name-only", "--format=", "HEAD") + if err != nil { + t.Fatal(err) + } + return out +} + +func TestGitCommit_HappyPathWritesNoteAndSkipsJunk(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + // A real source change, plus one junk file that must never be committed. + units := baseUnits(t) + units[tPage] = tUnit{tFolder, unitDoc(t, "Forms$Page", "Home", bson.E{Key: "Title", Value: "Again"})} + writeProject(t, dir, "11.13.0", units) + if err := os.WriteFile(filepath.Join(dir, "project-settings.user.json"), []byte("{}"), 0o644); err != nil { + t.Fatal(err) + } + + var out bytes.Buffer + if err := commitRun(mprPath, "retitle again", false, false, &out); err != nil { + t.Fatalf("commitRun: %v\n%s", err, out.String()) + } + if subj := commitSubject(t, dir); subj != "retitle again" { + t.Errorf("HEAD subject = %q", subj) + } + files := committedFiles(t, dir) + if strings.Contains(files, "project-settings.user.json") { + t.Errorf("user settings must not be committed:\n%s", files) + } + var n mxNote + if err := json.Unmarshal([]byte(noteOn(t, dir, "HEAD")), &n); err != nil { + t.Fatalf("HEAD has no valid note: %v", err) + } + if n.BranchName != "feature" || !n.HasModelerVersion { + t.Errorf("note header: %+v", n) + } + // The junk file is still on disk, untracked. + if _, err := os.Stat(filepath.Join(dir, "project-settings.user.json")); err != nil { + t.Error("junk file was deleted") + } + if !strings.Contains(out.String(), "git push origin feature "+mxMetadataRef) { + t.Errorf("must tell how to push branch and notes:\n%s", out.String()) + } +} + +func TestGitCommit_RepairsHashDrift(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + // Rewrite a unit file behind the index's back: the classic git checkout drift. + page := filepath.Join(dir, "mprcontents", tPage[:2], tPage[2:4], tPage+".mxunit") + raw, err := os.ReadFile(page) + if err != nil { + t.Fatal(err) + } + raw = append(raw[:len(raw)-2], raw[len(raw)-2]+1, raw[len(raw)-1]) + if err := os.WriteFile(page, raw, 0o644); err != nil { + t.Fatal(err) + } + + var out bytes.Buffer + if err := commitRun(mprPath, "drift", false, false, &out); err != nil { + t.Fatalf("commitRun must repair mismatched hashes and commit: %v\n%s", err, out.String()) + } + var out2 bytes.Buffer + if err := fixHashes(mprPath, false, &out2); err != nil { + t.Errorf("hashes still drift after commit: %v\n%s", err, out2.String()) + } + if subj := commitSubject(t, dir); subj != "drift" { + t.Errorf("HEAD subject = %q", subj) + } +} + +func TestGitCommit_RefusedWhileStudioProOpen(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + lock := mprPath + ".lock" + if err := os.WriteFile(lock, []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + + var out bytes.Buffer + if err := commitRun(mprPath, "nope", false, false, &out); err == nil { + t.Fatal("commit must be refused while the lock file exists") + } + if subj := commitSubject(t, dir); subj != "retitle home" { + t.Errorf("a commit was made anyway: %q", subj) + } +} + +func TestGitCommit_DryRunCommitsNothing(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + if err := os.MkdirAll(filepath.Join(dir, "theme"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "theme", "x.txt"), []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + before := commitSubject(t, dir) + + var out bytes.Buffer + if err := commitRun(mprPath, "dry", false, true, &out); err != nil { + t.Fatalf("dry run failed: %v\n%s", err, out.String()) + } + if got := commitSubject(t, dir); got != before { + t.Errorf("dry run committed: %q", got) + } + staged, err := gitOut(dir, "diff", "--cached", "--name-only") + if err != nil { + t.Fatal(err) + } + if strings.TrimSpace(staged) != "" { + t.Errorf("dry run left changes staged:\n%s", staged) + } +} + +func TestGitCommit_PushesBranchAndNotes(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + units := baseUnits(t) + units[tPage] = tUnit{tFolder, unitDoc(t, "Forms$Page", "Home", bson.E{Key: "Title", Value: "Ship"})} + writeProject(t, dir, "11.13.0", units) + remote := t.TempDir() + gitIn(t, remote, "init", "-q", "--bare") + gitIn(t, dir, "remote", "add", "origin", remote) + + var out bytes.Buffer + if err := commitRun(mprPath, "push me", true, false, &out); err != nil { + t.Fatalf("commitRun --push: %v\n%s", err, out.String()) + } + ls, err := gitOut(dir, "ls-remote", "origin") + if err != nil { + t.Fatal(err) + } + if !strings.Contains(ls, "refs/heads/feature") { + t.Errorf("branch was not pushed:\n%s", ls) + } + if !strings.Contains(ls, mxMetadataRef) { + t.Errorf("notes ref was not pushed:\n%s", ls) + } +} + +func TestGitCommit_NothingToCommit(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + + var out bytes.Buffer + if err := commitRun(mprPath, "empty", false, false, &out); err != nil { + t.Fatalf("nothing to commit must not be an error: %v", err) + } + if !strings.Contains(out.String(), "nothing to commit") { + t.Errorf("must say so:\n%s", out.String()) + } +} + +// gitInE runs git and lets it fail, for the commands whose failure is the +// fixture (a merge that conflicts). +func gitInE(t *testing.T, dir string, args ...string) (string, error) { + t.Helper() + cmd := exec.Command("git", append([]string{"-C", dir}, args...)...) + b, err := cmd.CombinedOutput() + return string(b), err +} + +// A staged path with spaces must reach the commit whole, and junk under a +// spaced path must still be left out. +func TestGitCommit_SpacedPathsStagedWholeAndJunkLeft(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + if err := os.MkdirAll(filepath.Join(dir, "my docs"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "my docs", "read me.txt"), []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.MkdirAll(filepath.Join(dir, "deployment"), 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "deployment", "old log.txt"), []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + + var out bytes.Buffer + if err := commitRun(mprPath, "spacey", false, false, &out); err != nil { + t.Fatalf("commitRun: %v\n%s", err, out.String()) + } + files := committedFiles(t, dir) + if !strings.Contains(files, "my docs/read me.txt") { + t.Errorf("spaced path not committed whole:\n%s", files) + } + if strings.Contains(files, "deployment/old log.txt") { + t.Errorf("junk was committed:\n%s", files) + } +} + +// An unresolved merge must refuse the commit: `git add -A` would otherwise +// stage the conflict markers and commit them. +func TestGitCommit_RefusedDuringUnresolvedMerge(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + if err := os.WriteFile(filepath.Join(dir, "notes.txt"), []byte("feature side\n"), 0o644); err != nil { + t.Fatal(err) + } + gitIn(t, dir, "add", "notes.txt") + gitIn(t, dir, "commit", "-q", "-m", "notes feature") + gitIn(t, dir, "switch", "-q", "-c", "other", "HEAD~1") + if err := os.WriteFile(filepath.Join(dir, "notes.txt"), []byte("other side\n"), 0o644); err != nil { + t.Fatal(err) + } + gitIn(t, dir, "add", "notes.txt") + gitIn(t, dir, "commit", "-q", "-m", "notes other") + gitIn(t, dir, "switch", "-q", "feature") + if _, err := gitInE(t, dir, "merge", "other"); err == nil { + t.Fatal("fixture: the merge was expected to conflict") + } + + var out bytes.Buffer + if err := commitRun(mprPath, "conflicted", false, false, &out); err == nil { + t.Fatal("commit must be refused during an unresolved merge") + } + if subj := commitSubject(t, dir); subj != "notes feature" { + t.Errorf("a commit was made anyway: %q", subj) + } +} + +// The lock guard honours the same override every other mxcli write honours, +// for the crash-left-a-lock case. +func TestGitCommit_LockOverrideEnvCommits(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + if err := os.WriteFile(mprPath+".lock", []byte("x"), 0o644); err != nil { + t.Fatal(err) + } + units := baseUnits(t) + units[tPage] = tUnit{tFolder, unitDoc(t, "Forms$Page", "Home", bson.E{Key: "Title", Value: "Crashed"})} + writeProject(t, dir, "11.13.0", units) + t.Setenv("MXCLI_ALLOW_STUDIO_PRO_OPEN", "1") + + var out bytes.Buffer + if err := commitRun(mprPath, "override", false, false, &out); err != nil { + t.Fatalf("commitRun with the override env: %v\n%s", err, out.String()) + } + if subj := commitSubject(t, dir); subj != "override" { + t.Errorf("HEAD subject = %q", subj) + } +} + +// A dry run writes nothing — not even the ContentsHash repair. +func TestGitCommit_DryRunLeavesHashDriftUntouched(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + page := filepath.Join(dir, "mprcontents", tPage[:2], tPage[2:4], tPage+".mxunit") + raw, err := os.ReadFile(page) + if err != nil { + t.Fatal(err) + } + raw = append(raw[:len(raw)-2], raw[len(raw)-2]+1, raw[len(raw)-1]) + if err := os.WriteFile(page, raw, 0o644); err != nil { + t.Fatal(err) + } + + var out bytes.Buffer + if err := commitRun(mprPath, "dry over drift", false, true, &out); err != nil { + t.Fatalf("dry run failed: %v\n%s", err, out.String()) + } + var out2 bytes.Buffer + if err := fixHashes(mprPath, false, &out2); !errors.Is(err, errHashIssuesRemain) { + t.Errorf("the dry run repaired the hashes (err %v):\n%s", err, out2.String()) + } +} + +func TestGitCommit_DryRunWithPushRefused(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + + var out bytes.Buffer + if err := commitRun(mprPath, "both", true, true, &out); err == nil { + t.Fatal("--dry-run with --push must be refused") + } +} + +// A misspelled -p must be refused, not turned into a "real" note with an +// empty ModelerVersion that no later run would ever touch again. +func TestGitDoctor_BadProjectPathRefused(t *testing.T) { + dir, mprPath := doctorNoteRepo(t) + doctorEnv(t, dir) + + var out bytes.Buffer + err := doctorRun(dir, mprPath+"-typo", "", "10.6.0", true, &out) + if err == nil || !strings.Contains(err.Error(), "project not found") { + t.Fatalf("want a project-not-found error, got %v", err) + } + if note := noteOn(t, dir, "HEAD"); strings.Contains(note, `"ModelerVersion":""`) { + t.Fatalf("a note with an empty version was written:\n%s", note) + } +} + +func TestGitDoctor_VersionFlagValidated(t *testing.T) { + dir := t.TempDir() + doctorEnv(t, dir) + gitIn(t, dir, "init", "-q", "-b", "main") + gitIn(t, dir, "commit", "-q", "--allow-empty", "-m", "docs") + + var out bytes.Buffer + err := doctorRun(dir, "", "", "banana", true, &out) + if err == nil || !strings.Contains(err.Error(), "--version") { + t.Fatalf("want a --version format error, got %v", err) + } +} diff --git a/cmd/mxcli/gitnote_test.go b/cmd/mxcli/gitnote_test.go index b9499345ec..361a1146a4 100644 --- a/cmd/mxcli/gitnote_test.go +++ b/cmd/mxcli/gitnote_test.go @@ -243,14 +243,14 @@ func writeProject(t *testing.T, dir, version string, units map[string]tUnit) { } defer db.Close() for _, q := range []string{ - `CREATE TABLE _MetaData (_ProductVersion TEXT)`, + `CREATE TABLE _MetaData (_ProductVersion TEXT, _BuildVersion TEXT, _SchemaHash TEXT)`, `CREATE TABLE Unit (UnitID BLOB PRIMARY KEY, ContainerID BLOB, ContainmentName TEXT, TreeConflict INTEGER, ContentsHash TEXT, ContentsConflicts TEXT)`, } { if _, err := db.Exec(q); err != nil { t.Fatal(err) } } - if _, err := db.Exec(`INSERT INTO _MetaData VALUES (?)`, version); err != nil { + if _, err := db.Exec(`INSERT INTO _MetaData VALUES (?, ?, '')`, version, version); err != nil { t.Fatal(err) } for id, u := range units { diff --git a/docs-site/src/tools/outside-studio-pro.md b/docs-site/src/tools/outside-studio-pro.md index 35cac97dfd..edfd2017ba 100644 --- a/docs-site/src/tools/outside-studio-pro.md +++ b/docs-site/src/tools/outside-studio-pro.md @@ -108,6 +108,60 @@ mxcli diag -p app.mpr prints mxcli's diagnostics followed by a *Project* section with both checks. +## One commit, all the steps: `mxcli git commit` + +Studio Pro's *Commit* does more than `git commit`: it keeps the ContentsHash +index in step, stages exactly the files a Mendix repository should ship, and +attaches the mx_metadata note. `mxcli git commit` is that sequence for commits +made outside Studio Pro: + +```bash +mxcli git commit -m "Add order history page" -p app.mpr +mxcli git commit -m "Fix stock filter" -p app.mpr --push +mxcli git commit -m "WIP" -p app.mpr --dry-run +``` + +In order it: + +1. repairs ContentsHash mismatches from the files on disk (`MISSING` and + `ORPHAN` entries have no automatic repair and refuse the commit); +2. stages everything, then unstages the files Studio Pro never commits: + `project-settings.user.json`, `*.mpr.lock`, `*.mpr.bak`, `.mendix-cache/`, + `deployment/`, `theme-cache/`, `.mxcli/`; +3. runs `git commit` with your message and your git identity; +4. computes the mx_metadata note from the new commit and attaches it, and + prints the `git push origin refs/notes/mx_metadata` command — + `--push` runs it for you, because the branch and the notes ref must travel + together. + +The commit is refused while Studio Pro has the project open (the `.mpr.lock` +beside it), and during an unresolved merge — `git add -A` would otherwise +commit the conflict markers. `--dry-run` verifies and reports (including +ContentsHash drift, which it does not repair — a dry run writes nothing), +stages, reports what it would commit, and unstages again without committing. + +## Auditing a repository: `mxcli git doctor` + +A repository that has known only plain git misses several things Studio Pro +wrote when it committed, and they are invisible one by one. `mxcli git doctor` +reports all of them in one read-only pass; `--fix` repairs: + +```bash +mxcli git doctor # report only; exits 1 while an issue remains +mxcli git doctor -p app.mpr --fix # repair, computing real notes from the model +mxcli git doctor --fix --version 10.21.0 +``` + +| Check | `--fix` | +|-------|---------| +| local git config: `core.autocrlf=False` (a checkout must not rewrite line endings inside `.mxunit` files), `mendix.commits-since-gc`, `mendix.lineEndingResetDone` | sets them | +| the remote URL: Studio Pro can only use HTTPS | rewrites SSH (`git@host:org/app.git`, `ssh://…`) to HTTPS; other non-HTTPS URLs are reported unchanged | +| the remote's `refs/notes/mx_metadata` | reported only — push the notes ref together with the branch | +| each commit on the checked-out branch and its mx_metadata note: missing notes make Studio Pro show `(unknown)`; a malformed note it cannot read at all | computes real notes from the model (`-p`), falling back to a template note with a `--version` ModelerVersion where the model at a commit cannot be read (MPR v1 history, a revision without the `.mpr`); when no version is known the newest valid note in history supplies it | + +Notes Studio Pro wrote are never touched, with or without `--fix`. `--fix` +does not push. + ## Commit metadata: `mxcli git note` Studio Pro attaches a git note under `refs/notes/mx_metadata` to every commit