[Exanite]

Summary of the PR

This PR reworks the generator to be configured through a C# API rather than a JSON file. The original reason for this was to allow more complex conditions to be easily expressed, such as for skipping mods/jobs on specific platforms.

In addition, this PR also reworks the dependency injection setup to be properly scope-based rather than using the keyed dependency resolution approach the generator was using before. This structurally avoids issues where state is accidentally shared between jobs.

The generator config is now found in a new Silk.NET.SilkTouch.Jobs project which is next to the SilkTouch project. Because Silk has a lot of bindings, I decided to split the config into multiple files. External consumers of SilkTouch should be able to use single file apps if more appropriate (should work, but I haven't tested yet).

---

This PR also reworks how copyright headers, using statements, and namespaces are injected into generated documents. These were previously implicitly handled by Mod and ModCSharpSyntaxRewriter, both of which were often accidentally used throughout the codebase. These are now replaced by DocumentFactory, which makes the process much more explicit. The idea is to call DocumentFactory.PopulateDocument before adding new documents to the project. ClangScraper also calls DocumentFactory.ReportDocument to generate a set of default using statements.

The behavior is otherwise mostly the same other than the need to call the DocumentFactory methods explicitly. One behavior I decided not to replicate is implicitly deciding the header to inject into new docs. DocumentFactory always uses the DefaultLicenseHeader defined in the job config rather than taking the longest header comment found.

Related issues, Discord discussions, or proposals

Original discussion with Perksey: https://discord.com/channels/521092042781229087/587346162802229298/1530907649561067693
Discord thread: https://discord.com/channels/521092042781229087/1542235869560438824

Further Comments

Next PR

This PR will be followed up by a PR that focuses on OpenXR and multi-platform ClangSharp scraping.

Tasks (from my tracker)

• Fix issues when running multiple jobs
  • Figure out why the headers are getting mixed up when running multiple jobs at the same time.
    • Seems to be related to TransformFunctions.
  • Figure out why the SDL handle structs are getting named as Silk.NET.SDL.ConditionHandle.gen.cs instead of ConditionHandle.gen.cs when running multiple jobs at the same time.
• Investigate how DI is set up in this project. I believe it is the core cause of the multi-job isolation issues. I need to verify this, but the jobs seem to share the same mod instances instead of being separated. This would explain why we need to access data by job key instead of it being simply injected into the mod instances.
• Rework registration/configuration system to be in pure C# instead of JSON
  • This also is related to reworking the DI setup.
  • This is something I volunteered to do.
  • See: https://discord.com/channels/521092042781229087/587346162802229298/1530907649561067693

These will be replaced by DocumentFactory:

• Remove Mod (the class, not the similarly named interface)
• Remove ModCSharpSyntaxRewriter
• Rework how using statements are added. Not sure why, but using statements are randomly added/removed between Windows/Linux. This causes noisy diffs.

Tasks

• Rebase after [3.0] Add OpenCL and OpenXR bindings #2590 is merged
• Migrate all job configs
• Create new executable project
• Change SilkTouch project to be a library
• Remove thread locals and similar API usages since these were hacks used to separate job-specific state in the previous setup
• Add a DocumentFactory class that handles the creation of new files
  • This is intended to replace the hacks used to initialize new C# documents with using statements and copyright headers. See my task tracker tasks above.
• Ensure that all bindings output changes are expected
• Fix missing using statement issue
  • My thoughts:
    • Fundamentally, the issue is because my new using statement injector is more conservative (on demand, rather than just update the entire project).
    • Probably fine to just require mods to declare the using statements they need.
    • Currently, using statements can be both implicitly gathered and explicitly declared, but might be worth always explicitly declaring them. Difference would be whether DocumentFactory.PopulateDocument also calls ReportDocument internally.
• Update documentation
• Self code review

Added during code review:

To complete within this PR:

• Investigate the two GetCommittedFileAsync comments
  • Completed during [3.0] Add multi-platform generation and finish initial OpenXR bindings #2628 in dc4b66a

To add to my task tracker, but sooner than later (last task has been moved):

• Change IModContext.SourceProject to be non-nullable.
  • Completed during [3.0] Add multi-platform generation and finish initial OpenXR bindings #2628 in eb51766
• Change config params that are often left as AddIncludes(new()) to be optional.
  • Completed during [3.0] Add multi-platform generation and finish initial OpenXR bindings #2628 in 5238b8d
• Open a follow up PR that removes Autofac. We discussed this and while MS DI isn't the best DI solution feature-wise, I think we can avoid adding another dependency without much downside.

To add to my task tracker (these have been moved):

• Add dedicated documentation section/folder for using the SilkTouch generator as a non-contributor.
• Add single file SilkTouch example project
• Add SilkTouch example project that implements a custom mod
• If possible, change the public API docs analyzer to also require API docs on internal APIs
  • This is to ensure that we keep our internal docs up to par even after we make everything internal instead of public.

[github-actions]

All native library builds modified in this PR shall attempt to be built by CI.

[Exanite]

Force pushing to rebase on develop/3.0.

[Exanite]

Going to post my current self code review comments. Not fully happy with how I'm handling using statements, so might revisit in the future, but it's fine for now.

Also going to work on updating docs before marking this PR ready for review.

[Perksey]

I don't think there's anything too objectionable in here