Repository navigation
Conversation
In preparation for adding support for JSON data extensions, this commit updates the scripts so that they store and build models using an in-memory representation that is independent of the YAML format currently used. This means tuples are represented as Python lists, where previously they were created as YAML-formatted strings in the merging process. Scalar values inside those tuples are represented as Python strings and booleans. Only the DataExtensionFile.write_yaml function now has to deal with YAML formatting concerns.
The `generate_mad.py` script accepts a flag that lets you choose between YAML or JSON, but YAML is still the default. We can change the default later, when CLI support for JSON is widely available. `convert_extensions.py` also still hard-codes `.yml` as the output extension/format.
nickrolfe
force-pushed
the
nickrolfe/mad-generator-json
branch
from
October 9, 2026 15:49
e478733 to
95c94ef
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is split into two commits:
DataExtensionFile.write_yamlfunction now has to deal with YAML formatting concerns.For now, YAML is still the default. We can change this when CLI support for JSON extensions is widely available and the existing models in this repo have been converted to JSON. (I will be making a separate PR with a conversion script).
I tested these changes by running both
misc/scripts/models-as-data/generate_mad.pyandjava/ql/src/utils/modelconverter/ConvertExtensions.pyand manually inspecting the output.