working on the generator code#167
Closed
webern wants to merge 27 commits into
Closed
Conversation
Delete the project in which core roundtrip, or corert tests were introduced.
Maintain AGENTS.md with fresh information. Also I had thought that MusicXML 4.1 was released but it is just the unreleased working version number, so fix references to it in instruction docs.
Add Jinja2 template rendering for the 101 simple-value elements, replacing the fake-CT path through generate_element_h/cpp. Templates and routing config live in gen/cpp/. Jinja2 is pinned in a new /opt/gen-venv Docker venv. New `make generate` target runs the generator inside Docker.
Move mutable sets (SYNTHETIC_OPTIONAL_GROUPS, etc.), static group dicts, WRAPPING_STREAMCONTENTS, and group_class_name() into a dedicated module.
Move CORE_ROOT_ATTRS, ATTRS_TYPE_ALIAS, ELEMENTS_DIR_SHARED_ATTRS, and resolve_attrs_name() into a dedicated module for attribute struct naming logic.
GROUPS_WITH_REAL_FROM_X_ELEMENT -> group_config.py DYNAMICS_MARKS, CHOICE_SKIP -> element_config.py
Move the 50-line SCORE_WRAPPER_FLAVOR_CONFIG dict (partwise vs timewise behavioral knobs) into its own module.
Per-attribute default value overrides are now expressed as nested TOML tables under [overrides.attr_default], loaded at import time by overrides.py into the same tuple-keyed dict.
XMLNS_PRESERVING_ATTRS, CHILD_INIT_VALUE_OVERRIDE, and ELEMENT_HAS_CONTENTS_ALWAYS_TRUE now live in TOML under [overrides], loaded by overrides.py at import time.
Replace 55-line generate_group_h f-string body with a group_h.j2 template, registered in config.toml under [categories.group].
Replace 185-line generate_group_cpp f-string body with a group_cpp.j2 template. Pre-renders ctor init list and fromXElementImpl (complex line-wrapping logic) in Python, passes the rest as structured data to the template.
Replace generate_attrs_h f-string body with attrs_h.j2 template, registered in config.toml under [categories.attrs].
Replace 125-line generate_attrs_cpp f-string body with attrs_cpp.j2 template. Pre-renders the ctor init list in Python, passes attr metadata as structured data.
SCORE_WRAPPER_FLAVOR_CONFIG (partwise/timewise behavioral knobs) now lives under [score_wrapper] in config.toml, loaded by score_config.py at import time.
CORE_ROOT_ATTRS, ELEMENTS_DIR_SHARED_ATTRS, and ATTRS_TYPE_ALIAS now live under [attrs] in config.toml, loaded by attrs_config.py.
OVERWRITE_FILE_STEMS, ELEMENT_CLASS_NAME_OVERRIDE, ELEMENT_VALUE_TYPE_OVERRIDE, SKIP_ELEMENTS, BESPOKE_FAMILY_OWNED, and TREE_ELEMENTS now live under [elements] in config.toml.
GENERATE_GROUPS, WRAPPING_STREAMCONTENTS, GROUPS_WITH_REAL_FROM_X, NESTED_OPTIONAL_SEQUENCE_AS_GROUP, EXTENSION_OPTIONAL_GROUP_RENAME, UNBOUNDED_SEQUENCE_AS_GROUP, and SYNTHETIC_UNBOUNDED_GROUP_IMPORT_- GROUP_AFTER now live under [groups] in config.toml.
Core-dev coverage
|
| Metric | Coverage | Covered / Total |
|---|---|---|
| Lines | 68.7% | 23306 / 33936 |
| Functions | 60.8% | 4624 / 7601 |
| Branches | 49.5% | 17880 / 36118 |
Download HTML report - unzip and open index.html.
Commit d40f1a0db0f69d69f0166c1da1dfe8cbcbb3ff9d.
gen-quality
|
webern
added a commit
that referenced
this pull request
Jun 14, 2026
Initially I tried using AI to reverse-engineer the pseudo-hand-rolled original codegen. It worked but resulted in a 12k lines of unredeemable Python garbage with no room to maneuver from there. So, I started over and developed a `gen/` program from the ground up using AI. I tried to keep it agnostic as to language target, so I think targeting other languages and use-cases is a real possibility now. Early on, I used Go and C as language targets to force the AI to think about extensibility. Those targets exist under `gen/test`, but are intended more as `gen/` program regression tests than for actual MusicXML use. For the replaced `mx::core` code, I prioritized compile time and better use of C++ features like `variant` and `option`. AI wanted to drop the `ezxml` abstraction and I guess it was time to let it go, so `pugixml` is promoted to `mx::core` interaction. A `test-core-dev` target was used to allow the the AI to innovate on `mx::core` without worrying about `mx::impl` and `mx::api` (in fact, I deleted those layers during code-gen). Then I replaced those layers and burned tokens to preserve the `mx::impl` algorithms targeting the new set of `mx::core` classes. ## References - Closes #157 - Closes #158 - Progresses #58 - Closes PR #167 - Closes PR #168 ## Follow-ups: - surface more features in the `mx/api` layer. top priority is probably SMuFL - better packaging and distribution
Owner
Author
|
Superseded by #169 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.