Add systematic tests to mg7#32
Conversation
Truth value is reported in #28. Processes metadata are reported in tests/check_xsec_processees/reference.json Tolerance and number of events can be set on the workflow file.
Especially for testing purposes, we make the workflow runnable when pushing to the repository.
|
Hi @Qubitol. Would it be possible to keep this within the test_manager.py framework ?
In principle then the CI can then use test_manager.py to launch such test (which will be nice when I would implement a nice action for running such tests (which is not the case for the moment). After that how do you want to proceed? |
|
I agree on all the points you raised, I will fix this to make sure it depends on the test manager. Regarding the failing: if it is related only to the physics processes themselves, then my idea would be to merge as failing CI, and we can check each process separately until it satisfies the cross section. |
|
Maybe we should define the threshold based on the MC integration error, i.e. something like |
The systematic cross-section CI self-skipped with "NNPDF23_lo_as_0130_qed LHAPDF data not found" because it relied on the NNPDF23 grid being pre-baked into the HEPTools cache, which is not guaranteed. The mg7 runtime reads the PDF grid directly from $LHAPDF_DATA_PATH/<set>/<set>_0000.dat (mg7/madevent.py, ms.PdfGrid), so it only needs the grid data -- not the LHAPDF C++ library. Add an install_lhapdf_pdfset composite action that installs the set with the lhapdf-management package into a cached, writable dir and exports LHAPDF_DATA_PATH, and wire it into the workflow in place of the fragile restore_heptools_lhapdf dependency. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per reviewer request, replace the standalone
run_check_xsec_processes.py driver with a unittest.TestCase
(tests/acceptance_tests/test_check_xsec_processes_mg7.py) that generates
one test_<section>_<id>_mg7 method per process from the reference table,
so the whole suite is discoverable and launchable through
tests/test_manager.py -- the single, locally-runnable entry point used by
the other mg7 acceptance tests.
The tolerance and event count stay settable via the MG7_XSEC_TOLERANCE /
MG7_XSEC_EVENTS environment variables. The workflow now launches each
section with a regexp, e.g.
./tests/test_manager.py test_ttx_decay_.*_mg7 -pA -t0
and the reference table moves next to the test as
check_xsec_processes_reference.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Yes, that's a good idea.
|
|
At this stage I would be agressive on when to run those. If this is too much, then we will be able to reduce that in the future |
|
Regarding the first point: we should have some generic way to change run card options for some processes, like cuts for the ones with decays, or number of events for other ones should we find that is necessary. Could be as simple as a list of key-value pairs added to the json, and then the |
|
I would propose to merge this, right away and then do a follow up PR where we handle the way to use the set command. |
Ok, good for me, then. |
3 participants
This implements #28.
Tests are defined in the metadata file
tests/check_xsec_processes/reference.json.All the processes listed in #28 are reported, but we may want to run a smaller set.
Additionally, the MadSpace installation has been made into a single shared action, that can be used by multiple workflows to build, install, and cache it, without code duplication.
TODO We need to decide when to run the workflow, if manually, on push, on push to dev/main, or similar.