Skip to content

[Pyomo.DoE] Add simultaneous design of multiple experiments#3866

Open
smondal13 wants to merge 188 commits into
Pyomo:mainfrom
smondal13:add-multiexperiment
Open

[Pyomo.DoE] Add simultaneous design of multiple experiments#3866
smondal13 wants to merge 188 commits into
Pyomo:mainfrom
smondal13:add-multiexperiment

Conversation

@smondal13
Copy link
Copy Markdown
Contributor

@smondal13 smondal13 commented Mar 2, 2026

Fixes # .

Summary/Motivation:

This PR adds a new DesignOfExperiments.optimize_experiments() API in pyomo/contrib/doe/doe.py to support simultaneous optimization of multiple experiments in one workflow. The motivation is to provide a multi-experiment DoE interface with stronger initialization options, clearer mode handling (template vs. user-initialized experiments), and richer diagnostics/results than the existing single-experiment path.

Changes proposed in this PR:

  • Added API optimize_experiments() for multi-experiment DoE optimization.
  • Added Cholesky-based D- and A-optimality objective and Greybox-based D-, A-, E-, and ME- optimality objective
  • Implemented two operating modes:
    • Template mode: pass one experiment and set n_exp.
    • User-initialized mode: pass a list of experiments; n_exp is inferred/validated.
  • Added optional LHS-based initialization (initialization_method="lhs") with controls for:
    • sample count, seed, candidate evaluation parallelism, combination fim metric scoring parallelism,
    • worker count, chunk size, parallel threshold, and optional wall-clock budget.
  • Added symmetry-breaking constraints for multi-experiment solves:
    • supports user-specified variable through sym_break_cons suffix,
    • falls back to the first experiment input with a diagnostic warning when not provided.
  • Expanded output for this API:
    • per-scenario and per-experiment results (designs, outputs, measurement errors, FIM/sensitivities),
    • aggregated FIM metrics, timings, settings, names, diagnostics, and structured run_info.
  • Added JSON-safe serialization via _DoEResultsJSONEncoder for numpy/Pyomo-enum values when writing results_file.

Note:

  • Added new documentation.md which describes the API. This documentation is to help the reviewers to understand the API and will not be merged into Pyomo:main

Remove before merging

  • documentation.md
  • rb_multi.py

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

…d replace `self.experiment` with `self.experiment_list[0]`. `doe/reactor_example.py` runs successfully.
…sing the `doe/reactor_multi_experiment.py`
…nt for `optimize_experiments()` and `compute_FIM()`
… both of sensitivity and optimize_experiments()
…A-opt gave different result for grid and optimization
@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt @blnicho , I have addressed the comments from @mrmundt. Wating for the CI tests to pass

Comment thread pyomo/contrib/doe/tests/test_greybox.py Outdated
@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt All the tests are passing except win/3.11 with

INFO: The following extensions were downloaded:
    [FAIL]  gjh
    [ OK ]  mcpp

Comment thread pyomo/contrib/doe/tests/test_doe_errors.py Outdated
smondal13 and others added 2 commits May 21, 2026 11:24
Co-authored-by: Miranda Mundt <55767766+mrmundt@users.noreply.github.com>
Comment thread pyomo/contrib/doe/tests/test_greybox.py Outdated
@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt and @blnicho , all the comments are addressed, and all the tests are passing now.

@slilonfe5
Copy link
Copy Markdown
Member

@mrmundt @blnicho, Shuvo has addressed all the comments. This looks pretty good to me. Any final review would be appreciated; otherwise, I think we are good to go.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 27, 2026

Codecov Report

❌ Patch coverage is 15.68627% with 86 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.07%. Comparing base (984f242) to head (6b3e5bf).

Files with missing lines Patch % Lines
pyomo/contrib/doe/rb_multi.py 5.55% 85 Missing ⚠️
pyomo/contrib/doe/grey_box_utilities.py 91.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3866      +/-   ##
==========================================
- Coverage   90.10%   90.07%   -0.04%     
==========================================
  Files         909      910       +1     
  Lines      108551   109249     +698     
==========================================
+ Hits        97813    98405     +592     
- Misses      10738    10844     +106     
Flag Coverage Δ
builders 28.97% <10.78%> (-0.13%) ⬇️
default 86.09% <10.78%> (?)
expensive 34.95% <10.78%> (?)
linux 87.61% <15.68%> (-2.00%) ⬇️
linux_other 87.61% <15.68%> (-0.01%) ⬇️
oldsolvers 27.89% <0.98%> (-0.16%) ⬇️
osx 83.06% <5.88%> (-0.02%) ⬇️
win 85.38% <15.68%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@adowling2
Copy link
Copy Markdown
Member

@smondal13 This PR is now showing merge conflicts.

Copy link
Copy Markdown
Member

@adowling2 adowling2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

Comment thread pyomo/contrib/doe/tests/test_doe_build.py Outdated
Comment thread pyomo/contrib/doe/tests/test_doe_build.py
Comment thread pyomo/contrib/doe/tests/test_doe_build.py
@smondal13
Copy link
Copy Markdown
Contributor Author

smondal13 commented May 28, 2026

I think this is practically ready. I requested a few small comments. Also, there is a merge conflict (I think).

@adowling2 I have addressed your comments. @adowling2 @mrmundt, I have resolved the conflicts. The conflicts were from @adowling2's recent PR #3867 merge into Pyomo:main.

@smondal13
Copy link
Copy Markdown
Contributor Author

@mrmundt @blnicho, the following two tests are failing. I think these two are not part of my PR

=========================== short test summary info ============================
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_any_solver - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set
FAILED pyomo/contrib/iis/tests/test_iis.py::TestIIS::test_write_iis_cplex - SystemError: <method 'write' of '_io.FileIO' objects> returned a result with an exception set

@blnicho
Copy link
Copy Markdown
Member

blnicho commented May 28, 2026

@smondal13 those failures were resolved in #3965 and I merged main into your PR to pick up the fix.

@smondal13
Copy link
Copy Markdown
Contributor Author

@blnicho @mrmundt this one is failing due to download issue

downloaded:
[FAIL] gjh
[ OK ] mcpp
Error: Process completed with exit code 1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready for design review

Development

Successfully merging this pull request may close these issues.

7 participants