Pyomo.DoE: fix trace/Cholesky initialization consistency

[adowling2]

Fixes # .

Summary/Motivation:

This PR adds advanced debugging/inspection controls for pyomo.contrib.doe.DesignOfExperiments.run_doe() while preserving default behavior for typical users fixes a trace-objective initialization inconsistency that can produce large residuals in Cholesky-related constraints at the start of the final NLP solve.

The motivating workflow showed:

• large early infeasibilities in optimize mode,
• inability to use max_iter=0 for final-model probing because scenario solves reused the same solver options,
• and inconsistent initialization among fim, L, fim_inv, L_inv, and cov_trace.

Changes proposed in this PR:

• Added an advanced run_config (ConfigBlock/dict) argument to run_doe(model=None, results_file=None, run_config=None):
  • scenario_solver_options
  • final_solver_options
  • final_solve
  • inspection.enabled
  • inspection.top_constraints
• Implemented scoped solver option application:
  • scenario-generation and square-initialization solves can use different options from the final optimization solve.
• Added optional structured residual reporting utility:
  • reports top violated active constraints with fields:
    • constraint_name
    • body
    • lower_bound
    • upper_bound
    • violation
    • constraint_type
• Added assemble-and-inspect path (final_solve=False) to inspect assembled NLP state without running final optimization.
• Fixed Cholesky jitter computation and added robust helper usage.
• Added _initialize_cholesky_from_fim() to re-synchronize L, L_inv, fim_inv, and cov_trace from current FIM immediately before final solve / inspection.
• Updated docstrings and added advanced usage documentation + example path in reactor_example.py.
• Added/updated regression tests in test_doe_initialization.py for jitter behavior and FIM synchronization.

Backward compatibility:

• Default usage (run_doe() with no run_config) remains unchanged.
• Advanced functionality is available only when run_config is provided.

Testing:

I ran:

• pytest -q pyomo/contrib/doe/tests/test_doe_debug.py
  • Result: 8 passed
• pytest -q pyomo/contrib/doe/tests/test_doe_solve.py -k "rooney_biegler_fd_central_solve"
  • Result: 1 passed, 17 deselected
• python3 -m unittest -q pyomo.contrib.doe.tests.test_doe_initialization
  • Result: 3 skipped in the current interpreter because SciPy/numpy are unavailable there
• python3 -m py_compile pyomo/contrib/doe/doe.py pyomo/contrib/doe/tests/test_doe_initialization.py
  • passed

New/updated tests cover:

• negative-jitter helper behavior,
• zero-jitter helper behavior,
• Cholesky/FIM inverse consistency after initialization in trace mode.

Outstanding Tasks (before removing WIP):

• Finalize proposed interface for advanced initialization diagnostics
• Decide the best way to document the proposed interface. Are examples sufficient or should we build out the Pyomo documentation?
• Run black
• @adowling2 carefully reviews proposed changes

Why the strikethought?

The original version of this PR included a new interface to run_doe() that makes debugging the initialization and, more broadly, solver failures easier. We decided that the new run_doe() interface added too much complexity and exposed some inconsistencies in other parts of the Pyomo solver interface. (This led to a real-time Codex experiment documented in #3871.) To expedite this current PR, I decided to remove these contributions and strike through the above parts of the PR text. These contributions are still available in the git history. Thus, this current PR only contributes a targeted patch to the Pyomo FIM initialization.

Legal Acknowledgement

By contributing to this software project, I have read the contribution guide and agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the BSD license.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[adowling2]

@smondal13 @sscini @slilonfe5 @snarasi2 @djlaky I found a small bug in how we initialize the Cholesky factorization when the FIM at the initial point needs "inertia correction". This PR fixes the initialization inconsistency. This PR also proposes an interface to allow for more advanced solver options when diagnosing initialization issues for the Pyomo.DoE (dynamic) optimization formulation.

Edit: This is a work in progress right now. I am not 100% happy with the proposed changes.

[adowling2]

@sscini @blnicho @mrmundt @smondal13 @slilonfe5 This is ready for review. This PR is now narrowly targeted at fixing a Pyomo.DoE FIM initialization issue.

[blnicho]

Is there a reason to have this as a method on the DesignOfExperiments class instead of a standalone utility function? It isn't using anything from the class instance.

Also, it's a 1-line method that is only being used once on L555. Why did you even abstract this out as a separate method?

[adowling2]

@blnicho @mrmundt Thank you for the great comments. I have implemented changes to address your comments and ran black. The tests are passing locally and are running on GitHub right now.

[adowling2]

Tests now pass! (I had failures on tests that were being skipped in my local environment.)

[blnicho]

@adowling2 I have one minor question but I'm also fine with merging as-is.

[blnicho]

Is there a more specific exception type that can be used here?

[mrmundt]

I'll push back and say that it doesn't matter for this test suite. This is a catch-all for, "In case anything else goes wrong while trying to run cyipopt."

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.11%. Comparing base (984f242) to head (7c073a5).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3867   +/-   ##
=======================================
  Coverage   90.10%   90.11%           
=======================================
  Files         909      909           
  Lines      108551   108579   +28     
=======================================
+ Hits        97813    97848   +35     
+ Misses      10738    10731    -7     

Flag	Coverage Δ
builders	29.10% <57.14%> (+<0.01%)	⬆️
default	86.11% <100.00%> (?)
expensive	35.13% <57.14%> (?)
linux	87.63% <96.42%> (-1.98%)	⬇️
linux_other	87.63% <96.42%> (+<0.01%)	⬆️
oldsolvers	28.04% <5.35%> (-0.01%)	⬇️
osx	83.06% <76.78%> (-0.01%)	⬇️
win	85.38% <96.42%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.