Skip to content

[FIX]: Fix get_test_results() lazy-load crashes and missing-result guard#1118

Open
x15sr71 wants to merge 4 commits into
CCExtractor:masterfrom
x15sr71:fix/get-test-results-lazy-load-and-null-guard
Open

[FIX]: Fix get_test_results() lazy-load crashes and missing-result guard#1118
x15sr71 wants to merge 4 commits into
CCExtractor:masterfrom
x15sr71:fix/get-test-results-lazy-load-and-null-guard

Conversation

@x15sr71
Copy link
Copy Markdown
Contributor

@x15sr71 x15sr71 commented Jun 6, 2026

Pleaxse prefix your pull request with one of the following: [FEATURE] [FIX] [IMPROVEMENT].

In raising this pull request, I confirm the following (please check boxes):

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.

My familiarity with the project is as follows (check one):

  • I have never used the project.
  • I have used the project briefly.
  • I have used the project extensively, but have not contributed previously.
  • I am an active contributor to the project.

Description

get_test_results() — Production Bug Report & Fix

get_test_results() crashes with HTTP 500 on test detail pages. Production access.log contains 3,087 HTTP 500 responses on /test/ URLs. error.log contains 5697 stack-trace frames involving SQLAlchemy lazy-loading, plus 57 confirmed AttributeError crashes from a missing null check.

Evidence — commands run on production VM

All commands run on the production VM at /var/www/sample-platform/.

5697 SQLAlchemy lazy-load stack-trace frames

grep -n "_emit_lazyload\|_load_for_state" /var/www/sample-platform/logs/error.log | wc -l
# 5697

Bug 1 stack trace (category.regression_tests)

grep -A 10 "line 76, in <listcomp>" /var/www/sample-platform/logs/error.log | grep -A 5 "category.regression_tests" | head -40

57 NoneType crashes (Bug 3)

grep -n "line 94" /var/www/sample-platform/logs/error.log | grep "controllers.py" | wc -l
# 57

Bug 3 stack trace with test IDs

grep -A 10 "line 94, in get_test_results" /var/www/sample-platform/logs/error.log | head -40

3087 confirmed HTTP 500 responses on test detail pages

grep " 500 " /var/www/sample-platform/logs/access.log | grep "/test/" | wc -l
# 3087

Bug 4 — 19 regression tests with non-zero expected_rc

SELECT COUNT(*) FROM regression_test WHERE expected_rc != 0;
# 19

# All 19 have regression_test_output rows confirming comparison was intended
SELECT rt.id, rt.expected_rc, rto.correct, rto.correct_extension
FROM regression_test rt
JOIN regression_test_output rto ON rto.regression_id = rt.id
WHERE rt.expected_rc != 0
ORDER BY rt.id;
# 19 rows returned, all with defined correct hashes

Bug 1 — Lazy-load crash on category.regression_tests (line 76)

Stack trace appearing thousands of times in logs/error.log (verified with grep -A 10 "line 76, in <listcomp>"):

mod_test/controllers.py, line 68, in get_test_results
    results = [{
mod_test/controllers.py, line 76, in <listcomp>
    } for rt in category.regression_tests if rt.id in test.get_customized_regressiontests()]
sqlalchemy/orm/attributes.py, line 482, in __get__
sqlalchemy/orm/strategies.py, line 912, in _load_for_state
sqlalchemy/orm/strategies.py, line 1046, in _emit_lazyload

Cause: Category.regression_tests uses SQLAlchemy's default lazy='select' (no explicit lazy= in mod_regression/models.py line 38). The lazy SELECT fires inside the list comprehension when SQLAlchemy attempts a lazy load outside a usable session context.

Fix: selectinload(Category.regression_tests) on the Category query loads all regression tests in one batched query within the active session.


Bug 2 — Lazy-load on regression_test_output.multiple_files (lines 93–94)

result_file.regression_test_output and .multiple_files are both lazy-loaded relationships (mod_regression/models.py lines 148 and 153). They are accessed inside the same list comprehension that triggers Bug 1, making them structurally identical failure points in the same session context. A separately isolated stack trace for this specific relationship was not extracted from logs during this investigation.

Fix: joinedload(TestResultFile.regression_test_output).selectinload(RegressionTestOutput.multiple_files) applied to the TestResultFile query — joinedload for the scalar relationship (uselist=False), selectinload for the collection hanging off it.


Bug 3 — result is None crash (line 94)

57 confirmed occurrences in error.log at controllers.py line 94:

mod_test/controllers.py, line 94, in get_test_results
    if result_file.got is not None and result.exit_code == 0:
AttributeError: 'NoneType' object has no attribute 'exit_code'

Three example occurrences with test IDs:

  • Test 4936 (2024-05-16)
  • Test 5132 (2024-07-03)
  • Test 5736 (2025-05-28)

Cause: TestResultFile rows are created by equality_type_request and TestResult rows by finish_type_request — separate HTTP calls from the VM. When a TestResultFile exists with got IS NOT NULL but no TestResult row was written, result is None and .exit_code raises AttributeError.

Fix: (result is None or result.exit_code == 0) — treats a missing result as worth checking for file mismatches rather than crashing. If the hash matches an accepted variant in multiple_files, it clears correctly even when result is None.


Bug 4 — File comparison skipped for non-zero expected_rc tests (line 91)

19 confirmed affected tests (SELECT COUNT(*) FROM regression_test WHERE expected_rc != 0 = 19, all expected_rc = 10).

Cause: result.exit_code == 0 skips the multiple-file variant check for any test where expected_rc != 0. All 19 affected tests have regression_test_output rows with defined correct hashes — proving comparison was always intended. The outer guard result_file.got is not None already ensures this block only runs when a mismatch was uploaded; the == 0 check was an undocumented additional gate with no semantic justification.

Fix: result.exit_code == result.expected_rc — runs the file check whenever the exit code matched expectation, zero or non-zero. Zero behavior change for all tests where expected_rc = 0.


Test update — test_data_for_test

Category is mocked in the existing test. Our change inserted .options(...) before .filter(...) in the query chain, making mock_category.query.filter.assert_called_once() invalid — .filter() is now called on the return value of .options(), not directly on .query. The assertion is updated to mock_category.query.options.assert_called_once() to reflect the new call chain. selectinload is also patched to prevent SQLAlchemy from rejecting MagicMock attributes when validating loader options in newer versions. No business logic or expected behavior changed.


Impact

Issue Before After
HTTP 500s on test detail pages 3,087 confirmed Prevented for these code paths
Lazy-load failures (category.regression_tests) Observed Prevented by eager loading
NoneType crashes at line 94 57 confirmed Prevented by null guard
Query pattern N+1 per category Batched (selectinload)

No behavior change for any test where data is complete.

@x15sr71 x15sr71 changed the title Fix get_test_results() lazy-load crashes and missing-result guard [FIX]: Fix get_test_results() lazy-load crashes and missing-result guard Jun 6, 2026
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud Bot commented Jun 7, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant