BROWSE
Detail
Conference Paper
SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution
- Authors
- Woojin Lee, Jin-Xia Huang
- Issue Date
- 2026-07
- Citation
- Findings of the Association for Computational Linguistics: ACL 2026, pp.7294-7316
- Publisher
- Association for Computational Linguistics
- Language
- English
- Type
- Conference Paper
- Abstract
- State-of-the-art code generation frameworks rely on mental simulation, where LLMs internally trace execution to verify correctness. We expose a fundamental limitation: the Mental-Reality Gap -- where models hallucinate execution traces and confidently validate buggy code. This gap manifests along two orthogonal dimensions: the Specification Gap (overlooking edge cases during planning) and the Verification Gap (hallucinating correct behavior for flawed code). We propose SolidCoder with a simple principle: don't imagine -- execute. The S.O.L.I.D. architecture addresses both dimensions by forcing edge-case awareness before algorithm design and replacing imagined traces with sandboxed execution using property-based oracles. With GPT-4o, SolidCoder achieves state-of-the-art pass@1 performance: 95.7% on HumanEval (+0.6%p), 77.0% on CodeContests (+4.3%p), and 26.7% on APPS (+3.4%p). Ablation reveals that edge-case awareness provides the largest individual gain, while execution grounding catches categorically different errors that specification improvements cannot address. These gains generalize to RL post-trained models, validating that bridging both gap dimensions is essential for robust code synthesis. We release our code and framework to facilitate future research.
- KSP Keywords
- Algorithm design, Code synthesis, Execution trace, Mental Simulation, Reality gap, code generation, fundamental limitation, robust code, state-of-The-Art
This work is distributed under the term of Creative Commons License (CCL)
(CC BY)
(CC BY)
