ETRI-Knowledge Sharing Plaform

KOREAN
논문 검색
Type SCI
Year ~ Keyword

Detail

Conference Paper Do LLM-Generated Web Apps Behave Correctly? A Contract-Grounded Benchmark for Behavioral Conformance
Cited - time in scopus Share share facebook twitter linkedin kakaostory
Authors
Chansung Park, Juyong Jiang, Jiasi Shen, Sunghun Kim
Issue Date
2026-10
Citation
Conference on Empirical Methods in Natural Language Processing (EMNLP) 2026, pp.45959-45992
Publisher
Association for Computational Linguistics
Language
English
Type
Conference Paper
DOI
https://dx.doi.org/10.18653/2026.emnlp-main.2018
Abstract
Large language models and coding agents can produce web applications that appear functional, yet surface plausibility is an unreliable proxy for behavioral conformance under interaction. We introduce ConformWeb, a contract-grounded protocol and benchmark that operationalizes public behavioral contracts to construct hidden interaction scenarios, evaluating whether agent-generated applications preserve strict conformance under dynamic execution. By decoupling hidden interaction sequences from the public behavioral scope, the protocol guarantees a fair held-out workflow evaluation without relying on undisclosed reference implementations. Evaluated across 24 validated instances scaling to 756 evaluation scenarios and 6,433 semantic steps, only 252 of the 2,160 total execution runs achieve full conformance, yielding a strict pass rate of 11.7%. While scenario-level conformance reaches 36.7% and step-level progress averages 49.4%, this quantitative gap demonstrates that generative systems frequently achieve substantial partial progress but fail to maintain the critical behavioral invariants required for end-to-end conformance. All code and data are released at https://github.com/deep-diver/ConformWeb.
KSP Keywords
Dynamic execution, End to End(E2E), Generative systems, Hidden interaction, Language Models, Pass rate, Web applications