cid ea37afc9478a40dd67a67fb3b62d7d66120d8dcb3cff75025fef1a55625dd33b · original · open access · via openalex
Title: THE PROBLEM OF CODE GENERATING DOMAIN-SPECIFIC LANGUAGES USING LARGE LANGUAGE MODELS (USING POLKIT AS AN EXAMPLE) Authors: Alexandr Nazimov Venue: Telecommunications and Comunications Year: 2026 Canonical: doi:10.21681/3034-4050-2026-2-43-49 URL: https://doi.org/10.21681/3034-4050-2026-2-43-49 Collector: openalex OA: true ABSTRACT Purpose of the study: to formalize the task of automatic generation of domainspecific language (DSL) code from natural language descriptions -referred to as Text2DSL -as an independent class of code generation problems, and to empirically evaluate the role of structured context in DSL code generation by a large language model.Methods of research: the study is based on an experimental evaluation conducted under two conditions (baseline mode and contextenhanced mode) using the PolkitBench dataset, which contains 4,204 verified pairs of natural language requests and corresponding Polkit rules.A threelevel AST validation procedure based on the esprima parser was employed, along with quantitative metrics of syntactic and semantic correctness.Results: the inclusion of structured context (BNF grammar, API specification, and a dictionary of valid identifiers) increases syntactic correctness from 80.5 % to 99.4 % (+23.4 %) and semantic correctness from 60.4 % to 95.9 % (+58.7 %).The results demonstrate that for the class of Text2DSL tasks, incorporating the formal specification of the target language into the prompt context constitutes a necessary and sufficient condition for achieving highquality DSL code generation without additional model finetuning. Scientific novelty: the study formalizes the Text2DSL problem as a distinct class of code generation tasks; introduces the PolkitBench dataset consisting of 4,204 verified pairs validated through a threelevel AST analysis; and provides empirical evidence of the critical role of structured context in enabling accurate DSL code generation by large language models.