Abstract
Agile software development heavily relies on user stories to capture requirements, but manually defining and extracting structured acceptance criteria (AC) remains error-prone and time-consuming. We present an end-to-end deep learning framework utilizing pre-trained Transformer models (specifically fine-tuned RoBERTa and T5 architectures) to automate the extraction of acceptance criteria from unstructured or semi-structured agile user stories. Utilizing a newly curated dataset of 1,200 real-world agile user stories across multiple industrial software projects, we formulate the extraction task as a joint sequence labeling and text-to-text generation problem. Our experimental results demonstrate that the proposed RoBERTa-BiLSTM-CRF model achieves an F1-score of 91.2% in identifying key acceptance parameters, while the T5-based generative model yields highly coherent Given-When-Then (Gherkin) scenarios with a BLEU score of 42.8. These results significantly outperform traditional rule-based and classic machine learning baselines. By automating the transition from informal user narratives to formal, testable acceptance criteria, our framework bridges the gap between requirements engineering and automated testing, reducing manual effort and minimizing semantic ambiguities in cloud-native software development workflows.
