Research Article

Transformer-Based Natural Language Processing for Automated Acceptance Criteria Extraction from Agile User Stories

143 reads
SCI J Softw Eng Cloud Comp, 2026, 1 (1), 2-8, ISSN

Abstract

Agile software development heavily relies on user stories to capture requirements, but manually defining and extracting structured acceptance criteria (AC) remains error-prone and time-consuming. We present an end-to-end deep learning framework utilizing pre-trained Transformer models (specifically fine-tuned RoBERTa and T5 architectures) to automate the extraction of acceptance criteria from unstructured or semi-structured agile user stories. Utilizing a newly curated dataset of 1,200 real-world agile user stories across multiple industrial software projects, we formulate the extraction task as a joint sequence labeling and text-to-text generation problem. Our experimental results demonstrate that the proposed RoBERTa-BiLSTM-CRF model achieves an F1-score of 91.2% in identifying key acceptance parameters, while the T5-based generative model yields highly coherent Given-When-Then (Gherkin) scenarios with a BLEU score of 42.8. These results significantly outperform traditional rule-based and classic machine learning baselines. By automating the transition from informal user narratives to formal, testable acceptance criteria, our framework bridges the gap between requirements engineering and automated testing, reducing manual effort and minimizing semantic ambiguities in cloud-native software development workflows.

Keywords natural language processing acceptance criteria Agile Software Engineering User Stories Transformer Models
Authors 3

The team behind this paper

3 authors, 3 institutions.

This paper ITMO University — Russia ITMO University 1 author Osaka University — Japan Osaka University 1 author Federal University of Technology — Nigeria Federal University of T… 1 author Prof. Elena Rostova — corresponding author ER Prof. Elena Rostova ✉ Dr. Haruki Tanaka HT Dr. Haruki Tanaka Dr. Amara Okechukwu AO Dr. Amara Okechukwu

Readership

143 reads over 5 months.

#3 most read in this journal this month
July 2026 November 2026

Bibliographic Information

Prof. Elena Rostova, Dr. Haruki Tanaka, Dr. Amara Okechukwu, (2026). Transformer-Based Natural Language Processing for Automated Acceptance Criteria Extraction from Agile User Stories, SCI Journal of Software Engineering and Cloud Computing, 1(1): 2-8
Bibtex Citation
@article{prof._elena_rostova2026sjsecc,
author = {Prof. Elena Rostova and Dr. Haruki Tanaka and Dr. Amara Okechukwu},
title = {Transformer-Based Natural Language Processing for Automated Acceptance Criteria Extraction from Agile User Stories},
journal = {SCI Journal of Software Engineering and Cloud Computing},
year = {2026},
volume = {1},
number = {1},
pages = {2-8},
doi = {},
url = {https://scimatic.org/show_manuscript/8515}
}
APA Citation
Rostova, P.E., Tanaka, D.H., Okechukwu, D.A., (2026). Transformer-Based Natural Language Processing for Automated Acceptance Criteria Extraction from Agile User Stories. SCI Journal of Software Engineering and Cloud Computing, 1(1), 2-8. https://doi.org/

Author Information

  • To change your profile photo, login to scimatic.org, go to your profile and change the photo.
  • Any photo that shows your face will do: we crop it to your head and shoulders and give it a plain background automatically.
  • To sign in, go to Sign in and enter the e-mail address registered with the article: we e-mail you a one-time sign-in link (there is no password).
  • In case of any other problem, contact your editor directly or write to us at info @ scimatic.org