user stories and natural language processing a systematic literature review

• Help & FAQ

User Stories and Natural Language Processing: A Systematic Literature Review

• Rekayasa Sistem Informasi
• Department of Mathematics
• Faculty of Science and Technology
• Information Systems

Research output : Contribution to journal › Article › peer-review

Context: User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. Objective: Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. Method: The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. Results: The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. Conclusion: NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence's context continues to be a challenge.

• Agile software development
• natural language processing
• systematic review

Access to Document

• 10.1109/ACCESS.2021.3070606

Fingerprint

• Digital libraries Engineering & Materials Science 100%
• Application Chemical Compounds 77%
• Software engineering Engineering & Materials Science 66%
• Defects Engineering & Materials Science 56%

T1 - User Stories and Natural Language Processing

T2 - A Systematic Literature Review

AU - Raharjana, Indra Kharisma

AU - Siahaan, Daniel

AU - Fatichah, Chastine

N1 - Publisher Copyright: © 2013 IEEE.

N2 - Context: User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. Objective: Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. Method: The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. Results: The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. Conclusion: NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence's context continues to be a challenge.

AB - Context: User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. Objective: Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. Method: The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. Results: The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. Conclusion: NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence's context continues to be a challenge.

KW - Agile software development

KW - natural language processing

KW - systematic review

KW - user story

UR - http://www.scopus.com/inward/record.url?scp=85103780650&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2021.3070606

DO - 10.1109/ACCESS.2021.3070606

M3 - Article

AN - SCOPUS:85103780650

SN - 2169-3536

JO - IEEE Access

JF - IEEE Access

M1 - 9393933

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

User Stories and Natural Language Processing: A Systematic Literature Review

2021, IEEE Access

Related Papers

Barbara Plank

Abstract. Agile software development puts more emphasis on working programs than on documentation. However, this may cause complications from the management perspective when an overview of the progress achieved within a project needs to be provided. In this paper, we outline the potential for applying natural language processing (NLP) in order to support agile development.

International Journal of Software Engineering and Its Applications

Vandana Gaikwad

Ahmed al-Hunaiyyan

Natural Language Processing (NLP) has demonstrated effectiveness in many application domains. NLP can assist software engineering by automating various activities. This paper examines the interaction between software requirements engineering (RE) and NLP. We reviewed the current literature to evaluate how NLP supports RE and to examine research developments. This literature review indicates that NLP is being employed in all the phases of the RE domain. This paper focuses on the phases of elicitation and the analysis of requirements. RE communication issues are primarily associated with the elicitation and analysis phases of the requirements. These issues include ambiguity, inconsistency, and incompleteness. Many of these problems stem from a lack of participation by the stakeholders in both phases. Thus, we address the application of NLP during the process of requirements elicitation and analysis. We discuss the limitations of NLP in these two phases. Potential future directions for...

Davide Dell'Anna

Konstantinos Tsilionis

IEEE Access

Riad Sonbol

Agile Software Development Quality Assurance

Maro Vlachopoulou

PURTEE KOHLI

This paper explain the novel technique of extracting useful keywords from user story and generating the use case diagram with the use of those keywords with Automated UML generation through text notation.

Seminal Contributions to Information Systems Engineering

RELATED PAPERS

IRJET Journal

Yannick Lebret

The Journal of Emergency Medicine

Clint Cappiello

The Pakistan Development Review

Ashfaque Hasan Khan

IFEOLU IDOWU

Rea Fulgosi Masnjak

Revista Chilena de Enfermería

Ricardo H. Pérez-Abarca

The European Physical Journal D

Yitzhak Maron

XIV Mostra de Iniciação Científica, Pós-graduação, Pesquisa e Extensão

vinicius Z de lima

Canadian Medical Association Journal

Steve Blevins

AL-IHKAM: Jurnal Hukum & Pranata Sosial

Serendipity in Rhetoric, Writing, and Literacy Research

Maureen daly Goggin

Folha de São Paulo / Ilustrada

Esther Hamburger

IJORER : International Journal of Recent Educational Research

Saidun Hutasuhut

Revista Argentina de Cardiología

Monica Diaz

Études littéraires africaines

Nathalie Courcy

Portal Militärgeschichte

Robert Dale

Abakar Abdallah

somayeh sadat shafiei

HAL (Le Centre pour la Communication Scientifique Directe)

Larisa Cebuc

Environmental Engineering and Management Journal

Irina Smical

Breast Care

Andrew M wardley

Biochemical Journal

Aran Incharoensakdi

International journal of comparative and applied criminal justice

Mahesh Nalla

See More Documents Like This

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

User Stories and Natural Language Processing: A Systematic Literature Review

• Scinapse’s Top 10 Citation Journals & Affiliations graph reveals the quality and authenticity of citations received by a paper.
• Discover whether citations have been inflated due to self-citations, or if citations include institutional bias.

User Stories and Natural Language Processing: A Systematic Literature Review

Content maybe subject to  copyright     Report

9  citations

6  citations

5  citations

4  citations

25,711  citations

2,696  citations

768  citations

484  citations

426  citations

Related Papers (5)

Trending questions (1).

User stories are short pieces of texts in a semi-structured format that express requirements in agile software development.

Ask Copilot

Related papers

Contributing institutions

Related topics

International Congress on Information and Communication Technology

ICICT 2023: Proceedings of Eighth International Congress on Information and Communication Technology pp 73–92 Cite as

Automating Systematic Literature Reviews with Natural Language Processing and Text Mining: A Systematic Literature Review

• Girish Sundaram 13 &
• Daniel Berleant 13  
• Conference paper
• First Online: 25 July 2023

552 Accesses

3 Citations

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 693))

Objectives : An SLR is presented focusing on text mining-based automation of SLR creation. The present review identifies the objectives of the automation studies and the aspects of those steps that were automated. In so doing, the various ML techniques used challenges, limitations, and scope of further research are explained. Methods : Accessible published literature studies primarily focus on automation of study selection, study quality assessment, data extraction, and data synthesis portions of SLR. Twenty-nine studies were analyzed. Results : This review identifies the objectives of the automation studies, steps within the study selection, study quality assessment, data extraction, and data synthesis portions that were automated, and the various ML techniques used challenges, limitations, and scope of further research. Discussion : We describe uses of NLP/TM techniques to support increased automation of systematic literature reviews. This area has attracted increase attention in the last decade due to significant gaps in the applicability of TM to automate steps in the SLR process. There are significant gaps in the application of TM and related automation techniques in the areas of data extraction, monitoring, quality assessment, and data synthesis. There is, thus, a need for continued progress in this area, and this is expected to ultimately significantly facilitate the construction of systematic literature reviews.

• Systematic literature review
• Text mining

This is a preview of subscription content, log in via an institution .

Buying options

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Systematic reviews. Georgetown University Medical Center. https://guides.dml.georgetown.edu/systematicreviews

Systematic reviews (2001) CRD’s guidance for those carrying out or commissioning reviews. CRD Report Number 4 (2nd edn). NHS Centre for Reviews and Dissemination, University of York

Google Scholar  

Glover J, Izzo D, Odato K et al (2006) EBM pyramid and EBM page generator. Trustees of Dartmouth College and Yale University

Kitchenham B, Charters S (2007) Guidelines for performing systematic literature reviews in software engineering. EBSE Technical Report EBSE-2007-01. Keele University. https://docs.edtechhub.org/lib/EDAG684W

Allen IE, Olkin I (1999) Estimating time to conduct a meta-analysis from number of citations retrieved. JAMA 282(7):634–635. https://doi.org/10.1001/jama.282.7.634

Article   Google Scholar  

Petticrew M, Roberts H (2006) Systematic reviews in the social sciences: a practical guide. Blackwell Publishing Co., Malden

Book   Google Scholar  

Shojani KG, Sampson M, Ansari MT et al (2007) How quickly do systematic reviews go out of date? A survival analysis. Ann Intern Med 147(4):224–233. https://doi.org/10.7326/0003-4819-147-4-200708210-00179

Marshall C, Kitchenham B, Brereton P (2018) Tool features to support systematic reviews in software engineering. E-Informatica Softw Eng J 12(1):79–115. https://doi.org/10.5277/e-Inf180104

Khangura S, Konnyu K, Cushman R et al (2012) Evidence summaries: the evolution of a rapid review approach. Syst Rev 1:10. https://doi.org/10.1186/2046-4053-1-10

Ganann R, Ciliska D, Thomas H (2010) Expediting systematic reviews: methods and implications of rapid reviews. Implementation Sci 5:56. https://doi.org/10.1186/1748-5908-5-56

Featherstone RM, Dryden DM, Foisy M et al (2015) Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Syst Rev 4:50. https://doi.org/10.1186/s13643-015-0040-4

Jonnalagadda SR, Goyal P, Huffman MD (2015) Automating data extraction in systematic reviews: a systematic review. Syst Rev 4:78. https://doi.org/10.1186/s13643-015-0066-7

O’Mara-Eves A, Thomas J, McNaught J et al (2015) Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev 4:5. https://doi.org/10.1186/2046-4053-4-5

Feng L, Chiam Y, Lo SK (2017) Text-mining techniques and tools for systematic literature reviews: a systematic literature review. In: 24th Asia-Pacific software engineering conference (APSEC 2017). https://doi.org/10.1109/APSEC.2017.10

Tsafnat G, Glasziou P, Choong MK et al (2014) Systematic review automation technologies. Syst Rev 3(74). https://doi.org/10.1186/2046-4053-3-74

Marshall IJ, Wallace BC (2019) Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev 8:163. https://doi.org/10.1186/s13643-019-1074-9

O’Connor AM, Tsafnat G, Gilbert SB et al (2018) Moving toward the automation of the systematic review process: a summary of discussions at the second meeting of the international collaboration for the automation of systematic reviews (ICASR). Syst Rev 7:3. https://doi.org/10.1186/s13643-017-0667-4

Beller E, Clark J, Tsafnat G et al (2018) Making progress with the automation of systematic reviews: principles of the international collaboration for the automation of systematic reviews (ICASR). Syst Rev 7:77. https://doi.org/10.1186/s13643-018-0740-7

Van Altena AJ, Spijker R, Olabarriaga SD (2019) Usage of automation tools in systematic reviews. Res Syn Meth 10:72–82. https://doi.org/10.1002/jrsm.1335

Olorisade BK, de Quincey E, Brereton OP et al (2016) A critical analysis of studies that address the use of text mining for citation screening in systematic reviews. In: EASE ’16: proceedings of the 20th international conference on evaluation and assessment in software engineering. ACM, Limerick, pp 1–11. https://doi.org/10.1145/2915970.2915982

Higgins J, Green S (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0. The Cochrane Collaboration. http://community.cochrane.org/handbook .

Richardson WS, Wilson MC, Nishikawa J et al (1995) The well-built clinical question: a key to evidence-based decisions. ACP J Club 123(3):A12–A13

Dawes M, Pluye P, Shea L et al (2007) The identification of clinically important elements within medical journal abstracts: patient–population–problem, exposure-intervention, comparison, outcome, duration and results (PECODR). Inform Prim Care 15(1):9–16

Kim S, Martinez D, Cavedon L et al (2011) Automatic classification of sentences to support evidence based medicine. BMC Bioinform 12(Suppl 2):S5

Razavi A, Matwin S, Inkpen D et al (2009) Parameterized contrast in second order soft co-occurrences: a novel text representation technique in text mining and knowledge extraction. In: 2009 IEEE international conference on data mining workshops, pp 71–6

Bragge P, Clavisi O, Turner T et al (2011) The global evidence mapping initiative: scoping research in broad topic areas. BMC Med Res Methodol 11(92). https://doi.org/10.1186/1471-2288-11-92

Snilstveit B, Vojtkova M, Bhavsar A et al (2016) Evidence and gap maps—a tool for promoting evidence informed policy and strategic research agendas. J Clin Epidemiol 79:120–129. https://doi.org/10.1016/j.jclinepi.2016.05.015

Arksey H, O’Malley L (2005) Scoping studies: towards a methodological framework. Int J Soc Res Meth 8:19–32

RTC Collaboration. Review Manager (RevMan) 4.2 for Windows. The Cochrane Collaboration, Oxford (2003)

Tsafnat G, Coiera E (2009) Computational reasoning across multiple models. J Am Med Info Assoc 16(6):768–774

Sim I, Detmer DE (2005) Beyond trial registration: a global trial bank for clinical trial reporting. PLoS Med 2(11):e365

Sim I, Tu SW, Carini S et al (2014) The ontology of clinical research (OCRe): an informatics foundation for the science of clinical research. J Biomed Inf 52:78–91. https://doi.org/10.1016/j.jbi.2013.11.002

Greenhalgh T, Peacock R (2005) Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. BMJ 331(7524):1064–1065. https://doi.org/10.1136/bmj.38636.593461.68

Marshal C, Sutton A, O'Keefe H et al (2022) The systematic review toolbox. http://www.systematicreviewtools.com .

Bowes D, Hall T, Beecham S (2012) SLuRp : a tool to help large complex systematic literature reviews deliver valid and rigorous results. In: Proceedings of the 2nd international workshop on evidential assessment of software technologies—EAST ’12, pp 33–36

Hernandes E, Zamboni A, Fabbri S et al (2012) Using GQM and TAM to evaluate StArt—a tool that supports systematic review. CLEI Electr J 15(1):2. http://www.scielo.edu.uy/pdf/cleiej/v15n1/v15n1a03.pdf

Fernández-Sáez AM, Bocco MG, Romero FP (2010) SLR-Tool—a tool for performing systematic literature reviews. In: ICSOFT 2010—proceedings of the 5th international conference on software and data technologies, pp 157–166

Molléri JS, Benitti FBV (2015) SESRA: a web-based automated tool to support the systematic literature review process. In: EASE ’15: proceedings of the 19th international conference on evaluation and assessment in software engineering, pp 1–6. https://doi.org/10.1145/2745802.2745825

Gurbuz HG, Tekinerdogan B (2018) Model-based testing for software safety: a systematic mapping study. Software Qual J 26:1327–1372. https://doi.org/10.1007/s11219-017-9386-2

Van Dinter R, Tekinerdogan B, Cagatay C (2021) Automation of systematic literature reviews: a systematic literature review. Inf and Software Tech 136:106589. https://doi.org/10.1016/j.infsof.2021.106589

Wohlin C (2014) Guidelines for snowballing in systematic literature studies and a replication in software engineering. In: EASE ’14: proceedings of the 18th international conference on evaluation and assessment in software engineering.. ACM, pp 1–10. https://doi.org/10.1145/2601248.2601268

Dybå T, Dingsøyr T (2008) Empirical studies of agile software development: a systematic review. Inf Softw Tech 50(9):833–859

Nguyen-Duc A, Cruzes DS, Conradi R (2015) The impact of global dispersion on coordination, team performance and software quality—a systematic literature review. Inf and Softw Tech 57:277–294

Brownlee J, Tour of evaluation metrics for imbalanced classification. https://machinelearningmastery.com/tour-of-evaluation-metrics-for-imbalanced-classification

Wallace BC, Trikalinos TA, Lau J et al (2010) Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics 11(1):55. https://doi.org/10.1186/1471-2105-11-55

Pham B, Jovanovic J, Bagheri E et al (2021) Text mining to support abstract screening for knowledge syntheses: a semi-automated workflow. Syst Rev 10:156. https://doi.org/10.1186/s13643-021-01700-x

Norman CR, Leeflang M, Porcher R et al (2019) Measuring the impact of screening automation on meta-analyses of diagnostic test accuracy. Syst Rev 8:243. https://doi.org/10.1186/s13643-019-1162-x

Dickson K (2017) Systematic reviews to inform policy: institutional mechanisms and social interactions to support their production. Dissertation. University College London. http://discovery.ucl.ac.uk/id/eprint/10054092/1/KD_PhD_FinalAugust2018_Redacted.pdf

Turing A (1950) Computing machinery and intelligence. Mind LIX (236):433–460. https://doi.org/10.1093/mind/LIX.236.433

Mo Y, Kontonatsios G, Ananiadou S (2015) Supporting systematic reviews using LDA-based document representations. Syst Rev 4:172. https://doi.org/10.1186/s13643-015-0117-0

Cohen AM, Ambert K, McDonagh M (2012) Studying the potential impact of automated document classification on scheduling a systematic review update. BMC Med Inform Decis Mak 12:33. https://doi.org/10.1186/1472-6947-12-33

Callaghan MW, Müller-Hansen F (2020) Statistical stopping criteria for automated screening in systematic reviews. Syst Rev 9:273. https://doi.org/10.1186/s13643-020-01521-4

Miwa M, Thomas J, O’Mara-Eves A et al (2014) Reducing systematic review workload through certainty-based screening. J Biomed Inf 51:242–253. https://doi.org/10.1016/j.jbi.2014.06.005

Basu T, Kumar S, Kalyan A et al (2016) A novel framework to expedite systematic reviews by automatically building information extraction training corpora. arXiv:1606.06424 [cs.IR] (2016). https://arxiv.org/abs/1606.06424

García Adeva JJ, Pikatza Atxa JM, Ubeda CM et al (2014) Automatic text classification to support systematic reviews in medicine. Expert Syst with Appl 41(4):1498–1508. https://doi.org/10.1016/j.eswa.2013.08.047

Ros R, Bjarnason E, Runeson P (2017) A machine learning approach for semi-automated search and selection in literature studies. In: EASE ’17: proceedings of the 21st international conference on evaluation and assessment in software engineering. Association for Computing Machinery, New York, pp 118–127. https://doi.org/10.1145/3084226.3084243

Frunza O, Inkpen D, Matwin S (2010) Building systematic reviews using automatic text classification techniques. In: Proceedings of the 23rd international conference on computational linguistics: poster, vol (COLING ‘10). Association for Computational Linguistics, pp 303–311

Timsina P, Liu J, El-Gayar O (2016) Advanced analytics for the automation of medical systematic reviews. Inf Syst Frontiers 18(2):237–252

El-Gayar OF, Liu J, Timsina P (2015) Active learning for the automation of medical systematic review creation. In: 21st Americas conference on information systems (AMCIS). Puerto Rico Aug 13–15. htttp://aisel.aisnet.org/amcis2015/BizAnalytics/GeneralPresentations/22

Halamoda-Kenzaoui B, Rolland E, Piovesan J et al (2021) Toxic effects of nanomaterials for health applications: how automation can support a systematic review of the literature? J of Appl Tox 42(1):41–51. https://doi.org/10.1002/jat.4204

Olorisade BK, Brereton P, Andras P (2019) The use of bibliography enriched features for automatic citation screening. J of Biomed Inf 94:103202. https://doi.org/10.1016/j.jbi.2019.103202

Bannach-Brown A, Przybyła P, Thomas J et al (2019) Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error. Syst Rev 8(1):23. https://doi.org/10.1186/s13643-019-0942-7

Bui D, Del Fiol G, Hurdle JF et al (2016) Extractive text summarization system to aid data extraction from full text in systematic review development. J Biomed Inf 64:265–272. https://doi.org/10.1016/j.jbi.2016.10.014

Tsafnat G, Glasziou P, Karystianis G et al (2018) Automated screening of research studies for systematic reviews using study characteristics. Syst Rev 7:64. https://doi.org/10.1186/s13643-018-0724-7

Norman C (2020) Systematic review automation methods. Université Paris-Saclay, Universiteit van Amsterdam https://tel.archives-ouvertes.fr/tel-03060620/document

Norman C, Leeflang M, Zweigenbaum P et al (2018) Automating document discovery in the systematic review process: how to use chaff to extract wheat. In: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan. https://aclanthology.org/L18-1582

Karystianis G, Thayer K, Wolfe M et al (2017) Evaluation of a rule-based method for epidemiological document classification towards the automation of systematic reviews. J of Biomed Inf 70:27–34. https://doi.org/10.1016/j.jbi.2017.04.004

Wallace BC, Kuiper J, Sharma A et al (2016) Extracting PICO sentences from clinical trial reports using supervised distant supervision. J Mach Lear Res 17:132

MathSciNet   Google Scholar  

Marshall IJ, Kuiper J, Wallace BC (2015) Automating risk of bias assessment for clinical trials. J Biomed Health Inf 19(4):1406–1412. https://doi.org/10.1109/JBHI.2015.2431314

Ma Y (2007) Text classification on imbalanced data: application to systematic reviews automation. Dissertation. University of Ottawa

Begert D, Granek J, Irwin B et al (2020) Towards automating systematic reviews on immunization using an advanced natural language processing-based extraction system. Can Commun Dis Rep 46(6):174–179. https://doi.org/10.14745/ccdr.v46i06a04

Scells H, Zuccon G, Koopman B (2019) Automatic boolean query refinement for systematic review literature search. In: The World Wide Web Conference (WWW ‘19). Association for Computing Machinery, New York, pp 1646–1656. https://doi.org/10.1145/3308558.3313544

Khabsa M, Elmagarmid A, Ilyas I et al (2016) Learning to identify relevant studies for systematic reviews using random forest and external information. Mach Learn 102:465–482. https://doi.org/10.1007/s10994-015-5535-7

Article   MathSciNet   Google Scholar  

Download references

Acknowledgements

Publication of this work was supported by the National Science Foundation under Award No. OIA-1946391. The content reflects the views of the authors and not necessarily the NSF. The authors are grateful to Deepak Sagaram, MD, for consulting on the list of articles regarding their relevance for inclusion and exclusion.

Author information

Authors and affiliations.

University of Arkansas at Little Rock, Little Rock, AR, 72204, USA

Girish Sundaram & Daniel Berleant

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Girish Sundaram .

Editor information

Editors and affiliations.

Department of Design Engineering and Mathematics, Middlesex University London, London, UK

Xin-She Yang

Department of Biomedical Engineering, University of Reading, England, UK

R. Simon Sherratt

Department of Computer Science and Engineering, Techno International Newtown, Chakpachuria, West Bengal, India

Nilanjan Dey

Global Knowledge Research Foundation, Ahmedabad, India

Supplementary Material

The supplementary material, including data Tables A1–A4, may be obtained at

https://dberleant.github.io/papers/sundber-supp.pdf .

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper.

Sundaram, G., Berleant, D. (2023). Automating Systematic Literature Reviews with Natural Language Processing and Text Mining: A Systematic Literature Review. In: Yang, XS., Sherratt, R.S., Dey, N., Joshi, A. (eds) Proceedings of Eighth International Congress on Information and Communication Technology. ICICT 2023. Lecture Notes in Networks and Systems, vol 693. Springer, Singapore. https://doi.org/10.1007/978-981-99-3243-6_7

Download citation

DOI : https://doi.org/10.1007/978-981-99-3243-6_7

Published : 25 July 2023

Publisher Name : Springer, Singapore

Print ISBN : 978-981-99-3242-9

Online ISBN : 978-981-99-3243-6

eBook Packages : Engineering Engineering (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Search code, repositories, users, issues, pull requests...

Provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

• Notifications

User Stories and Natural Language Processing: A Systematic Literature Review

indrakharisma/NLPUserStory

User Stories in Requirements Elicitation: A Systematic Literature Review

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Acta Informatica Pragensia 2023, 12(2) , 419-438  |  DOI: 10.18267/j.aip.212 554

Visualisation of User Stories in UML Models: A Systematic Literature Review

The use of agile methodology in software development projects is growing rapidly among industry professionals and academia. The Unified Modelling Language (UML) conventionally accompanies agile software development to model the software requirements. The user story is fundamental and should be identified to communicate the basic requirements between the development team and the stakeholders before the UML model such as the use case diagram, class diagrams and many others can be designed. However, there are several challenges associated with this process such as poorly organised user stories, natural language complexity and high time consumption to create them. A systematic literature review is conducted to grasp more knowledge about the utilisation of natural language processing (NLP) for UML model generation. A total of 198 papers were initially found in four online databases, namely Scopus, IEEE Xplore, ScienceDirect and ACM Digital Library, from the period 2018-2022. After removing duplicates, applying inclusion and exclusion criteria, and conducting the full-text assessment, only 20 papers are included as the primary studies. The primary studies are reviewed to discover several important pieces of information, namely the challenges of designing UML models, NLP tools and techniques used to generate UML models, UML models generated, and validation methods used for measuring the accuracy of generated models. Finally, this study discusses important elements related to UML model generation using NLP tools and techniques.

Keywords: Unified modelling language; Conceptual model; Natural language processing; Agile software development; User stories.

Received: December 13, 2022; Revised: March 7, 2023; Accepted: March 12, 2023; Prepublished online: March 12, 2023; Published: October 10, 2023   Show citation

• BibTeX (.bib)
• Bookends (.ris)
• EasyBib (.ris)
• EndNote (.enw)
• EndNote 8 (.xml)
• ISI WoS (.isi)
• Medlars (.medlars)
• Mendeley (.ris)
• MODS (.xml)
• Papers (.ris)
• RefWorks (.txt)
• RefManager (.ris)
• MS Word (.xml)
• Zotero (.ris)
• Cohn, M. (2004). User Stories Applied: For Agile Software Development. Addison Wesley Longman Publishing.
• Gebretsadik, K. K. (2020). Challenges and Opportunity of UML Diagram for Software Project development as a complete Modeling Tool. IOSR Journal of Mobile Computing & Application, 7(3), 46-48.
• Kitchenham, B. (2004). Procedures for Performing Systematic Reviews. Keele University Technical Report TR/SE-0401. https://www.inf.ufsc.br/~aldo.vw/kitchenham.pdf
• Ternes, B., Rosenthal, K., & Strecker, S. (2021). Automated assistance for data modelers combining natural language processing and data modeling heuristics: A prototype demonstration. In CEUR Workshop Proceedings, vol. 2958, (pp. 25-30). https://ceur-ws.org/Vol-2958/paper5.pdf

This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0) , which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.

Can large language models replace humans in systematic reviews? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages

Affiliations.

• 1 Trinity Centre for Global Health, Trinity College Dublin, Dublin, Ireland.
• 2 School of Psychology, Trinity College Dublin, Dublin, Ireland.
• 3 Department of Education, York University, York, UK.
• PMID: 38484744
• DOI: 10.1002/jrsm.1715

Systematic reviews are vital for guiding practice, research and policy, although they are often slow and labour-intensive. Large language models (LLMs) could speed up and automate systematic reviews, but their performance in such tasks has yet to be comprehensively evaluated against humans, and no study has tested Generative Pre-Trained Transformer (GPT)-4, the biggest LLM so far. This pre-registered study uses a "human-out-of-the-loop" approach to evaluate GPT-4's capability in title/abstract screening, full-text review and data extraction across various literature types and languages. Although GPT-4 had accuracy on par with human performance in some tasks, results were skewed by chance agreement and dataset imbalance. Adjusting for these caused performance scores to drop across all stages: for data extraction, performance was moderate, and for screening, it ranged from none in highly balanced literature datasets (~1:1) to moderate in those datasets where the ratio of inclusion to exclusion in studies was imbalanced (~1:3). When screening full-text literature using highly reliable prompts, GPT-4's performance was more robust, reaching "human-like" levels. Although our findings indicate that, currently, substantial caution should be exercised if LLMs are being used to conduct systematic reviews, they also offer preliminary evidence that, for certain review tasks delivered under specific conditions, LLMs can rival human performance.

Keywords: GPT; artificial intelligence (AI); large language models (LLMs); machine learning; natural language processing (NLP); systematic reviews.

© 2024 The Authors. Research Synthesis Methods published by John Wiley & Sons Ltd.

• Help & FAQ

User Stories and Natural Language Processing: A Systematic Literature Review

• Bachelor in Informatics
• Laboratory of Software Engineering
• Research Center for Artificial Intelligence and Health Technology (KATK)
• Bachelor in Information Technology
• Master Program in Master of Technology Management
• Doctoral Program in Computer Science
• Master Program in Informatics
• Laboratory of Intelligent Computing and Vision
• Institut Teknologi Sepuluh Nopember
• Universitas Airlangga

Research output : Contribution to journal › Article › peer-review

Context: User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. Objective: Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. Method: The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. Results: The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. Conclusion: NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence's context continues to be a challenge.

• Agile software development
• natural language processing
• systematic review

Access to Document

• 10.1109/ACCESS.2021.3070606

Other files and links

• Link to publication in Scopus

Fingerprint

• User Computer Science 100%
• Story Language Computer Science 100%
• Natural Language Processing Computer Science 100%
• Systematic Literature Review Computer Science 100%
• Exclusion Criterion Computer Science 18%
• Artifact Computer Science 18%
• Context Computer Science 18%
• Links Computer Science 9%

T1 - User Stories and Natural Language Processing

T2 - A Systematic Literature Review

AU - Raharjana, Indra Kharisma

AU - Siahaan, Daniel

AU - Fatichah, Chastine

N1 - Publisher Copyright: © 2013 IEEE.

N2 - Context: User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. Objective: Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. Method: The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. Results: The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. Conclusion: NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence's context continues to be a challenge.

AB - Context: User stories have been widely accepted as artifacts to capture the user requirements in agile software development. They are short pieces of texts in a semi-structured format that express requirements. Natural language processing (NLP) techniques offer a potential advantage in user story applications. Objective: Conduct a systematic literature review to capture the current state-of-the-art of NLP research on user stories. Method: The search strategy is used to obtain relevant papers from SCOPUS, ScienceDirect, IEEE Xplore, ACM Digital Library, SpringerLink, and Google Scholar. Inclusion and exclusion criteria are applied to filter the search results. We also use the forward and backward snowballing techniques to obtain more comprehensive results. Results: The search results identified 718 papers published between January 2009 to December 2020. After applying the inclusion/exclusion criteria and the snowballing technique, we identified 38 primary studies that discuss NLP techniques in user stories. Most studies used NLP techniques to extract aspects of who, what, and why from user stories. The purpose of NLP studies in user stories is broad, ranging from discovering defects, generating software artifacts, identifying the key abstraction of user stories, and tracing links between model and user stories. Conclusion: NLP can help system analysts manage user stories. Implementing NLP in user stories has many opportunities and challenges. Considering the exploration of NLP techniques and rigorous evaluation methods is required to obtain quality research. As with NLP research in general, the ability to understand a sentence's context continues to be a challenge.

KW - Agile software development

KW - natural language processing

KW - systematic review

KW - user story

UR - http://www.scopus.com/inward/record.url?scp=85103780650&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2021.3070606

DO - 10.1109/ACCESS.2021.3070606

M3 - Article

AN - SCOPUS:85103780650

SN - 2169-3536

JO - IEEE Access

JF - IEEE Access

M1 - 9393933