devops in practice a multiple case study of five companies

•   OuluREPO etusivu
• Oulun yliopisto
• Avoin saatavuus
• Näytä viite

DevOps in practice : a multiple case study of five companies

Lwakatare, lucy ellen; kilamo, terhi; karvonen, teemu; sauvola, tanja; heikkilä, ville; itkonen, juha; kuvaja, pasi; mikkonen, tommi; oivo, markku; lassenius, casper (2019-06-25).

Avaa tiedosto

Lucy Ellen Lwakatare, Terhi Kilamo, Teemu Karvonen, Tanja Sauvola, Ville Heikkilä, Juha Itkonen, Pasi Kuvaja, Tommi Mikkonen, Markku Oivo, Casper Lassenius, DevOps in practice: A multiple case study of five companies, Information and Software Technology, Volume 114, 2019, Pages 217-230, ISSN 0950-5849, https://doi.org/10.1016/j.infsof.2019.06.010

Tiivistelmä

Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects.

Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies.

Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis.

Results: Our analysis yielded some of the following results: (i) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure.

Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results

• Avoin saatavuus [31960]

Selaa kokoelmaa

Omat tiedot.

• ACRIS-ohjeet

DevOps in practice: A multiple case study of five companies

• Tietotekniikan laitos
• Professorship Lassenius Casper
• University of Oulu
• Tampere University
• University of Helsinki

Tutkimustuotos : Lehtiartikkeli › Article › Scientific › vertaisarvioitu

Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects. Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies. Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis. Results: Our analysis yielded some of the following results: (i) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure. Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results.

Pääsy asiakirjaan

• 10.1016/j.infsof.2019.06.010
• SCI_Lwakatare_DevOps_in_practice_INFSOF6157 Accepted author manuscript, 517 KB Lisenssi: CC BY-NC-ND

OpenUrl-saatavuus

Muut tiedostot ja linkit.

• Link to publication in Scopus

Sormenjälki

• Case Study Computer Science 100%
• DevOps Computer Science 100%
• Contexts Computer Science 50%
• Software Development Computer Science 25%
• Small and Medium Enterprise Computer Science 25%
• Software Change Computer Science 25%
• Synthesis Computer Science 12%
• Software Developer Computer Science 12%

Siteeraa tätä

T1 - DevOps in practice

T2 - A multiple case study of five companies

AU - Lwakatare, Lucy Ellen

AU - Kilamo, Terhi

AU - Karvonen, Teemu

AU - Sauvola, Tanja

AU - Heikkilä, Ville

AU - Itkonen, Juha

AU - Kuvaja, Pasi

AU - Mikkonen, Tommi

AU - Oivo, Markku

AU - Lassenius, Casper

PY - 2019/10/1

Y1 - 2019/10/1

N2 - Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects. Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies. Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis. Results: Our analysis yielded some of the following results: (i) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure. Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results.

AB - Context: DevOps is considered important in the ability to frequently and reliably update a system in operational state. DevOps presumes cross-functional collaboration and automation between software development and operations. DevOps adoption and implementation in companies is non-trivial due to required changes in technical, organisational and cultural aspects. Objectives: This exploratory study presents detailed descriptions of how DevOps is implemented in practice. The context of our empirical investigation is web application and service development in small and medium sized companies. Method: A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved. Data was mainly collected through interviews with 26 practitioners and observations made at the companies. Data was analysed by first coding each case individually using a set of predefined themes and thereafter perform a cross-case synthesis. Results: Our analysis yielded some of the following results: (i) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. (ii) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents. (ii) the delivery speed to production is affected by context factors, such as manual approvals by the product owner (iii) steep learning curve for new skills is experienced by both software developers and operations staff, who also have to cope with working under pressure. Conclusion: Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies. We discuss two practical implications of the results.

KW - Continuous deployment

KW - Development

KW - DevOps

KW - Operations

UR - http://www.scopus.com/inward/record.url?scp=85068546035&partnerID=8YFLogxK

U2 - 10.1016/j.infsof.2019.06.010

DO - 10.1016/j.infsof.2019.06.010

M3 - Article

AN - SCOPUS:85068546035

SN - 0950-5849

JO - Information and Software Technology

JF - Information and Software Technology

Assessment of DevOps Lifecycle Phases and their Role in DevOps Implementation using Best–Worst MCDM

• Original Research
• Published: 20 October 2023

Cite this article

• Ankur Kumar   ORCID: orcid.org/0000-0002-2417-9581 1 ,
• Mohammad Nadeem 2 &
• Mohammad Shameem 3  

136 Accesses

Explore all metrics

After DeBois popularized DevOps, there is now a focus on integrating software development and operations. However, DevOps lacks a widely accepted definition despite various interpretations in the literature. Practitioners possess a basic understanding, but academic conceptualization is necessary. This research paper presents a framework based on the continuity principle in DevOps Phases to elucidate the fundamental concepts underlying DevOps. The framework defines DevOps as a synthesis of phases that encompass the entire software development lifecycle, including continuous planning, development, testing, integration, delivery, and monitoring. The study aims to assess the importance of each phase in implementing DevOps practices within the software development lifecycle. To accomplish this, the best–worst method is employed to calculate priority weights for each stage. The analysis utilizes input values derived from the DevOps-related section of the HELENA Dataset, which is sourced from software development practitioners, particularly those engaged in hybrid development models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Future of software development with generative AI

Jaakko Sauvola, Sasu Tarkoma, … David Doermann

Ethics in the Software Development Process: from Codes of Conduct to Ethical Deliberation

Jan Gogoll, Niina Zuber, … Julian Nida-Rümelin

Comparative Analysis of Software Development Life Cycle Models (SDLC)

Abbreviations.

Multiple criteria decision making

Analytical hierarchy process

• Best–worst method

Best to others

Other to worst

Rezvani A, Khosravi P (2019) Emotional intelligence: the key to mitigating stress and fostering trust among software developers working on information system projects. Int J Inf Manag 48:139–150

Article   Google Scholar  

Gasparaite M, Naudziunaite K, Ragaisis S (2020) Systematic literature review of devops models. In: Quality of information and communications technology: 13th international conference, QUATIC 2020, Faro, Portugal, September 9–11, 2020, Proceedings 13. Springer International Publishing, Berlin

DileepKumar SR, Mathew J (2023) Ebola optimization search algorithm for the enhancement of devops and cycle time reduction. Int J Inf Technol 15(3):1309–1317

Google Scholar  

Nundlall C, Nagowah SD (2021) Task allocation and coordination in distributed agile software development: a systematic review. Int J Inf Technol 13:321–330

Gall M, Pigni F (2022) Taking DevOps mainstream: a critical review and conceptual framework. Eur J Inf Syst 31(5):548–567

Lwakatare LE et al (2019) DevOps in practice: a multiple case study of five companies. Inf Softw Technol 114:217–230

Luz WP, Pinto G, Bonifácio R (2019) Adopting DevOps in the real world: a theory, a model, and a case study. J Syst Softw 157:110384

Hemon-Hildgen A, Rowe F, Monnier-Senicourt L (2020) Orchestrating automation and sharing in DevOps teams: a revelatory case of job satisfaction factors, risk and work conditions. Eur J Inf Syst 29(5):474–499

Bilgaiyan S, Mishra S, Das M (2019) Effort estimation in agile software development using experimental validation of neural network models. Int J Inf Technol 11(3):569–573

Fitzgerald B, Stol K-J (2017) Continuous software engineering: a roadmap and agenda. J Syst Softw 123:176–189

Ahmad M, Alam MZ, Umayya Z, Khan S, Ahmad F (2018) An image encryption approach using particle swarm optimization and chaotic map. Int J Inf Technol 10:247–255

Grande R, Vizcaíno A, García FO (2023) Is it worth adopting DevOps practices in global software engineering? Possible challenges and benefits. Comput Stand Interfaces 103767

Sen A, Ahmed A et al (2018) Preserving privacy in internet of things: a survey. Int J Inf Technol 10:189–200

Mohammadi M, Rezaei J (2020) Bayesian best-worst method: a probabilistic group decision making model. Omega 96:102075

Brunelli M, Rezaei J (2019) A multiplicative best–worst method for multi-criteria decision making. Oper Res Lett 47(1):12–15

Article   MathSciNet   MATH   Google Scholar  

Kumar A, Nadeem M, Shameem M (2022) Assessing the maturity of DevOps practices in software industry: an empirical study of HELENA2 dataset. In: Proceedings of the 26th international conference on evaluation and assessment in software engineering

Akbar MA et al (2020) Identification and prioritization of DevOps success factors using fuzzy-AHP approach. Soft Comput 1–25

Noorani NM et al (2022) Factor prioritization for effectively implementing DevOps in software development organizations: a SWOT-AHP approach. Axioms 11(10):498

Rayhan A, Raharjo T, Suhanto A (2021) Challenges of DevOps implementation: a case study from technology companies in Indonesia. In: 2021 international conference on informatics, multimedia, cyber and information system (ICIMCIS). IEEE, 2021

Kuhrmann M, Tell P, Klünder J, Hebig R, Licorish S, MacDonell S HELENA stage 2 results

Tell P, Pfeiffer R-H, Schultz UP (2017) HELENA stage 2—Danish overview. In: Product-focused software process improvement: 18th international conference, PROFES 2017, Innsbruck, Austria, November 29–December 1, 2017, Proceedings 18. Springer International Publishing, Berlin

Rezaei J (2020) A concentration ratio for nonlinear best worst method. Int J Inf Technol Decis Mak 19(03):891–907

Download references

Author information

Authors and affiliations.

Department of Computer Science, Aligarh Muslim University, Aligarh, India

Ankur Kumar

Mohammad Nadeem

Department of Computer Science and Applications, Dr. Vishwanath Karad MIT World Peace University, Pune, Maharashtra, India

Mohammad Shameem

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ankur Kumar .

Ethics declarations

Conflict of interest.

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Kumar, A., Nadeem, M. & Shameem, M. Assessment of DevOps Lifecycle Phases and their Role in DevOps Implementation using Best–Worst MCDM. Int. j. inf. tecnol. (2023). https://doi.org/10.1007/s41870-023-01566-3

Download citation

Received : 14 June 2023

Accepted : 22 September 2023

Published : 20 October 2023

DOI : https://doi.org/10.1007/s41870-023-01566-3

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Continuous integration
• Find a journal
• Publish with us
• Track your research

DevOps in practice: A multiple case study of five companies

Summary  ( 6 min read), 1. introduction.

• DevOps, a portmanteau of development and operations, is an approach where software developers and operations work in close collaboration [1].
• Implementing and integrating effective deployment pipeline to an existing software release process is not only challenging but requires diverse skill-sets, including that of operations staff [21],[22],[23].
• First, it provides insights into the concept of DevOps and gives an enhanced definition of DevOps that is based on existing literature and interview data.
• In Section 4, presents an overview of the cases, including the software development context and techniques.

2. Background and Related Work

• Modern software development process is increasingly being characterised by frequent and rapid releases of software changes that enable fast feedback from end-users [3],[29].
• The need to attain the capability to deliver software frequently, fast and in an automated manner as soon as changes are checkedin into the mainline is a major motivation for organisations to adopt DevOps [4] and continuous practices [3].
• This section presents the DevOps concept and practices, as described in the literature.
• Empirical studies describing DevOps practices, benefits and challenges are presented to position this paper with the related work.
• As DevOps is intertwined with continuous practices, the authors only considered empirical studies that explicitly mentioned DevOps.

2.1. The DevOps concept

• DevOps concept, pioneered by practitioners like Patrick Debois [10], aims at tackling inefficiencies in software development, release and operations processes that are caused by organisational split between the processes [9].
• In such a context, the development and the operations units often experience conflicting goals of ‘agility vs. stability’ and face several challenges, including poor information flow and unsatisfactory test environments [6].
• Further, it is argued that there is no established causal link between the different forms of collaboration and software release time or quality in production [13].
• Clearly as noted by [32] there is a need to make additional inquiries from practitioners¢¢ understanding of DevOps if the authors are improve the definition but research needs to go beyond that to investigate its actual behaviour (methods, practices and tools) and consequences [35].
• This research makes an explicit inquiry of these aspects.

2.2. DevOps practices

• DevOps concept is associated with both technical and nontechnical practices.
• The common practices category is further split into collaborative practices involving human interactions and procedural practices that are mostly automated.
• DevOps practices done jointly between the development and operations teams are incorporated into the deployment pipeline, as the focus is not limited to the design and implementation of system features, but it also includes the consideration of environments and tools that are used to support the development, deployment and operations of software features [13].
• When software changes are available for release, a new VM image is created with the new system version in a process of baking an image.
• According to Bass et al. [13], these include monitoring and security.

2.3. Previous empirical studies

• This subsection presents DevOps practices, benefits and challenges from empirical studies published in academic forums.
• Tools such as Ansible, Chef and Puppet are reported to be used to facilitate automated provisioning of infrastructure [14][49], and makes transparent to developers the evolution of infrastructure and configurations [37].
• Similarly, continuous monitoring through log aggregation and monitoring tools has become an integral part of the deployment pipeline, and are made accessible to developers [23] [16] [37] [44] [28].
• Reported challenges include difficulties in implementing an automated deployment process when software changes involve changes to database [45].
• Furthermore, when DevOps adoption is championed from the bottom up, senior managers are to be convinced of the benefits DevOps [17].

3. Research Methodology

• This section describes the research methodology and steps taken in conducting this study.
• The study design, data collection and data analysis are further elaborated in the following subsections.

3.1. Study design and case selection

• The study applies a multiple-case study approach conducted between January 2016 and January 2017.
• The multiple-case study approach was selected because it gave the possibility to investigate DevOps at multiple levels of analysis and in a real context [51], [52].
• This is in contrast to the objective of generating theory and cause-effect relationship, which would require considering other appropriate research methods such as Grounded Theory [53] and Experimentation [54].
• Five company representatives nominated themselves and allowed researchers to conduct the study in their respective companies.
• Workshops were conducted by researchers to propose research questions, specify data collection process, identify roles suitable for interviews and develop an interview guide.

3.2. Data collection

• Data were collected mainly through interviews with 26 practitioners and observations at companies between March and May 2016.
• The data collection was first executed in a pilot case (Case D), and three researchers conducted the interviews with practitioners.
• Most interviews were done face-to-face on the company’s premises, except for one interview, which was done via Skype.
• During the interviews, one researcher mainly asked the questions while the others took notes.
• Tools Explicitly mentioned tools and their purpose of use before, during and after system deployment to production Rationale and motivation of DevOps Rationale given for adopting or using DevOps, including when DevOps was started, by whom and challenges that teams are solving through DevOps – 60 minutes.

3.3. Data analysis

• The data were analysed using a thematic coding technique [56].
• Data were first coded case by case and later on performing a cross-case synthesis, often used in multiple case studies [51] [52].
• The first round of pilot coding was done by six researchers who separately coded one similar transcript to determine a coding agreement level amongst them.
• After the first round 2 NVivo is a qualitative data analysis software.
• By the end of the workshop, the researchers had created a coding guideline document containing a list of basic principles and a codebook listing the predefined themes and their descriptions, showin in Table 2.

4. DevOps implementation and adoption in each case

• This section presents information about teams, processes and environments used to develop and operate software applications and services in each case.
• Table 3 gives a summary of the development context and Figure 1 shows the deployment pipelines of each case.
• Three cases were end user-facing services, of which the longest running service (Case C) had been in production for over a decade.
• At the time of the interviews, four cases (Cases A, B, C, F) were using the Amazon Web Services (AWS) cloud platform and Case D was transitioning to AWS from an internal private VMware-based cloud.

4.1. Case A - Road maintenance reporting tool

• The company behind Case A mainly provides digital business consultancy and solutions to its customers.
• The company has a separate operations team that is managing company’s third party provided cloud infrastructure and is located in another city and one member from the team interacts with Case A members.
• New software changes are first tested by the developer and then reviewed by another while in development server.
• Case A members interact with operations team through HipChat, particu- larly when setting-up new environments and accessing monitoring logs from CI server especially during build failures.
• The developed product is one JAR file that gets deployed to the server using a playbook, that copies the configuration template and the JAR file to the server, and then calls a system restart.

4.2. Case B - Occupational Health and Safety Services

• Case B provides productengineering services — software development, technical expertise and infrastructure — to an independent occupational health and safety research, development and specialist organisation.
• During each two-week sprint, estimated tickets are kept in priority order in a sprint backlog on the team¢¢s Kanban board.
• Tasks are implemented on feature branches and then merged to the master branch in GitHub.
• With the permission of the product owner, accepted changes/tickets are deployed to an acceptance testing environ- 7 AC CE PT ED M AN US CR IP T ment by push of a button.
• Case B’s customer project is considered a prime example of a successful implementation of DevOps practices, made possible by the customer’s willingness to adopt an AWS infrastructure and diverse skillest embedded into the team.

4.3. Case C - External Business Customer Access Point

• The company of Case C offers digital marketing and sales solutions that provide timely business and contact information.
• For the business customers, the application acts as a control panel to acquire and manage different services ordered from the company of Case C. Case C consists of four developers and one UX designer co-located together with a product owner.
• Developers can freely pick tasks from the backlog and begin work in local branches.
• The production deployment job starts a new instance, installs the required software and security updates, creates a AWS image and starts a new instances based on that image.
• As such, the transformation towards DevOps started in 2011, when the company decided to move from on-premise infrastructure to a cloud-based infrastructure offered by Amazon.

4.4. Case D - REST APIs

• The company of Case D is a media company, offering radio and television programmes across all telecommunication networks.
• In addition to other development teams, the company has a separate operations team that is building and supporting company’s VMware infrastructure.
• Freeze periods are used to ensure functionality of the services during holidays and weekends.
• From the repository, version-controlled deployment scripts built in-house are manually triggered in order for deployment to test or production environment.
• DevOps is reported to be infiltrating across the different parts of the company mainly due to enthusiastic practitioners who are considered as change agents to company’s ways of working.

4.5. Case E - Background Services for Products

• The company of Case E is a European cyber security company providing antivirus, security software and VPN services.
• The services have different purposes, use different technologies and are developed with different programming languages.
• Once built, the code is deployed on an alpha server and monitored.
• The authors had operations team and systems development team.
• Those were competing goals and that created friction.
• This section presents findings of cross-case analysis that identifies similarities in the descriptions of DevOps concept, practices and perceived impacts.
• Table 4 summarises common observed DevOps practices and other software development practices.
• Commonly perceived benefits and challenges are summarised in Table 5 and Table 6 respectively.
• From the perceived impacts, attention is given to the explanations since they are generally applicable and thus difficult to argue as a mere result of only applying DevOps.

5.1. A shared understanding of the DevOps concept (RQ1)

• DevOps, according to the majority of the interviewees, means ownership and responsibility of software development team to design, implement, test, deploy and maintain in production the web applications and services.
• In all Cases, the developers had the freedom and management support to learn, choose, improve or influence suitable development practices in the project.
• Thus, development focused on working on small features and getting them into production as soon as possible.
• Contrary to the other cases, the mind-set in Case A, also supported by management, was to deliver on time and with good quality instead the ‘deploy and fix later’ mind-set.

5.2.1. Automated deployment mechanism

• Deployment scripts facilitated the automatic installation of software to a target environment by software development teams.
• Production deployments could be executed by any developer, but often only after notifying other team members of such an intention (Cases C, D, E).
• Existing key development practices Trunk-based development Frequent merges of branches to the mainline in all cases to minimise effort and conflicts resulting from code merges.
• Implicit adaptation to agile Scrum and lean Kanban.
• Canary deployment used in Case E ensured testing in production environment with the users.

5.2.2. Infrastructure-as-code

• Provisioning and configuring of environments repeatedly and reliably was done as a part of the deployment process using tools, such as Amazon CloudFormation (Case B,E), Chef (Case C), Puppet (Case D) and Ansible (Cases A and B).
• Most are made such that they are easy to replace and build.
• You can rely that you can build any server again if something horrible happens.
• ‘If the authors are working on network services, for these they often have Docker-based system’(Developer, Case E) ‘The network has been a bit flaky at times.
• While provisioning scripts or templates once created are reused and that information for editing them is readily available from documentation, software developers saw it as a competence of operations staff which overtime they could acquire through learning and interactions with operations staffs.

5.2.3. Monitoring as a continuous activity performed by the development team

• Developers were actively monitoring the web applications and services they developed, both in production and in CI environments.
• Monitoring tools, such as New Relic, Kinesis and Graylog were used to provide monitoring automation, and the tools were configured to alert the teams in case of incidents or anomalies.
• Both developers and operations were responsible for reacting to alerts (Cases C, D, E).
• Monitoring activity was observed to be mostly established for the web services developed internally (Case C, D, E) and least practised in contracted web applications (Case A, B).
• ‘We have a department that has information screens about the hosted servers, and they react when the disc space is going to run out.the authors.the authors.

5.3.1. Perceived benefits

• B1. Improved delivery speed of software changes.
• Decreased communication problems, bureaucracy, waiting overhead due to removal of manual deployment hand-offs and organisational boundaries; Lowered human error in deployment due to automation and making explicit knowledge of operation-related tasks to software development Eased tension and increased collaboration between development and operations [22] [37] [16] [45] [14] [15].
• All cases perceived DevOps to increase production quality and reduce deployment risks and stress.
• Faster feedback guided development more rapidly towards and allowed the product management and customers to follow the progress of in-production features in a realistic way (Case B, C, D, E).

5.3.2. Perceived challenges

• In a similar vein, they did not have a centralised log of the containerised microservices.
• In Case C, external deadlines sometimes forced the team to publish updates that had quality issues.
• Determining the right level of quality was observed to be challenging for the developers.
• Lack of common understanding of DevOps concept [14] C5.

6. Discussion

• This sections discusses the findings of the study in light of existing literature.
• This study suggests two explanations that may apply in small and medium sized companies.
• As noted by [38], ], certain deployment approaches used with the tools pose some reliability concerns.
• Similar to their study, cloud infrastructure to a large extent is a major enabler of DevOps implementation; however, the use of public cloud remains challenging [22] and require acceptance of the client (for consulting companies) or regulations (for public entities).
• Secondly, the responsibility of product owner to manually test and approve new software features before deployment in production signicantly affects the delivery speed.

6.1. Validity and limitation of the study

• The study acknowledges several threats to validity, against which some mitigation strategies were employed.
• Threats to construct validity are related to whether the selected cases actually apply DevOps and whether the interview questions were interpreted in the same way by all interviewees and interviewers.
• External validity is used to define the domain to which the study ndings are applicable and thus generalizable.
• Furthermore, some experiences may not be observed in large companies and system domain due to large system scale, characteristics and other context factors.
• Furthermore, the respective companies were involved to allow them to give feedback in different phases of the research.

Did you find this useful? Give us your feedback

Content maybe subject to  copyright     Report

59  citations

41  citations

View 2 citation excerpts

Cites background from "DevOps in practice: A multiple case..."

... There are however challenges like the lack of appropriate skill-sets, fast-evolution of technology stack and tools, as well as resistance to change, associated with the DevOps adoption journey [3] [4] [16]. ...

... According to available literature, benefit derived from DevOps adoption include faster delivery, improved quality and security, and better collaboration [3] [4] [16]. ...

29  citations

View 3 citation excerpts

Cites background or methods from "DevOps in practice: A multiple case..."

... , blue-green deployment or rolling upgrade [10], [11]. ...

... ware changes frequently in production environment [10], [11]. ...

... Building on agile software development method, an important tenet in DevOps is automation of build, testing, deployment and operations processes (Figure 2) while also ensuring that all software artifacts are version controlled [10], [11]. ...

26  citations

View 1 citation excerpt

... Organization becomes a challenge for DevOps practices because of its rules and bureaucracy.(68,69) 8 of 26 KHAN AND SHAMEEM ...

103,789  citations

78,012  citations

3,620  citations

2,079  citations

889  citations

Related Papers (5)

Frequently asked questions (17), q1. what contributions have the authors mentioned in the paper "devops in practice: a multiple case study of five companies" .

This exploratory study presents detailed descriptions of how DevOps is implemented in practice.   A multiple-case study was conducted in five different development contexts with successful DevOps implementations since its benefits, such as quick releases and minimum deployment errors, were achieved.   Their analysis yielded some of the following results: ( i ) software development team attaining ownership and responsibility to deploy software changes in production is crucial in DevOps. ( ii ) toolchain usage and support in deployment pipeline activities accelerates the delivery of software changes, bug fixes and handling of production incidents.   The authors discuss two practical implications of the results.  

Q2. What are the future works in "Devops in practice: a multiple case study of five companies" ?

Future research should consolidate knowledge of other possible technologies or practices for validating deployment scripts.   Future research should investigate refactoring practices and the trade-offs in the balance between quality and speed in a non-open source context.  

Q3. What was the first step of analysis?

The first step of analysis involved the use of a thematic coding technique [56] to code the transcripts using a set of predefined themes.  

Q4. What is the main contribution of the paper?

As main contributions, the paper provides an enhanced definition of DevOps that gives emphasis to automation practices in addition to collaboration between software development and operations.  

Q5. Why is DevOps infiltrating the whole company?

DevOps is reported to be infiltrating across the different parts of the company mainly due to enthusiastic practitioners who are considered as change agents to company’s ways of working.  

Q6. What is the purpose of the tool?

The tool is a web-based application that displays information about road maintenance activities gathered from various other systems and is meant to modernise an existing solution with more advanced features, such as real-time follow up and map services.  

Q7. Why is a low quality code prone to problems?

Because functional testing protects the outward visible quality, a low internal quality may not cause visible issues for a long time, but could become problematic at some point.  

Q8. What is the negative aspect of the public cloud?

The negative aspect of the public cloud is that it imposes some technological and architectural restrictions to the system, which according to Cito et. al., [37] can be seen as a positive thing by developers in enforcing best practices and putting more focus on delivering customer value.  

Q9. What are the three categories of validity threats discussed in this study?

Three cate-15AC CEPT EDM ANUS CRIP Tgories of validity threats are discussed: construct validity, external validity and reliability [51] [52].  

Q10. What was the culture of the company?

At the company level, there was a strong culture of sharing knowledge in training groups dedicated to different themes, e.g., a DevOps training group where members meet and discuss topics, problems, etc. related to that theme —similar to community of practice in agile.  

Q11. What was the main focus of the DevOps mindset in all cases except Case A?

In all cases except Case A, the development mind-set was that of getting features to production as fast as possible and fixing them later on.  

Q12. What is the process of triggering deployment scripts?

From the repository, version-controlled deployment scripts built in-house are manually triggered in order for deployment to test or production environment.  

Q13. What is the importance of having a team that has all the skills and knowledge to develop,?

DevOps practices require that, in the least, the team as a whole has all necessary skills and knowledge to develop, integrate, test and deploy, which includes provisioning and upkeep target environments and infrastructure.  

Q14. What was the role of colocation in the development of the product?

Case B saw colocation of the development team with the customer to be an enrichment of the organisation and in support of the DevOps mind-set.  

Q15. What was the first round of pilot coding?

After the first round2 NVivo is a qualitative data analysis software.of pilot coding, poor coding agreement values (Kappa less than 0.4) were observed in most themes (8 out of 11).  

Q16. When did the company move to a cloud-based infrastructure?

As such, the transformation towards DevOps started in 2011, when the company decided to move from on-premise infrastructure to a cloud-based infrastructure offered by Amazon.  

Q17. How long did it take to bake an image?

Although the authors did not explore the implication of deployment approaches in detail, it was observed in studied cases that image baking took a long time (15 minutes in Case E) even for critical bugs could not be xed in less time.  

Trending Questions (2)

The provided paper does not directly answer the question of whether 5 case companies are enough for a qualitative study.

Our findings contributes to the overall understanding of DevOps concept, practices and its perceived impacts, particularly in small and medium sized companies.

Ask Copilot

Related papers

Contributing institutions

Related topics

DevOps Adoption Benefits and Challenges in Practice: A Case Study

• Department of Computer Science
• Empirical Software Engineering research group

Research output : Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

Publication series

Fields of science.

• 113 Computer and information sciences

Access to Document

• 10.1007/978-3-319-49094-6_44
• RiunguKalliosaari2016DevopsAdoptionBenefits Accepted author manuscript, 245 KB Licence: Other
• https://www.cs.helsinki.fi/u/jutiihon/publications/RiunguKalliosaari2016DevopsAdoptionBenefits.pdf

T1 - DevOps Adoption Benefits and Challenges in Practice: A Case Study

AU - Riungu-Kalliosaari, Leah

AU - Mäkinen, Simo

AU - Lwakatare, Lucy Ellen

AU - Tiihonen, Juha

AU - Männistö, Tomi

N1 - Conference code: 17

PY - 2016/11

Y1 - 2016/11

N2 - DevOps is an approach in which traditional software engineering roles are merged and communication is enhanced to improve the production release frequency and maintain software quality. There seem to be benefits in adopting DevOps but practical industry experiences have seldom been reported. We conducted a qualitative multiple-case study and interviewed the representatives of three software development organizations in Finland. The responses indicate that with DevOps, practitioners can increase the frequency of releases and improve test automation practices. DevOps was seen to encourage collaboration between departments which boosts communication and employee welfare. Continuous releases enable a more experimental approach and rapid feedback collection. The challenges include communication structures that hinder cross-department collaboration and having to address the cultural shift. Dissimilar development and production environments were mentioned as some of the technical barriers. DevOps might not also be suitable for all industries. Ambiguity in the definition of DevOps makes adoption difficult since organizations might not know which practices they should implement for DevOps.

AB - DevOps is an approach in which traditional software engineering roles are merged and communication is enhanced to improve the production release frequency and maintain software quality. There seem to be benefits in adopting DevOps but practical industry experiences have seldom been reported. We conducted a qualitative multiple-case study and interviewed the representatives of three software development organizations in Finland. The responses indicate that with DevOps, practitioners can increase the frequency of releases and improve test automation practices. DevOps was seen to encourage collaboration between departments which boosts communication and employee welfare. Continuous releases enable a more experimental approach and rapid feedback collection. The challenges include communication structures that hinder cross-department collaboration and having to address the cultural shift. Dissimilar development and production environments were mentioned as some of the technical barriers. DevOps might not also be suitable for all industries. Ambiguity in the definition of DevOps makes adoption difficult since organizations might not know which practices they should implement for DevOps.

KW - 113 Computer and information sciences

U2 - 10.1007/978-3-319-49094-6_44

DO - 10.1007/978-3-319-49094-6_44

M3 - Conference contribution

SN - 978-3-319-49093-9

T3 - Lecture Notes in Computer Science

BT - Product-Focused Software Process Improvement

A2 - Abrahamsson, Pekka

A2 - Jedlitschka, Andreas

A2 - Nguyen Duc, Anh

A2 - Felderer, Michael

A2 - Amasaki, Sousuke

A2 - Mikkonen, Tommi

PB - Springer International Publishing

T2 - Product-Focused Software Process Improvement, International Conference

Y2 - 22 November 2016 through 24 November 2016