Two case studies to explore pros / cons and to assess applicability of Nexus maturity model

Nexus maturity model, which is created as a structured gradual guideline for the companies wishing to adopt Nexus framework, is a new model, so the main purpose of this paper is to assess the model. Moreover, to explore pros and cons of the model to enlighten the researchers and implementers in the industry is aimed. Two case studies were conducted to collect data and to test three hypotheses. The second case study was executed 4 months later than the first case study which was completed with the same Nexus team members at the same company. During the case studies, structured interview technique was used via face-to-face meetings. Firstly, the results of the case studies show that this model can be applied easily by using the metrics to measure the success percentages of Nexus practices. Secondly, the practices with the low achievement percentage can be determined, the improvement strategies can be defined by means of the goals, objectives, practices of the model and investigation of the reasons as well. Lastly, with application of the improvement strategies, Nexus adoption success can be improved for higher levels by tracking Nexus maturity model elements. This paper presents the first application of Nexus maturity model in the industry, and also the first study to assess the model with its pros and cons in the literature.


INTRODUCTION
Globalized and rapidly changing markets have obliged organizations to be more simple, flexible and agile in order to quickly adapt to the change.Due to the development and transformation processes of these organizations, the management techniques of software development projects have been forced to evolve, especially in project-intensive companies.
According to a CHAOS report published in 2009, roughly 70% of information technology (IT) development projects have resulted in failure, mostly during the software development stage, due to poor communication between the clients and the stakeholders who play a key role in the product development (Dominguez, 2009).Moreover, the most important criteria for success according to this report is the inclusion of the clients at every stage of the project.The agile techniques came about as a "solution" to these cons of the classical project management methodologies like waterfall methodology.Despite sequential design processes, the agile techniques follow an incremental approach where developers start off with a simplistic project design, and then begin to work on small modules (Cobb, 2015).In this way, problems are identified and solutions are generated early, thus allowing for the creation of a product that ensures greater customer satisfaction.The concept of agility, which means continuously improving work and the infrastructure that enables it (Michael et al., 2003) emerged in 2001 with the Agile manifesto which contains the values and techniques regarding agility (Beck et al., 2011).Agile methods are techniques that enable quicker and more appropriate responses to customer needs by enabling the delivery of more frequent, smaller, iterative and incremental software developments.Dynamic Systems Development Method (DSDM), Feature-Driven Development (FDD), XP (Extreme Programming), Crystal (Crystal Clear Software Development), Scrum, Kanban and Scrumban are the most frequently used agile techniques (Stoica et al., 2013;Ahmed et al., 2010).
The concept of scalability has gained importance with the attempts of organizations to assess the success of agile methods in the projects they are implemented and their attempts to expand them within the organization.The scalability can be defined as the property of reducing or increasing the scope of methods, processes, and management according to the problem size (Laitinen et al., 2000).Scalable agile frameworks, which have come about in accordance with the nature of agile methods, enable agile transformations in organizations in which they are fully or partially implemented.
The following scaling agile frameworks have been published in the literature: Scrum of scrums Nexus was developed as a framework, and, due to the nature of frameworks, was structured on roles, rules, artifacts and events with the aim of drawing boundaries.On the path to becoming agile, it is necessary for organizations to be aware of their agility level and lay out a step-by-step roadmap in order to be promoted to higher agility levels.Nexus Framework does not provide the implementation strategies and the roadmap to the organizations which want to adopt Nexus.With this need, Nexus maturity model (MM) was developed in 2018 to provide detailed information about how Nexus can be adopted successfully (Firat and Can, 2018).
In this study, to explore pros and cons of Nexus MM implementation in a company is aimed.For this purpose, an international company is selected and two case studies were conducted at different times to measure the improvement of the Nexus teams by regarding Nexus MM.The second case study was conducted 4 months later than the first case study.Then the results were compared.

Nexus framework and Nexus maturity model
Nexus is a framework, not a methodology.The frameworks define the boundaries, they set the rules, definitions, events, roles and artifacts, but they don"t explain "how" questions.Also, Nexus is a new tool, so there is a limited number of an experienced individual in the industry.This limitation compels companies that wish to adopt Nexus framework to apply for coaching services which is a common method in agile transformation journey (Gandomani et al., 2014).With these main 2 reasons, Nexus MM was developed as a gradual roadmap and a well-structured guideline which consists of 117 practices, 22 objectives and 7 goals.Each item is located at the appropriate level in the model.The levels, goals, objectives and the number of the practices of Nexus MM are presented in Figure 1.Due to the extent of the material, only the numbers of the practices are showed next to the objective"s name.The definition of each practice listed in the Nexus maturity model is provided in a supplementary material, available at: http://avesis.yildiz.edu.tr/esincan/dokumanlarNexus MM is a guiding tool for Nexus teams consisting of 3 to 9 scrum teams.The model recommends that Nexus teams should apply the practices in the model by starting from level 2 and progressing step by step to reach the higher levels.Each high level practice is supported by the lower level practices and thus the increase in achievement percentage of the lower level practices is important to climb the maturity steps.The main goal of the model is to give direction to the companies and to provide opportunity to determine their own improvement strategies during Nexus adoption process.Two case studies were conducted at 2 different times in the next section to assess this feature of the model.In order to test expected results, 3 hypotheses were defined: H1: The achievement percentages of the practices can be defined with application of Nexus MM among Nexus teams.H2: For the practices with the low achievement percentage, the improvement strategies can be generated by making use of the model elements.H3: The progress in Nexus adoption can be tracked with  the application of these improvement strategies.

Case study
The Nexus MM was implemented in a large international corporation which is at the second year of its agility journey.The case company is a corporation headquartered in the Middle East area and located in 8 different regions including Silicon Valley.It services all the software development projects of a bank company which was established in 1924, employs more than 25,000 engineers, experts, professions and executives and has 1373 branches worldwide.This software company is also a research and development center to develop new products.The company initiated a change program in 2015 and the goal is to readjust the company"s methods and processes related to product/project management to be more agile.Within the scope of this corporate change program, the company started firstly with ten scrum teams in 2015.Then, in 2016, the company increased the number of agile teams working on software development and created 30 Scrum teams.In 2016, the company carried out research to select an agility framework for large settings and then decided to implement the Nexus framework on the path to become agile.In the last quarter of 2016, the company continued its operations with 10 Nexus teams (30 Scrum teams) and in 2017 increased the number of agile teams to 15 Nexus teams (50 Scrum teams).The number of total projects, the number of projects managed with the waterfall methodology, and the number of Scrum and Nexus teams since 2014, when the company entered the agile transformation period, are shown in Figure 2.

Application of the Nexus maturity model
To assess Nexus MM, the 9 Nexus teams were determined to join this case study.Each team of the six Nexus teams consisted of 3 scrum teams and the other each Nexus team (3 Nexus teams) consisted of 4 scrum teams, so the study was conducted with a total of 18+12=30 scrum teams (208 employees).This case study was carried out in two stages.At both stage, the aim was to acquire data regarding Nexus maturity levels by presenting the participants with questionnaires which were prepared based on the indicators of the practices in the model.It was important to reach a consensus on the answers and to attain accurate results from each Nexus team.Because, some of the participants were inexperienced in the team and did not have prior experience with Nexus, the moderator of the case study had to make sure the individuals properly understood the questions.For this reason, the "structured interview technique" was used to collect information from the participants.Face-to-face interviews that consisted of 2 sessions were conducted with each Nexus team.The sessions were attended by the scrum master of each scrum team, the product owner of each Nexus team, and the representatives from the development teams appropriate for the study.Each two-session interview took 4 h and those were organized on 9 different days, making a total of 36 h.A total of 30 scrum masters, 9 product owners, and 35 development team representatives attended the meetings.Before the interviews, the indicators were created in order to measure the achievement levels of the Nexus MM practices.As an example, the indicators used for defining the achievement percentage (Ap) of the second practice "cycle of sprints of development occurs" in level 2, are listed below; Indicator1: Sprints has maximum length of 30 days or less Indicator2: Sprints start with Nexus sprint planning meeting Indicator3: Daily Scrum meeting happens every sprint-day Indicator4: Nexus daily scrum meeting happens every sprint-day Indicator5: Nexus sprint review meeting occurs before the retrospective Indicator6: Sprints end with Nexus sprint retrospective meeting For these 6 indicators, the participants gave a percentage value and then the arithmetic mean was calculated for the indicators aforementioned to define the achievement percentage of the practice "cycle of sprints of development occurs".For the practice (Pr) measured by 3 indicators, the formula is: This process was executed for each practice and for each Nexus team.In the end, for each Nexus team, the average achievement values were calculated on each practice, afterwards the arithmetic averages of all Nexus teams were obtained for each practice.The second stage started 4 months later.In both meeting, it was expected for participants to say the percentage values in five-fold were 85, 60, 15 etc. for each indicator as the achievement value.
The results of the first stage assessments are summarized in Table 1.All values were calculated by arithmetic mean of all Nexus teams.
Table 1 is separated into 4 sections according to ISO/IEC 15504-2 standards (ISO/IEC 2003).As seen on Table 1, in case the average values are calculated, two practices were achieved less than 15% at level 5. Additionally, 4 practices at level 5, 1 practice at level 4 and 1 practice at level 3 were achieved less than 50%.In order to get a progress in maturity level, it was decided that these 8 practices should be prioritized and; primarily, the improvement strategies were defined for them.According to the results of the first case study, the practices with the low achievement percentage are listed in Table 2.The reasons of these low level achievement percentages were examined and the improvement strategies were identified in regarding their reasons as seen in Table 3.The improvement strategies are implemented for 4 months by all teams.4 months later, the second case study was conducted with the same people and the same format.In the end, the results obtained are shown in Tables 4 and 5.

RESULTS
To assess H3, the improvement from the first situation to the second situation in each level, the results are compared in Table 6.The aims of the first case study are; (1) To define "as is" situation and to state the practices having low Ap, (2) To determine the practices which need improving primarily, (3) To explore the reasons for failure of these practices, and (4) To develop improvement strategies to reach higher maturity level In order to test Hypothesis 1, the average achievement percentages of 9 Nexus teams for each practice in Nexus MM were calculated.The practices with Ap less than 50% were stated.So, H1 is validated.In order to test Hypothesis 2, the reasons for low Ap were identified with the comments from the participants in the case studies.By thinking over the reasons in Table 3 and utilizing the goal and objectives of Nexus MM, the improvement strategies were developed.So, H2 is validated.In order to test Hypothesis 3, the second case study was conducted.As seen in Table 6, the increase in Aps of the practices were observed through the application of the improvement strategies for 4 months.So, H3 is validated.

Levels Practice name AP 3
Nexus integration team members are responsible for coaching and guiding the scrum teams in a Nexus 25 4 All products are managed with compliance to all goals, objectives and practices which is defined quantitatively 35 5 Lessons learned are recorded 0 5 The occured impediments are analyzed daily 10 5 Agreement on how to visualize and track the identified actions 25 5 Nexus sprint retrospective is done as a 3-stage meeting 45 5 Causal analysis retrospective is made and then the corrective actions are taken against impediments 45 5 High level of energy for all scrum teams is observed 45 Lessons learned are to be recorded every Nexus sprint retrospective on an excel sheet by the selected participant 5 The impediments are identified daily but not analyzed because it is supposed that analyzing takes long time Every sprint day, 3-4 minutes are to be arranged for analyzing of the impediments 5 The teams don"t want to spend time on visualisation and tracking of identified actions Every Nexus sprint retrospective, 3 participants are to be selected as the responsible for visualizing and tracking of identified actions 5 The first two stages are held but for the third stage the time isn"t remained because Nexus sprint retrospective meetings are held afternoon Nexus sprint retrospective meetings are to be started before midday 5 Mostly, causal analysis aren"t done at the retrospectives, because when the team started to talk about the causes, the topic gets longer and distributed and the team can not reach any solution at the end of the meeting Nexus Scrum masters are to provide mentoring support to the teams, so they can do causal analysis without waste of time.Also, for causal analysis, some techniques are to be used like fishbone diagram 5 Mostly, the teams have concern about that they can not complete the work which they committed at the planning meeting at the end of the sprint.This concern makes them stressful At Nexus sprint planning meetings, the teams are to calculate the points with the margins agreed on while scoring Table 4.The levels, names and new achievement percentages of the practices whose achievement percentages are less than 50 at the first case study.

Levels Practice name AP 3
Nexus integration team members are responsible for coaching and guiding the scrum teams in a Nexus 60 4 All products are managed with compliance to all goals, objectives and practices which is defined quantitatively 65 5 Lessons learned are recorded 90 5 The occured impediments are analyzed daily 70 5 Agreement on how to visualize and track the identified actions 45 5 Nexus sprint retrospective is done as a 3-stage meeting 85 5 Causal analysis retrospective is made and then the corrective actions are taken against impediments 75 5 High level of energy for all scrum teams is observed 70

CONCLUSIONS
This study was based on creating an example of the use of Nexus MM in different organizations and questioning its applicability.A sample case study was conducted with the method of testing 3 hypotheses established.Because Nexus MM is a new model, the introduction of a sample application is another important goal of this paper.It has been tested and verified that the Nexus MM was created as a guide for the companies to use as a tool for their progress towards adopting Nexus framework better (Firat and Can, 2018).Since the company in which the case studies were conducted is a successful company in the international software development market, the application percentages are high.However, for the future, it is imperative to pursue the higher achievement rates, and for this, the improvement strategies must be defined constantly.Agile transformations are the iterative and incremental organizational development processes.For this reason, the companies that decide to adopt the Nexus framework need to make measurements over the Nexus maturity model, identify the reasons for failure, and develop the improvement strategies periodically to reach higher maturity levels.

LIMITATIONS
The biggest advantage of the model is its detailed structure, which means the model provides detailed information for each step of the adoption.Every goals, objectives and practices are well-defined level by level; thus, it can be used easily without any need for a consultant.The second advantage of the model is that it provides a roadmap that fits the desired goal, since the model clearly shows which practices foster the goal through which objective.
There are two main advantages of the case studies; they shed light on how Nexus MM should be used in business life.It provides information about the path to be followed and the time to be spent.The case studies also emphasize that the model is a tool which teams can implement easily without getting any support; moreover, this study encourage Nexus teams to take advantage of using and improving Nexus MM.
The more detailed the indicators defined to measure the practices in the model, the longer it will take to define the indicators and collect the percentages from the teams.For this reason, care should be taken to ensure that the indicators or questionnaires to be determined for the practices to be measured are as clear and concise as possible.Otherwise, applying the model may take a long time.Moreover, after the method of collecting data from the teams is determined, the participants (questionnaire respondents, meeting participants, interviewees, etc.) should be selected meticulously.
The data should be collected from the most knowledgeable team members of Nexus, the members who have been in Nexus teams from the beginning and, if possible, the different titles like scrum master, product owner, development team member etc.If the meeting method is preferred, it should be emphasized that the number of participants should not be more than optimum number.In these case studies, a structured interview technique was chosen as the data collection technique and face-to-face meetings were held.For future studies, it is also possible to measure the indicators in the form of questionnaire and to collect data by questionnaire method.
The most important contribution of the model is not to define only the maturity level of the Nexus teams.On the contrary, in compatible with agile transformation spirit, the main goal is always to be better instead of labeling the teams as "mature at level 3".In this paper, the measurements at the practice level were presented and hypotheses H1, H2 and H3 were constructed to assess the model in accordance with its main aim.Moreover, since the collected data are quantitative values, it is possible to infer and calculate Aps for goals and objectives, if desired.
The most important limitation of this study was the time.Only a 4-month recovery period has been observed.But agile transformation process and Nexus adoption process takes longer time.The firms can determine their own time period themselves.They can experience an iterative improving process by making these measurements multiple times, with more frequent or less frequent time intervals.
Lastly, because Nexus MM is a new model, the execution of case studies in different organizations can show the companies more alternative ways to use it more easily and to explore potential risks behind Nexus adoption.

Figure 2 .
Figure 2. Number of total projects, ongoing projects managed with waterfall methodology, scrum and Nexus teams in the case company over the years between 2015 and 2017, May.
(SoS) by Ken Scwaber in 2004, Disciplined Agile Delivery (DAD) by Scott Ambler in 2007, Scaled agile framework (SAFe) by Dean Leffingwell in 2008, LeSS (Large Scale Scrum) by Craig Larman and Bas Vodde in 2008, Spotify by Henrik Kniberg and Anders Ivarsson in 2012, Rage (Recipes for Agile Governance in the Enterprise) by Thompson in 2013, and Nexus by Ken Schwaber in 2015.Although the Nexus Framework is the most recent scaled agile framework, it has been widely preferred because it is simple, easy to apply, not diversified according to the number of teams (unlike LeSS) and overlaps to a great extent with scrum values and practices.

Table 1 .
The results of the first case study.

Table 2 .
The Levels, names and achievement percentages of the practices whose achievement percentages are less than 50.

Table 3 .
The reasons and improvement strategies for the practices.

Table 5 .
The results of the second case study.

Table 6 .
The comparison of the results of two case studies (before and after improvement strategies).