Chapter 3 - Research Methodology 3.1 Introduction This chapter describes the approach taken for data gathering and analysis. The findings discussed in the Literature Review are used as the basis for developing the research methodology. Section 3.2 discusses the research problem. A conceptual framework is then presented in section 3.3, identifying the independent and dependent variables. Section 3.4 presents the hypotheses to be tested and section 3.5 discusses the design of the questionnaire. Lastly, the methods of data collection are discussed in the section 3.6. 3.2 Research problem The author intends to analyze whether Scrum is an effective methodology for software development in the Sri Lankan context measure its effectiveness with respect to meeting quality, cost and time targets identify the factors which would have an impact on the effectiveness of project management in the Sri Lankan context. 3.3 Conceptual framework (Research design) The factors highlighted by Chow and Cao (section 2.3) are used as the starting point for an attempt to verify their applicability to the Sri Lankan context, as having an impact on the effectiveness of Scrum. In this research the author assumes that the effectiveness of a project can be defined as the overall perception of success of the project. As recommended in [18], effectiveness can be measured on four dimensions which depict the overall perception of success of a particular project, namely Quality (i.e. delivering a working product) Scope (meeting all requirements of the customer) Timeliness (delivering on time) Cost (within estimated cost and effort) 15
Due to the difficulty of measuring the Scope of a project within the limited period of this study, this aspect was not considered in this research. Table 3-2 lists the final dimensions that are used for measuring the success of a project. Dimension Success of a project Table 3-1: Attributes of project effectiveness Attribute 1. Time (delivering on time) 2. Cost (delivering within estimated cost and effort) 3. Quality (delivering good product or project outcome) Based on the initial research model (refer section 2.3), the following independent and dependant factors are considered in this research. The project factor is discarded as there are only a few companies which use Scrum and thus the project base is small and immature. The conceptual framework for this research is defined below. 16
Independent variables Dependent variables Organizational factors Management support Organizational environment Team environment People factors Team capability Customer involvement Perceived success of Scrum in local software development industry Time Cost Process factor Project management process Quality Technical factors Delivery strategy Scrum adherence level Figure 3-1: Research model 17
3.4 Hypotheses The following hypotheses have been derived based on the conceptual framework. These hypotheses will be tested using data collected from the above identified sample. 1. There is a relationship between strong management support and effectiveness of Scrum in project management in Sri Lanka. H o : There is no relationship between strong management support and 2. There is a relationship between Agile-friendly organizational environment and H o : There is no relationship between Agile-friendly organizational environment and 3. There is a relationship between Agile-friendly team environment and effectiveness of Scrum in project management in Sri Lanka. H o : There is no relationship between Agile-friendly team environment and 4. There is a relationship between a team of high capability and effectiveness of Scrum in project management in Sri Lanka. H o : There is no relationship between a team of high capability and effectiveness of Scrum in project management in Sri Lanka. 18
5. There is a relationship between strong customer involvement and effectiveness of Scrum in project management in Sri Lanka. Ho: There is no relationship between strong customer involvement and 6. There is a relationship between Agile project management process and H o : There is no relationship between Agile project management process and 7. There is a relationship between a good delivery strategy and effectiveness of Scrum in project management in Sri Lanka. H o : There is no relationship between a good delivery strategy and effectiveness of Scrum in project management in Sri Lanka. 8. There is a relationship between Scrum adherence level and effectiveness of Scrum in project management in Sri Lanka. H o : There is no relationship Scrum adherence level and effectiveness of Scrum in project management in Sri Lanka. 19
3.5 Questions for the survey Table 3-2 represents the attributes and the questions pertaining to the factors derived from the literature survey and [30]. The survey is enclosed in Appendix A. Factor Table 3-2: Questions for the survey Decomposed Question Related question Factors number Organizational 1. Management Management support 3A(12.1), 4A(12.1) support 2. Organizational Organization and 2(4), 3A(12.16), environment Scrum 4A(12.16) 3. Team Management influence 3A(12.4), 3A(12.2), capability on changing priorities 3A(12.8), Team attitude 3A(12.20), Level of 3A(12.24), communication 4A(12.4), 4A(12.2), Team self-managed 4A(12.8), SM influence on 4A(12.20), making decisions 4A(12.24) People 4. Team Cross functional team 3A(12.10), environment Team and Sprint burnt- 3A(12.15), down 3A(12.18), Team self-policed use 3A(12.23), of process and rules 3A(12.25), Self-managing team 3A(12.26), Dedicated team 3A(12.27), Conflict management 3A(12.28), within team 4A(12.10), Internal development processes Improved team processes 4A(12.15), 4A(12.18), 4A(12.23), 4A(12.25), 20
Dependency 4A(12.26), management 4A(12.27), 4A(12.28) 5. Customer Customer feedback 3A(12.22), involvement 4A(12.22) Process 6. Project Product owner's 3A(12.3), 3A(12.5), management involvement 3A(12.6), 3A(12.7), process Team disturbed in a 3A(12.14), Sprint 3A(12.17), Scrum tools used 3A(12.19), Changing requirements 3A(12.21), Effectiveness of SM 4A(12.3), 4A(12.5), Surprises occurrence 4A(12.6), 4A(12.7), Team and PO 4A(12.14), 4A(12.17), 4A(12.19), 4A(12.21) Technical 7. Delivery On-time deliverables 3A(12.13), strategy 4A(12.13) 8. Scrum Scrum roles in team 3A(2), 3A(3), adherence level Daily Scrum 3A(4), 3A(5), Daily Scrum duration 3A(6), 3A(7), No of team members 3A(8), 3A(9), Deliverables 3A(10), 3A(11), Recent Sprint duration 3A(13), Sprint planning 4A(2), 4A(3), meeting duration 4A(4), 4A(5), Retrospective meeting 4A(6), 4A(7), duration 4A(8), 4A(9), 4A(10), 4A(11), 4A(13) 21
Table 3-3 represents the attributes and the questions pertaining to the calculated effectiveness score. Factor Table 3-3: Questions for measuring effectiveness score Decomposed Question Related question Factor Effectiveness Time Actual time frame vs estimated time frame number 3A (12), 4A (12) Cost Actual budget vs estimated budget 3A (11), 4A (11) Quality Achieving quality objectives 3A (9), 4A (9) 3.6 Data collection This section describes the population and how a sample was selected for analysis. 3.6.1 Population, sample and sample selection Population Sampling Population Table 3-4: Population and sample Software professionals working in Sri Lankan software development organizations that use Scrum. Software professionals who are working in small, medium or large scale software organization that use Scrum, where the organization is a member of one of the following: Sri Lanka Association for Software Industry (SLASI), Software Exporters Association (SEA) or Export Development Board (EDB). Out of the few companies that are known to use Scrum in Sri Lanka, the following companies were selected for the research: Creative Solutions ExileSoft Aepona ecollege Teamworks IFS 22
These organizations represent large organizations as well as small organizations, and also a reasonably broad range of years of operation in Sri Lanka. All of these organizations are also headquartered outside Sri Lanka, and this analysis only considered the Sri Lankan operations. 3.6.2 Process of data collection Figure 3-2 below presents an overview of the data collection and the processing conducted in this research. Figure 3-2: Data collection process 23
3.6.3 Methods of data collection The author employed two methods, initially conducting semi-structured interviews with ScrumMasters from each company (refer section 3.6.1) and later collecting the team's perception via an online survey. This facilitated the involvement of many team members, allowing the comparison of their opinions. At the same time, through the use of open-ended questions, it permitted maintaining a qualitative view on the project environment, without having to conduct interviews with every single team member. 3.6.3.1 Preliminary data collection The preliminary data collection consisted of responses from three selected people who played the role of ScrumMaster, team member and the Product Owner. The online survey was sent to them. The author gathered suggestions for clarifications and refined the questionnaire. The suggestions are mentioned in Appendix C. 3.6.3.2 Questionnaire The research was conducted using an anonymous web-based survey based on (http://www.esurveyspro.com/) and offered over a period of two months in September and October 2009 for the employees who are involved with Scrum in any role. The invitations were sent by email to six companies (refer section 3.6.1). The questions were intended to understand respondents demographics, company information, level of Scrum adherence, Scrum usage and their perceptions of why Scrum works well or poorly on their software teams. (refer Appendix A) 3.6.3.3 Interviews Interviews were held with ScrumMasters from six companies (refer section 3.6.1). These interviews were made in order to get a view of what made them choose Scrum. Where Scrum was adopted recently, the challenges and barriers they faced during implementation of Scrum were also investigated. The criteria for selecting a ScrumMaster for a given project, was also discussed. (refer Appendix B) 24