Survey on Agile Implementation of the BI Systems

Business intelligence (BI) is a technique that helps organisations to effectively analyse, manipulate and store data. It takes historical and present data from various sources and presents the data to the users anytime, anywhere to help them make smart and effective decisions. However, the cross functional nature of BI systems that covers the length and breadth of the organization, pose an issue with effective implementation. Various Traditional methodologies have been used to implement BI systems however have encountered countless fail-ures leading the practitioners to look up to Agile methodologies to overcome the shortcomings. Since different companies have different requirements, out of the box Agile solutions do not address the requirements effectively. As a result, use of Agile methodologies for BI implementation also face lot of issues. To justify this claim we conducted a survey of agile practitioners doing BI implementation. This paper aims at presenting the findings of the study focused on identifying the gaps in implementing BI systems using Agile methodologies. It also presents the results for the survey conducted to capture the methodologies used by the organisations and practitioners and issues in BI implementation. In future, this captured information would be utilised into formulating a framework that can work along with Agile methodology to help address the issues faced with Agile methodologies for BI Implementations.


Introduction
Information is vital to decision making. Current and accurate information help make effective decisions. Although the main focus of every organisation is growth in revenue, organisations use their knowledge bank to make strategic decisions. Such decisions help them utilise their resources productively, reduce project costs, maximize profits and minimize the risks [1]. Business Intelligence (BI) enables employees to have access to useful information anytime anywhere. Organisations use BI to gain competitive advantage over other organisations mainly by harvesting the benefits of having a bird's eye view of their entire current collection of data and forecast information. BI implementation is quite complex in nature. It integrates data from different levels of an organization and multiple factors have to be considered for data integration such as cleansing of data, modelling of the data warehouse, creating reports, dashboards to name a few [2]. Since BI projects involve people and data from different levels of organization and external sources, using traditional methodologies to implement BI projects can encounter issues [3]. After implementing BI systems using traditional methods, lightweight Agile methodologies have been adapted to implement the BI projects. The iterative nature of Agile methodologies has greatly helped implement better BI solutions [4]. However, every methodology comes with flaws. There are many Agile methodologies that are that are being used for the implementation of BI solutions namely scrum, lean, extreme programming and latest being SAFE, the most popular being the lightweight Scrum methodology. This research aims to identify and compare various methodologies that exists for BI project implementations. As BI is fairly a new concept, it is interesting to explore various Traditional and Agile methodologies that are being adapted by the organisations for its implementations. We took a twofold approach for this project. Firstly, an extensive literature review was conducted to identify the findings and gaps and then to verify the literature findings we conducted a survey conducted on industry professionals who had experience in Agile BI implementation. Agile methodologies have successfully implemented various kinds of software projects, this study will explore the scenarios in which organisations implement BI using Agile. This research is also targeted towards finding the areas that benefits the most through Agile methodology and look at what kind of measurement indexes the organisations use to measure the difference between the Traditional methodology and Agile methodology and how the benefits of Agile is calculated. Hence, we make the following claims with the implementation of using Agile for BI projects: • Out of the box Agile may not suite every type of organisational scenario • BI is relatively new, companies have very little knowledge and experience with the application. • BI projects are cross functional involving people and data from different departments hence, coordination can become an issue. This survey is designed to find whether use of Agile for BI implementation is successful or not. It aims to identify any drawbacks that comes with use of the Agile methodologies by the organisations. Finally, the purpose of this study is to identify based upon the survey responses if framework that can be used in conjunction with Agile methodologies that will aid in overcoming the gaps identified in this study. The paper is structures as follows: section 2 provides the definitions and methodologies used for BI implementation, section 3 outlines the research objective, methodology containing detail on research design to fulfil the research objective is given in section 4, Section 5 covers the mapping of survey to research objectives, survey results, analysis of results and the final outcome of the survey with recommendations. Section 6 concludes the paper. The main contributions of this paper are the justification and confirmation of the claim that out of box Agile that has been successfully used for software development is not being utilised for BI implementation and the major issues identified. The future work would focus on the development of the Agile framework that can be used for BI implementation.

Literature Review
In this section, the background on business intelligence, its implementation in organisations and the existing methodologies used has been elaborated. It has been identified that majorly, two methodologies have been used for BI implementation. Hence, we have scoped this paper at traditional and agile methodologies. We present the comparison of these two methodologies for BI implementation and how agile improves BI implementation. A.
What is BI? "Business Intelligence (BI) can be defined as a set of procedures, strategies, policies and technologies that is used to collect, store, manipulate and analyse data from internal and external sources for making strategic and operational decisions" [5]. BI applications uses historical and new data stored in Data Warehouses (DW) or Data Marts. As a result, in many cases BI and DW are collaboratively known as BI/DW. Organisations sometimes construct smaller data marts as a subset of data warehouses to store department specific or data dedicated to a specific requirement. The data warehouses are multidimensional databases and the data marts are constructed around it. Organisations produce large amount of structured and semi-structured data [6] every day. The semistructured data do not have the structure that fits into relational databases. It can be represented in any form ranging from emails, conversations, spreadsheets etc. Surprisingly, organisations produce lot of useful semi-structured data. In a typical BI architecture, as shown in Figure 1 applications extract structured and semi-structured data from internal and external sources using the Extract process. Such data is cleansed and transformed into required format that is acceptable by the target database using Data Transformation process. The next process i.e. Load process loads the data into the target database (Data Warehouse). The target databases or data warehouses are usually RDBMS, but with the increase in the volume of data, various data engines such as MapReduce engines [7] are now being utilised. This process of Extract, Transform and Load is known as the ETL process and is the most important part of the BI application. Data is then represented to the users through front-end applications in the form of reports allowing users to get a visual representation of data that is used for business decision making as shown in Figure  2. B.
Traditional approach of BI Implementation Among all the traditional approaches, this paper will focus on the "Waterfall Methodology" as it has been used extensively for BI implementation. Similar to any other software project, using a waterfall methodology for BI project implementation also requires a detailed documentation of functional business needs [9] before any development begins. It requires the users and the project team to document all the report requirements and have a formal sign-off before they move on to the next phase from the requirement gathering phase. Waterfall Methodology follows a non-iterative cascading sequence from business needs to planning, development, testing, implementation and then maintenance as depicted in Fig  The same sequence is followed when implementing BI systems using waterfall methodology. A BI project is 80% data effort and 20% coding effort [11] and waterfall methodology gives more importance to the deliverable rather than data analysis. C.
Agile approach of BI Implementation The term Agile literally refers to as ability to move quick and easy. Likewise n the article "Agile BI-the future of BI" [10], Agile methodology for BI implementation has been described as "an approach that combines processes, methodologies, organizational structure, tools and technologies that enable strategic, tactical and operational decision-makers to be more flexible and more responsive to the fast pace of changes to business and regulatory requirements" [10]. When it comes to Agile for BI implementation "iterative and incremental" [12] terms are generously used. This characteristic greatly differentiates this methodology from the traditional methodologies. The most common form of Agile used for BI implementation is Scrum where the requirements are broken down and the stakeholders get a fully workable end product after every iteration (also known a sprint). The Figure 4 depicts the typical flow of Agile BI implementation.  [10] There is constant communication between the project team and the stakeholders which help accommodate changing requirements and changing scope of the project. Communication is the most important part of Agile methodology. D.
Comparison between Traditional approach and Agile approach When it comes to comparison between the two methodologies they are miles apart in the sense that they follow different guidelines and have different method of implementation. Waterfall methodology starts with extensive requirement gathering phase, whereby project managers come up with definitive work breakdown structure (WBS) [12]. The WBS outlines all the various duties and milestones of the project team members. It also provides a framework that the team follows to complete the project. Based on the tasks outlined in WBS, the team calculate the man hours required to complete the implementation project. The project manager then comes up with the rough estimate of the project timeline, milestones and costs. Once the estimates have been accepted the work is delegated to the team members (BI developers in our case). It is noteworthy that all the tasks in the WBS are being defined by the project manager and not the team members who are going to perform the work. Waterfall methodology heavily depends on the project manager to understand the requirement, break them into tasks and delegate the tasks to the team members to ensure that the project completes within the timeline and within budget. The project requirement is more or less in the frozen state with very little room for modification. The Agile methodology on the other hand, is an incremental iterative approach which does not ask for all the requirements upfront or freeze them. Instead, the project team takes the most important functionalities that the user requires from the system as a starting point and keeps learning the requirement as the project progresses. They deliver working BI reports that the users can test in small timeframes and keep taking their feedbacks unlike waterfall methodology where user involvement is very minimal. With every release the accuracy of the BI reports and the requirements of the users become more and more distinct. E.
Limitations of Traditional Methodologies for BI Implementation BI system is a cross functional system and covers the breadth and depth of the business. It not only involves IT team but also involves people and data from all the various departments of the business. BI projects are completely data driven projects which functions by integrating data from these various sources. Since BI implementations are still in its infancy in most organizations, the requirements are not very clear. Traditional waterfall models works best for "predictable" systems. Waterfall methodology works best when the requirement of the project is very clear with minimal or no modification afterwards. Waterfall methodology also considers that there is no existing data and that the system is being constructed from the scratch. But seldom is such a case in any BI projects. Furthermore, the traditional methodologies require elaborate requirement gathering and clearly outlined work breakdown structure (WBS). As the requirement of any BI project cannot be fully captured at first instance and the team can encounter various changes as the project progresses, the elaborate project plan made at the beginning of the project is bound to fail. The analysis phase of the waterfall methodology does not consider the importance of data analysis separately. Like any other software projects, the analysis phase in waterfall model implementation of BI systems focuses on requirements gathering, with no extra focus on data analysis which is a problem in BI projects because data analysis is the most important part of BI projects. Since BI projects are 80% data and 20% coding effort [11], which the traditional methodology does not take extra effort to accommodate this requirement of BI systems. In waterfall methodology, there is a long gap between when users provide the requirements to when they get to view the working system. This is due to the fact that during the design and development phase there is user involvement. Design and development phase focus on the entire system design and development which is time consuming task. Once the development is complete, the users are involved again for testing [3]. This is the major drawback with the waterfall methodology for BI implementation as during that long gap and lack of communication users lose trust that the system still meets their requirements. As the user feedbacks are not taken during the process of design and development and until the final system has been completed and this implementation has no room for error modification. During this time new BI requirements may be identified which cannot be accommodated into the system development process. The requirement for the completion of each phase of waterfall methodology to move on to the nest phase poses a problem when it comes to the BI projects. As BI projects spans over various departments and individuals, taking the requirements from one user and freezing them to make an application out of it will result in a BI application that only works for a department or an individual. Due to sequential nature of waterfall methodology, the need for proper data analysis and data modelling for BI projects is not catered by traditional waterfall methodologies. This poses to be one of the major drawback of waterfall methodology when it comes to implementing BI projects since data management [11] is the most important and tedious procedure in BI implementations. F.
How does Agile overcome the failures? Agile attempts to cater for most of the issues faced by the waterfall methodology. The iterative and incremental design of the Agile methodology helps tackle most of the issues [12]. Agile methodology does not consider taking all the requirements upfront and this provides room for learning as you go and accommodate modifications. The user requirements are broken down into achievable tasks and the project team delivers the users with working reports after a short period of time (sprints). It works progressively with techniques that can accommodate changes and uses a development approach that is metadata driven [13]. Agile helps solve the issue of providing output to the users in a short time. Learning as you go in Agile helps users to improve their requirements and project team to provide better and improved reports to the users [14]. It helps to refine the requirements and keep only those that are of value to the organization [15]. Agile captures the essence of business where the requirements never remain the same and allow for modifications [16]. It catches the risks as it happens. Unlike waterfall model where the project manager delegates all the tasks to the project team members, it is the team members who is organising the tasks to provide better and quick results in Agile methodology. It also regards the 80-20 rule [12] of BI projects. Coding begins only when 80% of the clarity on the data is obtained. Any software project is prone to errors and unknown issues and BI projects are not different. The iterative and incremental nature of Agile methodologies however, helps to detect the errors in the process or system early on risking only a small part of the project. The errors can be detected early on and corrected as they are detected due to continuous review and feedback via constant communication. The iterative nature of Agile also helps to improve the quality of the BI solutions provid-ed through series of quality testing and testing from the users as well. Through the literature, Agile methodology seems to be a logical solution for BI system implementation however we were keen to identify the status of agile BI implementation in the industry. The later sections will confirm how the Agile fills these gaps through the survey conducted.

Research Objective
The objective of the research has been broken down into 8

A.
Research Design An extensive literature review was conducted to understand the current trends, different methodologies and how each methodology worked. The research objective of the project and the survey requirement highly influenced the choice of research design for this research. Keeping in mind the requirement of formulating a survey and deriving results from the survey, the research design has been selected based on the previous work that has been completed for Agile methodology for BI project implementations. The survey questions were used to endorse the knowledge gained from the literature review. The survey questionnaire with multiple choice questions and short answer question was circulated to industry participants with sound experience in BI system implementations. Most questions had multiple choice options along with spaces that allowed user to enter other options if it existed. Additionally, some short answer questions were included as well to capture the views of the participants. Matrix questions have been used to identify the areas that benefited the most from Agile methodology and to look at the different ceremonies of Scrum as well. Since the survey questions are the most important part of the study therefore, the questions were designed in a concise and definite manner to ensure that the participants understood the purpose of the question. The finalized questioners were then reviewed and tested rigorously to ensure that it was working and made sense before it was circulated for survey. This was done mainly to check whether the questions were framed correctly to achieve the required output or not and also to see the correctness of the technical details that was used in the questioners. The main aim of the whole study is to examine the use of existing methodologies for BI implementation including Agile methodology, and look at the issues they face while using the different methodologies. Subsequently, this study also aims at looking at the areas Agile benefits the most and also the risks and preventive measures that can be taken while using Agile methodology.

B.
Survey participants As the study heavily relied on the responses from the participants, identifying appropriate participants was very crucial. With this in mind, participants from the industry with sound experience in BI project implementations using various methodologies, especially Agile methodology was identified to take up the survey. Since BI system implementation is relatively a new area and is not that prevalent we did not use a sampling techniques to choose the participants as number of survey participants identified were not too large. A total of 19 participants took the survey and all 19 participants had good experience with BI system implementations using various methodologies. Although face to face interview can help to clear doubts better, the survey for this study was however, conducted through Survey Monkey. This was done mainly because participants belonged to various industries and the duration of the project was very short. Also, the industry participants were not available to provide face to face interview as the participants were contacted through a professional networking site known as Linkedin and they belonged to different geographical locations. The survey consisted of twentyfour questions with a focus to answer the research questions stated. C.
Data analysis The survey that was conducted for the purpose of this study helped to obtain quantitative data. Before beginning the analysis, data validation was conducted to check whether all the questions were answered or not. As all the questions were marked mandatory, participants did not skip any question. Some statistical tools have been used to analyze the collected data. The techniques that have been utilized for the analysis are as follows: a.
Indicator analysis This survey had matrix rating question which integrated 9 questions into one question. Matrix rating works by assigning weights to every answer and the formula to calculate the rating question was Univariate analysis In order to make the survey simple and concise this survey had maximum of multiple choice questions whereby the options were provided after careful study and feedbacks from the industry professionals. This was done to be able to collect all the relevant information and derive a definitive conclusion.

c.
General insight This survey also included five open ended questions at the end of the survey. This was done to allow the participants to freely express their views instead of being tied to predetermined answers. The responses collected was categorized based on the occurrence of the similar type of answers. The open ended questions were targeted towards finding the participants views and their opinion.

Survey Results & Analysis
Majority of the survey participants had the role of Scrum Team member or Scrum Master. We carefully evaluated the results.60% of the participants had more than 3 years of experience in implementing BI systems using Agile. It was clearly evident from the responses timeline was the factor that contributed towards choosing Agile for BI implementation. Since Waterfall methodology does take longer duration to deliver results and Agile can deliver workable solution after every sprint, 68.42% participants reported timeline being the factor for choosing Agile for BI system implementation. Contrastingly when it came to the quality of the system that was delivered using Agile, 46.67% i.e. half of the participants expressed that Agile improved the quality of the delivered system.
One open ended question was included to investigate if they had a repository before implementation or, was the data warehouse built during implementation. 43% of the participants stated that they had existing repository indicating that the new BI project they implemented using Agile was only used for reporting purpose. Further the survey also reflected that Agile was used for both reporting and data warehouse modelling. 100% of the participants reported that the use of Agile methodology improved the implementation of BI systems. In order to identify the effectiveness of Agile to foresee any issues, some of the prominent answers from the participants were: • It helped to identify faults and bugs early in the system and could see the performance and functionality of the system after the first sprint. Continuous integration was used to ensure the modules delivered were functional and the bugs identified were fixed in the upcoming sprints.
• It helped to foresee the resources required for the completion of project within scheduled timeline and also the knowledge required to complete it. • Issues were identified timely much earlier in the projects which helped to widen scope at a later phase • Raised risks earlier. Higher visibility. Better collaboration and team work. We can conclude from the responses that the iterative nature of Agile does help to detect errors and problems early on.
For any other additional methodology for their BI Project implementation along with Agile, many responses indicated that a mix of methodologies were used for implementing BI systems with Agile. One of the participant stated the reason being unclear and mismatch requirement as being one of the reasons for choosing a hybrid methodology. Although, some did insist they followed strictly Agile. However, 53.3% of the users indicated that additional information was required while using Agile for BI implementation. Most of the participants followed the standard norms of Agile methodology for project implementations. The survey also indicated that out of the box Agile methodology was customized to suite the requirement of the organization. Surprisingly, Agile-Waterfall combination was followed by half the participants for BI Implementation.
Three of the main issues that was faced during implementing their Business Intelligence systems using Agile methodology and the corrective measures from the responses are as illustrated in the Figure 6: The results of the survey helped solidify some of the information obtained from the literature review. Information such as the most popular Traditional methodology and the Agile methodology. Through the literature review Waterfall methodology and Scrum was identified as the most popular traditional methodology and Agile methodology respectively. This was confirmed by the responses whereby almost all the participants confirmed this information. There is extensive material available that talks about the advantages of Agile methodology over waterfall methodology. However, this survey has helped to quantify the areas in a BI system implementation that is being benefited the most by using Agile. As expected there is very high improvement in the quality of the system that is delivered. This is mainly due to the iterative process of Agile. As mentioned in the literature review, learning as you go in Agile helps users to improve their requirements and project team to provide better and improved reports to the users. It helps to refine the requirements and keep only those that are of value to the organization. Agile captures the essence of business where the requirements never remain the same and allow modifications. It catches the risks as it happens. Further, it is remarkable to see that organisations are customizing the out of the box Agile to fit in the requirements of different organizations. It is also noteworthy to see that organisations are using Agile-Waterfall methodology together for some BI projects. Some participants expressed that ever changing requirements and too many enhancements and offshore team coordination lead to the agile waterfall methodology The fact that reporting is a lot dependent on the build of the data warehouse, and nothing is producible without the schema / data being in the DWH, that's why an agile -waterfall methodology was used. The three components-Tools, People and Process emerged as most crucial for successful implementation of a Business Intelligence projects using Agile methodology. Tools: Understanding the requirement of the organization will help identify the most appropriate BI tools that suits the organization and helps implement BI with minimal or no customization to the implementation methodology. Understanding which methodology to use also helps achieve the most favored result. People: Skill gap, involvement of the project stakeholders and management buy-in were the factors that affected the delivery and the quality of the solutions delivered indicating People as a defining factor. Considering all the information from literature review and the results from the survey it can be concluded that they are the main factor that contributes to the success of a project. The success of a project relies on the people involved in the project. Some of the factors that play important role are Getting the management support, Appropriate Stakeholder involvement and Well trained dedicated team for the implementation. Processes: A set of well-defined Processes was used to overcome issues pertaining to technical skill gap, identifying dependencies of the project before hand and allocating appropriate resources. This was identified during analyzing most prominent issues for implementing their Business Intelligence systems using Agile methodology.

Conclusion
Based on the information from the literature review and the survey results, we can conclude that 3 most important components form the defining factor for Agile BI implementation. Of course, these three components are to be used with Agile methodology for a better BI implementation. These three components form the framework that are to be used together with Agile for efficient implementation of BI. Although Agile methodology puts a lot of focus on data, a proper and detailed requirement gathering at the beginning of the project can help plan the appropriate number of team members, effective number and duration of sprints. A framework that combines and aligns these three noticeable factors along with out of the box Agile methodology can help to identify gaps and dependencies beforehand which is the focus of our future work.