The term Data Warehouse was coined by Bill Inmon in 1990. His definition is “A warehouse is a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management’s decision making process”. Easy for him to say. This was going to free the Information Technology department from the drudgery of creating literally thousands of reports for end users, the end users would get their information themselves (Self Service BI?).
Here we are over 20 years later and have we achieved this lofty goal? The single biggest complaint of end users today is the lack of access to timely, accurate data. So clearly we have not made data universally available to the user.
You can not go to your local BI store and buy a Self Service BI solution. In reality, Self Service BI is a lot like beauty, it is in the eye of the beholder. Each user may have a different view of what it is.
Over the years, it has become apparent that there are predominately two different types of BI users; casual users and power users. I have heard it said that there is an 80/20 ratio between casual and power users, but I believe the the ratio is closer to 95/5. I have been in organizations with hundreds of casual users and maybe 4-5 power users. From this perspective, you must create at least two self service solutions to support all of your users.
Self Service BI to the casual user
Casual users are those BI consumers who use standard interactive reports (static reports are dead, useless in a self service model) to help them make decisions in their daily work. These include executives, managers, suppliers, consumers and line workers. They need the ability to select data and drill down to a certain level of detail.
To these people, Self Service BI is the ability to customize data to a specific view or another way to look at it is ad-hoc reporting. This means the ability to select specific attributes (i.e. customer, geographic area, products, etc) and create a report or chart of the selected data. Once this is done, they may want to create a spread sheet or slide presentation with the data. The IT department is out of the loop as far as they are concerned.
By the way, they want to be able to manage all this functionality with a few minutes or less of training.
In this situation, the important part is the creation of basic reports and charts that give the user the ability to customize them to their current requirements. There are many tools that can be used to do this, some better than others.
However, IT is the group that creates these basic applications, not the end user. It is very important that the IT group that creates the base reports and charts understand the needs of their specific end user. They may very well create a different application for each department within the enterprise, and it is highly likely that the information in each department’s application overlaps other applications. Additionally, demands for new data occur regularly, so IT must adopt agile methodologies and agile tools (more on this in another blog) to keep up with the demand.
While the IT development effort can be high depending on the tool they use, the payback is also high because IT will not be burdened with daily requests for ad-hoc reports. In many (most?) organizations, the effort to provide ad-hoc reports to the end user consumes more time that the effort needed to maintain the data and BI applications. Usually a backlog of ad-hoc requests is created that prevents IT from providing the user with the information they need when they need it. This is truly a lose-lose situation, no one is happy and of course IT is fall guy. You must dump ad-hoc reporting on the end user.
The tools to support this type of self service BI vary. The user needs to be able to easily select the data they want, delete columns they do not want and save the report format for re-use. In this scenario I have no problem letting the user create a spread sheet of the data they have selected and use the spread sheet for their advanced analytics. The reason for this is that many people who do their own analytics already know how to navigate and build spread sheets, their problem is getting the correct data in the spread sheet.
One recommendation is to not give the users one humongous report and let them select from all the data in the universe. This is tedious and they will come back at you with ad-hoc requests. Create the base reports by subject matter. For example, if you are presenting sales, only present the 10 or 15 data elements that effect sales.
Self Service BI to the power user
The power user is an entirely different animal. These are forward thinking people such as strategic business planners, market analysts and product developers. Their primary requirement is “give me a pile of data and a stick I poke it with to see what falls out”.
The power user’s tools will, in many cases, be different from the casual user. They will want to build reports and charts from scratch. Their main dependence on IT is data. IT must provide them with current, accurate data in a format they can easily manage.
Data governance can be a problem with these users as they may distort data inadvertently as they create their applications. But at the same time, they can lift some of the support burden from IT’s shoulders. Consider power users allies not adversaries.
One problem with power users is once they fall in love with a tool, moving to a new one is very traumatic (breakups can be difficult). They will come to your organization with their own tool bag. You may run into the situation where different power users in the same department want different tools. This can escalate into a very sensitive topic and needs to be handled with the utmost diplomacy. The problem with allowing multiple tools is the inability to share information and could result in silos of information that are never properly reconciled to insure their validity.
Another problem with this type of solution is the potential for the same business term (analytic) to mean two different things. I learned a valuable lesson on the first data warehouse I worked on and that is “business terms do not mean the same to all people in the same business”. We spent 3 months nailing down what a sale was. It may sound trivial, but people can make wrong decisions if they don’t understand what makes up a specific value or analytic.
Also it is possible that the power user may not conform to enterprise defined KPIs. It is ok if the user wants to create new ways to measure company successes, but if he changes the analytics behind a KPI, he must either rename it or make the change available to everyone so they are all on the same page.
Where does IT fit into Self Service BI?
The Information Technology department is what makes this all happen. There must be a complete buy in by IT. They must also realize that their role is to serve the end user. They may provide some guidance to the casual user but will probably need a hands off approach to the power user. If they adopt the attitude of a responsive data servant they can be very popular with the client community. Listen to the user and provide them with as much as you can.
They will need to balance infrastructure costs with client requirements. Charge back helps but too much emphasis on the cost can subvert the dissemination of BI throughout the enterprise.
To be successful, it is my belief that IT must develop agile development methodologies, create agile architectures and begin using agile development tools. They must assume that there will be a requirement for new data every day and architect their data processes to support it. This is not a trivial effort. The new data may be a new data element from an operational system that needs to be added to the warehouse or mart, it may be a RSS feed, it may be a spreadsheet maintained by the CEO’s office, it may be literally anything and you will have little time to get it to the end user. You will not be successful with this day one, but you must keep this goal in mind as you architect your ETL, data warehouse, data marts, etc.
A huge challenge will be to eliminate reporting chaos. It is conceivable that with self service BI, everyone can create their own report to tell them the same information. You could wind up with millions of reports for a few users. IT will need to implement tools and procedures that will identify the data on reports, the frequency they are used and how people are using them to effectively manage the environment.
What’s in the tool box?
I have avoided mentioning tools and am not going to recommend any now, mainly because you need to find what fits your needs. I’ll be more than happy to help you do that. Here at Datavibes, we have relationships and a knowledge base that reflects all of the well known vendors and some of the lesser known.
Since most people today can search the Web, I believe that to make Self Service BI a universal reality, the tool manufacturers must implement similar functionality so the user can quickly extract real and accurate information in an intuitive manner. No cryptic formats and convoluted expressions.
This is the holy grail. Anything less will require too much support by the IT department and require too much education to the end user.
Illusion or reality?
The answer lies in your expectation. If you think any person can jump on the system and request any information, then it is probably an illusion. The reality is, Self Service BI is available within established guidelines and can enhance peoples work experience and improve the enterprise’s performance. But today, it requires foresight and a lot of effort from IT to make it happen.
