I recently read an article published by Gartner (Gartner Says Megavendors Are Slowing Business Intelligence Revenue Growth but Increasing Usage by Bundling Applications) which I think highlights a serious hidden impact of BI industry trends that will have serious unintended consequences for many companies.

“Bundling” of BI with other enterprise apps, together with the natural downward pricing pressure that bundling creates, would generally be good for promoting a competitive market. But that is not the practical effect of “megavendor” bundling behavior. Instead, it may in fact cause companies to make decisions which end up being significantly more expensive in the medium/long term (either via higher implementation or missed opportunity costs).

On closer inspection, the actions of these large vendors is not to promote greater penetration of BI, it is generally a defensive move to curtail the encroachment on their market from independent alternative products, or on their core enterprise apps market by other megavendors. When an SAP or Oracle, now portraying themselves as major purveyors of BI, are challenged by the market regarding their objectivity vis-a-vis what used to be independent technologies, they are forced to act in a way that does anything possible to prevent the attrition of their customer base by taking these types of actions to stall the natural evolution of the market (and technology).

The best test of a technology, and catalyst for its positive advancement, is via the competitive pressures of an objective market. “Bundling” sales gimmicks described in the Gartner paper obfuscate the true costs of BI investment, as customers settle for technologies which may not exploit the best advances available in the market. In this situation, it is the customer that ultimately loses – either through higher direct services costs or missed opportunity to deliver a project that yields a real competitive advantage.

Yesterday, LucidEra, a pioneering vendor in the SaaS-based BI space, announced that it will be winding down its operations by the end of the month. In light of that announcement, it’s interesting to consider what it will really take for the on-demand, or SaaS-based, BI market to take off.

Conceptually, the idea of SaaS-based BI is a good one. It takes advantage cost-effective, low maintenance storage and compute capacity available in the cloud. It enables organizations to begin developing BI solutions without significant up-front investment. It promises superior ease-of-use, and it eliminates reliance on IT for solution delivery.

Other enterprise applications (Salesforce.com is the go-to example) have made it successfully to the cloud. What is holding BI back? In other words, what will it take for SaaS-based BI to be ready for prime time?

The way I see it, SaaS-based BI faces two major hurdles:

#1… SaaS-Based BI Hasn’t Solved the Data Integration Problem. The biggest challenge that users of SaaS-based BI face is how to get their data into the cloud. Most SaaS-based BI vendors support a simple flat file upload. Others offer primitive batch upload from a source RDBMS. These are fine options if all your data is in one nice, neat data set and that data set rarely changes or gets updated. But that’s rarely the case in the real world. Today’s SaaS-based solutions make unrealistic assumptions about the quality of the data that gets uploaded. There is no capability to do even simple data transforms or logical data mapping to establish data relationships. It is simply required that all fields across disparate sources to share the same name – a completely unrealistic assumption in the real world. There is also no recognition of the need to map data from multiple sources into one canonical data event for consolidated reporting and analysis purposes.

Because data loading is either done manually or, at best, in a scheduled batch, users find they are moving large amounts of data into the cloud unnecessarily and redundantly. Load times for large data sets are unavoidably slow. And there is no way to support any kind of low latency or real-time BI.

Suprisingly, these solutions have also failed to make it easy to integrate data in the cloud with data from on-premise apps. What should be an area of opportunity for on-demand BI solutions is in actuality a weakness.

And of course there are naturally issues around data security that are intrinsic to any cloud-based implementation. Are you really comfortable moving essential corporate data into the cloud? Even if you are already using other on-demand applications, there will likely always some corporate data you don’t want in the cloud under any circumstances. How do you govern and control this? Again, the simplistic tools offered to date rely on the user to solve the problems of redaction and other data reduction techniques to ensure only the data which is appropriate to cross the firewall is actually allowed in the cloud.

These are difficult and fundamental problems to solve. Solving them requires overcoming fundamental architectural hurdles. And even if that can be achieved, there is still a significant amount of design and development work that needs to happen from a user interface perspective to support the complex data transformation and integration challenges that are inherent to any serious BI solution.

#2… The End User Experience is Lacking. Right now, SaaS-based BI solutions deliver only very primitive data visualization and analysis features. Most fall far short of delivering highly interactive dashboards, with alerts, multi-path drill-downs, real-time data refresh, maps, synchronized component behavior, and other rich features. Eventually, they will bridge this gap. This functionality, is of course, all browser-based, so delivery is not an issue. Development is the barrier. Most traditional BI tools require a heavy client to build these rich dashboards – something that won’t work for a SaaS-based solution. But with enough time and effort, it is possible to design even the richest, most interactive dashboards in a browser. Altosoft, in fact, offers 100% browser-based dashboard and report development.

Customers seem to implicitly recognize these limitations. It’s true that many (typically smaller) organizations are playing with BI in the cloud. But adoption for serious BI projects, especially at major organizations, is negligible, or nonexistent. To overcome this the market will need to respond with a fresh approach to linking disparate systems and data sources behind the firewall with cloud-based deployment architectures making sure that the cloud-based tools (e.g. the visualization and analysis tools) meet or exceed those of their on-premise competitors.

One day, inevitably, the benefits of SaaS-based business intelligence will be realized. And the value will be real. But that day is not yet here. And it won’t get here by setting the bar so low that you can trip and still make it over. The tipping point will be when vendors offer solutions that raise the bar beyond the current on-premise offerings – not use the challenges of the cloud to excuse a lack of innovation.

You have to admire the marketing audacity of our newest partner, Netezza. They seem to relish attacking the big “megavendors” in their space. They do it tenaciously, but also intelligently and often humorously. One of their latest efforts is case in point. The weblog-based site hits the nail on the head. Traditional solutions have failed to solve the business problem of getting the right information to the right decision maker at the right time. It’s almost unbelievable that after twenty, maybe even thirty years of business intelligence technology, organizations are sometimes still struggling to cope with even the most basic reporting requirements.

Netezza’s aggressiveness is understandable given their position. They offer a superior solution. But they’re the little guy – certainly an increasingly large little guy, but still small compared to the Oracles and IBMs of the world. If Netezza doesn’t challenge the megavendors, how are they going to get a prospective customer’s attention? If they don’t get the customer’s attention, how will the customer find out how much better off they could be with Netezza?

IT buyers – particularly in the BI space – have been fed a steady diet of false expectations and overinflated promises over the years. They are fed up to the point where any vendor’s claim is met with apathy. Of course, this plays right into the hands of the largest vendors. Customer apathy is what they want. It stifles innovation and keeps users dependent, masking the failure of these vendors to innovate and solve the fundamental problems the industry faces.

A mutual partner of Netezza and Altosoft recently remarked to me that the two companies are in a similar position. Both have innovated to offer significantly superior — faster, easier, and more cost effective – solutions. Both have the challenge of getting the word out above the steady (and deliberate) noise created by the large vendors in the space. Netezza has shown the effectiveness of an aggressive, but intelligent, approach. They set an excellent example.

It always surprises me how many buyers choose not to demand a POC from their vendor. I speculate there are a couple reasons for this:

1. Buyers think that POCs consume too much time and effort. They consume internal resources that just aren’t readily available; and
2. Buyers have conducted POCs in the past and have found that real product implementations have little relation to a vendor’s POC performance. So they dismiss the value of the POC.

To a certain extent, prospective buyers are right to think this way – with every POC there is both cost and risk to the buyer. That makes it all too easy for buyers to either talk themselves out of running a POC entirely, get convinced by a lazy vendor that a POC isn’t necessary, or run a halfhearted, token POC that gives the prospective vendor an easy layup win, but creates no value for the buyer.

Done right, a POC is invaluable. Rather than a source of cost and risk, it should be the ultimate cost saver and risk mitigator. Properly executed, it’s by far the best way find out whether a vendor’s sales claims are legitimate. Not only that, but there are other clear-cut benefits (quantifiable value) that you get from a POC:

1. A POC is a free, personalized product training session, underwritten by the vendor’s sales budget.
2. A successful POC should yield an impressive deliverable that can be used internally to gain support for an essential project.

If you are in the market for a business intelligence or data integration solution, it is a best practice to always run a proof of concept (POC) with your prospective vendor.

The trick is to constrain the POC effectively so your internal cost is limited, and to keep a sharp eye out for vendor sleight of hand and obfuscation — what my Irish grandmother would have called “shenanigans”.

AVOID VENDOR SHENANIGANS

How can you avoid shenanigans and get the most out of your POC? Here are a few helpful suggestions:

1. Insist that size doesn’t matter. If your vendor isn’t willing to do a POC because you, or your project, is deemed “too small” of an opportunity, it should tell you a couple things. First, it reveals how that vendor is likely to treat you in the future – as a low priority. Be wary. Second, it should make you suspicious that their solution isn’t as easy to use as they claim – because if it were really that easy, the vendor would have minimal downside (cost) and should be willing to do the POC.
2. Make the POC as real as possible. This is essential! If possible, test the vendor against your real data sources. Do not make it easy for the vendor by moving all your data in advance to a nice staging database with easy access! If it isn’t realistic to work directly against the data sources in the POC, do your best to come up with an environment that at least replicates the complexity of the real one.
   • Make sure you have more than one data source ready to go. Your project will probably will require analyzing data from more than one source. So should your POC. Most BI vendors can handle a single data source easily. Bringing together data from multiple sources gives them trouble. Test them.
3. Ask to see the queries. Make sure you get a good look at the queries they are running against your data.
   • How complex are the queries? How efficient? Are the queries optimized to minimize the burden on your source systems? Or will they need to rely on those systems to perform joins, aggregations, and calculations?
   • Does their architecture effectively isolate the source systems from the burden of identical, repetitive queries? Or will your production systems get hit with a new query every time a user refreshes a dashboard?
   • How much control do you have over when and how often queries run?
4. Take control. Make sure you drive the product installation, configuration, and solution design. Force the vendor to give you control of the mouse. Remember, it’s a learning experience for you. The best way to maximize it is to get hands on. Before the POC starts, you may want to work remotely with the vendor to do a clean product installation on your test machine to get things ready. There’s nothing wrong with that so long as you know what is involved in the install. As for the configuration and design – make sure you take ownership. That’s the only way to be sure you don’t fall victim to vendor tricks and clever shortcuts.
5. Ask questions. Lots of questions. Your best opportunity to get a straight answer is when you have control of the mouse.
6. Be prepared. Know the key metrics you want to measure in advance. Have a good understanding of how the metrics and data you want to see in dashboards and reports relate to your chosen data sources. But whatever you do, do not deliver a list of KPIs or a dashboard or report mockup to your vendor in advance. Make them work from a clean install! Don’t let them do any prep work. If you do, you open the door to vendor shenanigans.
7. Pay attention to the deployment process. Make sure you understand what it takes to move from product configuration into production. This will tell you a lot about the product’s architecture and ease-of-use.
8. Force the vendor to make changes. When you’re finally looking at pretty, interactive dashboards or reports, ask the vendor to start making changes. Because change is inevitable, you need to know what is involved.
   • Tell the vendor to add a new data source and join that data with existing data sources.
   • Have them add new dimensions.
   • Ask them to create new metrics, and change the way that current metrics are calculated.
   • Request that they change the manner in which data is visualized or presented to end users.
9. Operate on a timeline. At Altosoft, we typically do a one day POC. Sometimes two. Other vendors probably require more time. Ask your vendor how much time they need, negotiate this with them up front, and hold them to completing the POC in the requisite time frame!
10. Be prepared. A well designed POC should challenge your vendor. But you also need to hold up your part of the bargain. Be prepared for the POC. Have your test environment ready to go when your vendor arrives. If the vendor has provided a specification, make sure it is configured accordingly. And make sure that you are ready with the correct authentication to systems and data sources involved in the POC.

Now, here is what you absolutely should not do:

1. Don’t get suckered. The vendor may suggest that you export your data and send it to them before the POC so they can prepare. Don’t do it! Many vendors will take your data back to their lab, massage it, optimize it, have a team of experts build their solution in advance, and then script out the POC presentation like it’s a Broadway production. Classic vendor shenanigans!! Don’t allow it. Make the vendor work from real data, in your real systems, with no advance preview. Better yet, grab the mouse from the vendor and have them talk you through every step. No possibility of shenanigans there!

At Altosoft, we do a lot of POCs. After all, there’s not much downside to us. It’s easy to offer customers a free POC when you have confidence in your product, and you can use the product to create a great deal of value in a very short time. That’s what Altosoft is all about.

We typically offer a standard one-day POC session. The POC typically includes product install, connection to multiple data sources, data mapping, metrics definition, and development of a few dashboards and reports. We let the customer drive the process. It’s tight, well constrained, and people find it very productive.

THE VIRTUAL POC

But recently we’ve created a new POC variant variant that is faster and, for some customer prospects, even more exciting. We call this a “virtual POC”. Basically, it’s a POC that we deliver remotely, using a web meeting format. Here’s how it works:

We start off by helping you through the install process. This takes place prior to the actual POC web meeting. Installation is generally fast and easy.

Once you’re installed the product, we kick off the web meeting. Basically, over the next two hours, we talk you through the process of connecting to data sources, mapping data, defining metrics, setting up user permissions, deploying the solution, designing dashboards and reports, and finally visualizing and analyzing your data. It’s a fast, completely repeatable process. And since you’re driving, you get a great sense of how the product works. Of course, if you prefer, you can also hand control over to us for portions of the POC.

At the end of the process, we leave you with a working system. Not only that, but you have the knowledge and skills to extend it as you see fit. You can play with the system to your heart’s content.

The virtual POC is faster than the traditional POC, so the downside is that our prospects don’t get quite as much face-to-face training time. But it gives buyers a chance to get hands on with the product and experience how fast and easy delivering serious BI solutions can be with Altosoft.

Looks Good

Altosoft typically delivers in our free, one day (sometimes two day) POC. I figured I’d take the opportunity to provide a bit more detail here.

First of all, a lot depends on how prepared a prospective buyer is going into the POC. If the buyer is well prepared and understands what they want to measure, and is familiar with the data sources that will support those measurements, then we will accomplish a great deal in a short period of time. Our solutions are built for speed and efficiency. In other words, if you know what you want, we enable you to get there very, very quickly.

And everything we do is completely transparent to the customer. We will never do a portion of the work off site. The buyer sees everything, and learns throughout the process.

In the end, every POC is different. But there are some common characteristics of just about all our one or two day POCs.

Here’s what we do do.

First, we install the product. This is a fast, easy process assuming that the prospective buyer is ready with a properly configured machine.

Next, we connect to data sources. In just about every POC, we connect to multiple data sources. Typically, we deal with two or three relational databases (or existing warehouses), plus we like to show how we can easily incorporate Excel as a data source as well. Since dealing with multiple sources is one of the strong suits of our product, we consider showing this functionality to be important to POC success.

The next step is event mapping and metrics definition. A typical POC involves defining multiple “business events”. These are basically the key business activities that you need to monitor, analyze, and report on. For example, a “new order”, a “product shipment” or a “client approval” could all be business events. Business events can (and often do) incorporate data from our multiple sources. Business events then spawn multiple KPI metrics. For example, “number of new orders”, “discounted dollar value of new orders”, etc.

So multiple events (typically 2-5) create a number of separate KPI metrics (typically 5-10). Each metric can then be analyzed across multiple dimensions. For example, you could analyze “discounted dollar value of new orders” across multiple dimensions (e.g. region/country/city).

The reason we don’t do more events and KPIs is simple – time. In almost all POCs, the actual time we have for development is 2-3 hours at the most. The rest of the time is spent on discussions, explaining to the users how the product works, learning the data, inventing KPIs, trying different visualization options, presenting the results to the business users, etc. If you want to see more KPIs, we can run into a second day…. But for the most part, once you’ve seen us build a few KPIs, you’ll have enough of a handle on it that adding more would be overkill. In fact, you should be able to build them yourself without too much trouble.

Once we’ve built the KPI metrics, we determine who is allowed to see the data. We demonstrate how to set up multiple users or roles, with privileges based on functional access (e.g. right to define KPIs, right to create alerts), privileges to view data, and access to data dimensions.

At this point, we deploy the system and start building our dashboards and/or reports. Typically, in a POC we focus on dashboard development, although we can also demonstrate how to develop, schedule, and distribute a report as well. But dashboards are generally more interesting.

We typically build out a number of dashboards that display our KPI metrics in an organized, informative way. All the dashboard components are interactive, drillable, sortable, etc. We can also link components together, so that you can refresh an entire dashboard based on filtering by a certain dimension. Naturally, we’ll also add your branding into the dashboard to give it a personalized look and feel.

On top of the dashboard, we’ll define a rules-based alert and show you how alerts can be used to automatically monitor new business activities as they occur. We set up the data refresh rate based on your preferences. For example, if you want to monitor changes on a minute-by-minute basis, we can accommodate that. If new data triggers an alert, you can use the incident management functionality within the system to resolve the problem.

So to summarize, in a typical one/two day Altosoft POC you’ll see:

• A solution that joins data from multiple data sources (RDBMS, warehouse, Excel).
• A variety (5-10) of unique metrics that measure your business performance.
• Metrics all have multiple dimensional breakdowns for analysis.
• Demonstration of multiple user roles.
• Near real-time data, data source permitting. (Real-time data is only available for event-based sources.)
• A sample report.
• Multiple interactive web dashboards featuring a variety of ways to visualize data (charts, graphs, maps, etc.).
• Automated alerting based on your rules.
• Workflow-based incident management.
• Your branding.

And we’ll leave the solution with you so you can continue to work with it, show it, even add to it. Since we’re very willing to let you control the mouse during our POC, you will hopefully feel very comfortable with our technology by the end of the day. Because that’s really the goal – to position you to achieve success with the product, without needing to have us around to hold your hand.

Naturally, if you require a more elaborate POC deliverable, we can handle that as well.

Now, what don’t we do?

Typically, we don’t do SharePoint integration. Although if we’re given a few extra hours and the right access to your SharePoint server, we can make it happen. Basically, every dashboard and every component (chart, graph, etc.) that we’ve built with you will be accessible natively through SharePoint as web parts. You can easily use them to build SharePoint composites.

We also don’t typically do business process intelligence. We don’t feel too bad about not offering this as part of the one day POC, because our competition can’t handle process analysis at all. If you really want to dig into our unique ProcessMart capability, we can discuss extending the POC for an extra day.

So as you can see, it’s possible to accomplish quite a bit in just a few hours! This shouldn’t be too surprising given that many of our actual full projects take just a week or two.