An Ounce of Prevention: Forging Healthy BI

Rebecca Jozwiak: Ladies and gentlemen, hello and welcome to Hot Technologies of 2016! Today’s title: “An Ounce of Prevention: Forging Healthy BI.” I am your stand-in host today, Rebecca Jozwiak; Eric Kavanagh had an appointment today, he couldn’t make it for our last webcast of the year. Hit me up on Twitter, I will try to keep up and multitask @RebeccaJozwiak.

This year is hot, or maybe I should say that this year has been hot. We’ve done a hundred webcasts I think this year and on this program alone, we’ve covered everything from high availability database, data modeling, mainframe integration – that one was pretty cool – security, streaming analytics, editing analytics, embedding analytics, data catalogs, you name it. And we’ve probably covered it in some way, shape or form at some time.

And today’s topic, you know BI, you might be asking yourself, “What’s so hot and interesting about BI?” Well you know there are a ton of different components and processes that feed business intelligence. And as we get new data forces – as we get, you know, disparate offsite force systems – we got to find a way to bring that together as fast and seamlessly as possible. And any hiccup in performance is going to impact the end user’s experience. And that’s definitely not something you want in your organization.

So, with us today, we have our own chief analyst, Dr. Robin Bloor; we have our data scientist calling in from Australia, Dez Blanchfield; and we have Stan Geiger from IDERA. Now IDERA’s done quite a few programs with us this year – thank you to IDERA – and this is Stan’s first. So, welcome, Stan.

And with that I’m going to pass the ball to Robin Bloor.

Robin Bloor: OK, well thank you for the ball. Monitoring BI, I figured that I would actually talk around the BI topic. It’s been the last couple of years, have really been the year of analytics and therefore to a certain extent, BI has taken a little bit of a back seat as regards to exposure. But in actual fact, BI always was – and to a certain extent continues to be – the main thread of the back systems of an organization, which is the way that I think of it. BI is the back loop for corporate systems.

This diagram, which we have used for about three or four years now, just the idea that pretty much all of reporting in BI, actually comes out of the activity of insight, which is analytical investigation of data. This kind of represents, you know, from the insight comes stuff predicts analytics which is foresight, the hindsight which is reporting and the oversight stuff that you can think of as dashboard.

The reason I add optimization here, is that optimization is a very special area of analytics, and one incidentally where mathematics does not solve the problem entirely. The begetting of BI. The desire for knowledge begets user requests various IT capabilities. User requests beget analytics projects. Analytics projects beget data lakes. Data lakes plus analytics beget insights. Insights beget BI. And BI is normally implemented as a piece of software sitting over a database.

If you think of a full BI platform – this is by the way much simplified – you realize it’s a very complicated activity. You will or you may get external data going through staging area or internal data feeding the staging area you get the activities of the governance of the data, data cleansing and ingest activities in order to serve it up. The exploitation of the data is basically reporting analytics depending on what you are trying to achieve. And a lot of the analytics actually lead to action.

And if you actually look at the list down the right-hand of the slide, when you think that each one of these things isn’t necessarily just a capability, it’s actually a product, they are products, several products, maybe quite a few of products in pretty much every area mentioned. You’ve got alerts, reports, forecasting, dashboards, score cards, performance management, OLAP, visualization, attractive visualization. BI portals put together multitudes of these things. Animation: data animation can reveal things that you can’t sell in any other way. The data exploration, which is more analytics than interactive exploration. And data mining, and text mining and video mining are all kind of analytics. What’s called predictive analytics nowadays, is in my opinion, become BI.

And this is one of those things that happens, and it’s kind of, if you like, consequence, of you know, you investigate things and then you discover patterns and then you realize that certain patterns are regular and then you, in a sense, in some kind of reporting capability, and therefore it becomes BI. So you know about three or four years ago people were saying that predictive analytics – only a few companies invested in that, whereas, now days, there is a fair amount of predictive analytics which has just become a part of the reporting function or even automated decision-making function that takes place in some operational systems.

And you know with data stream we have the real-time analytics time series geoanalytic. And all of this machine running cognitive computing and various software developments are all really more part of analytics. It’s a huge area, when you kind of think about it. You can look at it in a different way and this is just, you know, to divide between two specific activities. BI activity, which is some kind of reporting going on towards various issues and various departments. The analytical activity, which is in really in my opinion, it’s corporate R&D as regards data.

If you actually look at this, there’s the operational apps and the office apps, work flowing. They will be passing information to BI apps or analytical apps, data flow management. And then there’s all these data flows and data sources from the outside. It’s a very complicated operation and that’s just represented in overview.

BI disruption is worth mentioning the, you know what’s happened in the past few years. You’ve seen data volumes go up, a number of data sources: streaming’s become a reality; the introduction of unstructured data; social data, which tends actually to be unclean data; IoT data; data provenance; compute power; power of parallelism; machine learning; new analytic workloads – all of this stuff has actually been very disruptive in the area of BI. It’s disruptive in the sense that, you know, the older technologies have not necessarily been able to take all this stuff on board. It’s not disruptive, it’s actually enhancing in the sense that more and more can be delivered by BI. BI is basically not a static situation, is what that slide is saying.

And also worth emphasizing – but is not much being said here – that the internet of things, you’ve been driven architectures in real-time everything. A part of the future of BI landscape, and this is just to emphasize, BI isn’t going away. BI is probably getting bigger and bigger, and becoming an important parcel of more and more of the activity of an organization.

So, BI of the user, by the user, for the user – that’s the whole point of BI. The issues in summary that need to be addressed in terms of having what you would call a coherent BI environment – I think I’ve listed them all here: data flow integration automation, performance timeliness, data coverage, data forces, also the structured/unstructured data split, data cleansing, data access skills, knowing how to get to the stuff and use it, visualization, sharability and actionability. And that’s kind of a summary overview. The point that I may as well make, because that’s the point of the whole presentation: unless the BI service is dependable and timely, it isn’t a service, so that’s why we have BI on this ring.

I shall now pass the ball to Dez.

Dez Blanchfield: Thank you Robin, always a tough act to follow. While last show of the year, and doing hundreds of these things, this should be exciting. So my take on this is, as Robin analyzed, the general topic of business intelligence has matured dramatically, significantly; it’s a whole new ballgame.

Once upon a time, BI was one of the core systems that we might use, in that you know we would have a finance system, we’d have an HR system and arrange its silos. These days, business intelligence often is an umbrella that goes across them all and gives us a single pane of glass to view the world with. This is not your parents’ BI; this is something that’s grown up a lot in the last decade, particularly the last three to five years. And more so because of the dramatic improvements in infrastructure and availability, the advent of cloud, the understanding [inaudible], analytics, big data.

These are things that have just made it possible to do new, amazing things with data in general, gaining access to data invisibility to data sources and presenting them in a way that we hadn’t really imagined before. Often, I feel like we kind of sort of gone from spreadsheeting to this new magic that we haven’t defined yet and continues to grow. My point of this is that the modern BI platforms have really complex moving parts. And to that point, we’re not now just using a single database platform to see one BI engine. That BI engine is often looking at lots and lots of different components whether it’s manual ingesting on a regular basis through batches of data or real-time feeds into other systems.

It’s more than just keeping the lights on, is my key point with this particular part of my discussion. You know, once upon a time just keeping a BI platform up and running and making it available was adequate. These days that isn’t really the case, and it’s important to be able to not just keep the complex parts moving, but also actually make them form really well. Because as Robin highlighted, a BI platform that isn’t forming well is going to impact a great deal and I’m going to get into that a little bit more.

The types of things we need to be thinking about, when we think about modern business intelligence platforms, is we’re not just keeping the database going, not just keeping the software platform available, it’s reachable by end users and can print reports. It’s about the performance availability, you know, can I reach it and is it performing quickly? All the way through things like, you know, what’s going into the BI platform, the core of the data and the data sources. Not just what’s in the database, it’s seeing a BI platform, but what are the other sources of data that are available to it?

The underlying database platforms themselves are a natural given, but what about database warehousing environments? The enterprise data warehouse platforms that we are looking at, are they available, are they performing well? You may have a standalone environment for your BI to protect it and control it and secure it, and it forms very well, but what if all the environments around it are not working well?

Particularly now, we’ve got BYOD, mobility, you know, people aren’t just sitting in front of PC desktops anymore, they’re doing it when they’re out on the road. These days when we think about the types of things we’re doing with the BI platform, it’s real time, it isn’t just a kid sitting down at the end of quarter or the end of the month or end of the week and writing up reports and copying those on the clipboard and running around. We’re doing it real time. The transition from desktop PCs to laptops to tablets or iPad and smartphones, you know we’ve got it in our hands, it’s real time. If we’ve got a question about what’s going on in an organization, we can ask it immediately.

And more often than not we’re prompting ourselves, getting push alerts on what’s happening, particularly if it’s a, whether it’s in sales and HR logistics, whether it’s a designer ad, R&D departments, whether it’s in development. You know you even have software development teams that are now getting real-time visibility of what’s happening in the organization and acting in a more agile format to get things out the door quickly.

If you know if you’ve got a large end-of-the-year Christmas sale coming, for example, there might be a new feature that someone’s asked for and the development team’s being required to push it out the door. And they’re watching the BI platform, all that’s coming through, seeing how it’s performing and what the business is looking like in real time. Then there’s the security platform and the governances around it. There’s all the usual things like control and user access and system access. And you know, even just environmental things like, are people having to log in, is it responding quickly enough?

And then the user interface platform. You know, these days things are often, not just some local client, and there’s a web interface; people come from different access points. They might be using a desktop PC with an old graphic user interface, or they might be using a tablet or mobile device or a phone that’s using a web interface and are each of those presentation lines working? The various analytic platforms that are connected to the tools that we are using. The BI platform might provide some functionality. Or they might use other tools to do analytics in some of the data that’s coming out of the BI platform.

And then the basic reporting and monitoring of dashboard tools, which again may be native to the BI platform, they may be fitting from the BI platform. These are really complex things to think about and that’s why, when I talk to people, I keep saying that this is not BI as you used to know it: this is a whole new generation of BI, and there are some really big challenges that come with it.

So, the challenges of managing these BI platforms, particularly at scale now, you know once upon a time it would be just a stamp on a system, as I mentioned. Now they’re very large and complex systems, they’re lots of moving parts. And it’s not just a case of one subject matter expert managing the BI platform or one database administrator system administrating. Often they’re very large, complex teams where they have platforms. And particularly because now they’re using it more heavily, we demand performance from our platforms.

And in my view, and I think many people agree with me, business intelligence platforms are often the lifeblood of the organization, and that is if we have a performance hit of some form, that performance hit will impact the bottom line, so much so, that organizations are now often, in the same way if we can’t get an email we can’t communicate, if our phones are out we can’t talk to inter office and so forth or between states and regions or internationally. If we lose the BI platform or if it’s not performing very well, in effect it means we’re flying blind. And that’s a real risk to any organization.

And if we end up with performance issues, we end up with people just, it impacts adoption. People aren’t going to use a tool; they’ll go back to pen and paper and fundamental things. They are parting with bad data or bad information. Overall I think these issues impact all kinds of things, particularly the decision making company wide: the ability to make accurate, timely decisions. These days we’re making data-driven decisions around the organizations, as you’ve heard the coinage, and that is a reality. It’s driven by a business intelligence platform so we can make decisions at all of our organizations at any scale, at any size, at any time, based on the data we have available, so data-driven decisions come out of BI platforms and ability to get them to see time and information.

The reporting on it, for example – whether it’s daily, hourly, weekly or monthly – it’s really important that platform is responding quickly enough to provide timely reporting. Everything that runs in the organization on reporting, if it’s not running quickly, you know we don’t run a batch job that takes a week anymore, we want to see it in real time. And so that impacts our real capacity to run the business, you know, the fundamental reason we all turn up at 9:00 o’clock on Monday morning, or these days 24/7.

And then there’s some other things that flow out of it, which often people don’t think about it, because we often think about it from a commercial or from technical point of view. The general staff performance and morale, the people’s ability to do their job, the reason they are willing to jump out of bed in the morning and come to work is often driven by their access and availability and the performance of the BI platforms.

Not often thought about, but one of the things I see in KPI across organizations now that I’m working with, is that we put key performance indicators against all the elements that drive business intelligence platforms. Because if any of those KPIs start to fail or hurt, we can directly relate that correlates that against morale and performance of staff in general and that underpins the ability for the business to operate. And then there’s some issues around compliance and governance. Can we report all the way through the organization how we are going across risk, across governance, across compliance, you know, are we doing the right thing right now at any point in time? How do we know that, how do we prove that?

If we think about some of these things, the CIO has a lot of responsibilities in that space alone. And you got CFOs managing the finance component, you’ve got CMOs managing marketing and other activities around their real time. We’ve got new roles, like CEOs becoming a thing and getting a seat in the boardroom. Chief data officer, we’ve got risk officers, a chief risk officer, often the CRO had a particular focus on compliance, governance and risk. Now they’re thinking about more in data terms. We’ve got new roles, like one I’ve seen recently called a CAO, a chief analytics officer, all from actuary clear through analytics and data analytics thinking about what’s going in the BI tools, what’s coming out of it, what it looks like, is it accurate, is it performing well?

You know, when we think about it, that very broad brush stroke, when we look at it, we realize that any particular single performance issue across our entire ecosystem can impact almost every other part of it. It’s kind of like a house of cards, in many ways; people don’t like discarding that. If you pull one out, the rest of the thing, it will collapse. It’s like a domino effect: you knock one over, it will push everything else over. It’s very important your entire BI platform remains stable, secure and online at all times.

So to summarize, some of the things that I see around the place that can impact performance and leading to our frantic idea talking about a big platform and where it addresses these things. In many ways this brings me to my favorite one-liner, the Donald Rumsfeld conundrum here, and that is, “There are no unknowns or no unknowns and there are often unknown unknowns.” Things that can impact the core performance of the BI platforms. Whether it’s a sales platform or it’s a marketing, finance, human resource, operations, logistics, planning, forecasting, reporting – all these core functions of the business that we take for granted, do invariably pull information reporting and real-time visibility from the business intelligence platforms. Some part of it in some way, whether it’s in operations, whether it’s planning, whether it’s design, whether it’s strategy, whether it’s historical reporting, current real-time reporting, or predictive in the future over the horizon, crystal ball gazing.

Any of those back-end platforms, any of the front end, if the response time is slow, or the load averages in the platform start to get hurt for some reason. Particularly around the end of the year, Christmas is a classic for that, which is timely, given the time that we are in now. Whether it’s a particular workload that’s running, whether it’s batches that are being run, or it’s a backup that’s being run somewhere, whether we’re doing maintenance or upgrades of patches of some security fix, particularly now we’re worrying about security more and more because of the risk of data breach and data loss, and so forth, given that this is a real hot topic in any industry these days. Particularly social platforms and email platforms losing billions of accounts, which is becoming a reality. General system administration, someone deciding that they are flushing out some logs or deleting some unused data.

You know, running some of things ad hoc often can generate workload that has a small impact. Or maybe it’s just an event where you’re merging acquisitions or ingesting some new data, particularly now that we’re bringing data sources from outside the business. Someone doing a manual data import, sometimes that generates a real load in the back end that we haven’t accounted for and everything else slows down. Maybe we’re exporting data for some particular reason, maybe we’re doing something as simple as a mail run and we’re dumping data to provide to a mailing house. Again, these are things that often don’t get through get run and have a real impact on the business. And so the trick here is bringing tools so that the data can monitor that and keep track of it.

And so, to that end, my overall view of the world is that these complex systems require smart tools to manage that performance. And I believe we’re about to hear a really great story that way to approach that and particular talk for that, and with that in mind, I’m going to hand it to our friend at IDERA, take it away, let’s hear what you got.

Stan Geiger: Alright, let me share my screen real quick here. So here at IDERA, as you may or may not know, we develop tools for monitoring several administrative-type tools: everything from monitoring to backing up. Basically, what we’ve got is a tool called Business Intelligence Manager that is responsible for managing the health and the performance and the availability of your Microsoft BI stack. That’s what we’re going to talk about today and build off what Dez was talking about.

I like to put this up, this is kind of the old-school view of business intelligence. I mean a lot of this is still adequate today. It’s just expanded with things like the internet of things and now we’ve got NoSQL databases out there and Hadoop file systems where we’re storing unstructured data, and you know, that basically, you start talking about those being your data sources for business intelligence aggregations, if you want to call them.

And here in the middle is where we have, we might have ETL processes where we go out and pull data from all these disparate sources and we put them in data stores that we then can do business intelligence operations, things like the data scientist can run an R script against the data doing analysis, visualization, things like Microsoft Power BI, you’ve got products like Tableau, things like that, that can sit on top of this data. And you know those things are basically on the presentation side of this. They consume the data. We’re talking about the platform level and it all starts with, you know, kind of your data stores.

And as Dez talked about, you know, performance of those is critical to being able to run your business now. It used to be that you know your products or whatever you sold was your key component. Now in the competitive landscape it seems like how well you use the information you have at your hand to make quicker business decisions is really where kind of business intelligence lies now.

So, talk a little bit about the Microsoft platform architecture, you know, across their BI stack, basically you’ve got three areas. And I’m going to include the database component in there too, because that’s where typically your data stores and your data warehouse type of applications reside, on the database side. And that would be where your data storage and integration, you know, pulling data. Microsoft is integration services for an ETL platform in the BI stack; it’s got analysis services as your multidimensional cube, aggregation data store, and now with tabular, as part of analysis services in another way of viewing that aggregation data.

Microsoft has the average data warehouse now in the cloud. And on the presentation side, I used to be a BI architect, and we always joked that Excel is the BI tool of the masses, right, so you know you still have a lot of Excel connecting to analysis services. And then on the presentation side we got reporting services as Microsoft’s tool for reporting.

So, you know, Dez put up the quote from Rumsfeld. It’s similar to the Noam Chomsky quote, you know, “You don’t know what you don’t know.” How do you know whether your BI platform is performing like you would like it to be, or like the users would like it to be, or whether services are up or down? And I can’t get, you know, I’m connected to other services and I can’t get to it. Usually that’s when the phone or the email start coming to the DBAs or the DevOps team, when the users start complaining.

But really, you would like to be proactive about that; you would like to know what’s going on; you would like to be alerted when things are starting to get to a point where users become impacted and the business becomes impacted because they can’t get to the information they needed to. You don’t ever want to be in that situation where, you know, there’s a fire and you turn around and, you know, ten thousand acres are on fire and, you know, you got a garden hose.

So when we talk about platform monitoring, we want to be able to monitor availability: are those resources up or down, are they running? Be able to do some root cause identifications based on availability. We also want to look at performance. We want to look at performance not only of the resource itself, but at the server level, maybe the hardware level. Because you know things going on that piece of iron or the VM or the virtualized environment that you’re running these things in, you may have other things that are going on at the same time that your BI sources are running.

You want to be able to see what’s going on at that server level, and you want to be able to identify those bottlenecks and look at the level of performance at those resource levels. You know the other thing that people often forget is that you need to look at the utilization of those resources, you know, who’s connecting, who’s connected now, what are the active sessions are doing, what queries are running, what reports are running, what interactions are being done at that point? Because if you don’t look at that, how do you know how to troubleshoot performance issues? Basically, Laurent says, “Hey my CPU was at 95 percent,” well you know what, that could be good and that could be bad. It could be good in that my server is really cranking out and really performing at an optimal that I want. Or it could be that I’ve got one thing that’s tying up my server and everything is waiting on there. You need to know what users are doing.

One of my favorite quotes from the immortal Burt Gummer – who was played by, I can’t remember his name – in “Tremors” was, “When you need it and you don’t have it, you sing a different tune.” And that’s kind of the way we view it here at IDERA about our monitoring, performance monitoring products. You know, you don’t think you need it until you don’t have it, and then you’re singing a different tune.

I’m going to pull up the demo here. So, what we’re looking at here is our BI Manger tool. If you want to monitor your entire Microsoft BI platform, we have a companion to our Diagnostic Manager product, which has been around for over 10 years and is pretty well known out there in the industry, and it monitors basically the database platforms. The BI Manager product which we’re showing here, monitors the BI stacks. Those are your BI services, integration services, analysis services and reporting services. And one of the key things, as I mentioned, is that if you’re going to be proactive, you need to have an alerting mechanism. So, what we have in the product is the ability to specify alerts and specify thresholds for those alerts and the ability to be able to be notified, and to specific email groups or people that need to be notified when things happen.

You can see in the example here, I’ve got a set of alerts and you can see they happened over time here, and you can see what the alert is. You know, I have one of my analysis services is down for example, and I can set up email alerts on that, so somebody is alerted. This is the way you become proactive, is to, you know, be notified before things can get to a critical situation. And one of those things you need to be able to do is to be able to drill in and see what is going on.

I use this alert for example. I can click here and it will take me to that point in time and show me what was going on in my analysis services, for example, at that point in time. What we see here is when that alert was triggered, now I can see for example what was going on. You know, in this case, let’s see, I can see there were a lot of CPU thread switching going on – and I’m not going to go into these metrics in details – but you can see, I can see these peaks now, and I can see whether this was ramped up or ramped down after that, because it only happened and then it went back down, I’m probably not too concerned about that. You know, one of things we do here, is be able to monitor at the server level and at the service level, here at the analysis services.

And I’m just going to cover one more area in here. And I talked about user activity, one of the things I can see here, I can see actually what was being run against my analysis services instances that I’m monitoring here. I can look at these queries and actually see what they were doing and I can see performance metrics around there, and where that’s important is when I get the phone call, I said “Man, everything was running slow yesterday between 10 and noon. Man, what was going on?” I can set to that time period here and I can go back and actually see what the users were doing. Not only can I do that, but then I can actually to start looking around the platform itself to see what was going on. Things like, you know, the caching mechanism, things like that.

And then I began to correlate a story on what was happening there. And that allows me to go back and by figuring out what happened and then I can put things in place. I may want to change configuration; I may want to add more memory; I may need to make some changes around the platform itself to be able, so that doesn’t happen again. You know, one of the things we like to say is that it’s critical that you monitor this stuff over time. You know, when you look at some of these charts, you can look and see how your resources are doing over time and utilize that for capacity planning. I may need to ramp up my hardware to be able to handle the capacity that I’m starting to approach over time. You know, it’s real good, these tools are real good for capacity planning, not just correlating and identifying events and what users are doing.

The other thing we talked about is integrations services being the ETL platform. You know, in business intelligence you typically will take data from, you know, multiple platforms. You know, you could have your financials in an Oracle system; I could have an SAP environment over here; I can have some SQL Server databases running over here, but I may want to pull all of that data together into my data warehouse.

And I may use a tool like integration services. One of the things that is critical about that is the ability to see what’s going on as far as what’s running and whether what’s running and how it’s running and whether it’s failing. And one of the things we do here in the product is allow you to go in there for an integration services and we monitor all of your package execution, so if you running ETL processes, we monitor those, you know you can get, say you had a critical, your data warehouse, your dimension, and your facts. Your job that runs integration services that updates your data warehouse that updates your dimensions and facts every night. We would like to know whether that process failed. You know you can be notified on that through the product. I can go here, I can drill into it, look at the steps that ran with in there.

And I can see, for example, that this step failed. I can click on that step and then I can get the actual error message. What we like to say is that our goal is to shorten that window from the identification of a problem to fixing the problem. This allows you to drill into and get right the root cause of the problem. I can look at this error message here – and I happen to know what the error message is here, because I looked at it earlier – and by knowing that, I can go right to, I can go fix my process, kick it back off, rerun it and it shortens that window down. It’s all about getting to the problem, fixing it and getting things up and running again.

One of the other things we also provide here is the ability to look at these packages over time, and look at how they are running. And this kind of gives me an idea: if I see spikes and it starts over time to be running longer and longer, I may want to look at things like my maintenance window; my maintenance window may be shortening so that I may need to move things around. Such that this job where it used to take, you know, I may want to go in and start juggling around my schedule. Again, just information, so that you can keep things up and running in an acceptable service level to the people that need to get to that information from the BI standpoint.

On the reporting services side, we monitor the reporting services also. And one of the key things there, is you’d like to know what reports are running and how long they are taking. We can go in here and we monitor that, and whether they are subscription based or ad hoc, we monitor all those reports. What this allows you to do is to go in there, when some customer calls or an employee calls, or the executive calls, for that matter, “Hey I didn’t get my TPS report today.” You can go in here, well it failed, and then correlate, why did that fail? Well then you can see when that report ran, and you can then go into maybe it was against analysis services, maybe it was running against the database warehouse. I can go into Diagnostic Manager, kind of the sister product on the database, and I can go in there and say “Oh, here’s what happened.” I don’t know, memory was constrained, the report was running and it was a deadlock victim and it killed the connection and that’s why the report didn’t run. But if, you know, if I could kind of sum it up. The key thing is there, it goes back to you don’t know what you don’t know, right?

It’s important to have tools around to being able to monitor that environment. It’s not just about having a tool that notifies you when stuff breaks, it’s about having a tool that you can monitor and set thresholds and look at things over time so that you can be proactive about your BI infrastructure and your BI environment in order to be able to make changes or have a better user experience, or for people to get to the information in a timely manner that they need. Because that information becomes old really fast.

I guess that’s all I’ve got. So I’ll turn it back over to Rebecca, or whoever’s next.

Rebecca Jozwiak: I’m sure that Robin and Dez have a lot of questions for you and I do have a couple of good ones from the audience for you too. So, Dez why don’t you go ahead and fire away?

Dez Blanchfield: Absolutely, I have lots of them. My first one, just a couple of high-level ones, if you don’t mind, just to set the scene, so is it the case that you’re seeing BI mature rapidly and now there’s not so much a scramble, but certainly a lot more tension being focused around the organization and not just having a BI platform and just parking it. But are you seeing people now, are now just living and breathing with BI in their hands? I’ve noticed a lot of people just walking around with tablets and phones with their platform on their screen on a regular basis. Is that what you have seen?

Stan Geiger: Oh yeah, without a doubt. I mean my background is in BI – and I’ve been removed for about two years, since I’ve been in product management – and it’s just changed leaps and bounds. I mean Microsoft, I forget what the term they use, it’s about getting the data in the hands in of the user, right? You see things like power BI and mobile platforms and things like that. You’re exactly right, people walk around basically connected to the BI platforms now, and just being able to dynamically run stuff. Used to be we got reports, right, and dashboards, but now, you know, data changes so fast that these platforms are so much more flexible and are able to get that data quickly to the consumer.

Dez Blanchfield: I was at a logistics firm recently just catching up for lunch with a friend, and he was walking around with an iPad Pro, and I asked him how the experience was. And he’s like “I actually don’t use the device as a general thing,” and he hit the wakeup button and showed me. And on the screen, on the product they use, he had this live dashboard, and his job was to make sure the trucks were coming and going, delivering, logistics. This wasn’t even related to the Christmas spirit, this was just general day-to-day, they had hundreds of trucks coming in every day.

And he said that once upon a time, he would get up and arrive and there would be reports on his desk with graphs and things, kind of what he called the clipboard experience. Now he’s walking around with a digital clipboard. And so I asked him, leading up to the event, I said to him, “What happens if you end up with a performance issue?” Well, he went pale. He absolutely, he was like, “No, no, don’t jinx, don’t put the kibosh on me, no way!” He said if we have an hour outage, you know, he started lifting a flow on a fix, and his mind performance hits of BI equated to kind of like a disaster recovery. You know if we’re like offline from BI for an hour, you start thinking about cutting out to disaster recovery platforms.

So, it was interesting, and I’m interested to hear if you’re seeing it at other places. Do you think that, where do you think this sort of, you know, I mean the type of tools that you’re talking about are often something that might be used by a very technical part of the business, but are you seeing this thing as a focal point now, as in your product being demanded by chief data officers, chief risk officers and so forth, as opposed to maybe previously selling just to CIOs, CTO would be thinking about? Is it less a pinnacle part of the organization and more the commercial and seeing your execs outside the CIO’s office asking for access to this sort of tool?

Stan Geiger: Yeah, that’s a good point. I don’t know if you notice, but this product runs in a web console, not a Windows client. And one of the reasons is that, one of things that we’re finding is yeah, I mean I won’t name the customer, but it was a large entertainment firm and that was one of the whole reasons they were looking at this, because the CIO wants the CTO to be able to see when one of these platforms are up, and are they performing at a high level. Because they’re getting the nasty phone calls from the guy, when, if it’s down for five minutes, I’m now five minutes behind on my trucking, right?

Dez Blanchfield: Yeah, yeah, I definitely am seeing that now. And it’s a pleasant surprise, because I think, I don’t think it’s an Australian thing, I think it’s a global thing now. A couple of forums I’ve been to and I had the privilege of talking at a smart city conference the other day. And a number of people talking about what we would normally consider business intelligence, they were talking about it like big data, analytics, and a whole range of things, and new coinages, like you’re in marketing, in vendor land or the media hyped up. But really, when I brought it back a couple of times, I said “You know you’re really talking about fundamental business intelligence; you’re talking about the big data; you know you’re bringing this back to just some smarts about what’s happening in your company.”

You’re also finding that there is a shift in one of the things I notice in your talks, a number of pieces of the puzzle. Are you seeing a shift to people thinking outside their own computer of a data center, so that they’re now worried about what’s happening in their cloud platforms, or what’s happening in third-party platforms and how to monitor that. For example, I might be dependent on someone else’s system and I’m now starting to think more, “OK that’s great that I know my BI platform is working well, but what about their infrastructure, what about the bits that are feeding us?” Are you seeing a more broad view of the world in that sense?

Stan Geiger: Yeah, if I understand what you’re asking, we get customers all the time asking about, hey you know, we’re moving some of our platforms to the cloud. You know, typically, you know, you get charged on utilization. How do I know that I’m a getting my bang for my buck? Because I get a bill that says I get this much utilization, but I want to know that because I’m basically sharing resources, you know? If it’s a result of a poorly performing resources on their end. We do get asked by customers a lot about monitoring the, being able to monitor their cloud environment from the standpoint of, being able to know if they’re getting a bang for their buck in, on those platforms.

Dez Blanchfield: And are you seeing people, one of things, I was just watching a demo, the thing that stood out to me, I can now put a kind of dollar value against the KPIs inside the BI, so a lot of the companies I’m seeing, enterprise now, and are saying what does it cost if this platform’s offline? What does it cost if we don’t invest in this core infrastructure? If we don’t have X does it cost us Y? Are you seeing a conversation in the place now where it’s almost like a stead complete from your point of view, because it sells your product? That people now actually understand the kind of value and commercial bottom-line impact of not having their BI online and available? And that they now have become independent on them as a critical business system?

Stan Geiger: I think, you know, in my experience, that’s, the conversations are starting. But they’re having a hard time coming up with the dollar, with the ROI, basically dollar value and then being able to – all they can do, just in my experience, with customers – it’s like “Yeah, we know that we need to have these things up and running, but we haven’t been really able to figure out how to put a dollar on, if I’m down for a half an hour,” for example. But the conversations are starting, and I think over the next year or so, you know, we will start to get to the point. You know, we struggled for a long time early on, you know, trying to come up with ROI for having tools period, you know, 10 years ago.

Dez Blanchfield: Yeah, I think that it will almost be a metric build in the platforms. I haven’t seen it become an actual dashboard yet, but I have been in conversations where people turn around and say, “Why can’t we have a dashboard for that?” You know, almost a dashboard to show us what it costs, except the dashboard is not available.

From a technical standpoint, what does it look like? I think one of the questions I got asked a moment ago, just through the WebEx platform, “What does the – in a couple of minutes, just a really short version – what is the journey of going from not having a tool in place to having it in place? What is the time frame does it take to implement it and getting it running per proof of concepts? What does that look like? Can we download a free trial? What kind of resources do we need inside the organization? How do we go about putting it in place and doing the proof of concepts and seeing what the value of proposition is beyond the great demo you just provided?”

Stan Geiger: Yeah, so you know, one of the things we do, we have a trial buy, you know, we have by experiment. They can go to the IDERA website, download a fully functional version of the product, install it and run it for two weeks – and it’s a fully functional product. And it’s not real complex: it’s agentless, so there’s nothing to install after all the instances. You just basically have to download it, install it and then set it up and then register your BI instances that you want to monitor. And then we have a data collection process that runs on the, as a service on the client side. And it just goes out there by default, it’s set for every six minutes to collect data except for query data, which you can turn on the continually collect.

But anyway, long story short, it’s very easy to set up. There’s not a lot of pieces to it. You just basically install it, give it the credentials that it needs to be able to communicate to those platforms, register those instances or those platforms that you want to monitor and then you’re up and running.

Dez Blanchfield: When you’re doing the demo, the thing that really struck me as raw, that there are a couple of tsunamis coming at us, they’re kind of like freight trains running in a tunnel and the light on the other end is not good news. You know there’s a lot of talk about whether it’s on one end of the spectrum of smart cities and infrastructure, that’s becoming intelligent or whether it’s the IoT in general. Where do you see some of the biggest impacts hitting BI in the next three to five years? What kind of things should we be thinking about given that you’re at the bleeding edge of it? You’re talking to companies on a daily basis about the type of challenges is a thing, and I’m on the show now, and I’m thinking, where should I be focusing my time and effort to get a business case and model to get this tool in place? Is it companies getting smarter and being expected to deliver more? Is it that they need to think about what IoT even means to them? You know, if you’re AirBus and you’re making airplanes and you’re putting sensors in airplanes and these sensors are producing terabytes of data, where are some of these bigger impacts that you are seeing come through?

Stan Geiger: That’s a really good question. I do another presentation, I actually call it a data tsunami, instead of the data lake, because of just the shear amount of information that is collected now, right.

Go ahead?

Dez Blanchfield: No, no, I was going to say I’m keen to hear what your thoughts are on that.

Stan Geiger: Yeah so, one of the biggest issues that I hear from customers and just colleagues of mine that I keep in touch with, I’ve got this shear amount of data, how do I correlate all of this data to, in the organization in such a way that we can make sense, out of what this data means? You know, because there’s such a shear amount, you know people use the term unstructured data, but it’s only unstructured until the point that you need to use it to make a decision, then you structure, so then you can get relevant or meaningful information out of it.

And the biggest thing they struggle with is, I’ve got instrumentational data over here, how do I correlate that with the production data of the widgets that I’m building or the airplane parts? You know, I know that there is some correlation there, how do I get my head around that to be able to put that in some kind of a meaningful platform that people can make quick and good decisions that are relevant to the business? In other words, it’s just dealing with the shear amount of data and how to figure out how that data fits within the organization. I mean, obviously you can make some stovepipe decisions around that, but you know, there’s correlation there and I think that’s why data science is such a big deal right now.

Now we’ve developed, there’s this whole career and mindset around using this higher-level mathematics and things in machine learning on taking that data and trying to mine those correlations in that information out of there. So, I think that right now that’s one of the biggest challenges and I think that’s why you’re seeing data science and things like that growing so fast.

Dez Blanchfield: You know, you’ve actually led me to my last question before I hand it back to Dr. Robin Bloor, who I’m sure has got some great questions. Are you seeing not a shift too, but also a balancing out that the traditional enterprise IT shop has now had to kind of give way a little to the refocusing around data science?

And I’m seeing a heavy move, not away from, but you know, we used to be really anally fixated about how our IT shop was running in the COO’s office and whether the lights were flashing. I’m now seeing like a second genius pool arrive in organizations, particularly around the BI platforms, where we’ve got actuary, we’ve got statisticians, we’ve got data scientist as a whole. They now want as much as infrastructure as the rest of the business, if not more. You know, they’re saying “We want access to the data, but we also want to run it somewhere, and we don’t just want it in a Hadoop called Spark. And we don’t just want analytics platform.”

And as Robin alluded to, we now have machine learning being applied to things now by default; we’ve got cognitive computing. The likes of the Watson platform from IBM, we can now throw BI data at and say “what if.” Are you seeing like a rebalancing of where the focus and the technology component of the business is, and some more pressure being put on the team delivering BI infrastructure and services to now support yet another a big piece of the business around data science in particular? Is that something you’re seeing?

Stan Geiger: Yeah, no, that’s a great question and a great observation; you’re definitely seeing that. You know it used to be, for example if you had a BI group, it lived in the IT. It’s still that data science will live in that area, but it lived pretty much next door to the DBAs, right? You know, the guys responsible for all the data stores. Now you’re seeing this other, that grow out, and sometimes BI analytics groups and data science, you know, those groups are living outside of, I will call it on the business side, outside of the IT department.

So, what’s happening is, exactly like what you said: they’re using these platforms, and these platforms are needing to expand so much, they are putting a lot of pressure on the IT groups, or in some cases I always tell sales reps, when they sell these products, find the data analytics group, find the BI group, not just the IT folks. Their interest in their platforms being up and running, because their jobs depend on it.

Dez Blanchfield: Yeah, definitely, and I’m seeing that once upon a time, the CIO got removed from the boardroom, he was too technical and using acronyms that no one understood, now the CIO’s not just, the CIOs are back in the boardroom, but I’m seeing, you know, CMOs, chief marketing officers, chief data officers, chief risk officers and now analytic services are being brought in the boardroom and they are being asked these big questions of, you know, what if, where is going to, what’s over the horizon, what’s in the crystal ball? It’s predominantly analytics, but it’s analytics for the BI. I think this is an interesting point and time now. People are having to create a vocab and a language to describe what they’re asking for in a common way so that people understand it.

Thank you very much for your time on that. Really, really great answers to those questions. I probably threw a few curve balls at you there. I’m going to pass to Dr. Robin Bloor. I know we’ve got some questions that come from the audience as well, and we’re probably running short on time. Robin, can I pass to you? I know you’ve got a couple of curvy ones to throw as well.

Robin Bloor: Yeah, sure you can pass to me. What I would like you to do, Stan, if you can – we’ve done a number of these hot tags with the various of IDERA products, and it’s quite clear to me in one way or another this fits into the portfolio we’ve got. If I were in a situation – let’s imagine that I’m a corporate that has actually got very poor monitoring tools – and I have databases out there and various sorts and various BI applications, how does the portfolio monitoring and fixing tools that IDERA’s got? How does that all fit together?

Stan Geiger: Well, that’s a great question. You know, I think, we obviously have tools, you know, you mentioned the Diagnostic Manager on the database monitoring side. We’ve got BI Manager on the BI monitoring side, and then we’ve got other tools like Inventory Manager that you can run and it will go out there and find all of your instances, your SQL instances both BI and database out there. Now I can discover what I’ve got and then I can use the monitoring tools to point those for monitoring. And then we also have security tools, like Compliance Manager and Secure, to go out there for doing secure audits and things like that.

Now I can check, you know, if my environment is secure and I can audit my instances and make sure that they are set up correctly and you know, and then you know we’ve got other tools that do job management. Basically, our whole portfolio is all around being able to monitor, discover monitor, you know audit around that entire environment. Most of those tools fit in, as I mentioned, this is a web-console-based product, the BI manager is. Diagnostic Manager, our backup and recovery tool, fits in this web console. You can conceivably have a several of our products and they all fit within the same web console. It gives me a holistic view across my environment from my disaster recovery, from my monitoring, to, you know, my discovery of all the instances in my environment. So, there’s a pretty good synergy between the products there that fit into the web console, as I showed earlier.

Robin Bloor: Yeah, OK. I got that impression from the various hot tags we’ve done out there. The one thing I didn’t get from your demo, if I wanted an overview of exactly what is going on before I dug down into anything, was there a screen for that in the tool?

Stan Geiger: Yeah, because you know, with that short amount of time, basically what we do in every one of those services, you can look at things at the server level, so we monitor things at the server level. And then I can also drill down into the different areas: I can go down and look at users’ sessions, the active sessions that the users, everybody who’s connected, I can drill into it and see what they’re doing. So, if you were connected, I could click on your login or on your session, then I could see what you’re actually running or interfacing with that resource. For time, I didn’t go through that. Yeah, you can look at the different areas and drill into those areas.

Robin Bloor: Yeah, I was particularly impressed with the root cause identification, because I used to do a lot of that, when I was kind of, I don’t know it was about 25 years ago. So, life was slightly less complicated than it is now. I think things are kind of running out of time. Rebecca, do, have you got any questions from the audience?

Rebecca Jozwiak: I do have a couple of questions from the audience, and audience, thank you for sticking with us and spending time today on our last webcast of 2016. Stan, I think I know the answer to this question, and I think we’ve had someone from IDERA do a Hot Technologies on this very solution, and maybe you can talk a little bit about it. Does the BI Manager help at all with query performance tuning? And I’m pretty sure you have some other products that do that, is that right?

Stan Geiger: Yeah, so we have a couple of things, we the have SQL Doctor, which will go in and look at your database instances, and then we have another product called the SQL Workload Analyzer that looks at queries against the database side. Right now, we don’t have a comparable product for like queries that run against analysis services, but we’re looking at down the road developing a similar product, for looking at queries or optimizing queries on the analysis services side.

Now you do have the ability to be able to view those queries, but you would like to use the product similar SQL Workload Analyzer, which will actually go in analyze those queries. We’re looking at a comparable product for that analysis services piece.

Rebecca Jozwiak: OK, good. And I mean, I know every time you run any extra service or script or anything on a system it might – whether it’s a little or a lot – impact performance. But how would you kind of judge the way BI Manger impacts performance, or does it at all?

Stan Geiger: You know, I usually say it’s less than three percent. People always wonder what the overhead cost is. Because we’re agentless, so we don’t have something actually running on that server. You know, that obviously saves things. And even the query monitoring piece for analysis services is very low overhead; it’s less than three percent.

Rebecca Jozwiak: OK, good. And I know we’re going over the top of the hour a little bit. I did kind of want to ask this question, I know in your architecture slide – let me pull it up real quick – you did mention that a data security and encyclopedias, I’m having to assume that you’re using some other product for security. But one attendee does want to know, how do you deal with cyberattacks or what are you using for security?

Stan Geiger: What we’re using in the product? Obviously, I mean yeah, that’s a good question. I can tell you the high level, because we have a lot of government customers, but we’re using – I forget what the name of it is, and I should know it – but we’re using secure connections between our communicating, between the web console and things like the services and things like that. I apologize, I don’t know the specifics around that. But we have several, several government customers, that are, even DOD customers and so we’re really – how can you say – we’re hyper sensitive to that. We always try and use the latest protocols to protect against the outside intrusion into our products.

Rebecca Jozwiak: That’s interesting that you brought that up. And I’m not sure if this is a question, where you will say, “If I tell you I’ll have to kill you!” But another attendee is asking, does the military intelligence enjoy kind of better analysis, more secure protection than typical business intelligence? I’m not sure if you can answer that or not.

Stan Geiger: Well through the product, it’s across the board. I mean we don’t have, we don’t do – to my knowledge, unless they just don’t tell me – they get the same product that everybody else does. So, everybody gets the same security.

Rebecca Jozwiak: OK. Well, that’s good to know. Before we wrap up, Stan, is there any place that our folks can go to maybe get a free trial or download a light version?

Stan Geiger: Yep, just go to IDERA.com and go to the product section and then you can see BI Manager in there, and if you go to the page and register, you can get a two-week trial, and it’s a fully functional copy, so it’s not a light copy. Kind of our sales model is trial buy. So, we like our customers to be able to trial the real deal for, you know, a couple of weeks so they can make sure it works for them and their environment.

Rebecca Jozwiak: OK, good. We do have a few more questions from the audience; I will make sure that they do get forwarded to the presenter today if we didn’t get to your question online. With that, folks, we’re going to wrap it up. Thanks so much to Robin Bloor and Dez Blanchfield for doing our Hot Techs, as always. And thank you Stan, a really good demo.

And this is it for 2016, everyone we’ll kick it off in 2017 on January 10^th, with IDERA again in the Briefing Room this time. So, that should be another fun time. And everyone, hope you have a merry holiday, maybe you get to have a little R&R and have a great new year. And with that, folks, bid you farewell. Take care.