Breaking Silos: How to Consolidate, Cleanse and Use Your Data for Good
Destroying silos is the first step to gaining valuable insight from your data.
Take inventory at any medium or large business, and you’ll find vast amounts of data: financials, marketing details, employee metrics, sales figures, product information, customer support calls, business process outputs and much, much more. Whether this data is used for business KPIs, internal measures or (more likely) not used at all, it’s all sitting there in separate, proprietary databases, and it's growing every day. (To learn more about managing your databases, see 7 Reasons Why You Need a Database Management System.)
The thing is, hidden in that data is the answer to some important business questions. There are trends, experiments, approaches and changes just waiting to be uncovered. Use that data to illuminate your decision making, and you’re at the heart of what good business intelligence does. It drives positive action.
There’s just one issue. The answers aren’t just in one database. You need to bring together data from lots of different sources to get the whole story. As Ben Carpel, CEO from business data dashboard company Cyfe, put it in a recent blog post, “Keeping a pulse on your business’ sales, marketing, finances, web analytics, customer service, internal R&D, IT and more as isolated sources of data never gives you a complete picture. In other words, big data doesn’t lead to big insights if you can’t bring it together.”
Want to know if a new product will work with your current marketing strategy and customer base, for example? You’re going to need to dig into your CRM, marketing stats, customer support details and product development information, and they’re all in separate data silos. You need to take the metaphorical hammer to those silos, break them down, bring the data together, and see what it tells you.
Here’s how to do that.
Think About the Questions You Want Your Data to Answer
Consolidating and cleansing data to bring out business intelligence is no small task. That’s why the whole process needs to be driven by the questions you want to answer. Think about the insights you need to make better business decisions, and turn them into actionable questions.
Write down those specific questions and drill down into the details. How will the questions help you make better business decisions? Create a list of requirements from your data that will help you answer the question in a smart, actionable way.
What analysis will you need to do to turn the data into intelligence that can drive action? Forrester reports that only 29 percent of businesses turn analytics into action. Don’t be in the 71 percent.
Understand the Type and Location of the Data You Need to Source
Now you have your questions, it’s time to look for the data that can provide those insights.
Carry out a database audit to identify exactly what data your business is already collecting. For each database, understand the following:
- Where is this database located?
- What are the key features, inputs and outputs of this database?
- What data is being captured?
- Which of the data points recorded here can answer your business questions?
- What’s the best way to get this information out of the database?
- How can this data be combined with other sources to create better context and analysis?
When the audit is complete, you should have a comprehensive understanding of the various data your business is capturing, where it’s located, and the specific elements of the data that can drive business intelligence for your specific questions. (For more on understanding data, see Graph Databases: A New Way of Thinking About Data.)
Get Your Data into One Place
It’s time to break the silos. You need to consolidate all of the key information that can answer your questions into one, central repository. Create data flows to collect data and combine it in a data warehouse.
Once you have all the data in a central location, you’ll need to start combining it. The most effective way to do this is through a data integration tool. As Mike Ferguson of Technology Transfer puts it, the purpose here is “to integrate disparate master data maintained in multiple operational systems.”
The idea is to create a combined dataset that contains all the key information in one place. That means mapping individual data fields together, understanding the context of each data field, and developing individual data elements to show that data in a logical and cohesive way.
Consolidate and Cleanse Your Data
Once you have your dataset, you need to cleanse and verify it. The chances are your data is going to be “noisy.” It may have spurious information, outliers and other characteristics that need to be smoothed out. This part of the process is vital for data integrity, because you need to have confidence in your data if you’re going to use it for business decision making.
Run some preliminary analyses and reports so you can see what the data is telling you. Identify any odd outputs and drill down into the data to see where they’re coming from. You will also need to deduplicate your data so you’re not double-reporting.
Be careful when you’re cleansing your data, though. You don’t want to smooth it out so much that you miss important outliers and trends, as they too can often provide valuable insights. This idea of cleansing and consolidating data should be one of continuous improvement. Tweak, measure, understand what it’s telling you, and repeat. When you’re satisfied with the results, stop.
Turn Your Data into Actionable Business Intelligence
This is where you bring everything together and use it to drive business decisions. Use the cleansed, integrated data in your data warehouse to power your reporting and business intelligence tools.
The most important thing to do, though, is act. When you put the effort into breaking silos, creating integrated, verifiable data and drawing out smart insights, you’ll have the confidence to make better business decisions. That translates to more efficiency, less waste, happier stakeholders and an improved bottom line.