Go undercover with big data to boost your competitive intelligence

October 19, 20184 minute read

Select article text below to share directly to Twitter!


Competitive intelligence doesn’t just belong to the business side of your organization. Business can keep its love for the word “leverage” and weirdly named SWOT analyses, but competitive analysis is also an IT game, and it’s more like a necessary mechanism for survival than a game.

The technology landscape changes drastically every few years. It moves quickly and has created a bizarre productivity paradox. Even though researchers say innovation isn’t worth as much as it used to when it happens at such speed, you can’t just stop competing in technological innovation—and that’s where competitive intelligence comes in. According to Entrepreneur, it’s “understanding and learning what’s happening in the world outside your business so you can be as competitive as possible.” It includes gathering intelligence on your competitors, industry, and customers, and it all points to the fact that IT departments should be making the big decisions about tech.

But how do you know what’s happening in your competitors’ R&D labs and the minds of your customers when you can’t send out a fleet of undercover spies? The ethical answer, unsurprisingly, lies in big data.

3 things a top-secret data intelligence agent needs

There are a few different types of competitive intelligence in the world, but you should focus on gathering strategic and tactical insight on your competition and customers—namely, how your customers behave online and what your competition is doing to innovate. Here’s the breakdown of what you’ll need to interpret the data you collect.

1. Lots of data

You’ve got data—probably more data than you know what to do with—but you’ll want to primarily dig through data from transactions, social media, blog posts, industry sources, IoT sensors, survey panels, search engines, and benchmarks. You likely have a lot of insight on your customers, too, but it’s common for companies to work with third-party vendors to purchase even more industry insights.

With all this data, you’ll need tools for data storage and warehousing—and hopefully they aren’t a series of hard-to-access, messy data swamps. You should search for something centralized and scalable with hardcore processing power—like the combo of a cloud-based enterprise data warehouse appliance and a distributed, in-memory NoSQL database layer.

2. More technology and prying questions

Once you’ve got the raw data and it’s stored in a stable, accessible form of information architecture, you’ll need to check off the rest of the components for your big data stack. This consists of four parts:

  • Distributions and data warehousing
  • Data storage
  • Computing
  • Visualization and analytics

Now, it’s time to layer on analytical processing and visualization. You’ll need some sort of big data query engine and a self-service tool for turning these results into shiny graphs for your boss. Fortunately, there’s quite a few options for big data querying and visualization in the market that don’t require expert-level knowledge of Java programming, like Presto or Google BigQuery.

Once you’re all stacked up, the fun begins. You can start spying and asking targeted questions, like “What are our customers telling their friends?” or “What’s the theme of the competition’s marketing right now?” Uncovering the answers to these questions is the secret to innovating at the speed of light.

3. An effective hacker guard

Say you’ve got a massive volume of highly varied data flying around. Unstructured data leaves social media networks and IoT sensors, enters your data warehouse, and exits through your analytical query engine. You’re using graphs to understand how the IT team can innovate, disrupt, and generally differentiate through digital. But what about data security?

Adopting competitive intelligence probably won’t increase your data security risks. However, discussing privacy and risk is a critically important part of any conversation about big data, especially in light of GDPR.

When you’re building or reviewing your big data stack, security isn’t an optional consideration. You want to know your competitor’s next big move, but hackers want to know your customer’s social security numbers. Lock the bad guys out, and while you’re at it, make data security easier on yourself with smart office IT investments, like hacker-fighting printers, so you can keep enjoying your new role as a hybrid spy, data scientist, and business visionary.

Like it or not, the future of technology is coming. The role of the IT department is changing, moving from basement offices to the boardroom. As the strategic value of IT continues to increase and technology advancement accelerates, it’s time for IT to fully embrace competitive intelligence.

  • Recommended for you
  • Recommended for You