The issues that torpedo big data projects and how to avoid them


You’ve heard the stats that say many big data projects fail. So, what can you do to stand up an analytics solution with the confidence that it will help you achieve your business outcomes? Here are a few causes, and solutions.

Often, the issues that torpedo big data projects are a lack of experience and a shortage of skilled people. Without experience, companies often come to expect analytics solutions to do things they aren’t capable of doing. Or, they make the wrong choice about the products they need. Time is another issue.

Enterprise analytics systems are large-scale projects that require a lot of lead time to get the components and put everything together.

Implementing this yourself is like a recipe: You get step-by-step help, but you still need to bake the bread. And with that, comes decisions that are better made if you’re an experienced baker. For example, here are some of the tasks that remain after basic implementation:

  • Spec, acquire and assemble the hardware
  • Set up the network
  • Develop a security model and implement
  • Connect the cluster to your enterprise (Directory services, VPN, reporting tools, legacy applications, legacy data sources, etc).
  • Integrate third party development, analytics and visualization tools, both open source and commercial
  • Oversee operations and day-to-day management
  • Establish and manage backup and recovery
  • Perform ongoing tuning and maintenance

That may be more than you’re prepared to handle, especially if you don’t have the staff or the experience. Some platform-as-a-Service solutions like Cloudera Altus can specifically address the staffing and experience issues, but if you need even more control over the implementation you need to consider other solutions, such as those offered as a managed service.

Effectively managing a platform is as important as the work you put into deployment. For all the automation these tools can provide, a cluster is still a complex system and you may not be able to hire the right administrator skillset in the timeframe you are considering. In this case, it may be worth considering a managed service such as the DXC Analytics Platform. After deployment of the platform, a managed service team will monitor your cluster for any issues, and provide ; help with common administrative tasks, incident and problem management, service level reporting, license management, billing support, and change management.

Whether you decide to tackle standing up a Big Data platform by yourself, or enlist some expertise to assist you, once the platform is up and you are running analytics you will start unlocking the value hidden in your data and increasing your company’s Analytics IQ.

Jim Coleman is Analytics Platform Product Manager at DXC.

Toby Ferguson is Senior Partner Sales Engineer at Cloudera.

Speak Your Mind


This site uses Akismet to reduce spam. Learn how your comment data is processed.