A distributed file system, a MapReduce programming framework, and an extended family of tools for processing huge data sets on large clusters of commodity hardware, Hadoop has been synonymous with “big data” for more than a decade. But no technology can hold the spotlight forever.While Hadoop remains an essential part of the big data platforms, and the major Hadoop vendors—namely Cloudera, Hortonworks, and MapR—have changed their platforms dramatically. Once-peripheral projects like Apache Spark and Apache Kafka have become the new stars, and the focus has turned to other ways to drill into data and extract insight. Let’s take a brief tour of the three leading big data platforms, what each adds to the mix of Hadoop technologies to set it apart, and how they are evolving to embrace a new era of containers, Kubernetes, machine learning, and deep learning.Cloudera Enterprise Data HubCloudera was the first to market with a Hadoop distribution—not … [Read more...] about 3 big data platforms look beyond Hadoop
In the previous article, I’ve described why financial services companies should care about machine learning and what use cases they should pay attention to. Now, it’s time to see how exactly financial businesses can adopt this technology to make it a success.The problem is, in spite of all the advantages of AI and machine learning, even companies with deep pockets often have a hard time extracting the real value from this technology. Financial services incumbents want to exploit the unique opportunities of machine learning but, realistically, they have a vague idea of how data science works, and how to use it.Time and again, they encounter similar challenges like the lack of business KPIs. This, in turn, results in unrealistic estimates and drains budgets. It is not enough to have a suitable software infrastructure in place (although that would be a good start). It takes a clear vision, solid technical talent, and determination to deliver a … [Read more...] about How to make use of machine learning in finance?
Machine learning, a subset of artificial intelligence, is the practice of using algorithms and large data sets or Big Data to develop insights ranging from which movie a Netflix user may want to watch next to recommendations about cybersecurity incident handling.According to consulting firm McKinsey, “the unmanageable volume and complexity of the big data that the world is now swimming in have increased the potential of machine learning—and the need for it.”For security professionals, machine learning capabilities can increase responder productivity and enable leaner, more efficient security operations. Humans however, not machines, must direct and guide machine learning algorithms to achieve the business goals and objectives that the computers are given.Machine Learning, Big Data, and SecurityThe best way to understand how machine learning can be beneficial for security analysts is to perhaps look at another field with similar operational efficiency goals that is … [Read more...] about Machine learning: The perfect partner for security analysts
I’m a big fan of cloud-based machine learning and deep learning, and AI in general. After all, you can’t be a geek without imagining having a conversation with an artificially intelligent being that can answer questions and carry out your bidding!That’s said, I’m also seeing cloud-based machine learning and deep learning misapplied over and over again. All have easy fixes for the most part, and certainly cloud-based machine learning is here to stay. But use it wisely and appropriately.Here are the top three recurring mistakes that I’m seeing. 1. Not enough data to provide the training to the knowledge modelMachine learning, without any learning, is worthless. The true use case for machine learning is applying algorithms to massive amount of data and having certain patterns emerge that become the training for the machine-learning-based applications.So, no data, no learning. Although a machine learning application can gather data over time and … [Read more...] about 3 common machine learning mistakes to avoid
If you’re doing work in statistics, data science, or machine learning, the odds are high you’re using Python. And for good reason, too: The rich ecosystem of libraries and tooling, and the convenience of the language itself, make Python an excellent choice.But which Python? There are a number of distributions of the language, and each one has been created along different lines and for different audiences. Here we’ve detailed five Python incarnations, from the most generic to the most specific, with details about how they stack up for handling machine learning jobs.Anaconda PythonAnaconda has come to prominence as a major Python distribution, not just for data science and machine learning but for general purpose Python development as well. Anaconda is backed by a commercial provider of the same name (formerly Continuum Analytics) that offers support plans for enterprises. … [Read more...] about 5 Python distributions for mastering machine learning