Machine learning has been one of the biggest advancements in the history of computing, and now it is believed to be capable of taking on significant roles in the field of big data and analytics. Big data analysis is a huge challenge from the perspective of businesses. For example, activities such as making sense of huge volumes of varied data formats, data preparation for analytics and filtering redundant data can consume a lot of resources. Hiring data scientists and specialists is an expensive proposition and not within every company’s means. Experts believe that machine learning is capable of automating many tasks related to analytics – both routine and complex. Automating machine learning can free up a lot of resources that can be used in more complex and innovative jobs. It seems that machine learning has been heading in that direction. (To learn more about the use of machine learning, see The Promises and Pitfalls of Machine Learning.)
Automation in the Context of Information Technology
In the context of IT, automation is the linking of disparate systems and software so that they are able to do specific jobs without any human intervention. In the IT industry, automated systems can perform both simple and complex jobs. An example of a simple job could be integrating a form with a PDF and sending the document to the correct recipient, while provisioning an offsite backup could be an example of a complex job.
To do its job, an automated system needs to be programmed or given explicit instructions. Each time an automated system is required to modify the scope of its jobs, the program or the set of instructions need to be updated by a human being. While automated systems are efficient at their jobs, errors can occur due to various reasons. When errors occur, the root cause needs to be identified and corrected. Obviously, to do their jobs, automated systems are totally dependent on human beings. The more complex the nature of the job, the higher is the probability of errors and issues.
Usually, routine and repeatable jobs are assigned to automated systems. A common example of automation in the IT industry is automating the testing of web-based user interfaces. Test cases are fed into automation scripts and the user interfaces are tested accordingly. (For more on practical uses of machine learning, see Machine Learning & Hadoop in Next-Generation Fraud Detection.)
The argument in favor of automation has been that it performs routine and repeatable tasks and frees up employees to do more complex and creative tasks. However, it is also argued that automation has displaced a lot of jobs or roles formerly performed by humans. Now, with machine learning finding its way into various industries, automation could add a new dimension altogether.
Is Automation the Future of Machine Learning?
The very essence of machine learning is the ability of systems to continuously learn from data and evolve without the intervention of human beings. Machine learning is capable of behaving like the human brain. For example, a recommendation engine in an e-commerce website can assess a user’s unique preferences and tastes and offer recommendations on products and services that best fit the user’s choices. Given this ability, machine learning is considered ideal for automating complex tasks related to big data and analytics. It has already overcome the main limitation of the traditional automation systems which cannot operate without regular human intervention. There are multiple case studies to show that machine learning is capable of completing sophisticated data analysis tasks, as will be discussed later in this article.
As already pointed out, big data analysis is a challenging proposition for companies and it can be partially delegated to machine learning systems. From the perspective of a business, this can bring a lot of benefits such as freeing up of data science resources for more creative and critical assignments, higher volume of work completion, less time taken to complete tasks and cost effectiveness.
In 2015 MIT researchers began working on a data science tool that is capable of creating predictive data models out of huge volumes of raw data using a technique called the Deep Feature Synthesis algorithm. The algorithm, the scientists claim, can combine the best features of machine learning. According to the scientists, they have already tested the algorithm on three different data sets and are going to expand the scope of testing to more data sets. Describing how they do it, researchers James Max Kanter and Kalyan Veeramachaneni stated in a paper to be presented at an international data science and analytics conference, “Using an auto-tuning process, we optimize the whole pathway without human involvement, enabling it to generalize to different datasets.”
Let us examine how complex the task has been: the algorithm has a capability that is known as auto-tuning capability, with the help of which it derives or extracts insights or values from raw data such as age or gender, and after that, it can create predictive data models. The algorithm uses complex mathematical functions and a probability theory known as Gaussian Copula. So, it is easy to understand the extent of complexity the algorithm is able to handle. The technique has also won prizes in competitions.
Machine Learning Might Replace Jobs
It is being discussed throughout the world that machine learning might replace many jobs because it is performing tasks with the efficiency of a human brain. In fact, there is some concern that machine learning will replace data scientists – and there seems to be basis for such apprehensions.
For the common users who do not have the data analysis skills but still need analytics in their day-to-day lives in varying degrees, it is not feasible to have computers that are capable of analyzing huge data volumes and offering analytics. But natural language processing (NLP) technologies can overcome this limitation by teaching computers to accept and process the natural, spoken language of humans. That way, the common user does not need sophisticated analytics capabilities or skills.
IBM believes that the need for data scientists can be minimized or eliminated with its product Watson natural language analytics platform. According to its vice president for Watson Analytics and Business Intelligence, Marc Atschuller, “With a cognitive system like Watson you just bring your question – or if you don’t have a question you just upload your data and Watson can look at it and infer what you might want to know.”
Automation is the next logical step for machine learning and we have already been experiencing the effects in our day-to-day lives – in e-commerce websites, Facebook friend suggestions, LinkedIn networking recommendations and Airbnb search rankings. Considering the examples given, no doubt can be cast upon the quality of output produced by automated machine learning systems. For all its qualities and benefits, the thought of machine learning causing huge unemployment may seem a bit of an overreaction. Machines have been replacing human beings in many areas of our lives for several decades and yet, human beings have evolved and adapted to stay relevant in the industry. Depending on the perspective, machine learning, for all its disruptiveness, is just another such wave to which people will adapt.