Question

How can containerization be a good choice for a machine learning project environment?

Answer
By Justin Stoltzfus | Last updated: September 10, 2018

Some companies are moving toward containerization for machine learning projects, based on some of the benefits that container setups offer in terms of platforms and software environments.

Machine learning is complex – the algorithms themselves perform a lot of very detailed and complicated actions on data. However, the value proposition is, in some ways, pretty simple – the machine learning algorithms work on data coming in from storage environments.


Free Download: Machine Learning and Why It Matters


The use of containers involves how engineers put the data into the machine learning environment, and how the algorithms work.

Engineers can use container virtualization either to house the data, or to deploy the code that runs the algorithms. Although containers can be helpful for data, their main benefit probably comes in their use to house algorithm code.

Container architectures feature self-contained apps and codebases. Each container gets its own operating system clone, and it gets a full operating environment for the app or code function set that lives inside it.

As a result, the individual apps, microservices or codebases that are in each container can be deployed in very versatile ways. They can be deployed in different platforms and different environments.

Now, suppose you're trying to ramp up a machine learning project in which various algorithms have to work on various pieces of data in an iterative way. If you get tired of dealing with cross-platform challenges or dependency issues or situations where bare-metal deployment is difficult, containers can be the solution.

Essentially, the containers provide a way to host code. Experts talk about deploying the containers up against the stored data to get good results.

”(The apps) can be mixed and matched in any number of platforms, with virtually no porting or testing required,” David Linthicum writes in a TechBeacon article that expounds on the value of containers for machine learning projects, “because they exist in containers, they can operate in a highly distributed environment, and you can place these containers close to the data the applications are analyzing.”

Linthicum goes on to talk about exposing machine learning services as microservices. This allows external applications – container-based or not – to leverage these services at any time without having to move the code inside the application.

In a very basic sense, container deployment is all about making the functionality of the machine learning program more adaptable – doing away with silos and unnecessary connections – and again, dependencies – that can cripple a project. For a lean, mean machine learning project, if the individual parts of the algorithms or applications or functionality are housed inside containers, it's easy to micromanage these self-contained pieces and create complex machine learning product projects accordingly.

Share this Q&A

  • Facebook
  • LinkedIn
  • Twitter

Tags

Artificial Intelligence Emerging Technology Machine Learning

Written by Justin Stoltzfus | Contributor, Reviewer

Profile Picture of Justin Stoltzfus

Justin Stoltzfus is a freelance writer for various Web and print publications. His work has appeared in online magazines including Preservation Online, a project of the National Historic Trust, and many other venues.

More Q&As from our experts

Related Terms

Related Articles

Term of the Day

Session Cookie

A session cookie is a file containing an identifier (a string of letters and numbers) that a website server sends to a...
Read Full Term

Tech moves fast! Stay ahead of the curve with Techopedia!

Join nearly 200,000 subscribers who receive actionable tech insights from Techopedia.

Resources
Go back to top