How can a 'random walk' be helpful in machine learning algorithms?

Q:

How can a "random walk" be helpful in machine learning algorithms?

A:

In machine learning, a "random walk" approach can be applied in various ways to help the technology sift through the large training data sets that provide the basis for the machine's eventual comprehension.

A random walk, mathematically, is something that can be described in several different technical ways. Some describe it as a randomized collection of variables; others might call it a "stochastic process." Regardless, the random walk contemplates a scenario where a variable set takes a path that is a pattern based on random increments, according to an integer set: For example, a walk on a number line where the variable moves plus or minus one at every step.


Free Download: Machine Learning and Why It Matters


As such, a random walk can be applied to machine learning algorithms. One popular example described in a piece in Wired applies to some groundbreaking theories on how neural networks can work to simulate human cognitive processes. Characterizing a random walk approach in a machine learning scenario last October, Wired writer Natalie Wolchover attributes much of the methodology to data science pioneers Naftali Tishby and Ravid Shwartz-Ziv, who suggest a road map for various phasing of machine learning activity. Specifically, Wolchover describes a "compression phase" that is related to filtering out irrelevant or semi-relevant features or aspects in an image field according to the program's intended purpose.

The general idea is that, during a complex and multi-step process, the machine works to either "remember" or "forget" different elements of the image field to optimize results: In the compression phase, the program could be described as "zeroing in" on important features to the exclusion of peripheral ones.

Experts use the term "stochastic gradient descent" to refer to this type of activity. Another way to explain it with less technical semantics is that the actual programming of the algorithm changes by degrees or iterations, to "fine tune" that learning process that is taking place according to "random walk steps" that will eventually lead toward some form of synthesis.

The rest of the mechanics are very detailed, as engineers work to move machine learning processes through the compression phase and other related phasing. The broader idea is that the machine learning technology changes dynamically over the life span of its evaluation of big training sets: Instead of looking at different flash cards in individual instances, the machine looks at the same flash cards multiple times, or pulls flash cards at random, looking at them in a changing, iterative, randomized way.

The above random walk approach is not the only way that the random walk can be applied to machine learning. In any case where a randomized approach is needed, the random walk might be part of the mathematician or data scientist's tool kit, in order to, again, refine the data learning process and provide superior results in a quickly emerging field.

In general, the random walk is associated with certain mathematical and data science hypotheses. Some of the most popular explanations of a random walk have to do with the stock market and individual stock charts. As popularized in Burton Malkiel's "A Random Walk Down Wall Street," some of these hypotheses argue that the future activity of a stock is essentially unknowable. However, others suggest that random walk patterns can be analysed and projected, and it's no coincidence that modern machine learning systems are often applied to stock market analysis and day trading. The pursuit of knowledge in the tech field is and has always been entwined with the pursuit of knowledge about money, and the idea of applying random walks to machine learning is no exception. On the other hand, the random walk as a phenomenon can be applied to any algorithm for any purpose, according to some of the mathematical principles mentioned above. Engineers might use a random walk pattern to test an ML technology, or to orient it toward feature selection, or for other uses related to the gigantic, byzantine castles in the air that are modern ML systems.

Have a question? Ask us here.

View all questions from Justin Stoltzfus.

Share this:
Written by Justin Stoltzfus
Profile Picture of Justin Stoltzfus
Justin Stoltzfus is a freelance writer for various Web and print publications. His work has appeared in online magazines including Preservation Online, a project of the National Historic Trust, and many other venues.
 Full Bio