Tech moves fast! Stay ahead of the curve with Techopedia!
Join nearly 200,000 subscribers who receive actionable tech insights from Techopedia.
Bidirectional Encoder Representations from Transformers, or BERT, is a new type of natural language processing (NLP) deep learning strategy in which deep neural networks utilize bidirectional models with unsupervised language representation.
One of the most basic ways to understand BERT is that because of a shortage of training data, engineers innovated to allow an NLP platform to draw from wide bodies of existing text, with new links between words and other data units. Specifically, bidirectionality, combined with a masking strategy, allows multilevel deep learning neural networks to more effectively mine archives to create NLP models.
Google engineers utilized tools like Tensorflow to create the BERT neural network architecture in which bidirectional flows are used to pre-train the network. Google suggests that BERT can allow users to train a state-of-the-art question and answer system in 30 minutes on a cloud TPU, or to utilize a GPU structure to complete the same task in just a few hours.
As a particular deep learning strategy, BERT has a lot of promise in the field of NLP. Meanwhile, other types of advanced innovations are making headway in game theory and related disciplines – where Q learning and outcome-based learning are driving quick progress.
In that sense, BERT is one of several key tools used to supercharge what new artificial intelligence systems can do across the board.