Bidirectional Encoder Representations from Transformers (BERT)

Last Updated: July 5, 2020

Definition - What does Bidirectional Encoder Representations from Transformers (BERT) mean?

Bidirectional Encoder Representations from Transformers (BERT) is a type of natural language processing (NLP) deep learning strategy in which deep neural networks utilize bidirectional models with unsupervised language representation.

BERT is one of several key tools used to supercharge what artificial intelligence systems can do.

Techopedia explains Bidirectional Encoder Representations from Transformers (BERT)

One of the most basic ways to understand BERT is that because of a shortage of training data, engineers innovated to allow an NLP platform to draw from wide bodies of existing text, with new links between words and other data units. Specifically, bidirectionality, combined with a masking strategy, allows multilevel deep learning neural networks to more effectively mine archives to create NLP models.

Google engineers utilized tools like Tensorflow to create the BERT neural network architecture in which bidirectional flows are used to pre-train the network. Google suggests that BERT can allow users to train a state-of-the-art question and answer system in 30 minutes on a cloud TPU, or to utilize a GPU structure to complete the same task in just a few hours.

As a particular deep learning strategy, BERT has a lot of promise in the field of NLP. Meanwhile, other types of advanced innovations are making headway in game theory and related disciplines – where Q learning and outcome-based learning are driving quick progress.

Share this: