Margaret Rouse is an award-winning technical writer and teacher known for her ability to explain complex technical subjects simply to a non-technical, business audience. Over…
Bidirectional Encoder Representations from Transformers (BERT) is a deep learning strategy for natural language processing (NLP) that helps artificial intelligence (AI) programs understand the context of ambiguous words in text.
Applications that use BERT are able to predict the correct meaning of a synonym by processing text in both left-to-right and right-to-left directions simultaneouosly.
Google engineers used tools like Tensorflow to create the BERT neural network architecture. Until BERT, AI programs were unidirectional, which means they could only process text from left-to-right.
BERT's bidirectionality, combined with a masking strategy that teaches the programming how to predict the meaning of an ambiguous term, allows deep learning neural networks to use unsupervised learning techniques to create new NLP models.
This approach to natural language understanding (NLU) is so powerful that Google suggests that users can use BERT to train a state-of-the-art question and answer system in about 30 minutes as long as they have enough training data.
Techopedia’s editorial policy is centered on delivering thoroughly researched, accurate, and unbiased content. We uphold strict sourcing standards, and each page undergoes diligent review by our team of top technology experts and seasoned editors. This process ensures the integrity, relevance, and value of our content for our readers.
Margaret is an award-winning technical writer and teacher known for her ability to explain complex technical subjects to a non-technical business audience. Over the past twenty years, her IT definitions have been published by Que in an encyclopedia of technology terms and cited in articles by the New York Times, Time Magazine, USA Today, ZDNet, PC Magazine, and Discovery Magazine. She joined Techopedia in 2011. Margaret's idea of a fun day is helping IT and business professionals learn to speak each other’s highly specialized languages.
What are Integrated Payroll Services? Integrated payroll services are features and functionalities built into other HR software, most commonly an...
Aleksandar StevanovicSoftware Reviews Expert
What is the Work Opportunity Tax Credit? The Work Opportunity Tax Credit (WOTC) is defined by the U.S. Department of...
What is Earned Income Tax Credit? Earned Income Tax Credit (EITC) is a refundable tax credit awarded to workers and...
Trending NewsLatest GuidesReviewsTerm of the Day