On This Day in AI History

Google releases BERT for language understanding

Google AI released the BERT paper on arXiv on October 11, 2018, presenting a bidirectional Transformer pretrained with masked-language modeling that set new state-of-the-art results across many NLP benchmarks. BERT popularized the pretrain-then-fine-tune paradigm for language tasks. It was soon integrated into Google Search and became one of the most influential NLP models.