On This Day in AI History

A CHRONICLE OF MACHINE INTELLIGENCE

DARTMOUTH TO DEEP LEARNING · THE PROGRAMS, PAPERS & PEOPLE · 1956–PRESENT

← Back

"Attention Is All You Need" introduces the Transformer

Transformer · June 12, 2017
Transformer
System
Transformer
Series
Google
Venue
Google Brain / Google Research, Mountain View, California
Date
June 12, 2017
Lab / Organization
Google
Nickname
"Attention Is All You Need" introduces the Transformer
Researchers
Ashish Vaswani
Noam Shazeer
Jakob Uszkoreit

Google researchers posted 'Attention Is All You Need' to arXiv on June 12, 2017, introducing the Transformer architecture built entirely on attention mechanisms rather than recurrence or convolution. The design enabled far more parallelizable training and better long-range modeling. It became the foundation for virtually all modern large language models, including the GPT and BERT families.

Were you working in the field when it happened?

Comments

Loading…