AIGlossaryLab
⌕
← Back
ModelsAdvanced2014

Attention

Attention Mechanism

Attention helps an AI focus on the parts of the information that matter most for the current task.

Explained simply

Attention helps an AI focus on the parts of the information that matter most for the current task.

1Information
→
2Focus on key parts
→
3Connect meaning
→
4Better output

At a glance

Category
Models
Difficulty
Advanced
Introduced
2014

Real example

When translating a sentence, attention helps the model connect a pronoun with the correct earlier word.

Why it matters

It gives you a clear way to understand where this idea fits in AI and how it affects the tools you use.

Timeline

Origins

The ideas behind Attention begin developing.

2014

Attention becomes a recognised term or technique.

Wider use

Research and practical applications increase.

Modern AI

Attention becomes connected to newer AI systems and products.

Today

Attention remains relevant in the models area of AI.

Dates for broad concepts may describe the period when the idea emerged or became widely used, rather than one exact invention date.

Learn next

Continue with these connected terms:

TransformerA transformer is the neural-network design behind most modern language models. It is especially good at understanding relationships within information.→BERTBERT is a language model designed to understand the meaning of words by looking at the words before and after them.→LLMA large language model is an AI trained on huge amounts of text so it can understand and generate language.→

Simple infographic

A quick visual way to understand Attention.

1Information
→
2Focus on key parts
→
3Connect meaning
→
4Better output
1 of 6

Was this helpful?

Your feedback helps improve this explanation.