Attention Sinks and Retrieval Decay

3 selectedDifficulty 5-63 unseenView topic

Saved practice

Keep this quiz in your learner record

Answers count toward your profile, review queue, and next-topic suggestions. You can also use the quick practice below.

IntermediateNew

0 answered

3 intermediateAdapts to your performance

Question 1 of 3

120sintermediate (5/10)state theorem

Attention sinks (Xiao et al. 2023) are an empirical phenomenon in transformer language models. What are they?