Models
Datasets
Spaces
Docs
Enterprise
Pricing
Log In
Sign Up

Collections

Discover the best community collections!

Collections including paper arxiv:2404.07129

Interpretability

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - XAI - Attention - LayerNorm

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Phase Change - Delay - Classes and Labels

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - Training - ICL - Induction Circuit Evolution

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Head - Num Labels vs Classes - Loss

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Residual Head Hypothesis

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Phase Change Delay - Large Vocabulary Size

Larger vocab is better compression, but may result in longer training ICL phase change delays due to the slower Induction Head Copy Subcircuit (C)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Head - Copy vs QK Match

See figure 6: Classes vs labels in columns B and C. Subcircuit B delays phase change on number classes vs C delays on number of labels (dramatically)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Circuit - Data Dependent Learning

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - XAI - Induction Head - Phase Change - Components

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions

Paper • 2403.07809 • Published Mar 12, 2024 • 1

Interpretability

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Residual Head Hypothesis

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - XAI - Attention - LayerNorm

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Phase Change Delay - Large Vocabulary Size

Larger vocab is better compression, but may result in longer training ICL phase change delays due to the slower Induction Head Copy Subcircuit (C)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Phase Change - Delay - Classes and Labels

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Head - Copy vs QK Match

See figure 6: Classes vs labels in columns B and C. Subcircuit B delays phase change on number classes vs C delays on number of labels (dramatically)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - Training - ICL - Induction Circuit Evolution

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Circuit - Data Dependent Learning

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - ICL - Induction Head - Num Labels vs Classes - Loss

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3

Papers - XAI - Induction Head - Phase Change - Components

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Paper • 2404.07129 • Published Apr 10, 2024 • 3
pyvene: A Library for Understanding and Improving PyTorch Models via Interventions

Paper • 2403.07809 • Published Mar 12, 2024 • 1

Previous
1
2
3
4
Next

Company

TOS Privacy About Careers

Website

Models Datasets Spaces Pricing Docs