The Salt - Curated AI

Home
Notes
AI Notebooks
AI Repositories
Related Articles
Archive
About
Evaluating AdEMAMix: A New Optimizer for Faster, More Efficient LLM Training
But with hyperparameter values not easy to find!
 • 
Benjamin Marie
Qwen2-VL: How Does It Work?
One of the best VLMs for image captioning, visual question answering, optical character recognition (OCR), and multimodal chat.
 • 
Benjamin Marie
Q-GaLore: Pre-Train 7B Parameter LLMs from Scratch on a 16 GB GPU
Start now, get your model in 50 years!
 • 
Benjamin Marie
2
Most Popular
View all
LongRoPE: Towards Unlimited Context Length for the Transformer
 • Benjamin Marie
FineWeb-Edu: How to Make a Very High-Quality Dataset to Pre-train LLMs
 • Benjamin Marie
3
MoRA: A High-Rank Alternative to LoRA
 • Benjamin Marie
Why Can't We Compare the Perplexity of Two Different Models?
 • Benjamin Marie
1
Pre-train LLMs on Millions of Synthetic Instructions
 • Benjamin Marie
How Generative LLMs Achieve Top MMLU Scores without Generating Anything
 • Benjamin Marie
How Generative LLMs Achieve Top MMLU Scores without Generating Anything
what you think MMLU evaluates ≠ what MMLU really evaluates
 • 
Benjamin Marie
Mixture-of-Experts: Mixture-of-Head Attention and Embedding Model
The Weekly Salt #40
 • 
Benjamin Marie
Cancelling Attention Noise with Differential Transformer
The Weekly Salt #39
 • 
Benjamin Marie
Cross Capabilities of LLMs and Contextual Document Embeddings
The Weekly Salt #38
 • 
Benjamin Marie
LLMs Can Follow Instructions Without Instruction Tuning
The Weekly Salt #37
 • 
Benjamin Marie
SCoRe: Teach LLMs to Self-Correct
The Weekly Salt #36
 • 
Benjamin Marie
New Advances in Linear-time Sequence Modeling
The Weekly Salt #35
 • 
Benjamin Marie
The Salt - Curated AI
The Salt - Curated AI
The Salt offers weekly reviews and in-depth analyses of the latest AI papers. If you want to stay informed of recent progress in AI without reading much, The Salt is for you!
Recommendations
The Kaitchup – AI on a Budget
The Kaitchup – AI on a Budget
Benjamin Marie

The Salt - Curated AI

AboutArchiveRecommendationsSitemap
© 2024 Benjamin Marie
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great culture