The Salt - Curated AI

The Salt - Curated AI

Home
Notes
AI Notebooks
AI Repositories
Related Articles
deep dive
Archive
About

deep dive

Jet-Nemotron: Searching for the Best Attention Architecture
DeltaNet + Hardware-aware Search
 • 
Benjamin Marie
Magistral: Advancing Reasoning with Efficient GRPO Training
No More KL Penalty, No Need for a Reference Model
 • 
Benjamin Marie
Qwen3 Technical Report: Reasoning in Pre-Training and Post-Training
Plus a Brief Look at the Limitations of the Multilingual Evaluation
 • 
Benjamin Marie
Qwen2.5-VL: High-Resolution Vision Encoding with Efficient Windowed Attention
Also impressive in language generation tasks!
 • 
Benjamin Marie
TÜLU 3: The Post-Training Recipe
SFT + DPO + RLVR
 • 
Benjamin Marie
TÜLU 3's High-Quality Synthetic Datasets for Post-Training LLMs
Made by GPT-4o
 • 
Benjamin Marie
Go Zero-Shot for Cheaper LLM Evaluations
Unless you use a generative benchmark
 • 
Benjamin Marie
Evaluating AdEMAMix: A New Optimizer for Faster, More Efficient LLM Training
But with hyperparameter values not easy to find!
 • 
Benjamin Marie
Qwen2-VL: How Does It Work?
One of the best VLMs for image captioning, visual question answering, optical character recognition (OCR), and multimodal chat.
 • 
Benjamin Marie
Q-GaLore: Pre-Train 7B Parameter LLMs from Scratch on a 16 GB GPU
Start now, get your model in 50 years!
 • 
Benjamin Marie
2
Add Code to Your Training Data for Better LLMs
But not too much!
 • 
Benjamin Marie
How Generative LLMs Achieve Top MMLU Scores without Generating Anything
what you think MMLU evaluates ≠ what MMLU really evaluates
 • 
Benjamin Marie
© 2025 Benjamin Marie
Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture