CALM and the Revolt Against the Token
Continuous Autoregressive Language Models challenge the token-by-token bottleneck and hint at a different future for language generation.
Model Architecture
14 posts
Distillation attacks on large language models: motives, actors and defences
A concise guide to model distillation as both useful compression technique and strategic attack surface in the LLM economy.
From PDE Guarantees to LLM Inference: What BEACONS Gets Right About Reliability
BEACONS offers a model for reliability that AI systems badly need: explicit bounds, checkable guarantees, and less benchmark theater.
Engram, DeepSeek, and the return of “memory” as an architectural primitive
DeepSeek's Engram reframes memory as an architectural primitive, suggesting models may need recall structures rather than ever-larger layers.
Recursive Language Models: when “more context” stops meaning “more tokens”
Recursive language models challenge the idea that longer context alone solves reasoning over large documents and codebases.
Beyond Fine-Tuning: What Apple’s Multimodal Sensor Fusion Study Reveals About LLMs and User Privacy
Apple's sensor-fusion research hints at a privacy-sensitive future where models learn from multimodal context without simply grabbing more cloud data.
Beyond the Token Stream: Investigating Introspective Awareness in Large Language Models
Interpretability research asks whether LLMs can detect their own internal states, moving introspection from philosophy toward experiment.
Transformers Are Injective: Why Your LLM Could Remember Everything (But Doesn’t)
If transformers are theoretically invertible, the question shifts from whether models lose information to how they manage and suppress it.
Elon Musk's Vision: Turning Tesla's Idle Fleet into a Global AI Inference Powerhouse
Musk's idea of using idle Teslas for inference turns a car fleet into a provocative vision of distributed AI infrastructure.
Small Models, Big Brains: Why Less Might Be the Future of AI Reasoning
Tiny reasoning models challenge the assumption that scale is always the path to intelligence, especially on structured problems.
How DeepSeek's Mathematical Optimizations Complement NVIDIA's NCCL for Efficient AI Training
DeepSeek's mathematical optimizations show how model design and NVIDIA communication infrastructure meet inside efficient training.
The Top 10 Unsolved Challenges in AI: A 2024 Retrospective
A year-end inventory of ten unresolved AI problems that still define the frontier despite rapid progress.
Exploring MM1: Apple's Advancement in Multimodal Large Language Models
Apple's MM1 research is presented as a step toward AI systems that understand text and images together.
Multimodality in Large Language Models: A Key to Versatile and Specialized Task Performance
Multimodal LLMs are explained as a key step toward systems that can reason across text, images, and other signals.