Neural Networks Mar 2026 · #81 most cited · 68 citations

Mamba-3: Improved Sequence Modeling using State Space Principles

Aakash Lahoti et al.

arXiv:2603.15569

In short

Mamba-3 improves state-space sequence models, which keep a fixed-size memory instead of attending over everything, with a more expressive recurrence, complex-valued state updates and a multi-input multi-output design. It beats comparable linear models and matches Mamba-2’s perplexity with half the state.

Why it matters

Inference cost now dominates, which makes fast, constant-memory architectures matter again.

Read first

The 3 Field Guide ideas this paper leans on.

Neural Networks · Landmark Recurrent Neural Network A neural network architecture with loops that allow information to persist, designed for sequential data like text and time series. Language & LLMs · Landmark Transformer A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model. Shipping AI · Standard Inference Latency The time delay between submitting input and receiving output from a deployed model, critical for real-time applications.

Starting from scratch? The full route 13 ideas · basics first

Machine Learning ✓ understood
Building systems that learn patterns from data instead of following hand-written rules, getting better at a task as they see more examples.
Neural Network ✓ understood
A computational model inspired by biological neural networks, consisting of interconnected nodes (neurons) organized in layers that process information through weighted connections.
Recurrent Neural Network · read first ✓ understood
A neural network architecture with loops that allow information to persist, designed for sequential data like text and time series.
Dataset ✓ understood
A collection of data examples used for training, validating, or testing machine learning models.
Feature ✓ understood
A single measurable property of an example, such as a house's floor area or how many links an email contains, used as an input to a model.
Deep Learning ✓ understood
A subset of machine learning that uses neural networks with multiple layers (deep neural networks) to learn hierarchical representations of data.
Representation Learning ✓ understood
Learning useful features or representations of data automatically, rather than hand-crafting them.
Embedding ✓ understood
A list of numbers (a vector) that represents a word, sentence, image or other item, learned so that similar items end up close together.
Attention Mechanism ✓ understood
A technique that lets a neural network weigh every part of its input when producing each output, focusing on the parts most relevant at that step.
Transformer · read first ✓ understood
A neural network architecture, introduced in 2017, built from stacked self-attention and feed-forward layers; the basis of nearly every modern large language model.
Training ✓ understood
The process of fitting a model to data by repeatedly measuring how wrong its outputs are and adjusting its parameters to reduce that error.
Inference ✓ understood
Running a trained model on new inputs to get predictions, with its weights frozen: the stage of a model's life that users actually interact with.
Inference Latency · read first ✓ understood
The time delay between submitting input and receiving output from a deployed model, critical for real-time applications.

In the frontier

Rank: #81 of 100
Citations: 68; as of Aug 9, 2026
Published: Mar 2026

Topics: Model architecture , Long context and attention , Efficiency and serving

Selection: 1kpapers.com by Together AI, most-cited as of Aug 9, 2026

Nearby papers

Canon · 1997 Long Short-Term Memory LSTMs powered speech recognition, translation and text generation for twenty years, until transformers took over. Canon · 2014 Neural Machine Translation by Jointly Learning to Align and Translate Attention was born here, three years before it became “all you need”. Canon · 2017 Attention Is All You Need Every major LLM, and most modern vision and speech models, is a transformer. Canon · 2018 BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding It established “pre-train once, fine-tune everywhere”, and still powers much of search and classification.