[ICLR 2026] LongLive: Real-time Interactive Long Video Generation

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

Fast Multi-dimensional Sparse Attention

[ICML2025, NeurIPS2025 Spotlight] Sparse VideoGen 1 & 2: Accelerating Video Diffusion Transformers with Sparse Attention

[NeurIPS 2025] Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation

Trainable fast and memory-efficient sparse attention

Speed Always Wins: A Survey on Efficient Architectures for Large Language Models

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse–Linear Attention

[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference

Efficient triton implementation of Native Sparse Attention.

Code for paper: [ICLR2025 Oral] FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference

[CoLM'25] The official implementation of the paper <MoA: Mixture of Sparse Attention for Automatic Large Language Model Compression>

[ICLR 2026] SparseD: Sparse Attention for Diffusion Language Models

Advancing the frontier of efficient AI

Vortex: A Flexible and Efficient Sparse Attention Framework

[TIP-2025] Official Pytorch implementation of "Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution"

Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"

Dynamic Attention Mask (DAM) generate adaptive sparse attention masks per layer and head for Transformer models, enabling long-context inference with lower compute and memory overhead without fine-tuning.

The code implementation of paper "VORTA: Efficient Video Diffusion via Routing Sparse Attention"

Official repository for "SSA: Sparse Sparse Attention by Aligning Full and Sparse Attention Outputs in Feature Space"

A Curated Collection of Frontier Language Model Architectures

O(N) attention with a bounded inference KV cache. D4 Daubechies wavelet field + content-gated Q·K gather at dyadic offsets.

Toy Hydra prototypes: SSM + sparse attention + MoE + memory; synthetic benchmarks. Paper: https://arxiv.org/abs/2508.15099

Building Native Sparse Attention

HSPMN: Hybrid Sparse-Predictive Matter Network - LLM architecture optimized for Blackwell GPUs bridging O(N) and O(N^2) routing via ALF-LB

🦀 a rust project for reinventing current llm architecture to be highly efficient, more scalable and better performing ✨

Adaptive Sparse Attention Module with Flash Attention - 5.45x speedup on consumer GPUs

🔧 Optimize MoE model inference performance with automated Triton kernel tuning in the vLLM framework for various architectures and hardware setups.

Code for ACL 2025 paper: "Structural Deep Encoding for Table Question Answering"

Sparse attention via hypergraph partitioning for efficient long-context transformers