Notes topic想法主题
Research科研 →
Questions, evidence and the practice of research.研究问题、证据，以及科研过程中的认识。
Search搜索
Find a project, a specific problem or an idea. Search works in English and Chinese.查找项目、具体问题或观点，支持中英文搜索。
Notes topic想法主题
Learning学习 →
How understanding is built, tested and revised.理解如何建立、验证与修订。
Notes topic想法主题
Investing投资 →
Personal reflections on value, uncertainty and investment decisions.关于价值、不确定性与投资决策的个人认识。
Notes topic想法主题
Life生活 →
Observations about time, choices and everyday life.关于时间、选择与日常生活的观察。
Notes topic想法主题
Personal Finance理财 →
Personal thinking about budgets, saving and financial planning.关于预算、储蓄和财务规划的个人思考。
Project项目 · MoEFlow
MoEFlowMoEFlow →
An experimental MoE backend: follow the design, paired measurements, correctness gates and debugging records.实验性 MoE 执行后端：沿着设计、配对测量、正确性检查与排障记录深入项目。
Project项目 · vLLM
vLLMvLLM →
A reading and research notebook on irregular execution, CUDA Graph policies, memory and communication in LLM serving.围绕大模型服务中的不规则执行、CUDA Graph 策略、内存与通信展开的阅读及研究档案。
Project项目 · mini-DeepInfer
mini-DeepInfermini-DeepInfer →
DeepInfer's engine design in about 2,000 lines of PyTorch: follow a request from the queue to its next sampled token.用约 2,000 行 PyTorch 读懂 DeepInfer 的引擎设计：从请求入队一路跟到下一个采样 token。
Project项目 · DeepInfer
DeepInferDeepInfer →
An upstream proposal for an NVLink Engram fetch path on nodes without NCCL GIN.为不具备 NCCL GIN 的节点提交 NVLink Engram 读取路径提案。
Project项目 · TraceLab
TraceLabTraceLab →
Contributions to a session-aware vLLM workload runner and an SGLang backend adapter.参与支持会话的 vLLM 负载执行器与 SGLang 后端适配。
Project项目 · JittorInfer
JittorInferJittorInfer →
An upstream model-support proposal for the JittorInfer inference framework.面向 JittorInfer 推理框架的上游模型支持提案。
Working outline研究提纲 · vLLM
We Removed a CUDA Graph Fallback. It Got Slower.我们移除了 CUDA Graph 回退路径，结果反而变慢了 →
A working analysis of CUDA Graph coverage, irregular mixture-of-experts workloads, and the cost hidden behind a seemingly simpler execution path.一份关于 CUDA Graph 覆盖率、不规则混合专家负载，以及看似更简单的执行路径所隐藏成本的工作分析。
Working outline研究提纲 · vLLM
Communication Overlap Is Not Free通信重叠并非免费 →
A closer look at GPU contention, NCCL, and the limits of overlap in expert-parallel inference.重新审视专家并行推理中的 GPU 争用、NCCL，以及通信重叠的边界。
Working outline研究提纲 · MoEFlow
Building a Better Benchmark for MoE构建更可靠的 MoE 基准测试 →
A benchmark outline that treats routing distributions and tail behavior as first-class inputs.一份将路由分布与长尾行为视为一等输入的基准测试框架。
Working outline研究提纲 · vLLM
GPU Memory Behavior in Long-Context Inference长上下文推理中的 GPU 内存行为 →
Questions and instrumentation for understanding memory access patterns and bandwidth utilization in long-context serving.用于理解长上下文服务中内存访问模式与带宽利用率的问题和测量方法。
Working outline研究提纲
Profiling Grouped GEMM for Expert Computation面向专家计算的 Grouped GEMM 性能分析 →
A measurement plan for kernel-level grouped GEMM efficiency under imbalanced workloads.一份针对不均衡负载下内核级 Grouped GEMM 效率的测量计划。
Working outline研究提纲 · vLLM
Understanding Irregular MoE Execution on Modern GPUs理解现代 GPU 上的不规则 MoE 执行 →
Analyzing routing, token distribution, and execution behavior without reducing irregular workloads to an average case.分析路由、Token 分布与执行行为，而不把不规则负载简化成平均情形。
Working outline研究提纲 · vLLM
Lessons from LLM Serving Systems大语言模型服务系统经验 →
A concise index of systems questions to carry from one serving runtime to the next.一份可从一个服务运行时带到另一个运行时的系统问题简明索引。
Working outline研究提纲 · mini-DeepInfer
From 3% to 11.9% of DeepInfer: Three Prefill Kernels on 4 × B200从官方的 3% 到 11.9%：4 × B200 上的三项 prefill 优化 →
Paired, profiled measurements of the real DeepSeek-V4.1-Flash: tensor-core sparse attention, a fused indexer and a local-row MoE combine took mini-DeepInfer from 1,085 to 4,316 input tok/s/GPU — and showed where the remaining 8.4× lives.真实 DeepSeek-V4.1-Flash 上的配对、带 profile 的测量：tensor core 稀疏注意力、融合 indexer 和只合并本地行的 MoE，把 mini-DeepInfer 从 1,085 提到 4,316 输入 tok/s/GPU，并定位了剩余 8.4 倍差距所在。
Working outline研究提纲 · mini-DeepInfer
Reading DeepInfer in 2,000 Lines of PyTorch用约 2,000 行 PyTorch 读懂 DeepInfer →
mini-DeepInfer keeps DeepInfer's engine design — paged state, prefill/decode steps, zero-overhead scheduling and deterministic sampling — and drops the scale, so a DeepSeek-V4.1 request can be followed from the queue to its next token.mini-DeepInfer 保留 DeepInfer 的引擎设计——分页状态、prefill/decode 分步、零开销调度与确定性采样——去掉规模相关的部分，让一个 DeepSeek-V4.1 请求可以从入队一路跟到下一个 token。
Working outline研究提纲 · mini-DeepInfer
A DeepInfer-Style Runtime DesignDeepInfer 风格的运行时设计 →
The engineering boundaries, state invariants and verification rules mini-DeepInfer commits to, written before the implementation.mini-DeepInfer 要坚持的工程边界、状态不变量和验证方法，先于代码实现写成。
Working outline研究提纲 · mini-DeepInfer
Performance Report B1: A Fused Indexer Score性能分析报告 B1：融合 indexer 打分 →
A single fused kernel replaces the chunked einsum for indexer scoring: 12.2× faster per layer, +31% real 4-GPU prefill throughput, 3.1 GiB less peak memory.用一个融合 kernel 取代分块 einsum 做 indexer 打分：单层快 12.2 倍，真实四卡 prefill 吞吐 +31%，峰值显存降低 3.1 GiB。
Working outline研究提纲 · mini-DeepInfer
Performance Report A: Sparse Attention on Tensor Cores性能分析报告 A：稀疏注意力改用 tensor core →
Real V4.1 4-GPU prefill goes from 1,085 to 2,572 tok/s/GPU (2.37×) after the sparse-attention kernel moves from CUDA cores to tensor cores.稀疏注意力 kernel 从 CUDA core 改用 tensor core 后，真实 V4.1 四卡 prefill 吞吐从 1,085 提高到 2,572 tok/s/GPU（2.37 倍）。
Working outline研究提纲 · mini-DeepInfer
Performance Report C1: MoE Combine Over Local Rows, In-Place Dequantization性能分析报告 C1：MoE 只合并本地行，原地反量化 →
Combining only the local expert rows and dequantizing in place cuts routed experts from 526 to 252 ms/step and lifts real 4-GPU prefill to 4,316 tok/s/GPU.只合并本地专家的行并原地反量化，路由专家从 526 降到 252 ms/步，真实四卡 prefill 提高到 4,316 tok/s/GPU。
Working outline研究提纲 · mini-DeepInfer
mini-DeepInfer Technical Notes: From a Request to DeepSeek-V4.1mini-DeepInfer 技术说明：从请求到 DeepSeek-V4.1 →
A code tour of how mini-DeepInfer turns one request into reproducible V4.1 inference steps: lifecycle, attention state, paged blocks, ZOS, sampling and evidence.代码导览：mini-DeepInfer 如何把一个请求变成可复现的 V4.1 推理步骤——生命周期、注意力状态、分页 block、ZOS、采样与正确性证据。
Perspective技术观点
Nsight Compute Metrics for MoE KernelsMoE 内核的 Nsight Compute 指标 →
Profiling性能分析
Working outline研究提纲 · MoEFlow
One Interesting Finding About NCCL关于 NCCL 的一个有趣发现 →
Distributed Systems分布式系统
Perspective技术观点
GPU Stream Priorities in PracticeGPU 流优先级实践 →
CUDACUDA
Perspective技术观点
Triton Kernel Configurations for Grouped GEMMGrouped GEMM 的 Triton 内核配置 →
TritonTriton
Perspective技术观点 · MoEFlow
A Simple Way to Visualize Expert Load Imbalance可视化专家负载不均衡的简单方法 →
MoEMoE
Working outline研究提纲
Why 2 × 64 Is Not 1 × 128为什么 2 × 64 不等于 1 × 128 →
GEMMGEMM
Learning Paths学习路线
How to read this blog如何阅读这个博客 →
Project-driven learning: a short path in, a working mental model, and the ability to explore independently.以项目驱动学习：尽快入门，建立可运行的心智模型，具备独立探索的能力。
Learning Paths学习路线
From an expert-load model to a real project从专家负载模型走向真实项目 →
A reading order through the concepts, minimal experiment, measurement method and source materials.按顺序连接基本概念、最小实验、测量方法与原始材料。
Playbooks实用手册
Plan a paired benchmark you can inspect later设计一次日后能够复查的配对测试 →
Define the boundary, validate outputs and retain the individual observations.定义边界、验证输出，并保留每次观测。
Quickstarts最简入门
Find exact overlap before trusting an evaluation相信评测前，先检查精确重复 →
A small train/test overlap check with an explicit normalization rule. Python only.使用明确的规范化规则，检查训练集与测试集重叠。仅需 Python。
Quickstarts最简入门
A minimal experiment in expert-load imbalance专家负载不均衡的最小实验 →
Keep the total assignment count fixed and change only its distribution. Python standard library; no GPU required.固定总分配数，只改变分布。仅需 Python 标准库，无需 GPU。
Concepts核心概念
Useful work, padded work, and the critical path有效工作、填充开销与关键路径 →
Three distinctions to make before interpreting an optimization result.解读一次优化结果前，先分清三件事。
No matching content. Try a broader query or clear a filter.没有匹配内容，可以换个关键词或清除筛选。