Research archive研究归档 · 14 entries篇
Technical articles on AI systems, GPU performance, and distributed inference.关于 AI 系统、GPU 性能与分布式推理的技术文章。
No writing matches that filter.没有符合条件的文章。
Paired, profiled measurements of the real DeepSeek-V4.1-Flash: tensor-core sparse attention, a fused indexer and a local-row MoE combine took mini-DeepInfer from 1,085 to 4,316 input tok/s/GPU — and showed where the remaining 8.4× lives.真实 DeepSeek-V4.1-Flash 上的配对、带 profile 的测量：tensor core 稀疏注意力、融合 indexer 和只合并本地行的 MoE，把 mini-DeepInfer 从 1,085 提到 4,316 输入 tok/s/GPU，并定位了剩余 8.4 倍差距所在。
mini-DeepInfer keeps DeepInfer's engine design — paged state, prefill/decode steps, zero-overhead scheduling and deterministic sampling — and drops the scale, so a DeepSeek-V4.1 request can be followed from the queue to its next token.mini-DeepInfer 保留 DeepInfer 的引擎设计——分页状态、prefill/decode 分步、零开销调度与确定性采样——去掉规模相关的部分，让一个 DeepSeek-V4.1 请求可以从入队一路跟到下一个 token。
A single fused kernel replaces the chunked einsum for indexer scoring: 12.2× faster per layer, +31% real 4-GPU prefill throughput, 3.1 GiB less peak memory.用一个融合 kernel 取代分块 einsum 做 indexer 打分：单层快 12.2 倍，真实四卡 prefill 吞吐 +31%，峰值显存降低 3.1 GiB。
Real V4.1 4-GPU prefill goes from 1,085 to 2,572 tok/s/GPU (2.37×) after the sparse-attention kernel moves from CUDA cores to tensor cores.稀疏注意力 kernel 从 CUDA core 改用 tensor core 后，真实 V4.1 四卡 prefill 吞吐从 1,085 提高到 2,572 tok/s/GPU（2.37 倍）。
Combining only the local expert rows and dequantizing in place cuts routed experts from 526 to 252 ms/step and lifts real 4-GPU prefill to 4,316 tok/s/GPU.只合并本地专家的行并原地反量化，路由专家从 526 降到 252 ms/步，真实四卡 prefill 提高到 4,316 tok/s/GPU。
The engineering boundaries, state invariants and verification rules mini-DeepInfer commits to, written before the implementation.mini-DeepInfer 要坚持的工程边界、状态不变量和验证方法，先于代码实现写成。
A code tour of how mini-DeepInfer turns one request into reproducible V4.1 inference steps: lifecycle, attention state, paged blocks, ZOS, sampling and evidence.代码导览：mini-DeepInfer 如何把一个请求变成可复现的 V4.1 推理步骤——生命周期、注意力状态、分页 block、ZOS、采样与正确性证据。
A working analysis of CUDA Graph coverage, irregular mixture-of-experts workloads, and the cost hidden behind a seemingly simpler execution path.一份关于 CUDA Graph 覆盖率、不规则混合专家负载，以及看似更简单的执行路径所隐藏成本的工作分析。
A closer look at GPU contention, NCCL, and the limits of overlap in expert-parallel inference.重新审视专家并行推理中的 GPU 争用、NCCL，以及通信重叠的边界。
Analyzing routing, token distribution, and execution behavior without reducing irregular workloads to an average case.分析路由、Token 分布与执行行为，而不把不规则负载简化成平均情形。
A measurement plan for kernel-level grouped GEMM efficiency under imbalanced workloads.一份针对不均衡负载下内核级 Grouped GEMM 效率的测量计划。
Questions and instrumentation for understanding memory access patterns and bandwidth utilization in long-context serving.用于理解长上下文服务中内存访问模式与带宽利用率的问题和测量方法。
A concise index of systems questions to carry from one serving runtime to the next.一份可从一个服务运行时带到另一个运行时的系统问题简明索引。
A benchmark outline that treats routing distributions and tail behavior as first-class inputs.一份将路由分布与长尾行为视为一等输入的基准测试框架。
