Engineering ·
RL setup for calibrated probabilities in decisions
Radar briefing on a reinforcement learning setup for decision models that return calibrated probabilities instead of generated text. Publication date: October 4, 2026.
Radar
Daily bookmarks: relevant news, facts and discoveries.
Articles 1219
Engineering ·
VeriHarness verifies agent rollouts in long-horizon tasks
The paper presents VeriHarness, which checks agent claims in long tasks using a fixed base model, without reference answers or rubrics during testing.
Engineering ·
Distillation study: per-token KL direction matters more than rollout policy
A systematic study reports that the direction of per-token KL matters more than the rollout policy in on-policy versus off-policy distillation, questioning the idea that on-policy rollouts are inherently better.
Engineering ·
GOAT adds trainable priors to attention
Radar: a paper that reads standard attention as Entropic Optimal Transport with an implicit uniform prior and proposes GOAT, with a continuous trainable prior that stays compatible with optimized kernels such as FlashAttention.
Engineering ·
KohakuFA targets backward-pass precision problems
The publication claims cuDNN, FA4 and FlexAttention have precision problems in backward passes and presents KohakuFA as an alternative kernel, with a public GitHub repository for inspection.
Engineering ·
Looped Diffusion Transformer reuses blocks during denoising for image generation
Radar briefing dated 3 October 2026 on a text-to-image model that reuses Transformer blocks at each denoising step, with the performance claim stated by the original post.
Engineering ·
LoopCD: training-free contrastive decoding for Looped Transformers
Radar: the paper presents LoopCD, a training-free contrastive decoding method for recurrent Transformers that uses representations from earlier loops as weaker predictions to select tokens.
Engineering ·
LoopCD improves decoding in looped Transformers
Radar: LoopCD refines decoding in looped Transformers by using the gap between early and late predictions of recurrent states, with no extra training, and reports accuracy gains on AIME 2024 with the Ouro-2.6B-Thinking model.
Engineering ·
SFT with sampling for post-training: claims on generalization and forgetting
A post describes adding sampling to the post-training stack and claims SFT can compete with current methods, with better generalization and less forgetting in some cases.
Engineering ·
Training language models across multiple agent harnesses
A guide describes a proxy that records token IDs and sampled logprobs to train models with RL across four agent harnesses, with reported score gains and a comparison with imitation learning.
Engineering ·
When Transformers Need Chain of Thought
The article examines whether chains of thought do computational work in serial tasks; according to the post, removing them lowers accuracy to the shallow-circuit limit.
Engineering ·
MOPD combines capabilities from specialist models
Published on October 1, 2026, the text presents multi-teacher on-policy distillation (MOPD): a method that trains a student model on its own completions, using teachers’ per-token log probabilities and a reverse KL objective.
Engineering ·
Local inference: evaluate performance and exposure in the same test
Running a model close to data can change the architecture. It does not replace input processing, access control or real evaluation.
Engineering ·
Jagged Flash Attention with TLX on NVIDIA Blackwell
A PyTorch blog post describes Jagged Flash Attention, an attention kernel behind Meta's Generative Ads Model, developed with TLX for NVIDIA Blackwell. The post reports better performance than FlashAttention-4 on the GEM jagged shapes.
Engineering ·
Looped-DiT reuses Transformer blocks for image generation
Published on October 1, 2026, the post says Looped-DiT reuses Transformer blocks during denoising and that a 260-million-parameter model beat one 6.5 times larger on text-to-image benchmarks, with 4.9 times less inference computation.
Engineering ·
Radar: cache and attention techniques in LLMs
A post dated October 1, 2026 reports 890 bytes of global KV cache per token and a persistent cache eight times smaller in a language model.
Engineering ·
Radar: attention factorization and gradient flow
A September 30, 2026 publication argues that factoring S = WWᵀ changes gradient flow in attention and removes an information-exponent bottleneck that can stall unfactorized models.
Engineering ·
DeepEP-Ascend: communication for Huawei NPUs
Published on September 30, 2026, the item describes DeepEP-Ascend as a high-performance communication library for machine learning training and inference on Huawei Ascend NPUs.
Engineering ·
Efficient Transformer inference at scale
An article by Reiner Pope and coauthors discusses scaling Transformer inference efficiently, a relevant topic for engineers evaluating ways to serve these models.
Engineering ·
ProRL Agent separates rollouts from LLM training
Published on September 30, 2026, ProRL Agent is open-source infrastructure for multi-turn LLM agent rollouts. Its rollout-as-a-service model separates trajectory generation, environment orchestration, and evaluation from policy optimization.