0. 示例博客
Published:
这是一个示例博客文章。
Blogs
0. Other Thoughts
Published:
Some of Other Thoughts
1. Position Embedding
Published:
About Position Embedding: Sinusoidal PE & RoPE
2. Thought of Position Embedding
Published:
Sinusoidal PE vs. RoPE
3. Proximal Policy Optimization (PPO)
Published:
Introduction of Proximal Policy Optimization (PPO)
4. Group Relative Policy Optimization (GRPO)
Published:
Introduction of Group Relative Policy Optimization (GRPO)
5. KL Divergence：从熵到 Forward/Reverse KL
Published:
Introduction of KL Divergence, entropy, cross entropy, and the difference between Forward KL and Reverse KL.
6. 常见损失函数的推导：从 MLE 到 MSE / MAE / CE
Published:
Derivation of common loss functions from a unified MLE / NLL perspective: MSE, MAE, CE, and beyond.
7. 信息瓶颈视角：从互信息推导 MSE / MAE / KL / InfoNCE
Published:
Deriving MSE, MAE, KL, and InfoNCE losses from the Information Bottleneck framework, unified via mutual information.
8. On-Policy Distillation：从 SFT/RL 到 Self-Distillation
Published:
On-policy distillation (OPD): combining SFT’s dense supervision with RL’s on-policy property, plus a tour of self-distillation works (OPSD, SDFT, SDPO, CRISP, ExOPD, GAD).
9. On-Policy Distillation：图解速览（外部资料汇编）
Published:
A visual companion to blog 8. Image panels collected from an external author’s note (watermarks preserved).