CV

Education

Renmin University of China — Sep. 2022 – Present

Ph.D. in Artificial Intelligence, School of Artificial Intelligence | GPA: 3.94/4.0 | Beijing, China

Advisor: Prof. Yankai Lin | Co-advised by Prof. Zhiyuan Liu (THUNLP, Tsinghua University)

Research focus: Deep learning, reinforcement learning, model distillation for LLMs and LLM-based agent systems.

Dalian University of Technology — Sep. 2017 – Jul. 2022

B.Eng. in Digital Media Technology | GPA: 4.02/5.0 (90.02/100) | Rank: 1/99 (Comprehensive), 2/99 (Academic) | Dalian, China

Honors: National Scholarship (×2), Outstanding Graduate of Dalian, First-Class Academic Scholarship (×3)

Relevant Coursework: Computer Vision (100), Advanced Programming (98), Signal Processing (98), Discrete Mathematics (97), Probability Theory (95)

First Author | THUNLP & RUCBM | Baidu Search, Research Intern | Supervisor: Yankai Lin | Beijing, China

Reinforcement Learning with Verifiable Rewards (RLVR) has advanced LLM reasoning but remains constrained by inefficient exploration under limited rollout budgets, leading to low sampling success and unstable training. Discovered that many exploration failures arise not from problem difficulty, but from a small number of prompt tokens that introduce interference.
Proposed the Less Noise Sampling Framework (LENS), an online selective rollout framework: Stage 1 identifies and removes interference tokens within low-success prompts via Interference Score, producing purified prompts that yield a higher proportion of successful rollouts; Stage 2 transfers successful rollouts from the purification process to supervise policy optimization on the original noisy prompts, enabling the model to learn to ignore interference in real-world noisy prompting settings.
LENS significantly outperforms GRPO, achieving a Pareto improvement in performance–efficiency trade-offs, with an average +3.88% gain and over 1.6× faster convergence across 7 math reasoning benchmarks. Also surpasses both scaling exploration (GRPO with doubled rollouts) and zero-variance prompt filtering baselines (DAPO and GRESO) while using substantially fewer computational resources. Evaluated on Llama-3.2, Qwen-2.5/3 and Qwen-3 model families.

First Author | THUNLP & RUCBM | Modelbest Inc., Research Intern | Supervisor: Yankai Lin | Beijing, China

Proposed Learning to Focus (LeaF), a two-stage framework that reduces the influence of distracting tokens to improve LLM reasoning accuracy and generation quality in complex tasks.
Confounding Token Detection: LeaF identifies confounding tokens via teacher–student gradient-based comparisons on sensitivity discrepancies, effectively pruning interfering tokens from training corpora and generating high-quality counterfactual samples through span pruning.
Causal Attention Distillation: By contrasting model outputs on original and counterfactual samples through a hybrid distillation loss (standard KL divergence + counterfactual KL divergence), LeaF guides the student model to learn genuine causal dependencies, thereby improving reasoning capability.
LLaMA-1B/3B-LeaF and Qwen2.5-1.5B-LeaF outperformed standard distillation on math reasoning (GSM8K, MATH, OlympiadBench) by +2.41%, on code generation (HumanEval+, LeetCode, LiveCodeBench) by +2.48%, and on multi-hop reasoning (2WikiMQA, Musique, HotpotQA) by +3.24%.
Interpretability experiments confirm that LeaF enhances model attention to critical information while reducing reliance on irrelevant tokens, improving model reasoning capability.

First Author | THUNLP & RUCBM | Tencent WeChat, Research Intern | Supervisor: Yankai Lin | Beijing, China

Addressed the conflict problem in multi-objective alignment for LLMs. Innovatively introduced multi-objective preference tokens to specify optimization directions (e.g., helpfulness, honesty), transforming multi-objective optimization into a conditional multi-objective optimization problem, enabling controllable optimization over different objectives and mitigating the "alignment tax" issue.
Proposed Controllable Preference Optimization (CPO), consisting of two stages: Controllable Preference Supervised Fine-Tuning (CPSFT) and Controllable Direct Preference Optimization (CDPO). CPSFT controls generation quality via preference tokens; CDPO maximizes preferences along selected dimensions while constraining specific dimensions, effectively reducing inter-objective conflicts.
Experiments demonstrate that CPO surpasses traditional methods (SFT, PPO, DPO, Curry-DPO) across all three objectives (helpfulness, honesty, harmlessness), achieving dual improvements in both controllability and overall performance. Also introduced the UltraSafety dataset, addressing safety data gaps in UltraFeedback.

Yiju Guo, Tianyi Hu, Zexu Sun, Yankai Lin. "Less Noise, More Voice: Reinforcement Learning for Reasoning via Instruction Purification." In ACL 2026.
Yiju Guo, Wenkai Yang, Zexu Sun, Ning Ding, Zhiyuan Liu, Yankai Lin. "Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruning." In NeurIPS 2025.
Yiju Guo*, Ganqu Cui*, et al. "Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment." In EMNLP 2024. (128 citations)
Zexu Sun, Yiju Guo, Yankai Lin, et al. "Uncertainty and Influence Aware Reward Model Refinement for RLHF." In ICLR 2025.
Wenkai Yang, Weijie Liu, Ruobing Xie, Yiju Guo, et al. "LaSeR: RL with Last-Token Self-Rewarding." In ICLR 2026.
Shengda Fan*, Xuyan Ye*, Yupeng Huo, Zhi-Yuan Chen, Yiju Guo, et al. "AgentProcessBench: Diagnosing Step-Level Process Quality in Tool-Using Agents." In KDD D&B Track 2026.

Programming: Python (primary), C/C++
Deep Learning Frameworks: PyTorch, Hugging Face Transformers, DeepSpeed, Megatron-LM
Model Training & Optimization: SFT, Knowledge Distillation, RLHF/RLAIF (PPO, GRPO, DPO); Attention mechanism optimization
Distributed Systems: Multi-GPU distributed training (VeRL, OpenRLHF), Linux, Git, Docker
Data Engineering: Large-scale training data construction, preference data annotation (UltraSafety), agent process reward data annotation (AgentProcessBench), and pipeline development

2020, 2021 — National Scholarship (×2), Ministry of Education of China
2022 — Outstanding Graduate of Dalian, Dalian Municipal Government
2019–2021 — First-Class Academic Scholarship (×3), Innovation Scholarship (×3)
2020 — MatherCup National Mathematical Modeling Challenge, First Prize (National Level)
2020 — National College Mathematical Modeling Competition, Second Prize (National Level)
2019, 2021 — MCM/ICM (International Mathematical Modeling Contest), Honorable Mention (×2)