Value-Based Pre-Training with Downstream Feedback
Shuqi Ke, Giulia Fanti
NeurIPS 2026 · 2026
Conference version · Learning What to Predict: Downstream-Guided Task Design for Continued Pretraining
News
2026-04
Our pretraining project was recently approved for an ACCESS computing allocation, supported by NSF.
2026-03
Visit the Simons Institute for the Theory of Computing as part of the FCL program.
Selected Publications
Characterizing the Training Dynamics of Private Fine-Tuning with Langevin Diffusion
Shuqi Ke, Charlie Hou, Sewoong Oh, Giulia Fanti
TMLR · 2025
Reason for Future, Act for Now: A Principled Architecture for Autonomous LLM Agents
Zhihan Liu, Hao Hu, Shenao Zhang, Hongyi Guo, Shuqi Ke, Boyi Liu, Zhaoran Wang
ICML 2024 · 2024
Talks
Better Pre-Training with Downstream Feedback
Deep Learning: Classics and Trends · 2026-05-08
Teaching
- Fall 2026 · Teaching Assistant · CMU 18734 / 17731: Foundations of Privacy
Service
- Reviewer: ICML, ICLR, NeurIPS, AISTATS, TMLR, IEEE TSP
“Every paradigm shifting idea that humans have ever had began as heresy and lunacy.”