I am on the industry job market, and I hope to start interviewing from early 2025!

I am a fifth year PhD student at Carnegie Mellon University’s Machine Learning Department (CMU MLD). I am currently actively conducting research on LLM Agents, for application to both computer tasks and embodied domains. I am advised by Professors Ruslan Salakhutdinov and Yonatan Bisk.

My research is funded by the Apple AI/ ML scholars fellowship 2023.

I earned my B.S. and M.Eng from MIT EECS; I was awarded the Charles & Jennifer Johnson Thesis Award upon graduation. I have interned at Apple and Meta research. My research have been featured in CMU news multiple times; the most recent coverage is here.

Link to my up-to-date CV is here.

News

Jul. 3. 2024: Situated Instruction Following, my work during Meta internship, is accepted to ECCV 2024!

April. 1. 2024: Habitat 3.0 is accepted to ICLR 2024!

Dec. 21. 2023: I finished my internship at Meta.

Sep. 21. 2023: SPRING is accepted to NeurIPS 2023!

May. 30. 2023: I started my internship at Meta; I work with Roozbeh Mottaghi and am co-mentored by Devendra Singh Chaplot.

April. 10. 2023: Invited talk on Don’t copy the teacher at Yonsei University Vision and Learning Lab. Thank you for inviting me!

Mar. 9. 2023: I have been chosen as a recipient of the Apple AI/ ML scholars fellowship 2023!

Dec. 15. 2022: I finished my internship at Apple; Our paper is accepted as a main conference paper to IROS 2023. Thank you, Jian Zhang, Hubert Tsai, and the team!

Nov. 1. 2022: Don’t Copy the Teacher is accepted to EMNLP 2022!

May. 30. 2022: I started my internship at Apple.

Jan. 20. 2022: FILM is accepted to ICLR 2022!

Jan. 5. 2022: Invited talk on FILM at GIST Computer Vision Lab! Thank you for inviting me!

Jul. 6, 2021: I have been chosen as the second place recipient of the Charles & Jennifer Johnson Artificial Intelligence and Decision Making M.Eng Thesis Award for 2021, for my Master’s Thesis “Towards Knowledge-Based, Robust Question Answering.” Thank you once again, Pete and Preethi!

Publications

2024

Situated Instruction Following
ECCV 2024
So Yeon Min, Xavi Puig, Devendra Singh Chaplot, Tsung-Yen Yang, Akshara Rai, Priyam Parashar, Ruslan Salakhutdinov, Yonatan Bisk, Roozbeh Mottaghi

AgentKit: Flow Engineering with Graphs, not Coding
Arxiv Pre-print
Yue Wu, Yewen Fan, So Yeon Min, Shrimai Prabhumoye, Stephen McAleer, Yonatan Bisk, Ruslan Salakhutdinov, Yuanzhi Li, Tom Mitchell

GOAT: Go to any thing
RSS 2024
Matthew Chang, Theophile Gervet, Mukul Khanna, Sriram Yenamandra, Dhruv Shah, So Yeon Min, Kavit Shah, Chris Paxton, Saurabh Gupta, Dhruv Batra, Roozbeh Mottaghi, Jitendra Malik, Devendra Singh Chaplot

Habitat 3.0: A co-habitat for humans, avatars and robots
ICLR 2024
Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, Vladimír Vondruš, Theophile Gervet, Vincent-Pierre Berges, John M Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, Roozbeh Mottaghi

2023

SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning
NeurIPS 2023
Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Tom Mitchell, Yuanzhi Li

Plan, Eliminate, and Track–Language Models are Good Teachers for Embodied Agents
Arxiv Pre-print
Yue Wu, So Yeon Min, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Yuanzhi Li, Tom Mitchell, Shrimai Prabhumoye

Object Goal Navigation with End-to-End Self-Supervision
IROS 2023
So Yeon Min, Yao-Hung Hubert Tsai, Wei Ding, Ali Farhadi, Ruslan Salakhutdinov, Yonatan Bisk, Jian Zhang

EXCALIBUR: Encouraging and Evaluating Embodied Exploration
CVPR 2023
Hao Zhu, Raghav Kapoor, So Yeon Min, Winson Han, Jiatai Li, Kaiwen Geng, Graham Neubig, Yonatan Bisk, Aniruddha Kembhavi, Luca Weihs

2022

Don’t Copy the Teacher: Data and Model Challenges in Embodied Dialogue
EMNLP 2022
So Yeon Min, Hao Zhu, Yonatan Bisk, Ruslan Salakhutdinov

FILM: Following Instructions in Language with Modular Methods
ICLR 2022
So Yeon Min, Devendra Chaplot, Pradeep Ravikumar, Yonatan Bisk, Ruslan Salakhutdinov

Before 2022

Entity-Enriched Neural Models for Clinical Question Answering
Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing
Bhanu Pratap Singh Rawat, Wei-Hung Weng, So Yeon Min, Preethi Raghavan, Peter Szolovits

Advancing Seq2seq with Joint Paraphrase Learning
Proceedings of the 3rd Clinical Natural Language Processing Workshop
So Yeon Min, Preethi Raghavan, Peter Szolovits

TransINT: Embedding Implication Rules in Knowledge Graphs with Isomorphic Intersections of Linear Subspaces
Automated Knowledge Base Construction 2020
So Yeon Min, Preethi Raghavan, Peter Szolovits

Towards knowledge-based, robust question answering
Master’s Thesis