I am on the industry job market, and will start interviewing from early 2025!

I am a fifth year PhD student at Carnegie Mellon University’s Machine Learning Department (CMU MLD). I am actively conducting research on Embodied and LLM Agents, for application to both physical and computer domains. I am advised by Professors Ruslan Salakhutdinov and Yonatan Bisk.

My research is funded by the Apple AI/ ML scholars fellowship 2023.

I earned my B.S. and M.Eng from MIT EECS; I was awarded the Charles & Jennifer Johnson Thesis Award upon graduation. I have interned at Apple AI/ML and Meta FAIR. My research have been featured in CMU news multiple times; the most recent coverage is here.

Link to my up-to-date CV is here.

Talks

Sep 30, 2024: Invited Talk at the Multimodal Agents Workshop at ECCV 2024!
The topic of the talk was “Progress and Challenges in Non-parametric and Parametric Components of Embodied Agents.”
The slides can be seen here

April. 10. 2023: Invited Talk at the Yonsei University Vision and Learning Lab!

Jan. 5. 2022: Invited Talk at the GIST computer vision group!

Publications

The years (e.g., 2025) refer to the expected or actual dates when the conference takes place at the scheduled venue.

2025

Training Belief and Confidence Aware LLM Agents
First author work in progress

2024

Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation
Arxiv Pre-print
So Yeon Min*, Quanting Xie*, Tianyi Zhang, Aarav Bajaj, Ruslan Salakhutdinov, Matthew Johnson-Roberson, Yonatan Bisk

Tools Fail: Detecting Silent Errors in Faulty Tools
EMNLP 2024
Jimin Sun, So Yeon Min, Yingshan Chang, Yonatan Bisk

Situated Instruction Following
ECCV 2024
So Yeon Min, Xavi Puig, Devendra Singh Chaplot, Tsung-Yen Yang, Akshara Rai, Priyam Parashar, Ruslan Salakhutdinov, Yonatan Bisk, Roozbeh Mottaghi

AgentKit: Flow Engineering with Graphs, not Coding
COLM 2024
Yue Wu, Yewen Fan, So Yeon Min, Shrimai Prabhumoye, Stephen McAleer, Yonatan Bisk, Ruslan Salakhutdinov, Yuanzhi Li, Tom Mitchell

GOAT: Go to any thing
RSS 2024
Matthew Chang, Theophile Gervet, Mukul Khanna, Sriram Yenamandra, Dhruv Shah, So Yeon Min, Kavit Shah, Chris Paxton, Saurabh Gupta, Dhruv Batra, Roozbeh Mottaghi, Jitendra Malik, Devendra Singh Chaplot

Habitat 3.0: A co-habitat for humans, avatars and robots
ICLR 2024
Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yen Yang, Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, Vladimír Vondruš, Theophile Gervet, Vincent-Pierre Berges, John M Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, Roozbeh Mottaghi

2023

SPRING: GPT-4 Out-performs RL Algorithms by Studying Papers and Reasoning
NeurIPS 2023
Yue Wu, So Yeon Min, Shrimai Prabhumoye, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Tom Mitchell, Yuanzhi Li

Plan, Eliminate, and Track–Language Models are Good Teachers for Embodied Agents
Arxiv Pre-print
Yue Wu, So Yeon Min, Yonatan Bisk, Ruslan Salakhutdinov, Amos Azaria, Yuanzhi Li, Tom Mitchell, Shrimai Prabhumoye

Object Goal Navigation with End-to-End Self-Supervision
IROS 2023
So Yeon Min, Yao-Hung Hubert Tsai, Wei Ding, Ali Farhadi, Ruslan Salakhutdinov, Yonatan Bisk, Jian Zhang

EXCALIBUR: Encouraging and Evaluating Embodied Exploration
CVPR 2023
Hao Zhu, Raghav Kapoor, So Yeon Min, Winson Han, Jiatai Li, Kaiwen Geng, Graham Neubig, Yonatan Bisk, Aniruddha Kembhavi, Luca Weihs

2022

Don’t Copy the Teacher: Data and Model Challenges in Embodied Dialogue
EMNLP 2022
So Yeon Min, Hao Zhu, Yonatan Bisk, Ruslan Salakhutdinov

FILM: Following Instructions in Language with Modular Methods
ICLR 2022
So Yeon Min, Devendra Chaplot, Pradeep Ravikumar, Yonatan Bisk, Ruslan Salakhutdinov

Before 2022

Entity-Enriched Neural Models for Clinical Question Answering
Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing
Bhanu Pratap Singh Rawat, Wei-Hung Weng, So Yeon Min, Preethi Raghavan, Peter Szolovits

Advancing Seq2seq with Joint Paraphrase Learning
Proceedings of the 3rd Clinical Natural Language Processing Workshop
So Yeon Min, Preethi Raghavan, Peter Szolovits

TransINT: Embedding Implication Rules in Knowledge Graphs with Isomorphic Intersections of Linear Subspaces
Automated Knowledge Base Construction 2020
So Yeon Min, Preethi Raghavan, Peter Szolovits

Towards knowledge-based, robust question answering
Master’s Thesis