NSF Graduate Research Fellow and AAAI-MAKE conference chair
Enabling people through conversational agents
MIT Media Lab Ph.D. · Apple · Hypha
Research engineer and rapid prototyper specializing in knowledge graphs, retrieval, and agentic systems.
Knowledge graphs
Retrieval & RAG
Agentic systems
Research
Conversational agents for procedural tasks
At the MIT Media Lab I designed an agent that adapts commonsense knowledge to guide people through tasks.
Design
AI-powered experiences at Apple
In Apple’s Intelligence Design group I designed and prototyped LLM and multimodal experiences.
Production
Agents that ship
At Hypha I build agents that extract covenant terms from loan documents and check compliance against live data, plus the interfaces analysts use to review their output.
• Covenant extraction
• Agent evaluation
• Analyst review UI
From research to production
My Ph.D. work fine-tuned language models to generate and consume structured knowledge. Now I build and evaluate LLM agents and experiences in production.
Assistant
How can I help you today?
You
I want to build a chair. Where do I start?
Assistant
Let’s lay out the parts first. Do you have the seat, four legs, and the screws?
Publications & mentorship
10
Selected publications, at venues including EMNLP, ACL, and HRI
5
Students mentored in research and thesis work
Featured
A Conversational Agent for Dynamic Procedural Interactions
My Ph.D. thesis at MIT (2023): an agent that adapts commonsense knowledge to guide people through procedures, handling errors as they happen.
Pedro Colón-Hernández, Ph.D.
MIT Media Lab, Personal Robots Group
A bit more about me
What do you work on?
I’m a research engineer and rapid prototyper specializing in knowledge graphs, retrieval, and agentic systems. I build and evaluate LLM agents and the experiences people use to work with them.
Where have you worked?
I’m a Member of Technical Staff at Hypha (Cortex Group). Before that I was an HI Designer and then Senior HI Designer in Apple’s Intelligence Design group, and a prototyping intern in Siri’s Conversational Interaction group. I also did research at the MIT Media Lab and MIT Lincoln Laboratory.
What is your background?
I hold a Ph.D. and an M.S. in Media Arts and Sciences from MIT, and a B.S. in Computer Engineering (magna cum laude) from the University of Puerto Rico Mayagüez.
What tools do you use?
Mostly Python, with PyTorch, PyTorch Lightning, and Hugging Face for model work, and TypeScript, React, and React Native for interfaces. I build with AI-first development workflows using tools like Claude Code and Codex.
Can I see your resume?
Yes, here is my resume.
Pedro Colón