For a full list of publications, check my google scholar page.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control
- Training robots on internet scale image/text datasets
- Paper, Github
- Covered on the front page of New York Times
RT-1: Robotics Transformer for Real-World Control at Scale
SayCan: Grounding Language in Robotic Affordances
- Using large language models for planning in robotics
- Demos, Paper - Best paper at RSS 2022 Scaling Robot Learning Workshop, Oral at CoRL 2022
Self Driving Data Visualization
- Data visualization for status quo on the race to build the self driving car.
- Demos - 2020, 2019
RL bots on StarCraft 2
- Built RL bots to play StarCraft 2 by adapting A3C algorithm and training in Deepmind-Blizzard StarCraft 2 environment
- Video demos, Paper
Multimodal emotion detection
- Built a multimodal emotion recognition system with team of 3 in collaboration with Emotech AI.
- Co-invented gated recurrent attention, beat human performance on CREMA-D and paper accepted at CoNLL ’18
- Video demos, Paper
CubeRover: A Private 2kg Rover for Lunar Exploration
- Helped build a 2kg rover to explore the moon as part of a NASA-CMU-Astrobotic collaboration
- Adapted ORBSLAM for localization and augmented radio communication to send images and interrupts between rover and lander
- Media coverage : DesignWorld - Next generation of moon rovers, NASA CubeRover
- Localization video demo
Blockchain for Cars
- Built a decentralized app based on ethereum blockchain for secure and dynamic vehicle identity, finalist at World Blockchain Hackathon 2018, San Francisco
Decriminalising LGBTQ in India
- Prior to September 2018, identifying as LGBTQ made one an automatic criminal given that “unnatural sex” was felonious
- Petitioned the Supreme Court in the legal challenge against section 377 of the Indian constitution leading to historic verdict decriminalizing alternative sexuality in India
- Media Coverage : NDTV National Interview, Print
- Created a subscription based e-commerce demand aggregator in 2014, won Start-Up Weekend at IIM Calcutta and associated with 6+ investors
- Two takeaways: Decided to go to grad school to become an effective technical founder; Lots of ideas make money, but is it worth spending your life on?