Projects

For a full list of publications, check my google scholar page.

AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents

• Orchestrating robotics agents for real world operation using a Robot Constitution
• Paper, Website
• Forbes, The Verge, Business Insider

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

• Training robots on internet scale image/text datasets
• Paper, Website
• Covered on the front page of New York Times, Forbes

RT-1: Robotics Transformer for Real-World Control at Scale

• Adapting transformers for fast and scalable robot manipulation
• Paper, Github, Google AI Blog

SayCan: Grounding Language in Robotic Affordances

• Using large language models for planning in robotics
• Demos, Paper - Best paper at RSS 2022 Scaling Robot Learning Workshop, Oral at CoRL 2022

Self Driving Data Visualization

• Data visualization for status quo on the race to build the self driving car.
• Demos - 2020, 2019
• Code

RL bots on StarCraft 2

• Built RL bots to play StarCraft 2 by adapting A3C algorithm and training in Deepmind-Blizzard StarCraft 2 environment
• Video demos, Paper

Multimodal emotion detection

• Built a multimodal emotion recognition system with team of 3 in collaboration with Emotech AI.
• Co-invented gated recurrent attention, beat human performance on CREMA-D and paper accepted at CoNLL ’18
• Video demos, Paper

CubeRover: A Private 2kg Rover for Lunar Exploration

• Helped build a 2kg rover to explore the moon as part of a NASA-CMU-Astrobotic collaboration
• Adapted ORBSLAM for localization and augmented radio communication to send images and interrupts between rover and lander
• Media coverage : DesignWorld - Next generation of moon rovers, NASA CubeRover
• Localization video demo

Blockchain for Cars

• Built a decentralized app based on ethereum blockchain for secure and dynamic vehicle identity, finalist at World Blockchain Hackathon 2018, San Francisco
• Code

Decriminalising LGBTQ in India

• Prior to September 2018, identifying as LGBTQ made one an automatic criminal given that “unnatural sex” was felonious
• Petitioned the Supreme Court in the legal challenge against section 377 of the Indian constitution leading to historic verdict decriminalizing alternative sexuality in India
• Media Coverage : NDTV National Interview, Print

Commbuyn

• Created a subscription based e-commerce demand aggregator in 2014, won Start-Up Weekend at IIM Calcutta and associated with 6+ investors
• Two takeaways: Decided to go to grad school to become an effective technical founder; Lots of ideas make money, but is it worth spending your life on?