Kausik Sivakumar

kausik [at] seas [dot] upenn [dot no spam] edu

I am currently a Robotics Intern at Tutor Intelligence where make robot arms use color and depth sensory data to manipulate everyday objects.

I am broadly interested in Manipulation - in a world where robots are no longer confined to controlled environments but are envisioned as seamless companions in our daily lives, the pursuit of enabling them to comprehend and manipulate objects akin to human proficiency becomes paramount. My goal is to understand and attempt to unravel how robots could practically acomplish this behavior.

I was a Robotics student at the University of Pennsylvania (UPenn). I was part of the PAL group advised by Prof. Dinesh Jayaraman and Prof. Osbert Bastani where I ventured into the space of robot learning. I received the Outstanding Research Award from UPenn for my contributions toward AI.

I am actively seeking opportunities in the fields of manipulation, computer vision, and robot learning, with a strong desire to contribute and drive innovation. Please drop me an email if you are interested in my profile

News

Aug 7, 2023	I joined Tutor Intelligence as a robotics intern. Excited to understand practical challenges that arise in manipulation!
May 11, 2023	I received the Outstanding Research award from UPenn for my contributions toward Robot Learning! Excited for the opportunity to make robots perceive and act in the real world
Apr 3, 2023	I am graduating on May 15,2023, and currently on the lookout for research engineer roles that focus on robot learning/ deep learning/ computer vision/ reinforcement learning. Feel free to reach out in case you find my profile interesting.
Mar 15, 2023	Our paper on policy-aware model learning has been accepted to L4DC 2023 conference! More information about the conference - L4DC
Dec 1, 2022	Our paper on learning policy-aware models for model-based reinforcement learning has been submitted to L4DC 2023

Publications

L4DC

Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Yecheng Jason Ma*, Kausik Sivakumar*, Jason Yan, Osbert Bastani, and Dinesh Jayaraman

L4DC, 2023

Abs arXiv PDF

Standard model-based reinforcement learning (MBRL) approaches fit a transition model of the environment to all past experience, but this wastes model capacity on data that is irrelevant for policy improvement. We instead propose a new “transition occupancy matching” (TOM) objective for MBRL model learning: a good environment model has the property that the current policy experiences the same distribution of transitions, whether deployed in the real environment or inside the model. We derive TOM directly from a novel lower bound on the standard reinforcement learning objective. To optimize TOM, we show how to reduce it to a form of importance weighted maximum-likelihood estimation, where the automatically computed importance weights identify policy-relevant past experiences from a replay buffer, enabling stable optimization. TOM thus offers a plug-and-play model learning sub-routine that is compatible with any backbone MBRL algorithm. On various Mujoco continuous robotic control tasks, we show that TOM successfully focuses model learning on policy-relevant experience and drives policies faster to higher task rewards than alternative model learning approaches