Kausik Sivakumar

kausik [at] seas [dot] upenn [dot no spam] edu

I am currently a Robotics Engineer at Tutor Intelligence. Here, I have broadly worked on computer vision, and motion planning problems for robotic arms. These are few of my projects here that I am proud of

Built the company’s primary visual servoing method, that brought our grasping accuracy to under 3mm under real-world kinematic uncertainitites
Improved our motion planning stack through employing TOPPRA and building an online method to blend subsequent motion plans
Fine tuned deep learning based segmentation method - Segment Anything Model to improve scene understanding and automate robot arm picks

Before this, I was a Robotics student at the University of Pennsylvania (UPenn). I was part of the PAL group advised by Prof. Dinesh Jayaraman and Prof. Osbert Bastani where I ventured into the space of robot learning. I received the Outstanding Research Award from UPenn for my contributions toward AI.

I am actively seeking opportunities in the fields of manipulation, computer vision, and robot learning, with a strong desire to contribute and drive innovation. Please drop me an email if you are interested in my profile

News

Aug 7, 2023	I joined Tutor Intelligence as a robotics intern. Excited to understand practical challenges that arise in manipulation!
May 11, 2023	I received the Outstanding Research award from UPenn for my contributions toward Robot Learning! Excited for the opportunity to make robots perceive and act in the real world
Apr 3, 2023	I am graduating on May 15,2023, and currently on the lookout for research engineer roles that focus on robot learning/ deep learning/ computer vision/ reinforcement learning. Feel free to reach out in case you find my profile interesting.
Mar 15, 2023	Our paper on policy-aware model learning has been accepted to L4DC 2023 conference! More information about the conference - L4DC
Dec 1, 2022	Our paper on learning policy-aware models for model-based reinforcement learning has been submitted to L4DC 2023

Publications

L4DC

Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Yecheng Jason Ma*, Kausik Sivakumar*, Jason Yan, Osbert Bastani, and Dinesh Jayaraman

L4DC, 2023

Abs arXiv PDF

Standard model-based reinforcement learning (MBRL) approaches fit a transition model of the environment to all past experience, but this wastes model capacity on data that is irrelevant for policy improvement. We instead propose a new “transition occupancy matching” (TOM) objective for MBRL model learning: a good environment model has the property that the current policy experiences the same distribution of transitions, whether deployed in the real environment or inside the model. We derive TOM directly from a novel lower bound on the standard reinforcement learning objective. To optimize TOM, we show how to reduce it to a form of importance weighted maximum-likelihood estimation, where the automatically computed importance weights identify policy-relevant past experiences from a replay buffer, enabling stable optimization. TOM thus offers a plug-and-play model learning sub-routine that is compatible with any backbone MBRL algorithm. On various Mujoco continuous robotic control tasks, we show that TOM successfully focuses model learning on policy-relevant experience and drives policies faster to higher task rewards than alternative model learning approaches