Sayan Mondal

Welcome to my corner of the digital universe! I am on an exciting journey as I pursue my M.S. in Robotics degree at The Robotics Institute, Carnegie Mellon University. My academic exploration began with a foundation in Mechanical Engineering, where I earned my B.Eng. from Jadavpur University, India, in 2016.

Venturing further into the realm of engineering, I obtained my M.S. in Mechanical Engineering from the University of California, San Diego, in 2020. It was during this transformative period that I had the privilege of contributing to groundbreaking research in the Gravish Lab, enriching my understanding of mechanical systems and their applications. Additionally, I worked under Prof. Henrik I. Christensen on autonomous driving projects in the Contextual Robotics Institute at UCSD, honing my skills in the dynamic field of robotics.

Embarking on my academic journey, I earned a second master's degree in Biomedical Engineering from Carnegie Mellon University in 2022. Immersed in the dynamic field of Biorobotics, I actively contributed to innovative projects at the Biorobotics Lab, refining my technical acumen. From exploring mechanical systems at UC San Diego's Gravish Lab to delving into Biorobotics at CMU, I am passionate about pushing technology boundaries.

In my current research, I am exploring the vast landscape of robotics, delving into semantic perception, deep reinforcement learning, and planning. These cutting-edge topics fuel my curiosity and drive as I navigate the ever-evolving field, seeking to contribute to the forefront of innovation. Currently, I am also engaged in a research project on planning over skills, advised by Prof. Maxim Likhachev, further enhancing my expertise and insights into the exciting world of robotics. Join me in unraveling the possibilities at the intersection of engineering and innovation.

Email  /  Google Scholar  /  Github  /  LinkedIn  /  CV

B.E. Mechanical Engg.
JU

M.S. Mechanical Engg.
UCSD

M.S. Biomedical Engg.
CMU

M.S. Robotics
CMU RI

SAGE-Net framework

In recent years, predicting driver’s focus of attention has been a very active area of research in the autonomous driving community. Unfortunately, existing state-of-the-art techniques achieve this by relying only on human gaze information, thereby ignoring scene semantics. We propose a novel Semantics Augmented GazE (SAGE) detection approach that captures driving specific contextual information, in addition to the raw gaze. Such a combined attention mechanism serves as a powerful tool to focus on the relevant regions in an image frame in order to make driving both safe and efficient. Using this, we design a complete saliency prediction framework - SAGE-Net, which modifies the initial prediction from SAGE by taking into account vital aspects such as distance to objects (depth), ego vehicle speed, and pedestrian crossing intent. Exhaustive experiments conducted through four popular saliency algorithms show that on 49/56 (87.5%) cases - considering both the overall dataset and crucial driving scenarios, SAGE outperforms existing techniques without any additional computational overhead during the training process. The augmented dataset along with the relevant code are available as part of the supplementary material.

This masters thesis describes a novel underactuated robotic microgripper with two fingers.The design specifications, a thorough kinematic description of the gripper, and its static analysis are presented. The novelty of this gripper lies in the simplicity of its mechanism that can accomplish the task of picking up the target objects. What makes it unique is its ability to grasp objects that are either in the same plane as that of the gripper or are at a lower level. The gripper is equipped to be actuated by a single actuator. For preliminary evaluation of the gripper’s object manipulation capabilities, standard hexagonal nuts with varying weights, and sizes were selected.The success of grasping the nuts by the gripper at two different orientations were observed and studied. In this paper only one of the test cases has been shown in detail. In addition to that, a kirigami spring has been incorporated in the modified design of the gripper in order to enhance its grasping capabilities.

In the realm of deep reinforcement learning, achieving generalization over unforeseen variations in the environment often necessitates extensive policy learning across a diverse array of training scenarios. Empirical findings reveal a notable trend: an agent trained on a multitude of variations (termed a generalist) exhibits accelerated early-stage learning, but its performance tends to plateau at a suboptimal level for an extended period. In contrast, an agent trained exclusively on a select few variations (referred to as a specialist) frequently attains high returns within a constrained computational budget. To reconcile these contrasting advantages, we experiment with various combinations of specialists and generalists in the quadrupedal locomotion setting. Our investigation delves into determining the impact of each skill when they are trained to be specialists and the impact of combining them together into creating a more generalist agent.

Ballbot with 7DOF arms navigate and is capable of wall pushing

Several ballbots have been developed, yet only a handful have been equipped with arms to enhance their maneuverability and manipulability. The incorporation of 7-DOF arms to the CMU ballbot has presented challenges in balancing and navigation due to the constantly changing center of mass. This project aims to propose a control strategy that incorporates the arms dynamics. Our approach is to use a simplified whole-body dynamics model, with only the shoulder and elbow joints moving for each arm. This reduces the number of states and accelerates convergence. We focused on two specific tasks: navigation (straight and curved paths) and pushing against a wall. Trajectories were generated using direct collocation for the navigation task and hybrid contact trajectory optimization for pushing the wall. A time-variant linear quadratic regulator (TVLQR) was designed to track the trajectories. The resulting trajectories were tracked with a mean-average error of less than 4 cm, even for the more complex path. These experiments represent an initial step towards unlocking the full potential of ballbots in dynamic and interactive environments.

Logic Geometric Programming to solve relatively long-horizon TAMP problems

Sequential manipulation tasks are notoriously hard to plan for because they involve a combination of continuous and discrete decision variables. In this project we study and experimentally analyze Logic Geometric Programming (LGP), which is a trajectory optimization based approach to solve such problems.

Multi-object tracking using Deep-SORT for waste-tracking

This work presents a comprehensive approach to waste classification on a moving conveyor system, with a particular emphasis on leveraging sequential models to capture the spatial and temporal components inherent in the sequence of images. The utilization of sequential models contributes to enhanced stability and reliability in the classification results. Additionally, a robust multi-object tracking system has been developed to monitor the trajectory of waste materials across diverse conveyors, leveraging data from multiple cameras. The study's key collaboration was with Gateway Recycling facility, specializing in paper and plastic waste commodities. The project's success has paved the way for a significant collaboration with the recycling facility, contributing valuable insights and advancements in waste management practices. Furthermore, the research team, led by me, spearheaded an initiative for recycling facility automation. A team of five graduate students was assembled to automate the data-collection pipeline. This involved the development of an efficient system capable of identifying various grades of paper and plastic. Notably, the system addressed the challenge of differentiating between grades that even facility workers found visually challenging. The outcomes of this work showcase the potential of advanced technologies in waste management and underscore the importance of automation in streamlining recycling processes. The success of the project holds promise for improving the efficiency and accuracy of waste classification systems in recycling facilities, contributing to sustainable waste management practices.

MGAIL vs AMGAIL

In this paper we introduce a new concept named, AMGAIL to solve the imitation learning problem in an environment in which rewards are available. AMGAIL is based on MGAIL, but it replaces bad expert trajectories with good ones that we generate. We make use of the total rewards of the trajectories to detect how good or bad they are. We tested for 3 MuJoCo environments- Hopper-v1, HalfCheetah-v1, InvertedPendulum-v1. We expect that AMGAIL should perform better than vanilla MGAIL when the expert trajectories are a mix of experts of varying level because the algorithm is able to replace the weaker experts and in turn lower the variance. Our results generally confirm this.

Sampling-based planners for 5-DOFs arm to move from its start joint angles to the goal joint angles.

Point robot to catch a moving target in 2D grid worlds. (A* Search Algorithm)

Single-view image to 3D reconstruction

Neural Radiance Field and Neural Surfaces