Hi, I am a Ph.D. student at the Robotics Institute , School of Computer Science, Carnegie Mellon University working with Prof. Srinivasa Narasimhan. Currently, Iām exploring topics in the domain of computational imaging and vision.
Before starting my Ph.D., I was a Research Scholar at NEC Labs America, Media Analytics department where I worked with Prof. Manmohan Chandraker on problems related to trajectory prediction and Embodied AI.
In the past, I was associated with Robotics Research Center where I worked with Prof. K. Madhava Krishna and Prof. Vineet Gandhi. I spent the summer 2017 at IIIT Delhi, working with Prof. Sanjit Kaul where I first experienced the thrill of robotics by building a self-driving vehicle prototype.
I am always enthusiastic about discussing research and various other topics. Please feel free to reach out to me via email at snochurn at cs.cmu.edu.
07/24: Paper Shape from Heat Conduction accepted to ECCV 2024 (Oral) .
02/24: Paper on visible-thermal light transport accepted to CVPR 2024.
01/24: Paper on long-horizon object transport accepted to ICRA 2024.
08/22: Moved to Pittsburgh and started my PhD at RI, CMU.
10/21: Talk on Predicting simultaneous multi-hypotheses futures at Robotics Research Group, IIT BHU
02/21: Paper Divide and Conquer for Lane-Aware Diverse Trajectory Prediction is accepted to CVPR 2021 (Oral).
07/20: Paper SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction is accepted to ECCV 2020.
07/19: Joined NEC Labs America as a Research Scholar.
06/19: Paper Talk to the Vehicle: Language Conditioned Autonomous Navigation is accepted at IROS 2019.
04/19: Paper A Hierarchical Network for Diverse Trajectory Proposals is accepted at IV 2019.
04/19: Work on shrinking domain based control for planning is accepted at AIR 2019.
Shape from Heat Conduction Sriram Narayanan, Mani Ramanagopal, Mark Sheinin, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan European Conference on Computer Vision (ECCV), 2024 (Oral) Project Page Paper Thermal cameras measure the temperature of objects based on radiation emitted in the infrared spectrum. In this work, we propose a novel shape recovery approach that exploits the properties of heat transport, specifically heat conduction, induced on objects when illuminated using simple light bulbs. Although heat transport occurs in the entirety of an object's volume, we show a surface approximation that enables shape recovery and empirically analyze its validity for objects with varying thicknesses. We develop an algorithm that solves a linear system of equations to estimate the intrinsic shape Laplacian from thermal videos along with several properties including heat capacity, convection coefficient, and absorbed heat flux under uncalibrated lighting of arbitrary shapes. Further, we propose a novel shape from Laplacian objective that aims to resolve the inherent shape ambiguities by drawing insights from absorbed heat flux images using two unknown lights sources. Finally, we devise a coarse-to-fine refinement strategy that faithfully recovers both low- and high-frequency shape details. We validate our method by showing accurate reconstructions, to within an error of 1-2mm (object size ≤ 13.5cm), in both simulations and from noisy thermal videos of real-world objects with complex shapes and material properties including those that are transparent and translucent to visible light. We believe leveraging heat transport as a novel cue for vision can enable new imaging capabilities. |
A Theory of Joint Light and Heat Transport for Lambertian Scenes Mani Ramanagopal, Sriram Narayanan, Aswin C. Sankaranarayanan, Srinivasa G. Narasimhan Computer Vision and Pattern Recognition (CVPR), 2024 Project Page Paper Video We present a novel theory that establishes the relationship between light transport in visible and thermal infrared, and heat transport in solids. We show that heat generated due to light absorption can be estimated by modeling heat transport using a thermal camera. For situations where heat conduction is negligible, we analytically solve the heat transport equation to derive a simple expression relating the change in thermal image intensity to the absorbed light intensity and heat capacity of the material. Next, we prove that intrinsic image decomposition for Lambertian scenes becomes a well-posed problem if one has access to the absorbed light. Our theory generalizes to arbitrary shapes and unstructured illumination. Our theory is based on applying energy conservation principle at each pixel independently. We validate our theory using real-world experiments on diffuse objects made of different materials that exhibit both direct and global components (inter-reflections) of light transport under unknown complex lighting. |
Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport Sriram Narayanan, Dinesh Jayaraman, Manmohan Chandraker International Conference on Robotics and Automation (ICRA), 2024 Paper Video We address key challenges in long-horizon embodied exploration and navigation by proposing a new object transport task and a novel modular framework for temporally extended navigation. Our first contribution is the design of a novel Long-HOT environment focused on deep exploration and long-horizon planning where the agent is required to efficiently find and pick up target objects to be carried and dropped at a goal location, with load constraints and optional access to a container if it finds one. Further, we propose a modular hierarchical transport policy (HTP) that builds a topological graph of the scene to perform exploration with the help of weighted frontiers. Our hierarchical approach uses a combination of motion planning algorithms to reach point goals within explored locations and object navigation policies for moving towards semantic targets at unknown locations. Experiments on both our proposed Habitat transport task and on MultiOn benchmarks show that our method significantly outperforms baselines and prior works. Further, we validate the effectiveness of our modular approach for long-horizon transport by demonstrating meaningful generalization to much harder transport scenes with training only on simpler versions of the task. |
Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction Sriram Narayanan, Ramin Moslemi, Francesco Pittaluga, Buyu Liu, Manmohan Chandraker Computer Vision and Pattern Recognition (CVPR), 2021 (Oral) Paper Blog Talk Trajectory prediction is a safety-critical tool for autonomous vehicles to plan and execute actions. Our work addresses two key challenges in trajectory prediction, learning multimodal outputs, and better predictions by imposing constraints using driving knowledge. Recent methods have achieved strong performances using Multi-Choice Learning objectives like winner-takes-all (WTA) or best-of-many. But the impact of those methods in learning diverse hypotheses is under-studied as such objectives highly depend on their initialization for diversity. As our first contribution, we propose a novel Divide-And-Conquer (DAC) approach that acts as a better initialization technique to WTA objective, resulting in diverse outputs without any spurious modes. Our second contribution is a novel trajectory prediction framework called ALAN that uses existing lane centerlines as anchors to provide trajectories constrained to the input lanes. Our framework provides multi-agent trajectory outputs in a forward pass by capturing interactions through hypercolumn descriptors and incorporating scene information in the form of rasterized images and per-agent lane anchors. Experiments on synthetic and real data show that the proposed DAC captures the data distribution better compare to other WTA family of objectives. Further, we show that our ALAN approach provides on par or better performance with SOTA methods evaluated on Nuscenes urban driving benchmark. |
SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction Sriram Narayanan, Buyu Liu, Francesco Pittaluga, Manmohan Chandraker European Conference on Computer Vision (ECCV), 2020 Paper Video Short Talk We propose advances that address two key challenges in future trajectory prediction: (i) multimodality in both training data and predictions and (ii) constant time inference regardless of number of agents. Existing trajectory predictions are fundamentally limited by lack of diversity in training data, which is difficult to acquire with sufficient coverage of possible modes. Our first contribution is an automatic method to simulate diverse trajectories in the top-view. It uses pre-existing datasets and maps as initialization, mines existing trajectories to represent realistic driving behaviors and uses a multi-agent vehicle dynamics simulator to generate diverse new trajectories that cover various modes and are consistent with scene layout constraints. Our second contribution is a novel method that generates diverse predictions while accounting for scene semantics and multi-agent interactions, with constant-time inference independent of the number of agents. We propose a convLSTM with novel state pooling operations and losses to predict scene-consistent states of multiple agents in a single forward pass, along with a CVAE for diversity. We validate our proposed multi-agent trajectory prediction approach by training and testing on the proposed simulated dataset and existing real datasets of traffic scenes. In both cases, our approach outperforms SOTA methods by a large margin, highlighting the benefits of both our diverse dataset simulation and constant-time diverse trajectory prediction methods. |
Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars Sriram Narayanan*, Tirth Maniar*, Jayaganesh Kalyanasundaram, Vineet Gandhi, Brojeshwar Bhowmick, K Madhava Krishna International Conference on Intelligent Robots and Systems (IROS), 2019 Paper Video We propose a novel pipeline that blends encodings from natural language and 3D semantic maps obtained from visual imagery to generate local trajectories that are executed by a low-level controller. The pipeline precludes the need for a prior registered map through a local waypoint generator neural network. The waypoint generator network (WGN) maps semantics and natural language encodings (NLE) to local waypoints. A local planner then generates a trajectory from the ego location of the vehicle (an outdoor car in this case) to these locally generated waypoints while a low-level controller executes these plans faithfully. The efficacy of the pipeline is verified in the CARLA simulator environment as well as on local semantic maps built from real-world KITTI dataset. In both these environments (simulated and real-world) we show the ability of the WGN to generate waypoints accurately by mapping NLE of varying sequence lengths and levels of complexity. We compare with baseline approaches and show significant performance gain over them. And finally, we show real implementations on our electric car verifying that the pipeline lends itself to practical and tangible realizations in uncontrolled outdoor settings. In loop execution of the proposed pipeline that involves repetitive invocations of the network is critical for any such language-based navigation framework. This effort successfully accomplishes this thereby bypassing the need for prior metric maps or strategies for metric level localization during traversal. |
A Hierarchical Network for Diverse Trajectory Proposals Sriram Narayanan, Gourav Kumar, Abhay Singh, M. Siva Karthik, Saket Saurav, Brojeshwar Bhowmick, K. Madhava Krishna Intelligent Vehicles Symposium (IV), 2019 Paper Video Autonomous explorative robots frequently encounter scenarios where multiple future trajectories can be pursued. Often these are cases with multiple paths around an obstacle or trajectory options towards various frontiers. Humans in such situations can inherently perceive and reason about the surrounding environment to identify several possibilities of either manoeuvring around the obstacles or moving towards various frontiers. In this work, we propose a 2 stage Convolutional Neural Network architecture which mimics such an ability to map the perceived surroundings to multiple trajectories that a robot can choose to traverse. The first stage is a Trajectory Proposal Network which suggests diverse regions in the environment which can be occupied in the future. The second stage is a Trajectory Sampling network which provides a finegrained trajectory over the regions proposed by Trajectory Proposal Network. We evaluate our framework in diverse and complicated real life settings. For the outdoor case, we use the KITTI dataset and our own outdoor driving dataset. In the indoor setting, we use an autonomous drone to navigate various scenarios and also a ground robot which can explore the environment using the trajectories proposed by our framework. Our experiments suggest that the framework is able to develop a semantic understanding of the obstacles, open regions and identify diverse trajectories that a robot can traverse. Our comparisons portray the performance gain of the proposed architecture over a diverse set of methods against which it is compared. |
Gradient Aware - Shrinking Domain based Control Design for Reactive Planning Frameworks used in Autonomous Vehicles Adarsh Modh, Siddharth Singh, A. V. S. Sai Bhargav Kumar, Sriram Narayanan, K. Madhava Krishna Proceedings of the Advances in Robotics (AIR), 2019 Paper Video In this paper, we present a novel control law for longitudinal speed control of autonomous vehicles. The key contributions of the proposed work include the design of a control law that reactively integrates the longitudinal surface gradient of road into its operation. In contrast to the existing works, we found that integrating the path gradient into the control framework improves the speed tracking efficacy. Since the control law is implemented over a shrinking domain scheme, it minimizes the integrated error by recomputing the control inputs at every discretized step and consequently provides less reaction time. This makes our control law suitable for motion planning frameworks that are operating at high frequencies. Furthermore, our work is implemented using a generalized vehicle model and can be easily extended to other classes of vehicles. The performance of gradient aware-shrinking domain based controller is implemented and tested on a stock electric vehicle on which a number of sensors are mounted. Results from the tests show the robustness of our control law for speed tracking on a terrain with varying gradient while also considering stringent time constraints imposed by the planning framework. |
Conference Reviewer: ICRA 2020, IROS 2021, CVPR 2021, ICCV 2021, ICRA 2022, CVPR 2022, ECCV 2022, AAAI 2022, CVPR 2023, ICCV 2023, CVPR 2024, ECCV 2024
Divide-and-conquer for lane-aware diverse trajectory prediction Sriram Narayanan , Ramin Moslemi, Francesco Pittaluga, Buyu Liu, Manmohan Chandraker US Patent App. 17/521,139 Patent |
Multi-agent trajectory prediction Sriram Narayanan , Buyu Liu, Ramin Moslemi, Francesco Pittaluga, Manmohan Chandraker US Patent App. 17/187,157 Patent |
Simulating diverse long-term future trajectories in road scenes Sriram Narayanan , Manmohan Chandraker US Patent App. 17/090,399 Patent |
Method and a system for hierarchical network based diverse trajectory proposal Brojeshwar Bhowmick, K. Madhava Krishna, Sriram Narayanan, Gourav Kumar, Abhay Singh, M. Siva Karthik, Saket Saurav US Patent App. 16/894,411 Patent |
Intern | RA | Research Scholar | PhD |
Summer 2017 | Dec 2017 - Jul 2019 | Jul 2019 - Jul 2022 | August 2022 - Present |