bio photo

Research Scholar, NEC Laboratories America Inc.

Email Twitter LinkedIn Scholar

Hi, Iā€™m a Research Scholar at NEC Labs, Media Analytics department working with Prof. Manmohan Chandraker. My research interests span areas of vision, learning, and robotics. At NEC, I have worked on problems related to diverse simulation and prediction of trajectories for autonomous vehicles. My current direction focusses on problems (ex. Rearrangement, Social Navigation) in the domain of Embodied AI. My long term goal is build systems that can safely navigate and work alongside humans in doing day to day tasks.

Prior to joining NEC, I worked as a Research Assistant at Robotics Research Center with Prof. K. Madhava Krishna and Prof. Vineet Gandhi in problems related to perception and prediction for autonomous vehicles. I spent my Summer 2017 at IIIT Delhi, working with Dr. Sanjit Kaul gaining a good hands on experience in building a self-driving vehicle prototype.

Never give up on a dream just because of the time it will take to accomplish it. The time will pass anyway.

– Earl Nightingale


News

02/21: Paper Divide and Conquer for Lane-Aware Diverse Trajectory Prediction is accepted to CVPR 2021 (Oral).
00/21: Reviewer for CVPR 2021, IROS 2021 and ICCV 2021.
07/20: Paper SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction is accepted to ECCV 2020.
11/19: Served as a reviewer for ICRA 2020.
07/19: Joined NEC Labs America as a Research Scholar.
06/19: Paper Talk to the Vehicle: Language Conditioned Autonomous Navigation is accepted at IROS 2019.
04/19: Paper A Hierarchical Network for Diverse Trajectory Proposals is accepted at IV 2019.
04/19: Work on shrinking domain based control for planning is accepted at AIR 2019.


Publications

Divide-and-Conquer for Lane-Aware Diverse Trajectory Prediction
Sriram Narayanan, Ramin Moslemi, Francesco Pittaluga, Buyu Liu, Manmohan Chandraker
Computer Vision and Pattern Recognition (CVPR), 2021 (Oral)
Paper  

Trajectory prediction is a safety-critical tool for autonomous vehicles to plan and execute actions. Our work addresses two key challenges in trajectory prediction, learning multimodal outputs, and better predictions by imposing constraints using driving knowledge. Recent methods have achieved strong performances using Multi-Choice Learning objectives like winner-takes-all (WTA) or best-of-many. But the impact of those methods in learning diverse hypotheses is under-studied as such objectives highly depend on their initialization for diversity. As our first contribution, we propose a novel Divide-And-Conquer (DAC) approach that acts as a better initialization technique to WTA objective, resulting in diverse outputs without any spurious modes. Our second contribution is a novel trajectory prediction framework called ALAN that uses existing lane centerlines as anchors to provide trajectories constrained to the input lanes. Our framework provides multi-agent trajectory outputs in a forward pass by capturing interactions through hypercolumn descriptors and incorporating scene information in the form of rasterized images and per-agent lane anchors. Experiments on synthetic and real data show that the proposed DAC captures the data distribution better compare to other WTA family of objectives. Further, we show that our ALAN approach provides on par or better performance with SOTA methods evaluated on Nuscenes urban driving benchmark.

SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction
Sriram N N, Buyu Liu, Francesco Pittaluga, Manmohan Chandraker
European Conference on Computer Vision (ECCV), 2020
Paper Video Short Talk  

We propose advances that address two key challenges in future trajectory prediction: (i) multimodality in both training data and predictions and (ii) constant time inference regardless of number of agents. Existing trajectory predictions are fundamentally limited by lack of diversity in training data, which is difficult to acquire with sufficient coverage of possible modes. Our first contribution is an automatic method to simulate diverse trajectories in the top-view. It uses pre-existing datasets and maps as initialization, mines existing trajectories to represent realistic driving behaviors and uses a multi-agent vehicle dynamics simulator to generate diverse new trajectories that cover various modes and are consistent with scene layout constraints. Our second contribution is a novel method that generates diverse predictions while accounting for scene semantics and multi-agent interactions, with constant-time inference independent of the number of agents. We propose a convLSTM with novel state pooling operations and losses to predict scene-consistent states of multiple agents in a single forward pass, along with a CVAE for diversity. We validate our proposed multi-agent trajectory prediction approach by training and testing on the proposed simulated dataset and existing real datasets of traffic scenes. In both cases, our approach outperforms SOTA methods by a large margin, highlighting the benefits of both our diverse dataset simulation and constant-time diverse trajectory prediction methods.

Talk to the Vehicle: Language Conditioned Autonomous Navigation of Self Driving Cars
Sriram N N*, Tirth Maniar*, Jayaganesh Kalyanasundaram, Vineet Gandhi, Brojeshwar Bhowmick, K Madhava Krishna
International Conference on Intelligent Robots and Systems (IROS), 2019
Paper Video  

We propose a novel pipeline that blends encodings from natural language and 3D semantic maps obtained from visual imagery to generate local trajectories that are executed by a low-level controller. The pipeline precludes the need for a prior registered map through a local waypoint generator neural network. The waypoint generator network (WGN) maps semantics and natural language encodings (NLE) to local waypoints. A local planner then generates a trajectory from the ego location of the vehicle (an outdoor car in this case) to these locally generated waypoints while a low-level controller executes these plans faithfully. The efficacy of the pipeline is verified in the CARLA simulator environment as well as on local semantic maps built from real-world KITTI dataset. In both these environments (simulated and real-world) we show the ability of the WGN to generate waypoints accurately by mapping NLE of varying sequence lengths and levels of complexity. We compare with baseline approaches and show significant performance gain over them. And finally, we show real implementations on our electric car verifying that the pipeline lends itself to practical and tangible realizations in uncontrolled outdoor settings. In loop execution of the proposed pipeline that involves repetitive invocations of the network is critical for any such language-based navigation framework. This effort successfully accomplishes this thereby bypassing the need for prior metric maps or strategies for metric level localization during traversal.

A Hierarchical Network for Diverse Trajectory Proposals
Sriram N N, Gourav Kumar, Abhay Singh, M. Siva Karthik, Saket Saurav, Brojeshwar Bhowmick, K. Madhava Krishna
Intelligent Vehicles Symposium (IV), 2019
Paper Video  

Autonomous explorative robots frequently encounter scenarios where multiple future trajectories can be pursued. Often these are cases with multiple paths around an obstacle or trajectory options towards various frontiers. Humans in such situations can inherently perceive and reason about the surrounding environment to identify several possibilities of either manoeuvring around the obstacles or moving towards various frontiers. In this work, we propose a 2 stage Convolutional Neural Network architecture which mimics such an ability to map the perceived surroundings to multiple trajectories that a robot can choose to traverse. The first stage is a Trajectory Proposal Network which suggests diverse regions in the environment which can be occupied in the future. The second stage is a Trajectory Sampling network which provides a finegrained trajectory over the regions proposed by Trajectory Proposal Network. We evaluate our framework in diverse and complicated real life settings. For the outdoor case, we use the KITTI dataset and our own outdoor driving dataset. In the indoor setting, we use an autonomous drone to navigate various scenarios and also a ground robot which can explore the environment using the trajectories proposed by our framework. Our experiments suggest that the framework is able to develop a semantic understanding of the obstacles, open regions and identify diverse trajectories that a robot can traverse. Our comparisons portray the performance gain of the proposed architecture over a diverse set of methods against which it is compared.

Gradient Aware - Shrinking Domain based Control Design for Reactive Planning Frameworks used in Autonomous Vehicles
Adarsh Modh, Siddharth Singh, A. V. S. Sai Bhargav Kumar, Sriram N N, K. Madhava Krishna
Proceedings of the Advances in Robotics (AIR), 2019
Paper Video  

In this paper, we present a novel control law for longitudinal speed control of autonomous vehicles. The key contributions of the proposed work include the design of a control law that reactively integrates the longitudinal surface gradient of road into its operation. In contrast to the existing works, we found that integrating the path gradient into the control framework improves the speed tracking efficacy. Since the control law is implemented over a shrinking domain scheme, it minimizes the integrated error by recomputing the control inputs at every discretized step and consequently provides less reaction time. This makes our control law suitable for motion planning frameworks that are operating at high frequencies. Furthermore, our work is implemented using a generalized vehicle model and can be easily extended to other classes of vehicles. The performance of gradient aware-shrinking domain based controller is implemented and tested on a stock electric vehicle on which a number of sensors are mounted. Results from the tests show the robustness of our control law for speed tracking on a terrain with varying gradient while also considering stringent time constraints imposed by the planning framework.


Published Patents

Method and a system for hierarchical network based diverse trajectory proposal
Brojeshwar Bhowmick, K. Madhava Krishna, Sriram N N, Gourav Kumar, Abhay Singh, M. Siva Karthik, Saket Saurav
US Patent No. 2020/0387163
Patent  

Bachelor's Projects

Swarath - Self driving Car
Internship project at IIIT Delhi during Summer 2017, where I worked on building a SDV prototype. It started with designing controller and planner for the vehicle. Later built perception systems where sensor outputs were used to obtain occupancies. Further, integrated the planner and perception modules together. The internship ended with demonstrating autonomous navigation with GPS waypoints as input.
Video  
Building Perception, Planning and Control for Autonomous Vehicles
Bachelor thesis at IIIT Hyderabad, started with designing a low-level controller for a stock car from scratch. Built controllers for throttle, steering and brakes without CAN support. Further integrated perception (ORB-SLAM, LOAM) and planning (RRT*) modules. Demonstrated static obstacle avoidance in tight spaces which was showcased at Research and Development showcase held at IIIT. A part of my thesis culminated into a paper published at AIR 19.
Report Video  
Background Subtraction
Project work on Background Subtraction with Dr. Manigandan during bachelors. Designed a background subtraction algorithm based on mixture models.
Paper  
Bothoven - Eyantra Robotics Competition
A multi-robot task execution problem, where two robots simultaneously plan paths, coordinate and execute a task of playing an audio file in a concurrent fashion. First FFT is applied to decode the notes from an audio file and is fed as input to the robots. The robots are aware of the map but are unaware of the obstacles present on the map. The map is updated and replanning is done when any one of them encounters an obstacle.
Video  
Husky - Visual perception based Autonomous bot
Implemented obstacle avoidance and waypoint following in outdoor and indoor scenarios. Sensors like stereo camera, GPS, IMU and wheel encoders were used. Occupancy map was generated upon which planning was done using RRT for the bot to reach the goal autonmously.
Video  



I am privileged to be associated with the following
Intern RA Research Scholar
Summer 2017 Dec 2017 - Jul 2019 Jul 2019 - Present