Meng, Yue (孟岳)

I am a 5th year PhD student at MIT AeroAstro, working in REALM lab.
My research topic is using machine learning techniques for safe and robust robot control.
Before that, I was an AI Resident at IBM Thomas J. Waston Research Center.
I received an M.S. degree in ECE at UC San Diego and worked in ERL and WCSNG,
and a B.S. degree in the Department of Automation from Tsinghua University.

MIT UCSD Tsinghua Tsinghua Veterans Soccer San Diego Red Army Soccer Calblue Soccer Team
Amazon Robotics Microsoft Research Google
MIT-IBM Watson AI Lab Honda Research TuSimple



ConBaT: Control Barrier Transformer for Safe Policy Learning

Yue Meng, Sai Vemprala, Rogerio Bonatti, Chuchu Fan, Ashish Kapoor
IEEE Int. Conf. on Robotics and Automation (ICRA), 2024

We propose Control Barrier Transformer (ConBaT), an approach to learn safe behaviors from demonstrations. ConBaT is inspired by control barrier functions and uses a causal transformer to predict safe actions with a critic that requires minimal labeling. During deployment, we employ a lightweight optimization to ensure future states lie within the learned safe set. We apply our approach to control tasks and show that ConBaT results in safer policies compared to classical and learning-based baselines such as imitation learning, reinforcement learning, and model predictive control.

Signal Temporal Logic Neural Predictive Control

Yue Meng, Chuchu Fan
IEEE Robotics and Automation Letters (RA-L), 2023
[arXiv] [paper] [code][page] [video]

We learn a neural network controller to satisfy the requirements specified in Signal Temporal Logic (STL). Our controller learns to maximize the STL robustness score in training. In testing, our controller predicts within a planning horizon to satisfy STL. A backup policy is designed to ensure safety when our controller fails. On six tasks we outperform classical methods (MPC, STL-solver), model-free and model-based RL methods in STL satisfaction rate, especially on tasks with complex STL, while being 10X-100X faster than the classical methods.

Hybrid Systems Neural Control with Region-of-Attraction Planner

Yue Meng, Chuchu Fan
5th Annual Learning for Dynamics & Control Conference (L4DC), 2023
[arXiv] [paper] [code][page] [poster]

In this work, we propose a neural network (NN)-based method to control hybrid systems. Upon mode switching, we propose a differentiable planner to ensure the states after switching can land in next mode's RoA, hence stabilizing the hybrid system. With low running time, our controller achieves a higher stability/success rate over MPC, RL, common Lyapunov methods (CLF), linear quadratic regulator (LQR), quadratic programming (QP) and Hamilton-Jacobian-based methods (HJB).

Density Planner: Minimizing Collision Risk in Motion Planning with Dynamic Obstacles using Density-based Reachability

Laura Lützow, Yue Meng, Andres Chavez Armijos, Chuchu Fan
IEEE Int. Conf. on Robotics and Automation (ICRA), 2023
[arXiv] [video] [code]

We propose a density-based method which uses a neural network and the Liouville equation to learn the density evolution for a system with an uncertain initial state. We conduct motion planning experiments on simulated environments and environments generated from real-world data and outperform baseline methods such as model predictive control and nonlinear programming. While our method requires offline planning, the online run speed is 100times faster compared to model predictive control.

Density of Reachable States and How to Use it for Safe Autonomous Motion Planning

Yue Meng, Zeng Qiu, Md Tawhid Bin Waez, Chuchu Fan
NASA Formal Methods Symposium (NFM), 2022

Recent work provides a data-driven approach to compute the density distribution of autonomous systems’ forward reachable states online. In this paper, we study the use of such approach in combination with model predictive control for verifiable safe path planning under uncertainties. We design two challenging scenarios (autonomous driving and hovercraft control) for safe motion planning in environments with obstacles under system uncertainties. By leveraging the estimated risk, our algorithm achieves the highest success rate in goal reaching when enforcing the safety rate above 0.99.

Learning Density Distribution of Reachable States for Autonomous Systems

Yue Meng, Dawei Sun, Zeng Qiu, Md Tawhid Bin Waez, Chuchu Fan
Conference of Robotics and Learning (CoRL), 2021

In this work, we propose a data-driven method to compute the density distribution of reachable states for nonlinear systems. Our approach learns system dynamics and the state density jointly from trajectory data, guided by the fact that the state density evolution follows the Liouville partial differential equation. We show that our learned solution can produce a much more accurate estimate on density distribution, and can quantify risks less conservatively and flexibly compared with worst-case analysis.

Reactive and Safe Road User Simulations using Neural Barrier Certificates

Yue Meng, Zengyi Qin, Chuchu Fan
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2021.

We proposed a reactive agent model which can ensure safety by learning high-level decisions from expert data and a low level decentralized controller guided by the jointly learned decentralized barrier certificates. Empirical results show that our approach can achieve a significant improvement in safety while being similar to human behaviors.

AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition

Yue Meng, Rameswar Panda, Chung-Ching Lin, Prasanna Sattigeri, Leonid Karlinsky, Kate Saenko, Aude Oliva, Rogerio Feris
Int. Conf. on Learning Representations (ICLR), 2021.

This paper introduces an adaptive temporal fusion, AdaFuse, that dynamically fuses channels for strong temporal modelling. Extensive experiments on SomethingV1\&V2, Jester and Mini-Kinetics show AdaFuse achieves 40\% computation savings with comparable accuracy to state-of-the-art methods

AR-Net: Adaptive Frame Resolution for Efficient Action Recognition

Yue Meng, Chung-Ching Lin, Rameswar Panda, Prasanna Sattigeri, Leonid Karlinsky, Aude Oliva, Kate Saenko, Rogerio Feris
European Conf. on Computer Vision (ECCV), 2020.

This paper proposes a novel approach, AR-Net (Adaptive Resolution Network), that selects on-the-fly the optimal resolution for each frame conditioned on the input for efficient action recognition in long untrimmed videos. Extensive experiments on several action recognition datasets well demonstrate the efficacy of our method over state-of-the-art.

Learning 3D-aware Egocentric Spatial-Temporal Interaction via Graph Convolutional Networks

Chengxi Li, Yue Meng, Stanley H. Chan and Yi-Ting Chen
IEEE Int. Conf. on Robotics and Automation (ICRA), 2020.

This paper proposes a 3D-aware egocentric spatial-temporalinteraction framework for automated driving applications. Graph convolution networks (GCN) is devised for interaction modeling. Extensive experiments are conducted using Honda Research Institute Driving Dataset.

Localization and Mapping using Instance-specific Mesh Models

Qiaojun Feng, Yue Meng, Mo Shan and Nikolay Atanasov
IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2019.

This paper focuses on building semantic maps, containing object poses and shapes, using a monocular camera. Our contribution is an instance-specific mesh model of object shape that can be optimized online based on semantic information extracted from camera images.

SIGNet: Semantic Instance Aided Unsupervised 3D Geometry Perception

Yue Meng, Aman Raj, Samuel Sunarjo, Rui Guo, Tara Javidi, Gaurav Bansal, Dinesh Bharadia
IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.

This paper introduces SIGNet, a novel framework that provides robust geometry perception without requiring geometrically informative labels. SIGNet is shown to improve upon the state of art unsupervised learning for geometry perception by 30%

Dense Spatial Segmentation from Sparse Semantic Information

Qiaojun Feng, Yue Meng and Nikolay Atanasov
Workshop at Robotics: Science and Systems (RSS), 2018.

This paper develops an environment representation that affords reasoning about the occupancy of space, necessary for safe navigation, and about the identity of objects, necessary for complex task interpretation.

Cooperative Driving Strategies for Nonsignalized Intersections

Yue Meng, Li Li, Fei-Yue Wang, Keqiang Li and Zhiheng Li
IEEE Transactions on Vehicular Technology (TVT), 2017.

We study the difference between two major strategies of cooperative driving at intersections: the “ad hoc negotiation based” strategy and the "planning based" strategy. We carry out a series of simulations under different traffic scenarios for comparison. Results indicate the performance of a strategy mainly depends on the passing order of vehicles that it finds.


Collecting (click to see more)







Soccer (click to see more)


High school friends in THU-PKU Derby, 2014


Champion at Tsinghua Futsal Tournament, 2014


Champion at Automation Tournament, 2016


Champion at Chinese Alumni League, 2017


Champion at SV Alumni Soccer Tournament, 2019

Running (click to see more)


High school record, 2012 (3000m race)


Unlimited relay race champion at BUAA, 2014


1500m race champion, Tsinghua Sports Meeting, 2015


4*800m race champion, Tsinghua Sports Meeting, 2015


4*400m race champion, Tsinghua Sports Meeting, 2016