OMEGA LAB / 研究
研究方向
我们研究智能体如何感知、建模、推理和行动,涵盖机器人策略、世界模型、工具使用、多智能体工作流以及具身系统的安全。
多模态感知
表征物理世界
我们研究视觉感知、三维几何、多模态融合与场景表征,结合视觉、几何及其他传感信息来理解物体及其所处的环境。
UniT 能从任意数量的视角恢复公制尺度的几何,这里展示的是对 HKUST(GZ) 一处室内场景的重建。项目主页有更多重建结果、实验结果和在线演示。
了解 UniT代表性学术成果
Zhaonian Kuang, Rui Ding, Haotian Wang, Xinhu Zheng, Meng Yang, Gang Hua
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 40707–40716
Haotian Wang, Yusong Huang, Zhaonian Kuang, Hongliang Lu, Xinhu Zheng, Meng Yang, Gang Hua
IEEE Transactions on Pattern Analysis and Machine Intelligence(已投稿)
Ding, Rui; Kuang, Zhaonian; Zhou, Zongwei; Yang, Meng; Zheng, Xinhu; Hua, Gang
Proceedings of the AAAI Conference on Artificial Intelligence 40 (5), 3605–3613
Mengqi Lei, Siqi Li, Yihong Wu, Han Hu, You Zhou, Xinhu Zheng, Guiguang Ding, Shaoyi Du, Zongze Wu, Yue Gao
IEEE Transactions on Multimedia(已投稿)
VeXKD: The Versatile Integration of Cross-Modal Fusion and Knowledge Distillation for 3D Perception
2024Ji, Yuzhe; Chen, Yijie; Yang, Liuqing; Ding, Rui; Yang, Meng; Zheng, Xinhu
Advances in Neural Information Processing Systems (NeurIPS), 125608–125634
机器人学习与操作
学习行动策略
我们研发把推理与操作连接起来的视觉—语言—动作模型,研究运动生成、策略改进和高效推理。
代表性学术成果
Yunpeng Mei, Jiakai He, Hongjie Cao, Chenyu Wang, Xiaowen Zhu, Yihan Zhou, Jiamin Wang, Chenbo Xin, Peng Cheng, Yuxuan Yang, Yijie Wang, Xinhu Zheng, Gao Huang, Jie Chen, Gang Wang
NeurIPS 2026(已录用,海报)
Wenxuan Song, Jiayi Chen, Shuai Chen, Jingbo Wang, Pengxiang Ding, Han Zhao, Yikai Qin, Xinhu Zheng, Donglin Wang, Yan Wang, Haoang Li
European Conference on Computer Vision (ECCV), 361–378
Zhide Zhong, Haodong Yan, Junfeng Li, Xiangchen Liu, Xin Gong, Tianran Zhang, Wenxuan Song, Jiayi Chen, Xinhu Zheng, Hesheng Wang, Haoang Li
IEEE Transactions on Robotics (Early Access)
Wenxuan Song, Jiayi Chen, Wenxue Li, Xu He, Xinhu Zheng, Zhe Liu, Hesheng Wang, Haoang Li
IEEE/ASME Transactions on Mechatronics
世界模型与预测
建模世界如何变化
我们研究预测式世界模型和未来运动,关注行为、驾驶风格和物理动力学如何为规划提供依据。
代表性学术成果
Xiaoyun Qiu, Jingtao He, Yijie Chen, Yusong Huang, Haotian Wang, Yixuan Wang, Xinhu Zheng
IEEE Transactions on Intelligent Transportation Systems(已投稿);arXiv:2606.06014
S Shen, J Yang, H Lu, H Zhong, Q Zhang, X Zheng
arXiv 预印本 arXiv:2505.07855
Pengqin Wang, Meixin Zhu, Xinhu Zheng, Hongliang Lu, Hui Zhong, Xianda Chen, Shaojie Shen, Xuesong Wang, Yinhai Wang, Fei-Yue Wang
IEEE Transactions on Intelligent Vehicles
视觉语言导航
让语言落到空间里
我们研究智能体如何在物理环境中遵循语言指令,把空间感知、运动预测和规划结合起来,用于地面和空中导航。
代表性学术成果
Zhong, Zhide; Lu, Jia; Liu, Xiangchen; Yu, Runze; Zheng, Xinhu; Liu, Zhe; Wang, Hesheng; Li, Haoang
IEEE Transactions on Circuits and Systems for Video Technology 36 (8), 11717–11728
Tianfu Li, Wenbo Chen, Haoxuan Xu, Xinhu Zheng, Haoang Li
arXiv 预印本 arXiv:2603.17459
Tianshun Li, Tianyi Huai, Zhen Li, Yichun Gao, Haoang Li, Xinhu Zheng
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 17199–17206
Shi, Yuhong; Liu, Jianyi; Sun, Lihang; Zheng, Xinhu
IEEE Transactions on Circuits and Systems for Video Technology 35 (7), 6231–6244
意图与决策
理解行为与决策
我们对意图和人类驾驶行为建模,考察在真实世界的约束下,视觉证据和情境如何影响自主决策。
代表性学术成果
Xiaoyun Qiu, Haichao Liu, Yue Pan, Jun Ma, Xinhu Zheng
IEEE Transactions on Intelligent Transportation Systems, Early Access, 1–13
Xianda Chen, Xu Han, Meixin Zhu, Xiaowen Chu, PakHin Tiu, Xinhu Zheng, Yinhai Wang
IEEE Transactions on Intelligent Transportation Systems 26 (7), 9755–9766
Lu, Hongliang; Yang, Junjie; Zhu, Meixin; Lu, Chao; Chen, Xianda; Zheng, Xinhu; Yang, Hai
Transportation Research Part C: Emerging Technologies 172, 105030
多智能体系统
协调智能体与工具
我们研究智能体如何使用专用模型和工具、共享信息并协同完成任务,既包括用于分析和生成的数字工作流,也包括空地协同系统。
代表性学术成果
Jingyi Zheng, Yule Liu, Zifan Peng, Tianyi Hu, Yuemeng Zhao, Xinhu Zheng, Xinlei He
ACM Multimedia 2026(已录用)
Tianshun Li, Hongliang Lu, Haoang Li, Xinhu Zheng
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2026(已录用)
Feng, Zhaohan; Xue, Ruiqi; Yuan, Lei; Yu, Yang; Ding, Ning; Liu, Meiqin; Gao, Bingzhao; Sun, Jian; Zheng, Xinhu; Wang, Gang
Science China Information Sciences 69 (5), 151202
Yijie Chen, Yuzhe Ji, Haotian Wang, Xiaoyun Qiu, Ying-Cong Chen, Xinhu Zheng
Proceedings of the AAAI Conference on Artificial Intelligence 40 (35), 29376–29384
具身智能安全
检验并加固具身智能体
我们探究感知、规划和检测模型在后门、对抗和分布外威胁下如何失效,设计隐蔽而有效的攻击来暴露漏洞,并研究缓解策略,提升自动驾驶和多模态系统的鲁棒性。
代表性学术成果
Yifan Liao, Yule Liu, Zhen Sun, Zongmin Zhang, Yupeng He, Jiaheng Wei, Xinhu Zheng, Xinlei He
Network and Distributed System Security Symposium (NDSS)(已录用)
Yifan Liao, Yuxin Cao, Yedi Zhang, Wentao He, Yan Xiao, Xianglong Du, Zhiyong Huang, Jin Song Dong
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 34950–34960
Yifan Liao, Zhen Sun, Xiaoyun Qiu, Zixiao Zhao, Wenbing Tang, Xinlei He, Xinhu Zheng, Tianwei Zhang, Xinyi Huang, Xingshuo Han
arXiv 预印本 arXiv:2510.02803
查看完整的学术成果列表。
全部学术成果研究平台
面向具身智能研究的无人驾驶车辆、无人机、机器人操作平台、人形机器人、遥操作设备与 AI 算力。
查看研究平台