me3

Lingyun Yu

name

Associate Researcher [Google Scholar]
University of Science and Technology of China

Email: yuly AT ustc DOT edu DOT cn
IMCC
:[Group Link]
Address: Gaoxin Campus USTC, Fuxing Road, Hefei, Anhui, 230000, P. R. China


      My research interests span Talking face generation, Face forgery detection, Proactive defense, Speech Spoofing Detection, AIGC Content Generation and Security.

      欢迎来自数学、物理、信息、计算机等相关专业背景,对智能内容生成与安全方向感兴趣的同学保送/报考硕士研究生。也欢迎有意向读研、申请大创和暑研的本科生与我们联系,尽早加入实验室一起学习!

  News!

2024/08   Congratulation! Receiving the funding support from National Nature Science Foundation of China!
2024/08   One Journal paper accepted by CVIU!
2024/07   One Regular paper accepted by ACM MM2024!
2024/06   One Journal paper accepted by TMM!
2024/05   Journal papers accepted by TASLP& TIFS!
2024/03   One Journal paper accepted by T-CSVT!
2024/02   One Regular  paper accepted by CVPR2024!
2024/01   One Journal paper accepted by T-CSS!

 Work Experience

2022/10-present
Associate Researcher
University of Science and Technology of China

2020/08-2022/09
Associate Researcher
Hefei comprehensive national science center

2020/08-2022/09
Postdoctor
University of Science and Technology of China

  Education

2020/01-2020/04
Visiting Student
the University of Sydney, Australia (USYD)

2015/09-2020/07
Doctor of Control Science and Engineering
University of Science and Technology of China (USTC)

2011/09-2015/06
Bachelor of Electrical Engineering and Automation
China University of Mining and Technology (CUMT)

  Publications

After 2023

Symmetrical Siamese network for pose-guided person synthesis
Quanwei Yang, Lingyun Yu, Fengyuan Liu, Yun Song*, Meng Shao, Guoqing Jin, and Hongtao Xie
Computer Vision and Image Understanding

Control-Talker: A Rapid-Customization Talking Head Generation Method for Multi-Condition Control and High-Texture Enhancement
Yiding Li, Lingyun Yu*, Li Wang, and Hongtao Xie
ACM MULTIMEDIA 2024

IEIRNet: Inconsistency Exploiting Based Identity Rectification for Face Forgery Detection
Mingqi Fang, Lingyun Yu*, Yun Song, Yongdong Zhang, and Hongtao Xie
IEEE Transactions on Multimedia

Exploring Bi-Level Inconsistency via Blended Images for Generalizable Face Forgery Detection
Peiqi Jiang, Hongtao Xie*, Lingyun Yu, Guoqing Jin, and Yongdong Zhang
IEEE Transactions on Information Forensics and Security

Generalizable Speech Spoofing Detection Against Silence Trimming with Data Augmentation and Multi-task Meta-Learning
Li Wang, Lingyun Yu*, Yongdong Zhang,  Hongtao Xie*
IEEE/ACM Transactions on Audio, Speech and Language Processing.

Fast speech style adaptation with adjustable prosody and variable duration
Zhiqiang Hua, Lingyun Yu*, Chuanbin Liu,  Hongtao Xie, Dengdi Sun
Journal of University of Science and Technology of China (JUSTC)

A Detail-Aware Transformer to Generalisable Face Forgery Detection
Jiaming Li, Lingyun Yu*, Runxin Liu, Hongtao Xie
IEEE Transactions on Circuits and Systems for Video Technology

DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection
Yuhao Sun, Lingyun Yu*, Hongtao Xie, Jiaming Li, Yongdong Zhang
IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR2024)

STIDNet: Identity-Aware Face Forgery Detection with Spatiotemporal Knowledge Distillation
Mingqi Fang, Lingyun Yu∗, Hongtao Xie, Qingfeng Tan∗, Zhiyuan Tan, Amir Hussain, Zezheng Wang, Jiahong Li, and Zhihong Tian.
IEEE Transactions on Computational Social Systems 

RAIRNet: Region-Aware Identity Rectification for Face Forgery Detection
Mingqi Fang, Lingyun Yu*, Hongtao Xie, Junqiang Wu, Zezheng Wang, Jiahong Li, Yongdong Zhang.
In Proceedings of the 31th ACM International Conference on Multimedia 

High Fidelity Face Swapping via Semantics Disentanglement and Structure Enhancement
Fengyuan Liu, Lingyun Yu*, Hongtao Xie, Chuanbin Liu, Zhiguo Ding, Quanwei Yang,Yongdong Zhang.
In Proceedings of the 31th ACM International Conference on Multimedia 

Constructing Spatio-Temporal Graphs for Face Forgery Detection
Zhihua Shang, Hongtao Xie, Lingyun Yu*, Zhengjun Zha, and Yongdong Zhang.
ACM Transactions on the Web 

深度人脸伪造与检测技术综述
谢天,于灵云*,罗常伟,谢洪涛,张勇东
清华大学学报(自然科学版):1-16. DOI:10.16511/j.cnki.qhdxxb.2023.21.002.

2021-2022

Multimodal Learning for Temporally Coherent Talking Face Generation with Articulator Synergy
Lingyun Yu, Hongtao Xie and Yongdong Zhang.
IEEE Transactions on Multimedia 

Wavelet-enhanced Weakly Supervised Local Feature Learning for Face Forgery Detection.
Jiaming Li, Hongtao Xie*, Lingyun Yu, and Yongdong Zhang.
ACM MM2022 

Attention-Guided Transformation-Invariant Attack for Black-Box Adversarial Examples
Jiaqi Zhu, Feng Dai, Lingyun Yu, HongtaoXie*, Lidong Wang, Bo Wu, Yongdong Zhang.
International Journal of Intelligent Systems 

Dynamic-aware Federated Learning for Face Forgery Video Detection
Ziheng Hu, Hongtao Xie*, Lingyun Yu, Xingyu Gao*, Zhihua Shang, Yongdong Zhang.
ACM Transactions on Intelligent Systems and Technology 

Discriminative Feature Mining Based on Frequency Information and Metric Learning for Face Forgery Detection
Jiaming Li, Hongtao Xie*, Lingyun Yu, Xingyu Gao and Yongdong Zhang.
IEEE Transactions on Knowledge and Data Engineering 

REMOT: A Region-to-Whole Framework for Realistic Human Motion Transfer.
Quanwei Yang, Xinchen Liu, Wu Liu, Hongtao Xie, Xiaoyan Gu, Lingyun Yu, and Yongdong Zhang.
ACM MM2022

PRRNet: Pixel-Region relation network for face forgery detection
Zhihua Shang, Hongtao Xie*, Zhengjun Zha, Lingyun Yu, Yan Li, and Yongdong Zhang
Pattern Recognition

Before 2020

Multimodal Inputs Driven Talking Face Generation With Spatial-Temporal Dependency
Lingyun Yu, Jun Yu, Mengyan Li and Qiang Ling.
IEEE Transactions on Circuits and Systems for Video Technology

BLTRCNN Based 3D Articulatory Movement Prediction: Learning Articulatory Synchronicity From Both Text and Audio Inputs
Lingyun Yu, Jun Yu, and Qiang Ling.
IEEE Transactions on Multimedia

Mining Audio, Text and Visual information for Talking face generation
Lingyun Yu, Jun Yu
IEEE International Conference on Data Mining (ICDM) 2019 (Regular paper: 9.08% acceptance rate)

A realistic 3D articulatory animation system for emotional visual pronunciation
Lingyun Yu, Jun Yu
Multimedia Tools and Applications

Deep Neural Network Based 3D Articulatory Movement Prediction Using Both Text and Audio Inputs
Lingyun Yu, Jun Yu, Ling Qiang
International Conference on Multimedia Modeling

Synthesizing 3D Acoustic-Articulatory Mapping Trajectories: Predicting Articulatory Movements by Long-Term Recurrent Convolutional Neural Network.
Lingyun Yu, Jun Yu, Ling Qiang 
IEEE International Conference on Visual Communications and Image Processing (VCIP)

An Emotional Text-Driven 3D Visual Pronunciation System for Mandarin Chinese
Lingyun Yu, Changwei Luo, Jun Yu
Chinese Conference on Pattern Recognition (CCPR)

Synthesizing Photo-Realistic 3D Talking Head: Learning Lip Synchronicity and Emotion from Audio and Video
Jun Yun, Lingyun Yu
IEEE International Conference on Image Processing (ICIP)

Beauty Product Retrieval Based on Regional Maximum Activation of Convolutions with Generalized Attention.
Jun Yun, Lingyun Yu, et al.
ACM MM 2019 workshop

Bidirectional Attention-Recognition Model for Fine-grained Object Classification
Chuanbin Liu, Lingyun Yu, et al.
IEEE Transactions on Multimedia

  Awards

2022/10 荣获第二届广播电视和网络视听人工智能应用创新大赛的创新项目优秀 http://www.nrta.gov.cn/art/2022/11/1/art_113_62625.html
2020/05 Excellent graduates, USTC
2019/10 Suzhou Industrial Park Scholarship, USTC
2019/07 1st place, ACMMultimedia "AI Meets Beauty" challenge
2017/09 National Scholarships, USTC
2015/05 Excellent graduates, CUMT

  Fundation

2024/08 National Nature Science Foundation of China
2023/09 中国科学技术大学-校青年创新基金项目
2022/12 国家重点研发计划项目-子课题
2021/09 Hefei Postdoctoral Research Activities Foundation
2021/08 National Nature Science Foundation of China
2020/10 China Postdoctoral Science Foundation
2020/12 Anhui Postdoctoral Research Activities Foundation

 Professional Activities

Reviewer:
IEEE Transactions on Multimedia;
IEEE Transactions on Circuits and Systems for Video Technology;
ACM Transactions on Multimedia Computing, Communications, and Applications;
IEEE Transactions on Computational Social Systems;
Pattern Recognition;
IEEE/ACM Transactions on Audio, Speech and Language Processing;
Security and Communication Networks;
ACM International Conference on Multimedia;
Speech Communication; 
Frontiers in Psychology