News!

• 2024-12-13: We’re excited to have made a small contribution to DeepSeek-VL2, an advanced series of Mixture-of-Experts (MoE) Vision-Language Models. DeepSeek-VL2 improves upon its predecessor with superior performance in tasks like visual question answering and document understanding, offering competitive results with fewer parameters. Proud to be part of this cutting-edge project!

• 2024-11-22: Our latest work on the robustness of Vision-Language Models (VLMs), titled "Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks, has been submitted to CVPR 2025. This work identifies vulnerabilities in Vision-Language Models (VLMs) to noisy image attacks due to the lack of noise augmentation during training. It introduces Robust-VLGuard, a dataset to enhance robustness against Gaussian noise, and proposes DiffPure-VLM, a defense leveraging diffusion models to transform adversarial noise into Gaussian noise. The dataset and code will be open-sourced to advance secure VLM research.

• 2024-11-2: Grateful to have reached 100 citations on Google Scholar! This is a small milestone that encourages me to keep learning and contributing. Thank you to everyone who has supported my work along the way.

• 2024-8-30~9-4: I was thrilled to be an oral speaker, presenting two of our papers, DLAFormer and DRFormer, at ICDAR in Athens, Greece (Photos can be found here.). Athens is a beautiful city steeped in history, and it was wonderful to experience its rich cultural atmosphere. I also enjoyed meeting many new colleagues in the research community, making it a truly memorable and unforgettable journey.

• 2024-8-16: Our latest progress, the complete version of DLAFormer, titled "UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis", has been submitted to Pattern Recognition Journal.

• 2024-7-26: Detect-Order-Construct have been accepted by Pattern Recognition!

• 2024-6-29: DLAFormer, UniVIE, and DRFormer have been selected for oral presentation in ICDAR 2024!

• 2024-4-12: DLAFormer, UniVIE, and DRFormer have been accepted by ICDAR 2024!

• 2024-3-15: It's thrilling to share that Azure AI Document Intelligence has introduced support for Hierarchical Document Structure Analysis (HDSA), as detailed in our recently published paper, "Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis." This advancement opens up new possibilities for analyzing complex document structures more effectively. For more details on this breakthrough, check out our publication on arXiv and the official announcement.

• 2024-3-4: We released the source code of our solution for the Language-Enhanced Image New Category Discovery at the 2nd The Guangdong–Hong Kong–Macao Greater Bay Area International Algorithm and Programming Contest (Visual Prompt Tuning Challenge @ CVPR 2023 HIT Workshop (2023)).

• 2024-2-1: We submitted our new progress on Document Layout Analysis, named DLAFormer: A End-to-End Transformer for Document Layout Analysis, to ICDAR 2024.

• 2024-1-18: We released a unified label space approach for Visual Information Extraction, named UniVIE: A Unified Label Space Approach to Visual Information Extraction from Form-like Documents! In this work, we present a new perspective, reframing VIE as a relation prediction problem and unifying labels of different tasks into a single label space. This unified approach allows for the definition of various relation types and effectively tackles hierarchical relationships in form-like documents.

• 2024-1-16: We released a new technical paper for logical layout analysis in natural scenes, named Dynamic Relation Transformer for Contextual Text Block Detection!

• 2023-12-7: 2nd Prize, 2023 International Algorithm Case Competition (Visual Prompt Tuning Challenge @ CVPR 2023 HIT Workshop (2023)), 200,000 RMB bonus!

• 2023-11-14: We submitted our new progress on Hierarchical Document Structure Analysis to Pattern Recognition Journal.

• 2023-7-12: Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer is accepted by Pattern Recognition Journal!

• 2023-4-20: Two papers are accepted by ICDAR 2023!

• 2023-3-10: We proposed a new Dynamic Queries based Detection Transformer for more robust table structure recognition!

• 2022-12-1: 2nd Prize, 2022 International Algorithm Case Competition (Panoptic Scene Graph Challenge @ ECCV 2022 SenseHuman Workshop (2022)), 100,000 RMB bonus!

• 2022-9-1: A paper is accepted by ACM MM 2022!

Academic Services

Journal Reviewer

• International Journal on Document Analysis and Recognition

Conference Reviewer

• International Conference on Document Analysis and Recognition

Publications

LLM & Multimodal LLM

1. DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding

   Zhiyu Wu*, Xiaokang Chen*, Zizheng Pan*, Xingchao Liu*, Wen Liu*, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, Chong Ruan

   Cutting-edge Project

   PDF,Code

2. Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks

   Jiawei Wang*, Yushen Zuo*, Yuanjun Chai, Zhendong Liu, Yicheng Fu, Yichun Feng, Kin-man Lam

   Submitted to CVPR 2025

   PDF will be available soon.

Document Intelligence:

1. UniHDSA: A Unified Relation Prediction Approach for Hierarchical Document Structure Analysis

   Jiawei Wang, Kai Hu, Qiang Huo

   Submitted to Pattern Recognition

   PDF will be available soon.

2. DLAFormer: An End-to-End Transformer for Document Layout Analysis

   Jiawei Wang*, Kai Hu*, Qiang Huo

   2024 International Conference on Document Analysis and Recognition, Oral

   PDF

3. UniVIE: A Unified Label Space Approach to Visual Information Extraction from Form-like Documents

   Kai Hu, Jiawei Wang, Weihong Lin, Zhuoyao Zhong, Lei Sun, Qiang Huo

   2024 International Conference on Document Analysis and Recognition, Oral

   PDF

4. Dynamic Relation Transformer for Contextual Text Block Detection

   Jiawei Wang*, Shunchi Zhang*, Kai Hu*, Chixiang Ma, Zhuoyao Zhong, Lei Sun, Qiang Huo

   2024 International Conference on Document Analysis and Recognition, Oral

   Project page

5. Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysis

   Jiawei Wang, Kai Hu, Zhuoyao Zhong, Lei Sun, Qiang Huo

   Pattern Recognition, 2024 (Q1 Journal, IF 7.5)

   PDF,Dataset

6. DQ-DETR: Dynamic Queries Enhanced Detection Transformer for Arbitrary Shape Text Detection

   Chixiang Ma, Lei Sun, Jiawei Wang, Qiang Huo

   2023 International Conference on Document Analysis and Recognition, Oral

   PDF

7. A Hybrid Approach to Document Layout Analysis for Heterogeneous Document Images

   Zhuoyao Zhong, Jiawei Wang, Haiqing Sun, Kai Hu, Erhan Zhang, Lei Sun, Qiang Huo

   2023 International Conference on Document Analysis and Recognition, Poster

   PDF

8. Robust Table Structure Recognition with Dynamic Queries Enhanced Detection Transformer

   Jiawei Wang, Weihong Lin, Chixiang Ma, Mingze Li, Zheng Sun, Lei Sun, Qiang Huo

   Pattern Recognition, 2023 (Q1 Journal, IF 7.5)

   PDF

9. TSRFormer: Table Structure Recognition with Transformers

   Weihong Lin, Zheng Sun, Chixiang Ma, Mingze Li, Jiawei Wang, Lei Sun, Qiang Huo

   2022 ACM Multimedia, Poster

   PDF

Physics Related:

1. Study of nonequilibrium phase transitions mechanisms in exclusive network and node model of heterogeneous assignment based on real experimental data of KIF3AC and KIF3CC motors

   Yuqing Wang, Chang Xu, Molin Fang, Tianze Li, Liwen Zhang, Dasen Wei, Kaichen Ouyang, Tunyu Zhang, Chuzhao Xu, Haosong Sun, Yunzhi Wang, Jiawei Wang

   The European Physical Journal Plus, 2022

2. Physical mechanisms in impacts of interaction factors on totally asymmetric simple exclusion processes

   Yuqing Wang, Jiawei Wang, Binghong Wang

   International Journal of Modern Physics B, 2019

3. Stochastic dynamics in nonequilibrium phase transitions of multiple totally asymmetric simple exclusion processes coupled with strong and weak interacting effects

   Yuqing Wang*, Jiawei Wang*, Ziang Zhu*, Binghong Wang

   International Journal of Modern Physics B, 2019

4. Evolvement laws and stability analyses of traffic network constituted by changing ramps and main road

   Yuqing Wang, Chaofan Zhou, Jiawei Wang, Xinpeng Ni

   International Journal of Modern Physics B, 2019

5. A macroscopic model for VOC emissions process complemented by real data

   Yuqing Wang, Chaofan Zhou, Ziang Zhu, Jiawei Wang, Zimeng Wang, Chenhao Fang, Bin Jia

   Modern Physics Letters B, 2018

6. Control Strategies for Reducing VOCs Emission Process Based on Empirical Data

   Yuqing Wang*, Jiawei Wang*, Ziang Zhu*, Chaofan Zhou, Yiyao Kou, Jing Sun, Zhengwei Mei, Ziwu Li, Peng Wu, Donghu Wang, Si Zhang, Wenli Zhang

   2018 IEEE International Conference on Mechatronics, Robotics and Automation (ICMRA), Best Presentation Award