Advanced Technology Tutorial

1. Bei Li

Speaker:Bei Li
Affiliation:Meituan
Title:A Survey on Large Model Architecture Research
Abstract:This report provides a comprehensive survey of recent large model architectures from four dimensions. Regarding normalization methods, this paper explores how they enhance training stability and analyzes the limitations and improvement directions of mainstream Pre-Norm and Post-Norm structures. In terms of architectural improvements, it focuses on the exploration of non-attention architectures represented by Mamba and RWKV. For efficient long-text modeling, the report examines how variants such as sliding window and sparse attention effectively reduce computational complexity and memory overhead. Finally, it briefly discusses the recent applications and research progress of diffusion large models in text generation.
Bio:Bei Li, Ph.D., supervised by Professor Xiao Tong at Northeastern University, with research interests covering machine translation, foundation model optimization, and large language model post-training techniques. He currently works in Meituan's large language model group, focusing on foundation and post-training research. He has published over 30 papers at top-tier conferences including ACL, EMNLP, ICML, NeurIPS, ICLR, AAAI, and COLING, with over 2000 citations on Google Scholar. He has participated in the development of multiple open-source systems and has participated in machine translation evaluation tasks such as WMT, CCMT, and CWMT, achieving first place in multiple translation tracks. He has long served as a program committee member for top conferences including ACL, EMNLP, ICLR, ICML, and NeurIPS, and was recognized as an Outstanding Reviewer for EMNLP 2021. During his doctoral studies, he received multiple National Scholarships and was honored with Baidu Scholarship nomination and the Outstanding Doctoral Dissertation Award from the Chinese Information Processing Society of China.

2. Linfeng Zhang

Speaker:Linfeng Zhang
Affiliation:Shanghai Jiao Tong University
Title:AI Model Compression and Acceleration in Data Centers
Abstract:The computational cost of large models severely constrains their deployment applications. Generally speaking, the computational cost of models is jointly determined by their parameter count and data volume. Existing compression research mainly focuses on how to reduce the number of model parameters while ignoring compression in the data dimension. With the emergence of strong reasoning models and video generation models, data scale (number of tokens) has become the primary factor behind the persistently high computational costs of artificial intelligence. In this report, we will introduce several typical cases of model compression and acceleration in data centers for large models, multimodal large models, and image/video generation models.
Bio:Linfeng Zhang serves as an Assistant Professor at the School of Artificial Intelligence, Shanghai Jiao Tong University, leading the EPIC (Efficient and Precision Intelligent Computing) laboratory with qualifications to supervise master's and doctoral students. He received his Ph.D. from the Institute for Interdisciplinary Information Sciences at Tsinghua University in June 2024. During his doctoral studies, he received the Microsoft Scholar honor (twelve recipients annually in the Asia-Pacific region), Beijing Outstanding Graduate, Tsinghua University Outstanding Doctoral Dissertation, Tsinghua University Qihang Award Gold Prize (28 recipients university-wide), and Tsinghua University Jiang Nanxiang Scholarship (20 recipients university-wide). During his Ph.D., he published over 20 high-quality academic papers, including 13 as first author, with total citations exceeding 1900 (as of July 2024). In 2019, he first proposed the self-distillation algorithm, which is one of the representative works in the field of knowledge distillation. His research achievements have been applied in companies such as Beixiong Core, Huawei, and the Institute for Interdisciplinary Information Core Technologies. Since 2020, he has served as a reviewer for numerous academic conferences and journals including NeurIPS, ICML, ICLR, CVPR, ECCV, ICCV, AAAI, IJCAI, IEEE TPAMI, IEEE TCSVT, and IEEE TIP. He joined the School of Artificial Intelligence at Shanghai Jiao Tong University as an Assistant Professor in July 2024.

3. Chen Xu

Speaker:Chen Xu
Affiliation:Harbin Engineering University
Title:A Survey on Speech Processing in the Era of Large Language Models
Abstract:The rapid development of large language models is profoundly transforming the technical paradigm of speech processing, driving a revolutionary transformation from traditional cascaded architectures to end-to-end unified modeling. This report will systematically examine the core challenges faced by speech large language models, focusing on modeling paradigms, model architectures, and training strategies to introduce cutting-edge developments, and prospect future research directions, aiming to provide reference for researchers in related fields.
Bio:Chen Xu is an Associate Professor and Master's supervisor at Harbin Engineering University, with a Ph.D. from the Natural Language Processing Laboratory at Northeastern University. His main research interests include multimodal processing and artificial intelligence security. He has published over 30 papers in high-level conferences and journals such as ACL, EMNLP, ICLR, TASLP, and UAI. He has published one textbook in strategic emerging fields and leads multiple projects including the National Natural Science Foundation of China.

4. Xuebo Liu

Speaker:Xuebo Liu
Affiliation:Harbin Institute of Technology (Shenzhen)
Title:Frontiers in Machine Translation and Multilingual Processing in the Era of Large Models
Abstract:Large language models, as a new generation unified modeling framework, have broken through the boundaries of traditional tasks and greatly enhanced the capabilities of multilingual processing and cross-lingual translation. On the other hand, machine translation, as a core generation task, continuously drives the development of large models in key technologies such as data synthesis, evaluation methods, and agent collaboration. This report will focus on the bidirectional driving relationship between large models and machine translation, and discuss some highly promising research directions and innovative ideas that can quickly produce top-conference results.
Bio:Xuebo Liu is an Associate Professor and Doctoral Supervisor at the School of Computer Science and Technology/Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen). He is selected for the 10th China Association for Science and Technology Youth Talent Support Program (CAST funding), Pengcheng Peacock Plan Distinguished Position, and the university's Young Top Talent Program. He received his Ph.D. from the Faculty of Science and Technology, University of Macau in 2021. His main research interests include natural language processing, large language models, and machine translation. He has published over 40 papers at top NLP conferences such as ACL and EMNLP, and top machine learning conferences such as ICLR and NeurIPS, including over 30 papers as first or corresponding author. He holds 6 authorized invention patents as the first inventor. He serves as (Senior) Area Chair for the three major NLP conferences ACL 2022&2024-2025, EMNLP 2022-2025, NAACL 2024-2025, Area Chair for flagship domestic NLP conferences NLPCC 2024 and CCL 2023-2025, and as a reviewer for multiple international top conferences and journals. He leads projects including National Natural Science Foundation Youth Project, Guangdong Provincial Key R&D Program, Guangdong Provincial General Fund Project, Shenzhen Outstanding Science and Technology Innovation Talent Development Project, Shenzhen General Fund Project, CCF-Tencent Rhino-Bird Project, Tencent WeChat Rhino-Bird Special Project, Huawei Long-term Scientific Research Cooperation Project, and participates in National Natural Science Foundation Key Project, Shenzhen Peacock Team Project, Shenzhen Major Science and Technology Project, and large enterprise horizontal projects. He has won the First Prize of China Computer Federation Science and Technology Award for Technological Progress, the Second Prize of Macau Science and Technology Award for Technological Invention, Macau Graduate Science and Technology R&D Award, and the Outstanding Doctoral Dissertation Nomination Award from the Chinese Information Processing Society of China.

5. Libo Qin

Speaker:Libo Qin
Affiliation:Central South University
Title:A Survey on Chain-of-Thought Reasoning Research Progress in Large Models
Abstract:In recent years, reasoning large models represented by OpenAI o1 and DeepSeek R1 have attracted great attention. In this report, the speaker will provide an overview of the research progress in chain-of-thought reasoning, the core technology of reasoning large models. Meanwhile, the speaker will introduce cutting-edge directions such as cross-lingual chain-of-thought, cross-modal chain-of-thought, and long chain-of-thought.
Bio:Libo Qin is a Distinguished Professor and Doctoral Supervisor at Central South University, currently serving as Secretary-General of the Youth Working Committee of the Chinese Information Processing Society of China. His main research interests include natural language processing and large model reasoning. His research achievements have been selected for the World Artificial Intelligence Conference Young Outstanding Paper Nomination, Paper Digest High-Impact Papers, and EMNLP2022 MMNLU Workshop Best Paper. He serves as Executive Editor for ACL Rolling Review and (Senior) Area Chair for international conferences such as ACL, EMNLP, and NAACL.

6. Xiachong Feng

Speaker:Xiachong Feng
Affiliation:The University of Hong Kong
Title:Large Language Model-based Social Simulation: Progress and Challenges
Abstract:Large language models have injected new vitality into social simulation, enabling agent interactions in complex social contexts to possess advanced intelligent features such as language understanding, strategic reasoning, and situational adaptation, driving a paradigm shift from traditional rule-based modeling toward more complex and humanized agent simulation. This report will systematically review the latest progress of large language models in the field of social simulation, including key application scenarios such as role-playing, strategic games, and collaborative decision-making, exploring their potential in modeling social behavior, generating diverse scenarios, and evaluating policy effects. Meanwhile, the report also identifies key challenges currently faced, such as multi-agent consistency maintenance, long-term interaction stability, and the verifiability and reproducibility of simulation results. Finally, the report proposes future research directions, emphasizing the important significance of algorithmic mechanism design and theoretical framework construction for promoting the scientific development of this field.
Bio:Xiachong Feng is a Postdoctoral Research Fellow at The University of Hong Kong, with a Ph.D. from the Social Computing and Human-Robot Interaction Research Center at Harbin Institute of Technology and as a visiting student at National University of Singapore, supervised by Professor Qin Bing and Professor Xiachong Feng. His research interests include large language models and social agents. He has published multiple papers in top conferences and journals such as ACL, TASLP, and TMLR. He has received awards including three National Scholarships, CCL 2021 Best English Long Paper Award, TMLR Survey Award, and ICASSP 2023 MUG Competition Champion. He serves as program committee member and area chair for conferences including ICML, ICLR, and ACL Rolling Review.

7. Piji Li

Speaker:Piji Li
Affiliation:Nanjing University of Aeronautics and Astronautics
Title:A Survey on Brain Encoding and Decoding Technologies in the Era of Large Models
Abstract:The relationship between brain science and large models has evolved from unidirectional inspiration extraction to bidirectional collaborative innovation. Brain science provides the underlying logic of biological intelligence for large models, while large models provide super tools for analyzing complex systems in brain science. Brain Encoding refers to analyzing the relationship between external stimuli (such as images, language, etc.) and brain activity to establish mapping models from stimuli to neural signals. Brain Decoding aims to recover language, images, and other information from neural signals. This report surveys the exploration and achievements in the interdisciplinary research between large models and brain science from two dimensions: brain encoding analysis and brain decoding generation, and analyzes current challenges and future development directions.
Bio:Piji Li is a Professor and Doctoral Supervisor at the College of Computer Science and Technology/College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics. He completed his undergraduate and master's degrees at Shandong University and received his Ph.D. from The Chinese University of Hong Kong in 2018. He formerly served as a Senior Researcher at Tencent AI Lab's Natural Language Processing Center. His research mainly focuses on natural language processing, including pre-trained models, information retrieval, text mining, text generation, and dialogue systems. He has published over 50 academic papers at top conferences in related fields such as ACL, EMNLP, SIGIR, and WWW. He has been invited to serve as program committee member and reviewer for conferences such as ACL and NeurIPS, area chair for EMNLP2020 and IJCAI2021, and Associate Editor for the journal Neurocomputing. During his work in industry, he was responsible for algorithm development and product release of multiple important projects related to language understanding, text generation, and intelligent dialogue. He received the 2021 "Changkong Scholar" award from Nanjing University of Aeronautics and Astronautics and the 2021 Rising Star Award from ACM Nanjing Chapter. He leads and participates in multiple projects, such as National Key R&D Program, National Natural Science Foundation projects, CCF-Tencent Rhino-Bird Fund, CCF-Zhipu Large Model Fund, and CCF-Baidu PineCone Fund.

8. Peng Li

Speaker:Peng Li
Affiliation:Tsinghua University
Title:A Survey on Agent-based Autonomous Scientific Discovery Research
Abstract:Recent research indicates that large model agents can provide effective support for humans in various key aspects of scientific research. However, whether large model agents possess the capability to autonomously generate scientific discoveries remains an unresolved open question. This paper will systematically review the research progress and application practices of large model agents in core research processes such as literature review, proposal formulation, experiment execution, and systematic falsification, introduce the design concepts of several representative end-to-end autonomous scientific discovery systems, and further explore the main challenges and development prospects faced by artificial intelligence in achieving fully autonomous scientific discovery.
Bio:Peng Li is an Associate Research Fellow at Tsinghua University Institute for AI Industry Research. His main research interests include large models, agents, and AI4Math. He has published over 90 papers in important international conferences and journals in artificial intelligence, received the ACL 2023 Outstanding Paper Award, and has achieved first place on multiple internationally influential leaderboards, surpassing teams from Google Research and OpenAI. He leads projects including National Key R&D Program topics and National Natural Science Foundation general projects, and has served as area chair for important international conferences such as ACL, EMNLP, and NAACL. His research achievements have been applied in products with tens of millions of daily active users at Baidu and Tencent WeChat, achieving significant results. He received the First Prize of the Qian Weichang Chinese Information Processing Science and Technology Award from the Chinese Information Processing Society of China.

9. Yubo Chen

Speaker:Yubo Chen
Affiliation:Institute of Automation, Chinese Academy of Sciences
Title:A Survey on Knowledge Enhancement Research for Large Language Models
Abstract:In recent years, large language models have achieved remarkable progress in knowledge-intensive natural language processing tasks. This seems to indicate that large language models can spontaneously learn vast amounts of knowledge from corpora and implicitly store it in parameters. However, the underlying mechanisms of this phenomenon are still shrouded in many mysteries. How do large language models store and utilize knowledge? How can we modify knowledge in large language models on demand? How can we compensate for the knowledge deficiencies of large language models? These questions urgently need further exploration. This report will focus on introducing the foundational knowledge and recent research progress in knowledge mechanism analysis, knowledge editing, and knowledge enhancement for large language models.
Bio:Yubo Chen is an Associate Research Fellow at the Institute of Automation, Chinese Academy of Sciences. He is selected for the 5th China Association for Science and Technology Youth Talent Support Program, member of the Youth Innovation Promotion Association of the Chinese Academy of Sciences, Global Chinese AI Young Scholar, and Beijing Science and Technology Rising Star. He was consecutively selected for Stanford University's Global Top 2% Scientists list in 2023 and 2024. His research focuses on natural language processing, knowledge graphs, and large models. He has published over 80 academic papers in important international conferences and journals such as ACL, NeurIPS, and ICLR, with over 7200 citations on Google Scholar. One paper was selected as an ESI Highly Cited Paper, and two papers were selected as high-impact papers by ACL and EMNLP (Paper Digest selection). He received the Best Poster Paper Award at the International Semantic Web Conference ISWC 2023 (CCF B-class conference) and has won national conference best paper awards five times. He has published two academic monographs, "Knowledge Graph" and "Knowledge Graph: Algorithms and Practice," selected as textbooks for the 13th Five-Year National Key Book Publishing Program. He leads National Natural Science Foundation general and youth projects and participates in National Natural Science Foundation key projects and 2030 New Generation Artificial Intelligence major projects. He serves as Deputy Director of the Youth Working Committee of the Chinese Information Processing Society of China and has received the First Prize of the "Qian Weichang Chinese Information Processing Science and Technology Award" from the Chinese Information Processing Society of China and the First Prize of Beijing Science and Technology Progress Award.

10. Qingfu Zhu

Speaker:Qingfu Zhu
Affiliation:Harbin Institute of Technology
Title:A Survey on Embodied Planning Research for Large Models
Abstract:Embodied planning focuses on how agents understand complex goals, parse task intentions, and generate structured, executable actions. This report focuses on key issues such as hierarchical task decomposition, plan representation, and scheduling execution for large models in embodied scenarios, systematically reviewing key technical approaches including chain-of-thought, program-assisted planning, and tool-enhanced execution, summarizing the challenges they face such as data scarcity, feedback delays, and verifiability, providing reference for promoting efficient planning capabilities of general embodied intelligent agents.
Bio:Qingfu Zhu is an Associate Professor at the Faculty of Computing, Harbin Institute of Technology, and a joint Ph.D. from the University of California, Santa Barbara. He is a member of the Social Media Processing Professional Committee of the Chinese Information Processing Society of China. His main research interests include code large models and pre-training. He has published over 20 papers in the field of natural language processing, including top international conferences such as ACL, AAAI, and EMNLP. He leads National Natural Science Foundation youth projects and participates in multiple projects including National Key R&D Program and National Major Project of Science and Technology Innovation 2030 - "New Generation Artificial Intelligence". He led the development of the HIT Zhusuan Code Large Model and received awards including the First Prize of the Wu Wenjun Artificial Intelligence Science and Technology Progress Award from the Chinese Association for Artificial Intelligence in 2024.