Advanced Technology Tutorial

Retrieval-Enhanced Generation for Formalized Knowledge: Tong Xu

Speaker: Tong Xu
Title: Retrieval-Enhanced Generation for Formalized Knowledge
Time: July 25, 2024, 09:00-10:30
Abstract: Retrieval-Enhanced Generation has become a hot topic and has shown exceptional potential in various tasks such as question answering, dialogue generation, and text summarization. It offers a new and more precise perspective for addressing the issues of knowledge scarcity and complex reasoning in large models. Meanwhile, as cognitive engines of the AI era, knowledge graphs and other forms of formalized knowledge representation have shown a clear trend of deep integration with large model technologies, especially retrieval-enhanced generation technologies. Their methods of information extraction and induction are also evolving rapidly with the development of large model technologies. In this talk, we will review the related technologies of retrieval-enhanced generation and generative information extraction, and then summarize the latest advancements in how formalized knowledge aids large models through retrieval-enhanced generation.
Biography: Tong Xu is a Distinguished Professor and Ph.D. Supervisor at the University of Science and Technology of China, Chair of the Youth Working Committee of the Chinese Information Processing Society of China, and a recipient of the National Science Fund for Outstanding Young Scholars. His research focuses on multimodal knowledge learning. He has published over 70 papers in CCF-recommended A-level journals/conferences. He has received four international conference paper awards and has guided students to win over ten championships in domestic and international academic competitions/evaluations. In 2022, he received the Second Prize of the Anhui Provincial Science and Technology Progress Award.

Text-Speech Large Models: Chen Xu, Tong Xiao

Speakers: Chen Xu, Tong Xiao
Title: Text-Speech Large Models
Time: July 25, 2024, 10:30-12:00
Abstract: In recent years, multimodal models have made rapid progress and garnered widespread attention, especially with the release of GPT-4o, which has shown impressive performance in voice interaction capabilities. This talk will focus on the theme of "Text-Speech Large Models," covering various tasks such as speech recognition, speech translation, and speech synthesis. It aims to summarize the current state of technological development, analyze the core challenges and solutions in text-speech multimodal integration, and discuss cutting-edge research trends under the paradigm of large language models.
Biographies: Chen Xu is an Associate Professor at the School of Computer Science and Technology at Harbin Engineering University. He received his Ph.D. from Northeastern University in 2023. His research interests include natural language processing, speech processing, and AI security. He has published over 30 papers in high-level conferences and journals such as ACL, EMNLP, ICLR, AAAI, and IJCAI. He has participated in international evaluations like WMT News Translation, Quality Estimation, IWSLT Speech Translation, and various domestic evaluation tasks, achieving outstanding results.
Tong Xiao is a Professor and Ph.D. Supervisor at Northeastern University, Director of the Department of Artificial Intelligence in the School of Computer Science, and Director of the Natural Language Processing Laboratory. He is also the co-founder of NiuTrans. He received his Ph.D. in Computer Science from Northeastern University. From 2006 to 2009, he conducted research at Fuji Xerox and Microsoft Research Asia in Japan and completed a postdoctoral fellowship at the University of Cambridge, UK, from 2013 to 2014. He has published over 100 papers in high-level conferences and journals and authored the book "Machine Translation: Basics and Models." As the technical lead, he successfully developed open-source systems like NiuTrans and NiuTensor, winning over 30 championships in domestic and international evaluations such as WMT, CCMT/CWMT, and NTCIR. In 2016, he received the First Prize of the "Qian Weichang Chinese Information Processing Science and Technology Award" from the Chinese Information Processing Society of China and the CCF-NLP Youth Elite Award in 2021.

Large Model Agents and Security: Zhuosheng Zhang

Speaker: Zhuosheng Zhang
Title: Large Model Agents and Security
Time: July 25, 2024, 13:30-15:00
Abstract: With the rapid development of large-scale language models, constructing autonomous agents capable of dynamic environment perception, task planning, behavior decision-making, and tool manipulation is gradually becoming a reality. Current research has yielded rich results in areas such as system control, scientific research, software programming, and group collaboration. However, while these agents provide significant convenience in daily life, they also bring new security challenges. This talk will introduce the development status of large model agents, focusing on the security risks these agents face during dynamic environment interactions, and will discuss the latest advancements in agent security defenses and countermeasures.
Biography: Zhuosheng Zhang is a Tenure-Track Assistant Professor at Shanghai Jiao Tong University. His main research areas are natural language processing, large models, and security. His notable works include Automatic Chain of Thought (Auto-CoT), Multimodal Chain of Thought (MM-CoT), Multimodal GUI Agents (Auto-GUI), and Large Model Agent Security Evaluation (R-Judge). He has published over 70 papers in top-tier journals and conferences such as TPAMI, ICLR, ICML, ACL, and AAAI, with more than 30 as the first or corresponding author. His work has been cited over 4300 times on Google Scholar, and his open-source contributions have garnered over 10,000 stars on GitHub. He received the Young Outstanding Paper Award at the 2024 World Artificial Intelligence Conference, and three of his papers were selected by Paper Digest as the most influential papers at ICLR and AAAI. He has delivered tutorials on cutting-edge technologies at international conferences such as IJCAI, COLING, and CVPR. He was awarded the 2024 World Artificial Intelligence Conference Yunfan Award as a Brilliant Star, the 22023 Annual Doctoral Dissertation Incentive Program by the Chinese Information Processing Society of China, and was named one of the Top 100 AI Chinese Rising Stars in 2021.

Knowledge Mechanisms, Integration, and Editing in Large Models: Ningyu Zhang

Speaker: Ningyu Zhang
Title: Knowledge Mechanisms, Integration, and Editing in Large Models
Time: July 25, 2024, 15:00-16:30
Abstract: Mastering knowledge has always been a core pursuit in the development of AI systems. In this regard, large language models have demonstrated immense potential, acquiring and applying a broad spectrum of knowledge to a certain extent. However, our understanding of how large language models inherently acquire and store knowledge remains limited, and we are unable to promptly correct errors and harmful knowledge within these models. In this talk, I will explore the knowledge mechanisms of large language models and introduce cutting-edge methods for knowledge integration and editing within these models.
Biography: Ningyu Zhang is an Associate Professor at Zhejiang University and an Outstanding Young Scholar at the Qi Zhen Academy of Zhejiang University. He has published numerous papers in high-impact international academic journals and conferences, with six papers selected as high-impact papers by Paper Digest and one featured in a Nature sub-journal. He has led multiple projects funded by the National Natural Science Foundation, the Computer Society, and the Artificial Intelligence Society, and has received the Zhejiang Provincial Science and Technology Progress Award (Second Prize). He has won the Best Paper/Best Paper Nomination at IJCKG twice and the Best Paper Award at CCKS once. He serves as an area chair for ACL and EMNLP, an Action Editor for ARR, and a senior program committee member for IJCAI. He leads the development of the large language model knowledge editing tool EasyEdit (1.5k stars on GitHub).

Towards Scalable Large Model Auto-Alignment: Hongyu Lin, Bowen Yu, Yaojie Lu

Speakers: Hongyu Lin, Bowen Yu, Yaojie Lu
Title: Towards Scalable Large Model Auto-Alignment
Time: July 25, 2024, 16:30-18:00
Abstract: Aligning large language models (LLMs) with human needs is crucial in their development process. With the rapid advancement of large model technologies, models have increasingly approached or even surpassed human capabilities in many respects. In this scenario, traditional alignment methods based on manual annotation are becoming inadequate to meet the scalability requirements. Therefore, there is an urgent need to explore new sources of automatic alignment signals and technological approaches. In this report, we systematically review recent developments in automatic alignment methods for LLMs, aiming to discuss how effective and scalable automatic alignment can be achieved after LLMs surpass human capabilities. Specifically, we categorize existing automatic alignment methods into four major classes based on the sources of alignment signals and discuss the current status and potential developments of each class. Additionally, we explore the fundamental mechanisms for achieving automatic alignment and discuss key factors that make automatic alignment technologies feasible and effective in the construction of large models.
Biographies: Hongyu Lin is an Associate Researcher at the Chinese Academy of Sciences Institute of Software, affiliated with the Chinese Information Processing Laboratory. He obtained his PhD from the Institute of Software, Chinese Academy of Sciences in 2020. His primary research focuses on large language models, information extraction, and knowledge-based natural language understanding, with a particular interest in model alignment and knowledge mechanisms. Over the past years, he has published more than 60 papers in top international journals and conferences in natural language processing and artificial intelligence. He has led and participated in numerous national and departmental-level projects, including grants from the National Natural Science Foundation of China, and strategic pilot A-class projects of the Chinese Academy of Sciences. He has also collaborated on several corporate projects, including the CCF-Baidu Pinecone Fund and Tencent WeChat Rhino Bird Fund. He has received awards such as the Special Prize of the President of the Chinese Academy of Sciences, the Outstanding Contribution Award of the Chinese Information Processing Society of China, and the Qian Weichang Chinese Information Processing Science and Technology Award, First Prize. Bowen Yu is the Responsible Person for Aligning Thousand Questions at Alibaba Tongyi. He graduated with a PhD from the Institute of Information Engineering, Chinese Academy of Sciences in 2022. He has published over 50 papers in conferences such as ICML, ICLR, and ACL, cited over 2000 times, and served as an Area Chair for conferences like ACL and EMNLP. He led the development of the Qwen series Chat models and has consistently ranked in the top 10 globally in authoritative evaluations like LMSYS Chatbot Arena.
Yaojie Lu is an Assistant Researcher at the Chinese Academy of Sciences Institute of Software Chinese Information Processing Laboratory. He obtained his PhD from the University of Chinese Academy of Sciences in 2022. His primary research interests include large language models and information extraction. He is a member of the Youth Working Committee of the Chinese Information Processing Society of China and serves as a reviewer for important international conferences such as AAAI, ACL, and EMNLP. He has published over 30 papers in top international academic conferences and journals including AIJ, AAAI, ACL, EMNLP, and IJCAI. He has been awarded the Special Prize of the President of the Chinese Academy of Sciences and the Outstanding PhD Thesis Award of the Chinese Information Processing Society of China.