LLMs suffer from various limitations, such as hallucination, lack of robust reasoning, and difficult to control their outputs. Our work includes multi-perspective LLM self-reflection to enhance QA accuracy, addressing the order sensitivity of in-context learning, addressing the length bias of the Direct Preference Optimisation (DPO) algorithm, task embedding learning, prompt optimisation, and encouraging monosemanticity of LLM neurons.

Hanqi Yan, Xinyu Wang, Yanzheng Xiang, Junru Lu, Lin Gui, Yulan He

• A. Suzuki, Y. He, F. Tian and Z. Wang. Hallucinations are inevitable but statistically negligible , arXiv:2502.12187, 2025.
• Z. Shen, H. Yan, L. Zhang, Z. Hu, Y. Du and Y. He. CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation , arXiv:2502.21074, 2025.
• L. Zhang, J. Wu, D. Zhou and Y. He. PROPER: A Progressive Learning Framework for Personalized Large Language Models with Group-Level Adaptation , arXiv:2503.01303, 2025.
• J. Lu, J. Li, G. Shen, L. Gui, S. An, Y. He, D. Yin and X. Sun. RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following , arXiv:2502.11387, 2025.
• S. Qi, R. Cao, Y. He and Z. Yuan. Evaluating LLMs' Assessment of Mixed-Context Hallucination Through the Lens of Summarization , arXiv:2503.01670, 2025.
• J. Wu, W. Yin, Y. Jiang, Z. Wang, Z. Xi, R. Fang, L. Zhang, Y. He, D. Zhou, P. Xie and F. Huang. WebWalker: Benchmarking LLMs in Web Traversal , arXiv:2501.07572, 2025.
• Z. Hu, H. Yan, Q. Zhu, Z. Shen, Y. He and L. Gui. Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering , arXiv:2503.01606, 2025.
• S. Liang, L. Zhang, H. Zhu, W. Wang, Y. He and D. Zhou. RGAR: Recurrence Generation-augmented Retrieval for Factual-aware Medical Question Answering , arXiv:2502.13361, 2025.
• X. Wang, Y. Xiang, L. Gui and Y. He. GARLIC: LLM-Guided Dynamic Progress Control with Hierarchical Weighted Graph for Long Document QA , arXiv:2410.04790, 2024.
• J. Lu, S. An, M. Zhang, Y. He, D. Yin and X. Sun. FIPO: Free-Form Instruction-Oriented Prompt Optimization with Preference Dataset and Modular Fine-Tuning Schema , The 31st International Conference on Computational Linguistics (COLING), 2025.
• C. Zhang, L. Zhang, J. Wu, Y. He and D. Zhou. Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment , The 39th Annual AAAI Conference on Artificial Intelligence (AAAI), 2025.
• H. Yan, Y. Xiang, G. Chen, Y. Wang, L. Gui and Y. He. Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective , The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
• J. Lu, J. Li, S. An, M. Zhao, Y. He, D. Yin and X. Sun. Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence , The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
• Y. Zhou, J. Li, Y. Xiang, H. Yan, L. Gui and Y. He. The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis , The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024.
• H. Yan, Q. Zhu, X. Wang, L. Gui and Y. He. Mirror: A Multiple-Perspective Self-Reflection Method for Knowledge-Rich Reasoning , The 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024.
• X. Wang, H. Xu, L. Gui and Y. He. Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond , Findings of ACL, 2024.
• Y. Xiang, H. Yan, L. Gui and Y. He. Addressing Order Sensitivity of In-Context Demonstration Examples in Causal Language Models , Findings of ACL, 2024.
• S. Qi, Y. He and Z. Yuan. Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation , arXiv:2404.12041, 2024.
• H. Zhang, L. Gui, Y. Lei, Y. Zhai, Y. Zhang, Y. He, H. Wang, Y. Yu, K.F. Wong, B. Liang, R. Xu. COPR: Continual Human Preference Learning via Optimal Policy Regularization , arXiv:2402.14228, 2024.
• H. Zhang, Y. Lei, L. Gui, M. Yang, Y. He, H. Wang, R. Xu. CPPO: Continual Learning for Reinforcement Learning with Human Feedback , The 12th International Conference on Learning Representations (ICLR), 2024.
• H. Yan, L. Gui, M. Wang, K. Zhang and Y. He. Explainable Recommender with Geometric Information Bottleneck , IEEE Transactions on Knowledge and Data Engineering, to appear.
• J. Li, L. Gui, Y. Zhou, D. West, C. Aloisi and Y. He. Exploring Explainable Automated Student Answer Assessment with ChatGPT , Findings of EMNLP, 2023.
• H. Yan, L. Kong, L. Gui, Y. Chi, E. Xing, Y. He and K. Zhang. Counterfactual Generation with Identifiability Guarantees , The 37th Annual Conference on Neural Information Processing Systems (NeurIPS), 2023.
• J. Li, Z. Sun, B. Liang, L. Gui and Y. He. CUE: An Uncertainty Interpretation Framework for Text Classifiers Built on Pre-Trained Language Models , The 39th Conference on Uncertainty in Artificial Intelligence (UAI), Pittsburgh, PA, USA, Aug. 2023.
• J. Li, R. Zhao, Y. He and L. Gui. OverPrompt: Enhancing ChatGPT Capabilities through an Efficient In-Context Learning Approach , arXiv:2305.14973.
• H. Yan, L. Gui, M. Wang, K. Zhang and Y. He. Explainable Recommender with Geometric Information Bottleneck , arXiv:2305.05331.
• H. Li, H. Yan, Y. Li, L. Qian, Y. He and L. Gui. Distinguishability Calibration to In-Context Learning , Findings of EACL, 2023.
• H. Yan, L. Gui and Y. He. Hierarchical Interpretation of Neural Text Classification , Computational Linguistics, to appear.
• H. Yan, L. Gui, W. Li and Y. He. Addressing Token Uniformity in Transformers via Singular Value Transformation , The 38th Conference on Uncertainty in Artificial Intelligence (UAI), Eindhoven, Netherlands, Aug. 2022.