Dynamics of Neural Networks

These papers are concerned with the optimization trajectory of neural networks. We develop new methods for optimizing complex neural networks that differ from naive stochastic gradient descent. We also exploit the optimization trajectory to provide interpretation of the neural network's predictions.

• Xu Guo, Boyang Li, Han Yu, and Chunyan Miao. Latent-Optimized Adversarial Neural Transfer for Sarcasm Detection. The Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT) . 2021.
• Yuanyuan Chen, Boyang Li, Han Yu, Pengcheng Wu, and Chunyan Miao. HyDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks. The AAAI Conference on Artificial Intelligence (AAAI). 2021. [Code]
• Jianan Wang, Boyang Li, Xiangyu Fan, Jing Lin, and Yanwei Fu. Data-efficient Alignment of Multimodal Sequences by Aligning Gradient Updates and Internal Feature Distributions. The IEEE Winter Conference on Applications of Computer Vision (WACV). 2021. [Supplemental Material]

Small-data and Efficient Learning

These papers are concerned with learning from a small amount of training data. While excellent performance has been derived from scaling up training data and network sizes, training data of many tasks remain difficult to acquire. Thus, it is important to explore how to learn in the small-data / few-shot / zero-shot regimes, especially with the help of large networks.

• Anthony Meng Huat Tiong, Junnan Li, Boyang Li, Silvio Savarese, and Steven C.H. Hoi. Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training. Findings of the Conference on Empirical Methods in Natural Language Processing (Findings of EMNLP). 2022.
• Xu Guo, Boyang Li, and Han Yu. Improving the Sample Efficiency of Prompt Tuning with Domain Adaptation. Findings of the Conference on Empirical Methods in Natural Language Processing (Findings of EMNLP). 2022.
• Jun Chen, Han Guo, Kai Yi, Boyang Li, and Mohamed Elhoseiny. VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning. The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022.

Interpretability

• Yuanyuan Chen, Boyang Li, Han Yu, Pengcheng Wu, and Chunyan Miao. Hydra: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks. The AAAI Conference on Artificial Intelligence (AAAI). 2021. [Code]
• Adam Noack, Isaac Ahern, Dejing Dou, and Boyang Li. An Empirical Study on the Relation between Network Interpretability and Adversarial Robustness. Springer Nature Computer Science, 2(32). 2021. [Code]
• Isaac Ahern, Adam Noack, Luis Guzman-Nateras, Dejing Dou, Boyang Li, Jun Huan. NormLime: A New Feature Importance Metric for Explaining Deep Neural Networks. arXiv:1909.04200

Data-driven Initialization

• Yinan Zhang, Boyang Li, Yong Liu, Hao Wang, Chunyan Miao. Initialization Matters: Regularizing Manifold-informed Initialization for Neural Recommendation Systems. ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD). 2021. [Code]

Learning from Synthetic Data

• Zilin Du, Yunxin Li, Xu Guo, Yidan Sun, and Boyang Li. Training Multimedia Event Extraction With Generated Images and Captions. The ACM International Conference on Multimedia (ACM MM). 2023.

Debiasing Machine Learning

• Haoxin Li, Yuan Liu, Hanwang Zhang, and Boyang Li. Mitigating and Evaluating Static Bias of Action Representations in the Background and the Foreground. International Conference on Computer Vision (ICCV) (Oral Presentation). 2023.