Publications
Efficient Long Sequence Modeling via State Space Augmented Transformer [code]
Simiao Zuo*, Xiaodong Liu*, Jian Jiao, Denis Charles, Eren Manavoglu, Tuo Zhao and Jianfeng Gao
arxiv, 2022
DiP-GNN: Discriminative Pre-Training of Graph Neural Networks
Simiao Zuo, Haoming Jiang, Qingyu Yin, Xianfeng Tang, Bing Yin and Tuo Zhao
arxiv, 2022
Differentially Private Estimation of Hawkes Process
Simiao Zuo, Tianyi Liu, Tuo Zhao and Hongyuan Zha
arxiv, 2022
DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries
Simiao Zuo, Pengfei Tang, Xinyu Hu, Qiang Lou, Jian Jiao and Denis Charles
Conference on Information and Knowledge Management (CIKM), 2023
Context-Aware Query Rewriting for Improving Users’ Search Experience on E-commerce Websites
Simiao Zuo, Qingyu Yin, Haoming Jiang, Shaohui Xi, Bing Yin, Chao Zhang and Tuo Zhao
Association for Computational Linguistics, Industry Track (ACL), 2023
Machine Learning Force Fields with Data Cost Aware Training [code]
Alexander Bukharin, Tianyi Liu, Shengjie Wang, Simiao Zuo, Weihao Gao, Wen Yan and Tuo Zhao
International Conference on Machine Learning (ICML), 2023
SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process [code]
Zichong Li, Yanbo Xu, Simiao Zuo, Haoming Jiang, Chao Zhang, Tuo Zhao and Hongyuan Zha
International Conference on Machine Learning (ICML), 2023
Less is More: Task-aware Layer-wise Distillation for Language Model Compression [code]
Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen and Tuo Zhao
International Conference on Machine Learning (ICML), 2023
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance [code]
Qingru Zhang, Simiao Zuo, Chen Liang, Alexander Bukharin, Pengcheng He, Weizhu Chen and Tuo Zhao
International Conference on Machine Learning (ICML), 2022
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation [code]
Simiao Zuo, Qingru Zhang, Chen Liang, Pengcheng He, Tuo Zhao and Weizhu Chen
North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Self-Training with Differentiable Teacher
Simiao Zuo*, Yue Yu*, Chen Liang, Haoming Jiang, Siawpeng Er, Chao Zhang, Tuo Zhao and Hongyuan Zha
Findings of North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Adversarially Regularized Policy Learning Guided by Trajectory Optimization
Zhigen Zhao, Simiao Zuo, Tuo Zhao and Ye Zhao
Annual Learning for Dynamics & Control Conference (L4DC), 2022
No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models [code]
Chen Liang, Haoming Jiang, Simiao Zuo, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen and Tuo Zhao
International Conference on Learning Representations (ICLR), 2022
Taming Sparsely Activated Transformer with Stochastic Experts [code]
Simiao Zuo, Xiaodong Liu, Jian Jiao, Young Jin Kim, Hany Hassan, Ruofei Zhang, Tuo Zhao and Jianfeng Gao
International Conference on Learning Representations (ICLR), 2022
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach [code]
Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen and Tuo Zhao
Empirical Methods in Natural Language Processing (EMNLP), 2021
ARCH: Efficient Adversarial Regularized Training with Caching [code]
Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen and Tuo Zhao
Findings of Empirical Methods in Natural Language Processing (EMNLP), 2021
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization [code]
Chen Liang, Simiao Zuo, Minshuo Chen, Haoming Jiang, Xiaodong Liu, Pengcheng He, Tuo Zhao and Weizhu Chen
Association for Computational Linguistics (ACL), 2021
Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach [code]
Yue Yu*, Simiao Zuo*, Haoming Jiang, Wendi Ren, Tuo Zhao and Chao Zhang
North American Chapter of the Association for Computational Linguistics (NAACL), 2021
A Hypergradient Approach to Robust Regression without Correspondence
Yujia Xie*, Yixiu Mao*, Simiao Zuo, Hongteng Xu, Xiaojie Ye, Tuo Zhao and Hongyuan Zha
International Conference on Learning Representations (ICLR), 2021
Transformer Hawkes Process [code]
Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao and Hongyuan Zha
International Conference on Machine Learning (ICML), 2020
Tensor maps for synchronizing heterogeneous shape collections
Qixing Huang, Zhenxiao Liang, Haoyun Wang, Simiao Zuo and Chandrajit Bajaj
ACM Transactions on Graphics (TOG), 2019