Lijun Zhang's Publications

Stochastic Optimization | Convex Optimization | Online Learning | Randomized Algorithm | Compressive Sensing

Clustering | Active Learning | Dimensionality Reduction | LAMDA Publications | Home

Stochastic Optimization

  1. Stochastic Optimization for Non-convex Inf-Projection Problems [PDF, Supplementary, Bibtex]
    Y. Yan, Y. Xu, L. Zhang, X. Wang, and T. Yang
    In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), pages 10660 - 10669, 2020.

  2. VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning [PDF, Bibtex]
    F. Shang, K. Zhou, H. Liu, J. Cheng, I. W. Tsang, L. Zhang, D. Tao, and L. Jiao
    IEEE Transactions on Knowledge and Data Engineering (TKDE), 32(1): 188 - 202, 2020.

  3. Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the O(1/T) Convergence Rate [PDF, Bibtex]
    L. Zhang, and Z.-H. Zhou
    In Proceedings of the 32nd Annual Conference on Learning Theory (COLT 2019), pages 3160 - 3179, 2019

  4. $\ell_1$-regression with Heavy-tailed Distributions [PDF, Bibtex, arXiv]
    L. Zhang, and Z.-H. Zhou
    In Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pages 1076 - 1086, 2018.

  5. Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions [PDF, Supplementary, Bibtex]
    M. Liu, X. Zhang, L. Zhang, R. Jin, and T. Yang
    In Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pages 4678 - 4689, 2018.

  6. A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer [PDF, Bibtex]
    T. Yang, Z. Li, and L. Zhang
    In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018), pages 445 - 453, 2018.

  7. Empirical Risk Minimization for Stochastic Convex Optimization: O(1/n)- and O(1/n^2)-type of Risk Bounds [PDF, Bibtex]
    L. Zhang, T. Yang, and R. Jin
    In Proceedings of the 30th Annual Conference on Learning Theory (COLT 2017), pages 1954 - 1979, 2017.

  8. Sparse Learning with Stochastic Composite Optimization [PDF, Bibtex]
    W. Zhang, L. Zhang, Z. Jin, R. Jin, D. Cai, X. Li, R. Liang, and X. He
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 39(6): 1223 - 1236, 2017.

  9. Efficient Stochastic Optimization for Low-Rank Distance Metric Learning [PDF, Supplementary, Bibtex]
    J. Zhang*, and L. Zhang
    In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), pages 933 - 939, 2017.

  10. A Two-stage Approach for Learning a Sparse Model with Sharp Excess Risk Analysis [PDF, Bibtex]
    Z. Li, T. Yang, L. Zhang, and R. Jin
    In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), pages 2224 - 2230, 2017.

  11. Optimal Stochastic Strongly Convex Optimization with a Logarithmic Number of Projections [PDF, Supplementary, Bibtex]
    J. Chen, T. Yang, Q. Lin, L. Zhang, and Y. Chang
    In Proceedings of the 32nd Conference on Uncertainty in Artificial Intelligence (UAI 2016), pages 122 - 131, 2016.

  12. Stochastic Optimization for Kernel PCA [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, J. Yi, R. Jin, and Z.-H. Zhou
    In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016), pages 2316 - 2322, 2016.

  13. Lower and Upper Bounds on the Generalization of Stochastic Exponentially Concave Optimization [PDF, Errata, Bibtex]
    M. Mahdavi, L. Zhang, and R. Jin
    In Proceedings of the 28th Conference on Learning Theory (COLT 2015), pages 1305 - 1320, 2015.

  14. Efficient Distance Metric Learning by Adaptive Sampling and Mini-Batch Stochastic Gradient Descent (SGD) [PDF, Bibtex]
    Q. Qian, R. Jin, J. Yi, L. Zhang, and S. Zhu
    Machine Learning, 99(3): 353 - 372, 2015.

  15. Sparse Learning for Stochastic Composite Optimization [PDF, Bibtex]
    W. Zhang, L. Zhang, Y. Hu, R. Jin, D. Cai, and X. He
    In Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI 2014), pages 893 - 899, 2014.

  16. Linear Convergence with Condition Number Independent Access of Full Gradients [PDF, Bibtex]
    L. Zhang, M. Mahdavi, and R. Jin
    In Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 980 - 988, 2013.

  17. Mixed Optimization for Smooth Functions [PDF, Supplementary, Bibtex]
    M. Mahdavi, L. Zhang, and R. Jin
    In Advances in Neural Information Processing Systems 26 (NIPS 2013), pages 674 - 682, 2013.

  18. O(logT) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, R. Jin, and X. He
    In Proceedings of the 30th International Conference on Machine Learning (ICML 2013), pages 1121 - 1129, 2013.

Convex Optimization

  1. A Richer Theory of Convex Constrained Optimization with Reduced Projections and Improved Rates [PDF, Bibtex, Full Version]
    T. Yang, Q. Lin, and L. Zhang
    In Proceedings of the 34th International Conference on Machine Learning (ICML 2017), pages 3901 - 3910, 2017.

  2. SVD-free Convex-Concave Approaches for Nuclear Norm Regularization [PDF, Supplementary, Bibtex]
    Y. Xiao*, Z. Li, T. Yang, and L. Zhang
    In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI 2017), pages 3126 - 3132, 2017.

Online Learning

  1. Learning with Feature Evolvable Streams
    B.-J. Hou, L. Zhang, and Z.-H. Zhou
    IEEE Transactions on Knowledge and Data Engineering (TKDE), in press, 2020.

  2. Dynamic Regret of Convex and Smooth Functions [arXiv]
    P. Zhao, Y.-J. Zhang, L. Zhang, and Z.-H. Zhou
    In Advances in Neural Information Processing Systems 33 (NeurIPS 2020), to appear, 2020.

  3. Projection-free Distributed Online Convex Optimization with $O(\sqrt{T})$ Communication Complexity [PDF, Supplementary, Bibtex]
    Y. Wan*, W.-W. Tu, and L. Zhang
    In Proceedings of the 37th International Conference on Machine Learning (ICML 2020), pages 9818 - 9828, 2020.

  4. Online Learning in Changing Environments [PDF, Bibtex]
    L. Zhang
    In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020), Early Career, pages 5178 - 5182, 2020.

  5. Nearly Optimal Regret for Stochastic Linear Bandits with Heavy-Tailed Payoffs [PDF, Bibtex, arXiv]
    B. Xue*, G. Wang, Y. Wang, and L. Zhang
    In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020), pages 2936 - 2942, 2020.

  6. Minimizing Dynamic Regret and Adaptive Regret Simultaneously [PDF, Bibtex, arXiv]
    L. Zhang, S. Lu, and T. Yang
    In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), pages 309 - 319, 2020.

  7. A Simple Approach for Non-stationary Linear Bandits [PDF, Supplementary, Bibtex]
    P. Zhao, L. Zhang, Y. Jiang, and Z.-H. Zhou
    In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), pages 746 - 755, 2020.

  8. Bandit Convex Optimization in Non-stationary Environments [PDF, Bibtex, arXiv]
    P. Zhao, G. Wang, L. Zhang, and Z.-H. Zhou
    In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS 2020), pages 1508 - 1518, 2020.

  9. SAdam: A Variant of Adam for Strongly Convex Functions [PDF, Bibtex]
    G. Wang*, S. Lu, W. Tu, and L. Zhang
    In International Conference on Learning Representations (ICLR 2020), 2020.

  10. Adapting to Smoothness: A More Universal Algorithm for Online Convex Optimization [PDF, Bibtex]
    G. Wang*, S. Lu, Y. Hu, and L. Zhang
    In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI 2020), pages 6162 - 6169, 2020.

  11. Accelerating Adaptive Online Learning by Matrix Approximation[PDF, Bibtex]
    Y. Wan*, and L. Zhang
    International Journal of Data Science and Analytics (JDSA), 9(4): 389 - 400, 2020.

  12. Multi-Objective Generalized Linear Bandits [PDF, Bibtex]
    S. Lu*, G. Wang, Y. Hu, and L. Zhang
    In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI 2019), pages 3080 - 3086, 2019.

  13. Adaptivity and Optimality: A Universal Algorithm for Online Convex Optimization [PDF, Supplementary, Bibtex]
    G. Wang*, S. Lu, and L. Zhang
    In Proceedings of 35th Conference on Uncertainty in Artificial Intelligence (UAI 2019), pages 659 - 668, 2019.

  14. Adaptive Regret of Convex and Smooth Functions [PDF, Bibtex, arXiv]
    L. Zhang, T.-Y. Liu, and Z.-H. Zhou
    In Proceedings of the 36th International Conference on Machine Learning (ICML 2019), pages 7414 - 7423, 2019.

  15. Optimal Algorithms for Lipschitz Bandits with Heavy-tailed Rewards [PDF, Supplementary, Bibtex]
    S. Lu*, G. Wang, Y. Hu, and L. Zhang
    In Proceedings of the 36th International Conference on Machine Learning (ICML 2019), pages 4154 - 4163, 2019.

  16. Adaptive Online Learning in Dynamic Environments [PDF, Bibtex, arXiv]
    L. Zhang, S. Lu, and Z.-H. Zhou
    In Advances in Neural Information Processing Systems 31 (NeurIPS 2018), pages 1323 - 1333, 2018.

  17. Dynamic Regret of Strongly Adaptive Methods [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, R. Jin, and Z.-H. Zhou
    In Proceedings of the 35th International Conference on Machine Learning (ICML 2018), pages 5877 - 5886, 2018.

  18. Minimizing Adaptive Regret with One Gradient per Iteration [PDF, Bibtex]
    G. Wang*, D. Zhao, and L. Zhang
    In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI 2018), pages 2762 - 2768, 2018.

  19. Efficient Adaptive Online Learning via Frequent Directions [PDF, Bibtex]
    Y. Wan*, N. Wei, and L. Zhang
    In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI 2018), pages 2748 - 2754, 2018.

  20. Accelerating Adaptive Online Learning by Matrix Approximation[PDF, Supplementary, Bibtex]
    Y. Wan* and L. Zhang
    In Proceedings of the 22nd Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD 2018), pages 405 - 417, 2018.

  21. Improved Dynamic Regret for Non-degenerate Functions [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, J. Yi, R. Jin, and Z.-H. Zhou
    In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 732 - 741, 2017.

  22. Learning with Feature Evolvable Streams [PDF, Supplementary, Bibtex]
    B.-J. Hou, L. Zhang, and Z.-H. Zhou
    In Advances in Neural Information Processing Systems 30 (NIPS 2017), pages 1416 - 1426, 2017.

  23. Online Stochastic Linear Optimization under One-bit Feedback [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, R. Jin, Y. Xiao, and Z.-H. Zhou
    In Proceedings of the 33rd International Conference on Machine Learning (ICML 2016), pages 392 - 401, 2016.

  24. Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient [PDF, Supplementary, Bibtex]
    T. Yang, L. Zhang, R. Jin, and J. Yi
    In Proceedings of the 33rd International Conference on Machine Learning (ICML 2016), pages 449 - 457, 2016.

  25. Online Kernel Learning with Nearly Constant Support Vectors [PDF, Bibtex]
    M. Lin, L. Zhang, R. Jin, S. Weng, and C. Zhang
    Neurocomputing, 179: 26 - 36, 2016.

  26. Online Bandit Learning for a Special Class of Non-convex Losses [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, R. Jin, and Z.-H. Zhou
    In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI 2015), pages 3158 - 3164, 2015.

  27. Online Kernel Learning with a Near Optimal Sparsity Bound [PDF, Supplementary, Bibtex]
    L. Zhang, J. Yi, R. Jin, M. Lin, and X. He
    In Proceedings of the 30th International Conference on Machine Learning (ICML 2013), pages 621 - 629, 2013.

  28. Efficient Online Learning for Large-Scale Sparse Kernel Logistic Regression [PDF, Bibtex]
    L. Zhang, R. Jin, C. Chen, J. Bu, and X. He
    In Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI 2012), pages 1219 - 1225, 2012.

Randomized Algorithm

  1. Efficient Non-oblivious Randomized Reduction for Risk Minimization with Improved Excess Risk Guarantee [PDF, Bibtex]
    Y. Xu, H. Yang, L. Zhang, and T. Yang
    In Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI 2017), pages 2796 - 2802, 2017.

  2. Sparse Learning for Large-scale and High-dimensional Data: A Randomized Convex-concave Optimization Approach [PDF, Supplementary, Bibtex, Full Version]
    L. Zhang, T. Yang, R. Jin, and Z.-H. Zhou
    In Proceedings of the 27th International Conference on Algorithmic Learning Theory (ALT 2016), pages 83 - 97, 2016.

  3. Accelerated Sparse Linear Regression via Random Projection [PDF, Bibtex]
    W. Zhang, L. Zhang, R. Jin, D. Cai, and X. He
    In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016), pages 2337 - 2343, 2016.

  4. Theory of Dual-Sparse Regularized Randomized Reduction [PDF, Bibtex]
    T. Yang, L. Zhang, R. Jin, and S. Zhu
    In Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), pages 305 - 314, 2015.

  5. Random Projections for Classification: A Recovery Approach [PDF, Bibtex]
    L. Zhang, M. Mahdavi, R. Jin, T. Yang, and S. Zhu
    IEEE Transactions on Information Theory (TIT), 60(11): 7300 - 7316, 2014.

  6. Recovering the Optimal Solution by Dual Random Projection [PDF, Bibtex, Journal Version]
    L. Zhang, M. Mahdavi, R. Jin, T. Yang, and S. Zhu
    In Proceedings of the 26th Conference on Learning Theory (COLT 2013), pages 135 - 157, 2013.

Compressive Sensing and Matrix Completion

  1. Relative Error Bound Analysis for Nuclear Norm Regularized Matrix Completion [PDF, Bibtex]
    L. Zhang, T. Yang, R. Jin, and Z.-H. Zhou
    Journal of Machine Learning Research (JMLR), 20(97):1 - 22, 2019.

  2. A Simple Homotopy Algorithm for Compressive Sensing [PDF, Supplementary, Bibtex]
    L. Zhang, T. Yang, R. Jin, and Z.-H. Zhou
    In Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS 2015), pages 1116 - 1124, 2015.

  3. Efficient Algorithms for Robust One-bit Compressive Sensing [PDF, Supplementary, Bibtex]
    L. Zhang, J. Yi, and R. Jin
    In Proceedings of the 31st International Conference on Machine Learning (ICML 2014), pages 820 - 828, 2014.

Clustering

  1. Non-redundant Multiple Clustering by Nonnegative Matrix Factorization [PDF, Bibtex]
    S. Yang*, and L. Zhang
    Machine Learning, 106(5): 695 - 712, 2017.

  2. An Efficient Semi-Supervised Clustering Algorithm with Sequential Constraints [PDF, Bibtex]
    J. Yi, L. Zhang, T. Yang, W. Liu, and J. Wang
    In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2015), pages 1405 - 1414, 2015.

  3. A Single-Pass Algorithm for Efficiently Recovering Sparse Cluster Centers of High-dimensional Data [PDF, Supplementary, Bibtex]
    J. Yi, L. Zhang, J. Wang, R. Jin, and A. Jain
    In Proceedings of the 31st International Conference on Machine Learning (ICML 2014), pages 658 - 666, 2014.

  4. Semi-supervised Clustering by Input Pattern Assisted Pairwise Similarity Matrix Completion [PDF, Bibtex]
    J. Yi, L. Zhang, R. Jin, Q. Qian, and A. Jain
    In Proceedings of the 30th International Conference on Machine Learning (ICML 2013), pages 1400 - 1408, 2013.

  5. Locally Discriminative Coclustering [PDF, Bibtex]
    L. Zhang, C. Chen, J. Bu, Z. Chen, D. Cai, and J. Han
    IEEE Transactions on Knowledge and Data Engineering (TKDE), 24(6): 1025 - 1035, 2012.

Active Learning

  1. A Unified Feature and Instance Selection Framework Using Optimum Experimental Design [PDF, Bibtex]
    L. Zhang, C. Chen, J. Bu, and X. He
    IEEE Transactions on Image Processing (TIP), 21(5): 2379 - 2388, 2012.

  2. Active Learning Based on Locally Linear Reconstruction [PDF, Appendix, Bibtex]
    L. Zhang, C. Chen, J. Bu, D. Cai, X. He, and T. Huang
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 33(10): 2026 - 2038, 2011.

  3. G-Optimal Design with Laplacian Regularization [PDF, Bibtex]
    C. Chen, Z. Chen, J. Bu, C. Wang, L. Zhang, and C. Zhang
    In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI 2010), pages 413 - 418, 2010.

  4. Convex Experimental Design Using Manifold Structure for Image Retrieval [PDF, Bibtex]
    L. Zhang, C. Chen, W. Chen, J. Bu, D. Cai, and X. He
    In Proceedings of the 17th ACM International Conference on Multimedia (ACM Multimedia 2009), pages 45 - 53, 2009.

Dimensionality Reduction

  1. A-Optimal Projection for Image Representation [PDF, Appendix, Bibtex]
    X. He, C. Zhang, L. Zhang, and X. Li
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 38(5): 1009 - 1015, 2016.

  2. Graph Regularized Feature Selection with Data Reconstruction [PDF, Bibtex]
    Z. Zhao, X. He, D. Cai, L. Zhang, W. Ng, and Y. Zhuang
    IEEE Transactions on Knowledge and Data Engineering (TKDE), 28(3): 689 - 700, 2016.

  3. Locally Regressive Projections [PDF, Bibtex]
    L. Zhang
    International Journal of Software and Informatics (IJSI), 7(3): 435 - 451, 2013.

  4. Graph Regularized Sparse Coding for Image Representation [PDF, Bibtex]
    M. Zheng, J. Bu, C. Chen, C. Wang, L. Zhang, G. Qiu, and D. Cai
    IEEE Transactions on Image Processing (TIP), 20(5): 1327 - 1336, 2011.

  5. Robust Non-negative Matrix Factorization [PDF, Bibtex]
    L. Zhang, Z. Chen, M. Zheng, and X. He
    Frontiers of Electrical and Electronic Engineering in China, 6(2): 192 - 200, 2011.

  6. Discriminative Codeword Selection for Image Representation [PDF, Bibtex]
    L. Zhang, C. Chen, J. Bu, Z. Chen, S. Tan, and X. He
    In Proceedings of the 18th ACM International Conference on Multimedia (ACM Multimedia 2010), pages 173 - 182, 2010.

  7. Constrained Laplacian Eigenmap for Dimensionality Reduction [PDF, Bibtex]
    C. Chen, L. Zhang, J. Bu, C. Wang, and W. Chen
    Neurocomputing, 73(4-6): 951 - 958, 2010.