picture of Simon Lacoste-Julien

Simon Lacoste-Julien

Associate Professor and Canada CIFAR AI (CCAI) chair holder
Department of Computer Science and Operations Research (DIRO)
and Mila – Quebec Institute of Artificial Intelligence
Université de Montréal

VP Lab Director - SAIT AI Lab Montreal (SAIL Montreal)

Mila office:
6666 rue Saint-Urbain (map)
Montréal (QC) Canada
office: D-11 (2nd floor)

To send me a message, build my address by first using firstname.lastname (as written on my webpage -- don't forget the hyphen between the two last names!), and then use as server umontreaDOTca.

[Version française]

Prospective students: *do not email* me directly about doing a PhD or internship with me. Unfortunately, I receive hundreds of requests like these and cannot reply individually. If you are interested in doing a PhD, Master or internship with me, please apply first through the Mila admission website and put in your selection there that you are interested in working with me. If your application is succesful, I will contact you.

News:
I just started heading part-time a new machine learning research lab, SAIT AI Lab (SAIL) Montreal, from Samsung Advanced Institute of Technology (SAIT) located in Mila's corporate labs and am hiring research scientists. See the job posting for more info. [May 1st inauguration video]

NeurIPS 2018 Workshop on Smooth Games Optimization and Machine Learning.

Update August 2016: I just moved to the Université de Montréal.

(Until August 2016) I was a researcher at INRIA in the SIERRA project team which is part of the Computer Science Department of École Normale Supérieure in Paris.

I did my PhD in Computer Science at the University of California, Berkeley under the supervision of Michael I. Jordan, and (basically) a B.Sc. Triple Honours in Mathematics, Physics and Computer Science at McGill University. I then worked with Zoubin Ghahramani as a postdoc in the Machine Learning Group of the University of Cambridge. In September 2011, I got a Research in Paris fellowship to work with Francis Bach in the SIERRA project team, and then I joined as a researcher in September 2013.

CV (Jan 2019) | Google Scholar citation profile

Research Interests

Students and Postdocs

Alumni

Teaching

Grad classes at Université de Montréal: Previously:

Papers

Technical Reports

  1. [new!] GEAR: Geometry-Aware Rényi Information, J. Gallego, A. Vani, M. Schwarzer, S. Lacoste-Julien arXiv:1906.08325 [cs.LG], Jun 2019.
  2. [new!] A Tight and Unified Analysis of Extragradient for a Whole Spectrum of Differentiable Games, W. Azizian, I. Mitliagkas, S. Lacoste-Julien, G. Gidel arXiv:1906.05945 [cs.LG], Jun 2019.
  3. [new!] A Closer Look at the Optimization Landscapes of Generative Adversarial Networks, H. Berard, G. Gidel, A. Almahairi, P. Vincent, S. Lacoste-Julien arXiv:1906.04848 [cs.LG], Jun 2019.
  4. [new!] Gradient-Based Neural DAG Learning, S. Lachapelle, P. Brouillard, T. Deleu, S. Lacoste-Julien arXiv:1906.02226 [cs.LG], Jun 2019.
  5. [new!] Centroid Networks for Few-Shot Clustering and Unsupervised Few-Shot Classification, G. Huang, H. Larochelle, S. Lacoste-Julien arXiv:1902.08605 [cs.LG], Feb 2019.
  6. [new!] A Modern Take on the Bias-Variance Tradeoff in Neural Networks, B. Neal, S. Mittal, A. Baratin, V. Tantia, M. Scicluna, S. Lacoste-Julien, I. Mitliagkas arXiv:1810.08591 [cs.LG], Oct 2018.
  7. [new!] Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information, E. Larsen, S. Lachapelle, Y. Bengio, E. Frejinger, S. Lacoste-Julien, A. Lodi arXiv:1807.11876 [cs.LG], Jul 2018.
  8. [new!] Parametric Adversarial Divergences are Good Task Losses for Generative Modeling, G. Huang, H. Berard, A. Touati, G. Gidel, P. Vincent and S. Lacoste-Julien, arXiv:1708.02511 [cs.LG], August 2017.
  9. Convergence Rate of Frank-Wolfe for Non-Convex Objectives, S. Lacoste-Julien, arXiv:1607.00345 [math.OC], June 2016.
  10. An Affine Invariant Linear Convergence Analysis for Frank-Wolfe Algorithms, S. Lacoste-Julien and M. Jaggi, appeared at the NIPS 2013 Workshop on Greedy Algorithms, Frank-Wolfe and Friends, arXiv:1312.7864 [math.OC], December 2013.
  11. A Simpler Approach to Obtaining an O(1/t) Convergence Rate for the Projected Stochastic Subgradient Method, S. Lacoste-Julien, M. Schmidt and F. Bach, arXiv:1212.2002v2 [cs.LG], December 2012.
  12. Approximate Gaussian Integration using Expectation Propagation, J.P. Cunningham, P. Hennig and S. Lacoste-Julien, arXiv:11111.6832v1 [stat.ML], November 2011.
  13. A Kernel Approach to Tractable Bayesian Nonparametrics., F. Huszár and S. Lacoste-Julien, arXiv:1103.1761v3, [stat.ML], March 2011.
  14. Discriminative Machine Learning with Structure, S. Lacoste-Julien, PhD Thesis, University of California, Berkeley, 2009.
  15. A UC Berkeley class project which has been cited a few times as a tutorial: An introduction to Max-Margin Markov Networks. S. Lacoste-Julien, 2003.

Published Papers

  1. [new!] Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates, S. Vaswani, A. Mishkin, I. Laradji, M. Schmidt, G. Gidel, S. Lacoste-Julien arXiv:1905.09997 [cs.LG], to appear at Neural Information Processing Systems Conference (NeurIPS 2019), Vancouver, Canada, December 2019.
  2. [new!] Implicit Regularization of Discrete Gradient Dynamics in Deep Linear Neural Networks, G. Gidel, F. Bach, S. Lacoste-Julien arXiv:1904.13262 [cs.LG], to appear at Neural Information Processing Systems Conference (NeurIPS 2019), Vancouver, Canada, December 2019.
  3. [new!] Reducing Noise in GAN Training with Variance Reduced Extragradient, T. Chavdarova, G. Gidel, F. Fleuret, S. Lacoste-Julien arXiv:1904.08598 [stat.ML], to appear at Neural Information Processing Systems Conference (NeurIPS 2019), Vancouver, Canada, December 2019.
  4. [new!] A Variational Inequality Perspective on Generative Adversarial Networks, G. Gidel, H. Berard, P. Vincent and S. Lacoste-Julien, arXiv:1802.10551, International Conference on Learning Representations (ICLR 2019), New Orleans, USA, May 2019. [open reviews]
  5. [new!] Negative Momentum for Improved Game Dynamics, G. Gidel*, R. Askari*, M. Pezeshki, R. Le Priol, G. Huang, S. Lacoste-Julien and I. Mitliagkas, arXiv:1807.04740, International Conference on Artificial Intelligence and Statistics (AISTATS 2019), Naha, Okinawa, Japan, April 2019.
  6. [new!] Improved Asynchronous Parallel Optimization Analysis for Stochastic Incremental Methods, R. Leblond, F. Pedregosa, and S. Lacoste-Julien, Journal of Machine Learning Research (JMLR), 19(81): 1--68, December 2018.
  7. [new!] Quantifying Learning Guarantees for Convex but Inconsistent Surrogates, K. Struminsky, S. Lacoste-Julien and A. Osokin, Neural Information Processing Systems Conference (NIPS18), Montreal, Canada, December 2018.
  8. [new!] Learning from Narrated Instruction Videos, J.-B. Alayrac, P. Bojanowski, N. Agrawal, I. Laptev, J. Sivic and S. Lacoste-Julien, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 40(1), 2194 - 2208, September 2018. [project website]
  9. [new!] Scattering Networks for Hybrid Representation Learning, E. Oyallon, S. Zagoruyko, G. Huang, N. Komodakis, S. Lacoste-Julien and M. Blaschko, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), July 2018.
  10. [new!] Adaptive Stochastic Dual Coordinate Ascent for Conditional Random Fields, R. Le Priol, A. Piché and S. Lacoste-Julien, Uncertainty in Artificial Intelligence (UAI 2018), Monterey, CA, USA, August 2018. [project website]
  11. [new!] SEARNN: Training RNNs with Global-Local Losses, R. Leblond*, J.-B. Alayrac*, A. Osokin and S. Lacoste-Julien, International Conference on Learning Representations (ICLR 2018), Vancouver, Canada, April 2018. [open reviews] [project website] *Both authors contributed equally
  12. [new!] Frank-Wolfe Splitting via Augmented Lagrangian Method, G. Gidel, F. Pedregosa and S. Lacoste-Julien, International Conference on Artificial Intelligence and Statistics (AISTATS 2018), Lanzarote, Canary Islands, Spain, April 2018. (AISTATS oral!)
  13. On Structured Prediction Theory with Calibrated Convex Surrogate Losses, A. Osokin, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS17), Long Beach, USA, December 2017. [code] (NIPS oral!)
  14. Breaking the Nonsmooth Barrier: a Scalable Parallel Method for Composite Optimization, F. Pedregosa, R. Leblond and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS17), Long Beach, USA, December 2017. [project website] (NIPS spotlight!)
  15. Joint Discovery of Object States and Manipulating Actions, J.-B. Alayrac, J. Sivic, I. Laptev and S. Lacoste-Julien, International Conference on Computer Vision (ICCV 2017), Venice, Italy, October 2017. [project website]
  16. A Closer Look at Memorization in Deep Networks, D. Arpit*, S. Jastrzebski*, N. Ballas*, D. Krueger*, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio and S. Lacoste-Julien, International Conference on Machine Learning (ICML 2017), Sydney, Australia, August 2017. *Equal contribution.
  17. Frank-Wolfe Algorithms for Saddle Point Problems, G. Gidel, T. Jebara and S. Lacoste-Julien, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), Fort Lauderdale, Florida, USA, April 2017. [project website]
  18. ASAGA: Asynchronous Parallel SAGA, R. Leblond, F. Pedregosa and S. Lacoste-Julien, International Conference on Artificial Intelligence and Statistics (AISTATS 2017), Fort Lauderdale, Florida, USA, April 2017. [project website]
  19. PAC-Bayesian Theory Meets Bayesian Inference, P. Germain, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS16), Barcelona, Spain, December 2016.
  20. Minding the Gaps for Block Frank-Wolfe Optimization of Structured SVMs, A. Osokin*, J.-B. Alayrac*, I. Lukasewitz, P. Dokania and S. Lacoste-Julien, International Conference on Machine Learning (ICML 2016), New York City, USA, June 2016. *Both authors contributed equally. [project website]
  21. Beyond CCA: Moment Matching for Multi-View Models, A. Podosinnikova, F. Bach and S. Lacoste-Julien, International Conference on Machine Learning (ICML 2016), New York City, USA, June 2016. [code]
  22. Unsupervised Learning from Narrated Instruction Videos, J.-B. Alayrac, P. Bojanowski, N. Agrawal, I. Laptev, J. Sivic and S. Lacoste-Julien, Conference on Computer Vision and Pattern Recognition (CVPR 2016), Las Vega, USA, June 2016. [project website] (oral!)
  23. On the Global Linear Convergence of Frank-Wolfe Optimization Variants, S. Lacoste-Julien and M. Jaggi, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015. [code]
  24. Barrier Frank-Wolfe for Marginal Inference, R. Krishnan, S. Lacoste-Julien and D. Sontag, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015. [code]
  25. Variance Reduced Stochastic Gradient Descent with Neighbors, T. Hofmann, A. Lucchi, S. Lacoste-Julien, and Brian McWilliams, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015.
  26. Rethinking LDA: Moment Matching for Discrete ICA, A. Podosinnikova, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS15), Montreal, Canada, December 2015. [code] [project's web page]
  27. On Pairwise Costs for Network Flow Multi-Object Tracking, V. Chari, S. Lacoste-Julien, I. Laptev and J. Sivic, Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, USA, June 2015. [project website]
  28. Sequential Kernel Herding: Frank-Wolfe Optimization for Particle Filtering, S. Lacoste-Julien, F. Lindsten and F. Bach. International Conference on Artificial Intelligence and Statistics (AISTATS 2015), San Diego, California, USA, May 2015. MCMCSki IV poster prize honourable mention (2014).
  29. SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives, A. Defazio, F. Bach and S. Lacoste-Julien, Neural Information Processing Systems Conference (NIPS14), Montreal, Canada, December 2014.
  30. SiGMa: Simple Greedy Matching for Aligning Large Knowledge Bases, S. Lacoste-Julien, K. Palla, A. Davies, G. Kasneci, T. Graepel and Z. Ghahramani, 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2013), Chicago, USA, August 2013.
    Previous longer preprint: arXiv:1207.4525v1 [cs.AI], July 2012.
  31. Block-Coordinate Frank-Wolfe Optimization for Structural SVMs, S. Lacoste-Julien*, M. Jaggi*, M. Schmidt and P. Pletscher, International Conference on Machine Learning (ICML 2013), Atlanta, USA, June 2013. *Both authors contributed equally. [code (Matlab / Octave)]
  32. On the Equivalence between Herding and Conditional Gradient Algorithms, F. Bach, S. Lacoste-Julien and G. Obozinski, International Conference on Machine Learning (ICML 2012), Edinburgh, UK, June 2012.
  33. Approximate Inference for the Loss-Calibrated Bayesian, S. Lacoste-Julien, F. Huszár, and Z. Ghahramani, International Conference on Artificial Intelligence and Statistics (AISTATS11), Florida, April 2011.
  34. DiscLDA: Discriminative Learning for Dimensionality Reduction and Classification. S. Lacoste-Julien, F. Sha, and M. Jordan, Neural Information Processing Systems Conference (NIPS08), Vancouver, British Columbia, December 2008.
  35. Word Alignment via Quadratic Assignment. S. Lacoste-Julien, B. Taskar, D. Klein, and M. Jordan, Human Language Technology conference - North American chapter of the Association for Computational Linguistics (HLT-NAACL06), New York, June 2006.
  36. Structured Prediction, Dual Extragradient and Bregman Projections. B. Taskar, S. Lacoste-Julien, and M. Jordan, Journal of Machine Learning Research (JMLR), Special Topic on Machine Learning and Large Scale Optimization, 7, 1627-1653, 2006.
  37. Structured Prediction via the Extragradient Method. B. Taskar, S. Lacoste-Julien, and M. Jordan, Neural Information Processing Systems Conference (NIPS05), Vancouver, British Columbia, December 2005. [Longer version]
  38. A Discriminative Matching Approach to Word Alignment. B. Taskar, S. Lacoste-Julien, and D. Klein, Empirical Methods in Natural Language Processing (EMNLP05), Vancouver, British Columbia, October 2005.
  39. Meta-Modelling Hybrid Formalisms. S. Lacoste-Julien, H. Vangheluwe, J. de Lara and P. Mosterman, IEEE International Symposium on Computer Aided Control System Design, special section on multi-paradigm modelling. Taiwan, September 2004.