Research Article

On-line learning for very large data sets

  1. Léon Bottou1,*,
  2. Yann Le Cun2

Article first published online: 23 MAR 2005

DOI: 10.1002/asmb.538

Issue

Applied Stochastic Models in Business and Industry

Applied Stochastic Models in Business and Industry

Special Issue: Statistical Learning

Volume 21, Issue 2, pages 137–151, March/April 2005

Additional Information

How to Cite

Bottou, L. and Le Cun, Y. (2005), On-line learning for very large data sets. Appl. Stochastic Models Bus. Ind., 21: 137–151. doi: 10.1002/asmb.538

Author Information

  1. 1

    NEC Research Institute, 4 Independence Way, Princeton, NJ 08540, U.S.A.

  2. 2

    The Courant Institute of Mathematical Sciences, New York University, 715 Broadway, New York, NY 10003, USA

*NEC Research Institute, 4 Independence Way, Princeton, NJ 08540, U.S.A.

Publication History

  1. Issue published online: 23 MAR 2005
  2. Article first published online: 23 MAR 2005

SEARCH

SEARCH BY CITATION

ARTICLE TOOLS

Get PDF (153K)

Keywords:

  • learning;
  • convergence speed;
  • online learning;
  • stochastic optimization

Abstract

The design of very large learning systems presents many unsolved challenges. Consider, for instance, a system that ‘watches’ television for a few weeks and learns to enumerate the objects present in these images. Most current learning algorithms do not scale well enough to handle such massive quantities of data. Experience suggests that the stochastic learning algorithms are best suited to such tasks. This is at first surprising because stochastic learning algorithms optimize the training error rather slowly. Our paper reconsiders the convergence speed in terms of how fast a learning algorithm optimizes the testing error. This reformulation shows the superiority of the well designed stochastic learning algorithm. Copyright © 2005 John Wiley & Sons, Ltd.

Get PDF (153K)