Research Article

STXXL: standard template library for XXL data sets

  1. R. Dementiev1,*,
  2. L. Kettner2,†,
  3. P. Sanders1

Article first published online: 6 AUG 2007

DOI: 10.1002/spe.844

Issue

Software: Practice and Experience

Software: Practice and Experience

Volume 38, Issue 6, pages 589–637, May 2008

Additional Information

How to Cite

Dementiev, R., Kettner, L. and Sanders, P. (2008), STXXL: standard template library for XXL data sets. Softw: Pract. Exper., 38: 589–637. doi: 10.1002/spe.844

Author Information

  1. 1

    Fakultät für Informatik, Universität Karlsruhe, Karlsruhe, Germany

  2. 2

    Max Planck Institut für Informatik, Saarbrücken, Germany

  1. †

    Now at mental images GmbH, Berlin, Germany.

*Fakultät für Informatik, Universität Karlsruhe, Karlsruhe, Germany

  1. †

    Now at mental images GmbH, Berlin, Germany.

Publication History

  1. Issue published online: 4 APR 2008
  2. Article first published online: 6 AUG 2007
  3. Manuscript Accepted: 3 JUN 2007
  4. Manuscript Revised: 30 MAY 2007
  5. Manuscript Received: 26 JAN 2007

Funded by

  • DFG. Grant Number: SA 933/1-2

SEARCH

SEARCH BY CITATION

ARTICLE TOOLS

Get PDF (497K)

Keywords:

  • very large data sets;
  • software library;
  • C++ standard template library;
  • algorithm engineering

Abstract

We present the software library STXXL that is an implementation of the C++ standard template library (STL) for processing huge data sets that can fit only on hard disks. It supports parallel disks, overlapping between disk I/O and computation and it is the first I/O-efficient algorithm library that supports the pipelining technique that can save more than half of the I/Os. STXXL has been applied both in academic and industrial environments for a range of problems including text processing, graph algorithms, computational geometry, Gaussian elimination, visualization, and analysis of microscopic images, differential cryptographic analysis, etc. The performance of STXXL and its applications are evaluated on synthetic and real-world inputs. We present the design of the library, how its performance features are supported, and demonstrate how the library integrates with STL. Copyright © 2007 John Wiley & Sons, Ltd.

Get PDF (497K)