The copyright line for this article was changed on 18 February 2015 after original online publication.
Decoding billions of integers per second through vectorization
Article first published online: 28 MAY 2013
© 2013 The Authors. Software: Practice and Experience Published by John Wiley & Sons, Ltd.
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Software: Practice and Experience
Volume 45, Issue 1, pages 1–29, January 2015
How to Cite
2014), Decoding billions of integers per second through vectorization, Softw. Pract. Exper., 45, 1–29, doi:10.1002/spe.2203., and (
- Issue published online: 3 DEC 2014
- Article first published online: 28 MAY 2013
- Manuscript Accepted: 20 APR 2013
- Manuscript Revised: 18 APR 2013
- Manuscript Received: 17 SEP 2012
- index compression;
- vector processing
In many important applications—such as search engines and relational database systems—data are stored in the form of arrays of integers. Encoding and, most importantly, decoding of these arrays consumes considerable CPU time. Therefore, substantial effort has been made to reduce costs associated with compression and decompression. In particular, researchers have exploited the superscalar nature of modern processors and single-instruction, multiple-data (SIMD) instructions. Nevertheless, we introduce a novel vectorized scheme called SIMD-BP128⋆ that improves over previously proposed vectorized approaches. It is nearly twice as fast as the previously fastest schemes on desktop processors (varint-G8IU and PFOR). At the same time, SIMD-BP128⋆ saves up to 2 bits/int. For even better compression, we propose another new vectorized scheme (SIMD-FastPFOR) that has a compression ratio within 10% of a state-of-the-art scheme (Simple-8b) while being two times faster during decoding. © 2013 The Authors. Software: Practice and Experience Published by John Wiley & Sons, Ltd.