Henrique Andrade did this work while at IBM Research. He is now a vice president in the Securities Division at Goldman Sachs in New York.
A model-based framework for building extensible, high performance stream processing middleware and programming language for IBM InfoSphere Streams
Article first published online: 13 NOV 2011
Copyright © 2011 John Wiley & Sons, Ltd.
Software: Practice and Experience
Volume 42, Issue 11, pages 1363–1391, November 2012
How to Cite
Gedik, B. and Andrade, H. (2012), A model-based framework for building extensible, high performance stream processing middleware and programming language for IBM InfoSphere Streams. Softw: Pract. Exper., 42: 1363–1391. doi: 10.1002/spe.1139
- Issue published online: 2 OCT 2012
- Article first published online: 13 NOV 2011
- Manuscript Accepted: 24 SEP 2011
- Manuscript Revised: 19 SEP 2011
- Manuscript Received: 21 APR 2011
- data stream processing;
- model-based code generation
This work presents an extensive case study on the model-based design of a commercial-grade stream processing middleware (IBM's InfoSphere Streams) its runtime and language (SPL) compiler. The model-based underpinnings are pervasive throughout the whole environment, from describing inter-process communication interfaces and objects to the design of the extensibility mechanism in the runtime and language. In addition to many software engineering advantages such as consistent, uniform, and self-documented integration among the different parts of the system, we show intrinsic performance benefits to the platform derived from this design approach. First, we demonstrate how an incremental compilation strategy employed by the SPL compiler and rooted on the model description of the application, extracted by the compiler as part of the application building process, leads to better compile-time performance. Second, we discuss how the model-based code generation strategy employed by the SPL compiler also leads to increased runtime performance, by specializing the generated code to particular characteristics of the runtime environment. Finally, we show how the extensibility strategy used in the SPL language leads to automatic syntactic and semantic checks at compile time, while enabling behavioral reasoning and specific optimizations at runtime. Copyright © 2011 John Wiley & Sons, Ltd.