Research Article

You have free access to this content

Tracking provenance in a virtual data grid

  1. Ben Clifford1,
  2. Ian Foster1,2,
  3. Jens-S. Voeckler3,
  4. Michael Wilde1,2,*,
  5. Yong Zhao1

Article first published online: 21 AUG 2007

DOI: 10.1002/cpe.1256

Issue

Concurrency and Computation: Practice and Experience

Concurrency and Computation: Practice and Experience

Special Issue: The First Provenance Challenge

Volume 20, Issue 5, pages 565–575, 10 April 2008

Additional Information

How to Cite

Clifford, B., Foster, I., Voeckler, J.-S., Wilde, M. and Zhao, Y. (2008), Tracking provenance in a virtual data grid. Concurrency and Computation: Practice and Experience, 20: 565–575. doi: 10.1002/cpe.1256

Author Information

  1. 1

    Computation Institute, University of Chicago, Chicago, IL 60637, U.S.A.

  2. 2

    Math & Computer Science Division, Argonne National Laboratory, Argonne, IL 60439, U.S.A.

  3. 3

    USC Information Sciences Institute, Marina Del Rey, CA, U.S.A.

*Computation Institute, University of Chicago, Chicago, IL 60637, U.S.A.

Publication History

  1. Issue published online: 1 MAR 2008
  2. Article first published online: 21 AUG 2007
  3. Manuscript Accepted: 1 MAY 2007
  4. Manuscript Revised: 24 APR 2007
  5. Manuscript Received: 4 DEC 2006

Funded by

  • National Science Foundation. Grant Numbers: ITR-800864, PHY-0636265
  • U.S. Department of Energy. Grant Number: DE-AC02-06CH11357
  • NIH National Institute on Deafness and Other Communication Disorders. Grant Number: DC008638-01

SEARCH

SEARCH BY CITATION

ARTICLE TOOLS

View Full Article (HTML) Get PDF (217K)

Keywords:

  • Grid computing;
  • workflow;
  • data provenance

Abstract

The virtual data model allows data sets to be described prior to, and separately from, their physical materialization. We have implemented this model in a Virtual Data Language (VDL) and associated supporting tools, which provide for both the storage, query, and retrieval of virtual data set descriptions, and the automated, on-demand materialization of virtual data sets. We use a standardized data provenance challenge exercise to illustrate the powerful queries that can be performed on the data maintained by these tools, which for a single virtual data set can include three elements: the computational procedure(s) that must be executed to materialize the data set, the runtime log(s) produced by the execution of the computation(s), and optional metadata annotation(s) that associate application semantics with data and procedures. Copyright © 2007 John Wiley & Sons, Ltd.

View Full Article (HTML) Get PDF (217K)