Lockheed Martin-Developed Tool Enables Scientists to Efficiently Utilize Massive Data Stream From NASA's Solar Dynamics Observatory

News provided by

May 26, 2010, 11:00 ET

PALO ALTO, Calif., May 26 /PRNewswire/ -- The immense volume of data generated by the suite of instruments on NASA's Solar Dynamics Observatory (SDO) requires new tools for efficient identifying and accessing data that is most relevant to research investigations. Scientists at the Solar and Astrophysics Laboratory of the Lockheed Martin (NYSE: LMT) Advanced Technology Center (ATC) in Palo Alto have developed the Heliophysics Events Knowledgebase (HEK) to fill this need.

The HEK system combines automated data mining using feature detection methods and high-performance visualization systems for data markup. In addition, web services and clients are provided for searching the resulting metadata, reviewing results and efficiently accessing the data.

"We have so much high-resolution data from SDO that we need to make sure that we don't miss anything important. As a comparison, SDO generates as much data in a single day as our Transition Region and Coronal Explorer (TRACE) instrument – launched in 1998 – produces in five years," said Dr. Neal Hurlburt, Lockheed Martin astrophysicist and head of the Solar Software Group at the ATC. "The HEK tool enables us to characterize significant features and events that will be of interest to solar physicists, and make these data subsets easily accessible."

The HEK is designed to catalog interesting solar events and features and to present them to members of the solar physics community in such a way that guides them to the most relevant data for their purposes. This is a problem of data markup that is arising in many scientific and other fields. For example, the Monterrey Bay Aquarium Research Institute developed a similar system for annotating and cataloging video sequences of ocean fauna and activities, and various sports leagues have systems for cataloging clips of athletic events. The distinguishing factor for SDO is the large image format and complex event types.

The HEK was developed as an integrated metadata system designed with the following goals in mind:

To help researchers find data sets relevant for their topics of interest,
To serve as an open forum where solar and/or heliospheric features and events can be reported and annotated,
To facilitate discovery of statistical trends and/or relationships between different classes of features and events,
And to lighten the load on the SDO data system in terms of providing access to subsets of data that would be too large to transmit over the Internet.

To achieve these goals, HEK consists of registries to store metadata pertaining to observational sequences (the Heliophysics Coverage Registry, or HCR), to heliophysical events (Heliophysics Events Registry, or HER), as well as browse products such as movies. Interfaces for communications and querying between the different registries are also provided as web services.

The design and implementation of the HEK and its interfaces allow expansion beyond its primary function as a searchable database that contains metadata on solar events and on the observation sets from which these are derived. It can track temporary data products that users may wish to have online for a limited period. It is also possible to include numerical data sets based on assimilated observations, or even such data sets and events within them. In addition, it can also contain information on papers published based on certain data sets and events within them, so that for any given event users can be pointed to such publications, or likewise for a given publication users can be pointed to one or more data sets from one or more observatories.

Lockheed Martin developed the HEK system to address the immediate needs of SDO. However, the underlying motivation of devising a means for coping with enormous data sets resonates with many other missions and projects throughout modern science, and HEK could serve as a model.

Two of the SDO instruments were built at the Solar and Astrophysics Laboratory of the Lockheed Martin Advanced Technology Center (ATC) in Palo Alto. The Atmospheric Imaging Assembly (AIA), a suite of four telescopes, provides an unprecedented view of the solar corona, taking images that span at least 1.3 solar diameters in multiple wavelengths nearly simultaneously, at a resolution of 0.6 arc-seconds and at a cadence of 10 seconds or better. The Helioseismic and Magnetic Imager (HMI), designed in collaboration with Professor Philip Scherrer, HMI Principal Investigator, and other scientists at Stanford University, studies the origin of solar variability and attempts to characterize and understand the Sun's interior and magnetic activity. The third SDO instrument, the Extreme Ultraviolet Variability Experiment (EVE), measures fluctuations in the Sun's ultraviolet output. EVE was built by the Laboratory for Atmospheric and Space Physics at the University of Colorado.

The goal of SDO is to understand – striving towards a predictive capability – the solar variations that influence life on Earth and humanity's technological systems. The mission seeks to determine how the Sun's magnetic field is generated and structured, and how this stored magnetic energy is converted and released into the heliosphere and geospace in the form of solar wind, energetic particles, and variations in the solar irradiance.

SDO is the first mission and crown jewel in a fleet of NASA missions to study our Sun. The mission is the cornerstone of a NASA science program called Living With a Star (LWS). The goal of the LWS Program is to develop the scientific understanding necessary to address those aspects of the Sun and solar system that directly affect life and society.

The Solar and Astrophysics Laboratory at the ATC has a 47-year-long heritage of spaceborne solar instruments including the Soft X-ray Telescope on the Japanese Yohkoh satellite, the Michelson Doppler Imager on the ESA/NASA Solar and Heliospheric Observatory, the solar telescope on NASA's Transition Region and Coronal Explorer, the Solar X-ray Imager on the GOES-N, -O and -P environmental satellites, the Focal Plane Package on Hinode and an Extreme Ultraviolet Imager on each of the two spacecraft in NASA's Solar Terrestrial Relations Observatory. The laboratory also conducts basic research into understanding and predicting space weather and the behavior of the Sun including its impacts on Earth and climate.

The ATC is the research and development organization of Lockheed Martin Space Systems Company (LMSSC). LMSSC, a major operating unit of Lockheed Martin Corporation, designs and develops, tests, manufactures and operates a full spectrum of advanced-technology systems for national security and military, civil government and commercial customers. Chief products include human space flight systems; a full range of remote sensing, navigation, meteorological and communications satellites and instruments; space observatories and interplanetary spacecraft; laser radar; ballistic missiles; missile defense systems; and nanotechnology research and development.

Headquartered in Bethesda, Md., Lockheed Martin is a global security company that employs about 136,000 people worldwide and is principally engaged in the research, design, development, manufacture, integration and sustainment of advanced technology systems, products and services. The Corporation reported 2009 sales of $45.2 billion.