A Microsoft research meetup shows off three projects
Microsoft held its tenth annual Research Faculty Summit on Monday, and the focus was on data en masse -- processing it quickly and helping scientists make sense of it once it's gathered. Three projects shared the spotlight in Redmond.
Two of the three, Dryad and DryadLINQ, are intimately related. Both support high-performance computing. Dryad itself is an engine for making it easier to implement distributed applications on Windows HPC Server 2008 clusters. As its information page explains, "A Dryad programmer can use thousands of machines, each of them with multiple processors or cores, without knowing anything about concurrent programming."
DryadLINQ is a combination of Dryad and .NET's LINQ (Language-Integrated Query, extensions to C#). It's an abstraction layer that allows relative newbies to program for the Dryad system from within familiar .NET development tools.
A bit farther afield for most Betanews readers but fascinating nonetheless, the Project Trident Scientific Workflow Workbench makes one wonder if, 40 years after the first moon landing, it's Microsoft rather than NASA that's in on the most interesting explorations of strange new worlds. Project Trident is geared toward oceanographic studies -- processing a mighty wave (heh) of data currently being gathered from sensors, instruments, moorings, robots and cameras attached to fiber-optic cables on the ocean floor.
The sensors are part of the Ocean Observatories Initiative, underwritten by the NSF and managed by scientists at the University of Washington. The sensors lie along the ocean floor, gathering data Microsoft reps describe as "roughly equal to two simultaneous high-definition TV broadcasts going around the clock." All that data can tell us a lot about what's going on at the bottom of the sea -- the life, the seismic activity, all that.
In a similar vein, oceanographers at the Monterey Bay Aquarium Research Institute are using Project Trident to figure out the mechanics of typhoon intensification -- potentially lifesaving data for people who have nothing to do with computers. And for those who hate to see astronomical research shortchanged, Project Trident has a role there too; scientists at Johns Hopkins are using it to support the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS) project, which you may recognize as the project scanning the skies for celestial objects that might plonk us.
It doesn't take much to see how these three projects together could provide some impressive data-churn capabilities for projects that data a lot. In a prepared statement, Roger Barga, a Microsoft researcher and principal architect for the new tools, explained how it all clicks: "With the addition of DryadLINQ, our ability to interpret data has finally caught up with our ability to collect it.... While it is not necessary to couple Project Trident with Dryad, the combination provides a powerful system for processing very large volumes of data."