Thursday, April 2, 2009

Amazon Launches Data Processing Service

Seattle-based Amazon Web Services, the web services arm of Amazon.com, is taking on data processing with a new service it is calling Amazon Elastic MapReduce. According to Amazon, the new service is targeted at businesses, researchers, data analysts, and developers, and allows them to process "vast amounts" of data. The new service uses Amazon's EC2 computing cloud and S3 storage services, and can be applied to distributed computing applications such as web indexing, data mining, scientific simulation, and bioinformatics research. Amazon compared its new service to the Apache open source package Hadoop, which is widely used in parallel processing efforts to scale out distributed processing applications; in fact, Amazon said it is using a version of the Hadoop software implementation on its cluster to handle the new service. MapReduce is a programming model--developed at Google--designed to handle and process large data sets.