Fri, 1 August 2008 Episode 13 - Toby DiPasquale on Google, Map-Reduce, Hadoop, Amazon EC2 and more This week we feature an interview with Toby DiPasquale of Invite Media.  Toby and I discuss the Map-Reduce algorithm, which is the engine that powers Google's indexing and data processing systems.  We start off by discussing how Google started indexing pages, using traditional methods such as C/C++ routines.  Quickly this became unmanageable, as the amount of data to index outstripped the processing power and traditional data transformation paradigms. Toby and I then go into discussing Map Reduce, which was originally posited as a thesis and then published as a seminal paper in the community.  Map Reduce has been implemented by Google, and as we'll see in the podcast, others followed suit and created the Hadoop engine, a Java-based Map Reduce solution.  We talk about Hadoop and it's various subprojects, and then get into a discussion on Amazon EC2 and the Cloud Computing movement, including why it is valuable to organizations who want to scale from one to potentially dozens of CPUs. I'll post the show notes early next week at http://www.chariotsolutions.com/podcasts/techcast/shownotes.  Until then, enjoy the show and comments are always welcome. Note:  the podcast audio got a bit distorted on Toby's side, but I don't think it distracts too much.  Rather than re-record the interview I'm presenting it as-is. Direct download: TechCast-2008-08-01-TobyDipasquale.mp3 Category: techcast -- posted at: 8:10 AM Comments[0]

Listen Now!

Click the icon on the show title to download a show, or:

Subscribe (full feed)!

Sub-feeds

The TechCast	rss
Conference Sessions	rss
BizCast	rss
DevNews!	rss

Listener Feedback

New Survey!

There are lots of ways to get involved. Here are a few...

• Twitter @techcast or use hashtag #techcast in your comments!
• Email techcastfeedback at chariotsolutions.com
• Discuss on our Facebook Fan Page
• Delicious - submit a devnewsideas story
• View the delicious news items
• View our guests' tweets!

Archives

January
February
March
April
May
June
July
August

February
March
April
May
July
August
September
October
November
December

February
March
April
May
June
July
August
October
December

Favorite Sites

• Our Tweets
• Grails Podcast
• SpringSource Blogs
• Chariot Blog
• Hibernate / JBoss Bloggers
• Ruby Inside

Thanks for attending
Visit our ETE Community Site

About the TechCast...

We bring you interviews with project creators, architects and consultants, and feature major open source projects and initiatives, such as Spring, Flex and RIA technologies, Mule, Groovy/Grails, Rails, Scala, Cloud Computing (Amazon, Google), and much more.

About the host

Ken Rimple got into recording at an early age by watching his father work at radio stations in the Delaware Valley. He has more than twenty years experience in information technology and has a keen interest in emerging and innovative trends in software development, as well as interest in the people behind the technologies.

Disclosures

From time to time, we discuss news items related to specific companies and projects. We will make every attempt to disclose any relationships during our podcasts. Some of our partners include:

• SpringSource
• JBoss, a division of RedHat
• Sun / Oracle
• MulesSoft
• Engine Yard
• Apache
• Sonatype

Plugs and Feedback...

We are using Free Theme #3 and Free Theme #4 from podcastthemes.com. Mark Blasco works very hard at customizing themes for individual podcast, including This Week in Tech, MacBreak Weekly, and many others.

Please leave feedback via comments or email.

Syndication