Skip to main content's Web Crawler: A Massively Scaled Java Technology Application

No replies
Joined: 2003-06-09

from's innovative shopping search engine has done what many in the search engine community thought impossible: The company has successfully created a Java technology web crawler that may be the most sophisticated, massively scaled Java technology application in existence, obtaining information on over 3 billion web pages and writing well over 8 terabytes of data (and growing) on 30 fully distributed servers in seven days.