The Web Lab:
A Joint Project of Cornell University and the Internet Archive

• Home
  • Overview
  • Publications
• Researchers' Area
  • Tools
  • Documentation
  • Status
  • Registration
  • Login
  • Change password
• About the Web Lab
  • Contributors
  • Contact Us

Welcome to the Web Lab Researchers' Area

Replacement for Scidata1 database server

Over the summer, the Web Lab database server had serious hardware problems. The server has now been replaced with more powerful equipment and all databases restored

December 20, 2008

The Web Lab

The Web Lab is a research project and is continually under development.  Researchers are encouraged to contact members of the Web Lab team to discuss their research plans and to schedule large data analysis jobs.  For general information, send email to info@infosci.cornell.edu.

The Web Lab provides two major services: a relational database with metadata about Web pages, and a data analysis cluster running the Hadoop file system with support for map/reduce programming.  Small database tasks can be run from tools available on this Web site.  Large jobs should be discussed with the database administrator.  Use of the data analysis cluster requires authentication.

General information

• The Web Lab Hadoop cluster.
• Data base access tools that can be run from this Web site.
• Documentation about tools, formats, and services.
• Status of the system and the collections that are available.

Use Policy

The long term goal is to make the Web Lab available to all researchers, but at present we are not able to support researchers outside Cornell.  Use of the Web Lab is subject to the Use Policy of the Web Lab.