Internet Archive

Internet Archive
Formation 1996
Type online library
Website www.archive.org
Internet Archive headquarters is in the Presidio, a former US military base in San Francisco.
Mirror of the Internet Archive in the Bibliotheca Alexandrina

The Internet Archive (IA) is a nonprofit organization dedicated to building and maintaining a free and openly accessible online digital library, including an archive of Web. With offices located in the Presidio in San Francisco, California, and data centers in San Francisco, Redwood City, and Mountain View, CA, the archive includes "snapshots of the World Wide Web" (archived copies of pages, taken at various points in time), software, movies, books, and audio recordings. To ensure the stability and endurance of the Internet Archive, its collection is mirrored at the Bibliotheca Alexandrina in Egypt, so far the only library in the world with a mirror.[1] The IA makes its collections available at no cost to researchers, historians, and scholars. It is a member of the American Library Association and is officially recognized by the State of California as a library.[2]

Contents

History

The Internet Archive was founded by Brewster Kahle in 1996.

According to its website:

Most societies place importance on preserving artifacts of their culture and heritage. Without such artifacts, civilization has no memory and no mechanism to learn from its successes and failures. Our culture now produces more and more artifacts in digital form. The Archive's mission is to help preserve those artifacts and create an Internet library for researchers, historians, and scholars.

Wayback Machine

Examples from the Wayback
Machine's archives:

The Wayback Machine (web.archive.org) is a digital time capsule created by the Internet Archive. It is maintained with content from Alexa Internet. This service allows users to see archived versions of web pages across time—what the Archive calls a "three dimensional index."

Snapshots become available 6 to 12 months after they are archived. The frequency of snapshots is variable, so not all updates to tracked web sites are recorded, and intervals of several weeks sometimes occur.

As of 2006 the Wayback Machine contained almost 2 petabytes of data and was growing at a rate of 20 terabytes per month, a two-thirds increase over the 12 terabytes/month growth rate reported in 2003. Its growth rate eclipses the amount of text contained in the world's largest libraries, including the Library of Congress. The data is stored on Petabox rack systems manufactured by Capricorn Technologies.[3]

The name Wayback Machine is a reference to a segment from The Rocky and Bullwinkle Show in which Mr. Peabody, a bow tie-wearing dog with a professorial air, and his human "pet boy" assistant Sherman use a time machine called the "WABAC machine" to witness, participate in, and more often than not alter famous events in history.[4]

Archive-It

Users who want to archive material permanently and immediately cite an archived version can use the Archive-It system, a for-fee subscription service.[5] Data collected with Archive-It is periodically indexed into the general Wayback Machine. As of December 2007, Archive-It had created over 230 million URLs for 466 public collections, including government bodies, universities, and cultural institutions. Some of the organizations participating in Archive-It include the Electronic Literature Organization, the State Archives of North Carolina, the Texas State Library and Archives Commission, Stanford University, the National Library of Australia, the Research Libraries Group (RLG), and many others.

Media collections

In addition to web archives, the Internet Archive maintains extensive collections of digital media that are either public domain or licensed under a license that allows redistribution, such as Creative Commons licenses. The media are organized into collections by media type (moving images, audio, text, etc.), and into sub-collections by various criteria. Each of the main collections includes an "Open Source" sub-collection where general contributions by the public can be stored.

Moving image collection

Aside from feature films, IA's Moving Image collection includes: newsreels; classic cartoons; pro- and anti-war propaganda; Skip Elsheimer's "A.V. Geeks" collection; and ephemeral material from Prelinger Archives, such as advertising, educational and industrial films and amateur and home movie collections.

IA's Brick Films collection contains stop-motion animation filmed with LEGO bricks, some of which are 'remakes' of feature films. The Election 2004 collection is a non-partisan public resource for sharing video materials related to the 2004 United States Presidential Election. The Independent News collection includes sub-collections such as the Internet Archive's World At War competition from 2001, in which contestants created short films demonstrating "why access to history matters." Among their most-downloaded video files are eyewitness recordings of the devastating 2004 Indian Ocean earthquake. The September 11th Television Archive contains archival footage from the world's major television networks of the terrorist attacks of September 11th, 2001 as they unfolded on live television.

Some of the films available on the Internet Archive are:

Audio collection

Main article: Live Music Archive

The audio collection includes music, audio books, news broadcasts, old time radio shows and a wide variety of other audio files.

The Live Music Archive sub-collection includes 40,000 concert recordings from independent artists, as well as more established artists and musical ensembles with permissive rules about recording their concerts such as the Grateful Dead, and more recently, The Smashing Pumpkins.

Texts collection

The texts collection includes digitized books from various libraries around the world as well as many special collections. As of October 2008, the Internet Archive operated 18 scanning centers in five countries, digitizing about 1,000 books a day, financially supported by libraries and foundations.[13] As of November 2008, when there were about 1 million texts, the entire collection was over 0.5 petabytes, which includes raw camera images, cropped and skewed images, PDFs, and raw OCR data.[14]

Between about 2006 and 2008 Microsoft Corporation had a special relationship with Internet Archive texts through its Live Search Books project, scanning over 300,000 books which were contributed to the collection, as well as financial support and scanning equipment. On May 23, 2008 Microsoft announced it would be ending the Live Book Search project and no longer scanning books.[15] Microsoft will be making its scanned books available without contractual restriction and making the scanning equipment available to its digitization partners and libraries to continue digitization programs.[15]

Around October 2007 Internet Archive began importing book scans made by Google Books which are in the public domain.[16] As of November 2008 there are over half a million such imports representing almost half of the total books on Internet Archive. The books are identical to the copies found on Google, except without the Google watermarks.[17]

Open Library

The Open Library is another project of the Internet Archive. The site, still in beta, seeks to include a web page database for every book ever published, a sort of Open Source version of WorldCat. It holds the catalog records of nearly 20 million books, in addition to the full texts of about 400,000 public domain books, which are fully searchable.[18][19] Open Library is a free/open source software project, with its source code freely available on the Open Library site.

Controversies

Scientology sites

See also: Scientology versus the Internet

In late 2002, the Internet Archive removed various sites critical of Scientology from the Wayback Machine.[20] The error message stated that this was in response to a "request by the site owner."[21] It was later clarified that lawyers from the Church of Scientology had demanded the removal and that the actual site owners did not want their material removed.[22]

Archived web pages as evidence

United States civil litigation

Telewizja Polska

In an October 2004 case called "Telewizja Polska SA v. Echostar Satellite", a litigant attempted to use the Wayback Machine archives as a source of admissible evidence, perhaps for the first time. Telewizja Polska is the provider of TVP Polonia and EchoStar operates the Dish Network. Prior to the trial proceedings, EchoStar indicated that it intended to offer Wayback Machine snapshots as proof of the past content of Telewizja Polska’s website. Telewizja Polska brought a motion in limine to suppress the snapshots on the grounds of hearsay and unauthenticated source, but Magistrate Judge Arlander Keys rejected Telewizja Polska’s assertion of hearsay and denied TVP's motion in limine to exclude the evidence at trial.[23] However, at the actual trial, district Court Judge Ronald Guzman, the trial judge, overruled Magistrate Keys' findings, and held that neither the affidavit of the Internet Archive employee nor the underlying pages (i.e., the Telewizja Polska website) were admissible as evidence. Judge Guzman reasoned that the employee's affidavit contained both hearsay and inconclusive supporting statements, and the purported webpage printouts themselves were not self-authenticating.[24]

Healthcare Advocates, Inc.

In 2003, Healthcare Advocates, Inc. were defendants in a trademark violation lawsuit wherein the prosecution attempted to use archived web material accessed via the Internet Archive. When they lost that suit, the company turned around and attempted to sue the Internet Archive for violating the Digital Millennium Copyright Act (DMCA) and the Computer Fraud and Abuse Act. They claimed that since they had installed a robots.txt file on their website, it should have been avoided by the Internet Archive’s web crawlers but was not.[25] The initial lawsuit was filed on June 26, 2003, and they added the robots.txt file on July 8, 2003, so pages should have been removed retroactively. The lawsuit with Healthcare Advocates was settled out of court.[26]

Robots.txt is used as part of the Robots Exclusion Standard, a voluntary protocol the Internet Archive respects that disallows bots from indexing certain pages delineated by the creator as off-limits. As a result, the Internet Archive has removed a number of websites that are now inaccessible through the Wayback Machine. This is sometimes due to a new domain owner placing a robots.txt file that disallows indexing of the site. The administrators claim to be working on a system that will allow access to that previous material while excluding material created after the point the domain switched hands. Currently, the Internet Archive applies robots.txt rules retroactively; if a site blocks the Internet Archive, like Healthcare Advocates, any previously archived pages from the domain are also removed. In cases of blocked sites, only the robots.txt file is archived. This practice would appear to be detrimental to researchers looking for information that was available in the past.

However, the Internet Archive also states that, "sometimes a web site owner will contact us directly and ask us to stop crawling or archiving a site. We comply with these requests." [27] They also say, "The Internet Archive is not interested in preserving or offering access to Web sites or other Internet documents of persons who do not want their materials in the collection."[28]

Patent law

Main article: Internet as a source of prior art

The United States patent office and, provided some additional requirements are met (e.g. providing an authoritative statement of the archivist), the European Patent Office will accept date stamps from the Internet Archive as evidence of when a given web page was accessible to the public. These dates are used to determine if a web page is available as prior art for instance in examining a patent application.

Grateful Dead

In November 2005, free downloads of Grateful Dead concerts were removed from the site. John Perry Barlow identified Bob Weir, Mickey Hart, and Bill Kreutzmann as the instigators of the change, according to a New York Times article.[29] Phil Lesh commented on the change in a November 30, 2005, posting to his personal website:

It was brought to my attention that all of the Grateful Dead shows were taken down from Archive.org right before Thanksgiving. I was not part of this decision making process and was not notified that the shows were to be pulled. I do feel that the music is the Grateful Dead's legacy and I hope that one way or another all of it is available for those who want it.[30]

A November 30 forum post from Brewster Kahle summarized what appeared to be the compromise reached among the band members. Audience recordings could be downloaded or streamed, but soundboard recordings were to be available for streaming only. Concerts have since been re-added.[31]

Automatically entered contracts

On December 12, 2005, activist Suzanne Shell demanded Internet Archive pay her US$100,000 for archiving her website profane-justice.org between 1999 and 2004.[32] Internet Archive filed a declaratory judgment action in the United States District Court for the Northern District of California on January 20, 2006, seeking a judicial determination that Internet Archive did not violate Shell’s copyright. Shell responded and brought a countersuit against Internet Archive for archiving her site, which she alleges is in violation of her terms of service.[33] On February 13, 2007, a judge for the United States District Court for the District of Colorado dismissed all counterclaims except breach of contract.[32] The Internet Archive did not move to dismiss copyright infringement claims Shell asserted arising out of its copying activities, which will also go forward.[34] On April 25, 2007, Internet Archive and Suzanne Shell jointly announced the settlement of their lawsuit. The Internet Archive said, “Internet Archive has no interest in including materials in the Wayback Machine of persons who do not wish to have their Web content archived. We recognize that Ms. Shell has a valid and enforceable copyright in her Web site and we regret that the inclusion of her Web site in the Wayback Machine resulted in this litigation. We are happy to have this case behind us.” Ms. Shell said, “I respect the historical value of Internet Archive’s goal. I never intended to interfere with that goal nor cause it any harm.”[35]

Copyright situation in Europe

In Europe the Wayback Machine can sometimes violate copyright laws. Only the creator can decide where his content is published or duplicated, so the Archive would have to delete pages from its system upon request of the creator.[36] The exclusion policies for the Wayback Machine can be found in the FAQ section of the site. The Wayback Machine also retroactively respects robots.txt files.

Rescission of FBI National Security Letter

On May 8, 2008 it was revealed that the Internet Archive successfully challenged an FBI National Security Letter asking for logs on an undisclosed user.[37][38]

See also

Similar projects

Other

References

  1. ↑ The Internet Archive at the New Library of Alexandria
  2. ↑ " Internet Archive officially a library", May 2, 2007.
  3. ↑ Kanellos, Michael (July 29, 2005). "Big storage on the cheap", CNET News.com. Retrieved on 2007-07-29. 
  4. ↑ Green, Heather (February 28, 2002). "A Library as Big as the World", BusinessWeek. Retrieved on 2007-07-29. 
  5. ↑ Stefanie Olsen, "Preserving the Web one group at a time", CNet News.com, May 1, 2006
  6. ↑ Internet Archive: Details: Bronenosets Potyomkin (Battleship Potemkin)
  7. ↑ The Century of the Self
  8. ↑ Internet Archive: Details: Das Kabinett des Doktor Caligari ( The Cabinet of Dr. Caligari )
  9. ↑ Internet Archive: Details: The Kid
  10. ↑ Internet Archive: Details: M - Eine Stadt sucht einen Moerder
  11. ↑ Internet Archive: Details: The Power of Nightmares
  12. ↑ Why We Fight:
  13. ↑ "Books Scanning to be Publicly Funded", announcement by Brewster Khale, May 23, 2008.
  14. ↑ [http://blog.openlibrary.org/2008/11/24/bulk-access-to-ocr-for-1-million-books/ "Bulk Access to OCR for 1 Million Books", via Open Library Blog, by raj, November 24, 2008.
  15. ↑ 15.0 15.1 "Book search winding down", Live Search Blog. Official announcement from Microsoft. Last accessed May 23, 2008.
  16. ↑ Google Books at Internet Archive
  17. ↑ Books imported from Google have a metadata tag of scanner:google
  18. ↑ Gonsalves, Antone (December 20, 2006). "Internet Archive Claims Progress Against Google Library Initiative", InformationWeek. Retrieved on 2007-01-05. 
  19. ↑ "The Open Library Makes Its Online Debut", Chronicle of Higher Education, The Wired Campus (July 19, 2007). Retrieved on 2007-07-30. 
  20. ↑ Bowman, Lisa M (September 24, 2002). "Net archive silences Scientology critic", CNET News.com. Retrieved on 2007-01-04. 
  21. ↑ Jeff (23 September 2002). "exclusions from the Wayback Machine" (Blog). Wayback Machine Forum. Internet Archive. Retrieved on 2007-01-04. Author and Date indicate initiation of forum thread
  22. ↑ Miller, Ernest (24 September). "Sherman, Set the Wayback Machine for Scientology" (Blog). LawMeme. Yale Law School. Retrieved on 2007-01-04. The posting is billed as a 'feature' and lacks an associated year designation; comments by other contributors appear after the 'feature'
  23. ↑ Gelman, Lauren (17 November 2004). "Internet Archive’s Web Page Snapshots Held Admissible as Evidence". Packets 2 (3). http://cyberlaw.stanford.edu/packets002728.shtml. Retrieved on 2007-01-04. 
  24. ↑ Howell, Beryl A. (February 2006). "Proving Web History: How to use the Internet Archive" (PDF). Journal of Internet Law: 3–9. http://www.strozllc.com/files/Publication/fee98a34-d739-478b-a7db-6af37b757714/Presentation/PublicationAttachment/aae88469-9835-4fe4-ae5f-38637924314f/BAHPROVINGWEBHISTORY.pdf. Retrieved on 2008-08-06. 
  25. ↑ Dye, Jessica (2005). "Website Sued for Controversial Trip into Internet Past". EContent 28 (11): 8–9. 
  26. ↑ Bangeman, Eric (31 August 2006). "Internet Archive Settles Suit Over Wayback Machine". ars technica. http://arstechnica.com/news.ars/post/20060831-7634.html. Retrieved on 2007-11-29. 
  27. ↑ http://web.archive.org/collections/web/faqs.html#exclusions
  28. ↑ http://www.archive.org/about/faqs.php#2
  29. ↑ Jeff Leeds; Jesse Fox Mayshark (December 1, 2005). "Wrath of Deadheads stalls a Web crackdown", International Herald Tribune (republication of article from New York Times). Retrieved on 2007-01-04. 
  30. ↑ Phil Lesh (30 November 2005). "An Announcement from Phil Lesh" (Blog). Hotline. PhilLesh.net. Retrieved on 2007-01-05.
  31. ↑ Brewster Kahle; Matt Vernon (1 December 2005). "Good News and an Apology: GD on the Internet Archive" (Blog). Live Music Archive Forum. Internet Archive. Retrieved on 2007-01-05. Authors and date indicate the first posting in the forum thread
  32. ↑ 32.0 32.1 Lewis T. Babcock (13 February 2007). Internet Archive v. Shell (PDF), Civil Action No. 06cv01726LTBCBS.
  33. ↑ Claburn, Thomas (March 16, 2007). "Colorado Woman Sues To Hold Web Crawlers To Contracts", InformationWeek. Retrieved on 2007-07-29. 
  34. ↑ Samson, Martin. Internet Archive v. Suzanne Shell. via Phillips Nizer LLP
  35. ↑ Internet Archive and Suzanne Shell Settle Lawsuit, April 25, 2007.
  36. ↑ German lawyer about the Wayback Machine in a law paper, Journal of Internet Law: JurPC
  37. ↑ FBI rescinds secret order for Internet Archive records, CNet
  38. ↑ Nakashima, Ellen, "FBI Backs Off From Secret Order for Data After Lawsuit", Washington Post, May 8, 2008

External links