Sciencenet

Sciencenet is a distributed search engine at KIT – Liebel-Lab for scientific knowledge. The Sciencenet software (YaCy) is based on p2p technology developed by Michael Christen in collaboration with Liebel-lab at KIT.

Background

Scientific knowledge is spread across many databases, research institutes, educational websites and literature repositories. Today's search engines success is often based on popularity ranking. Popularity ranking is a powerful method for general search strategies, but less efficient for scientific knowledge. The Sciencenet search engine is especially for scientific websites and scientific content:.^[1] Its index is a collection of educational (e.g. of ".edu" / ".ac.uk" /".ac.au") and research dedicated sites (e.g. the German Helmholtz society, Max Planck institutes or Swiss and Austrian universities). This is the major difference from the global YaCy search engine ("Freeworld"), which serves as a global search engine.

Sciencenet architecture

Sciencenet is a network of Linux PCs running "YaCy" software. The "Sciencenet core cluster" is located at the Institute of Toxicology and Genetics (Liebel-Lab), KIT Karlsruhe (Karlsruhe Institute of Technology).

Sciencenet-YaCy software has been designed and tested to run in a network with at least 2000 CPUs, with up to 10 million webpages per single CPU, allowing to search large and/or distributed information repositories.

Mission

Any university / research institute is encouraged to use the software and contribute to the scientific network. The software automatically connects to the "core cluster" and contributes its search engine to the distributed sciencenet network. A high speed internet broadband connection (> 2 Mbit/s) and dedicated PCs are required. Ideally every research institute runs the free YaCy-Sciencenet software on 1–2 local PCs and provides its webpages to the network. As a side effect, wasted bandwidth from external search engines is reduced to a minimum.

The sciencenet ideally provides a free, open scientific search engine index to the community.

Sciencenet current status

• Currently (March 2010) 35 low cost Linux-PCs running Java and YaCy software share ~300,000,000 documents in a distributed peer2peer network.

External links

• The Sciencenet search portal

• See the current Sciencenet network overview.
• See the participating peers with detailed index information.
• Liebel-Lab
• yacy.net

References

1. ↑ ‘Sciencenet’—towards a global search and share engine for all scientific knowledge