Cyril Cleverdon

Cyril Cleverdon (9 September 1914 – 4 December 1997) was a British librarian and computer scientist who is best known for his work on the evaluation of information retrieval systems.

Cyril Cleverdon was born in Bristol, England. He worked at the Bristol Libraries from 1932 to 1938, and from 1938 to 1946 he was the librarian of the Engine Division of the Bristol Aeroplane Co. Ltd. In 1946 he was appointed librarian of the College of Aeronautics at Cranfield (later the Cranfield Institute of Technology and Cranfield University), where he served until his retirement in 1979, the last two years as professor of Information Transfer Studies.

With the help of NSF funding, Cleverdon started a series of projects in 1957 that lasted for about 10 years in which he and his colleagues set the stage for information retrieval research. In the Cranfield project, retrieval experiments were conducted on test databases in a controlled, laboratory-like setting. The aim of the research was to improve the retrieval effectiveness of information retrieval systems, by developing better indexing languages and methods. The components of the experiments were:

1. a collection of documents,
2. a set of user requests or queries, and
3. a set of relevance judgments—that is, a set of documents judged to be relevant to each query.

Together, these components form an information retrieval test collection. The test collection serves as a standard for testing retrieval approaches, and the success of each approach is measured in terms of two measures: precision and recall. Test collections and evaluation measures based on precision and recall are driving forces behind modern research on search systems. Cleverdon's approach formed a blueprint for the successful Text Retrieval Conference series that began in 1992.

Not only did Cleverdon's Cranfield studies introduce experimental research into computer science, the outcomes of the project also established the basis of the automatic indexing as done in today's search engines. Essentially, Cleverdon found that the use of single terms from the documents achieved the best retrieval performance, as opposed to manually assigned thesaurus terms, synonyms, etc. These results were very controversial at the time. In the Cranfield 2 Report, Cleverdon said:

This conclusion is so controversial and so unexpected that it is bound to throw considerable doubt on the methods which have been used (...) A complete recheck has failed to reveal any discrepancies (...) there is no other course except to attempt to explain the results which seem to offend against every canon on which we were trained as librarians.

Cyril Cleverdon also ran, for many years, the Cranfield conferences, which provided a major international forum for discussion of ideas and research in information retrieval. This function was taken over by the SIGIR conferences in the 1970s.

References

• Cyril Cleverdon (1960). "Report on the Testing and Analysis of an Investigation into the Comparative Efficiency of Indexing Systems" (PDF). The College of Aeronautics, Cranfield. 
• Cyril Cleverdon and Michael Keen, Factors Determining the Performance of Indexing Systems, Volume 2, The College of Aeronautics, Cranfield, 1966
• Stephen Robertson, In Memoriam Cyril W. Cleverdon, Journal of the American Society for Information Science 49(1):866, 1998