University Libraries Faculty Research & Creative Works

Evaluation of Web-Based Search Engines from the End-User's Perspective: A Pilot Study

Abstract

Due to the rapid development of the World Wide Web (WWW), Web-based search engines have become the most important tools for accessing global information resources on the Internet. Although these search engines are easily accessible and highly popular, systematic evaluation of their performance based on real users and their information problems has not been conducted. This study describes a pilot study using a systematic methodology proposed by Su (1997) to gather such data. Measures used to assess four Web-based search engines are based on five evaluation criteria: relevance, efficiency, utility, user satisfaction, and connectivity. Eleven participants, who were faculty, staff, and graduate students at the University of Pittsburgh, searched for their individual information topics on four search engines using simple search form. They tested two randomly selected hyperlinks from each set of twenty items retrieved for each engine. They made relevance assessments of the first twenty items retrieved by their best strategy on each engine. They chose and ranked five most relevant items from each set of the twenty items in relation to their individual information problems. Preliminary data analysis reveals that none of the four engines dominates in every aspect of the multidimensional evaluation. The study results shows that even though Lycos has the best relevance ranking performance and the highest precision ratio, Alta Vista has the best connectivity, the highest user's satisfaction ratios for precision and for three components of system features, and the highest value of the search result as a whole. One-way ANOVA test indicates that user's satisfaction ratings with online documentation (F=9.97 & p=0.000) and value of search results as a whole (F=3.65 & p=0.023) are significantly different among the four search engines.

Meeting Name

61st Annual Meeting of the American Society for Information Science, ASIS (1998: Oct. 25-29, Pittsburgh, PA)

Department(s)

Library and Learning Resources

International Standard Serial Number (ISSN)

0044-7870; 1550-8390

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 1998 John Wiley & Sons Inc., All rights reserved.

This document is currently not available here.

Share

 
COinS