Figure 1. Web Search Engines' Query Models (Source: Anthony, 2016)
This paper is on usability evaluation of web search engines using navigational query model examples from library and information services. The paper adapted laboratory experimental design in which a study population of five web search engines, namely: Ask.com, Bing, Excite, Google, and Yahoo! web search engines, were selected for the study, and twenty one master degree students from the Department of Library and Information Technology, Federal University of Technology Minna, Nigeria were recruited and trained to perform usability evaluation of the selected web search engines by submitting queries drawn from library and information services to the various web search engines and assessing/judging their performances based on the relevance of the output results. The instrument used for data collection was an observation template/form used to record the observations made by the participants. The instrument contains five queries each of which has six usability effectiveness constructs/statements developed. The study revealed that there is no significant difference in usability effectiveness between web search engines using navigational query model, i.e, overall the web search engines performed well, but at different levels of performances. Specifically, Google has the highest performance on usability effectiveness using navigational query model, as reflected in its mean score of 3.59. Yahoo! ranked second with a mean score of 3.50, the third is Ask.com which has the mean score of 3.29, followed by Bing with a mean score of 3.01, and lastly Excite which has a mean score of 2.73. Hence, it was recommended to Excite web search engine's designers to improve on their navigational query model for better performance and enhance their users' experience. This study will provide deeper understanding to search engine developers on the significance of usability performances of web search engines using navigational query models, so that they can put these into consideration when designing a web search engine.
Web search engines are important information retrieval tools, they are in fact the most widely used retrieval tools in the world today as compared to the traditional library catalogues, indexes, and abstracts. They range from general to subject-specific, and are pervasive in nature and use. A web search engine is a web-based information retrieval tool designed to search for information in various formats on the World Wide Web (WWW) and File Transfer Protocol (FTP) servers. The search outputs are usually displayed in a list of results and are often called hits, the format of the output information may consist of web pages, images, videos, links, and other types of file formats. Some web search engines also mine data available in databases or open directories. Unlike web directories, which are maintained by human editors, web search engines operate automatically under the control of some set of algorithms or may be a mixture of algorithmic and human input (Jadhav, Gupta, & Pawar, 2011).
Web search engines are one of the most widely used methods for navigating in cyberspace. According to Jain (2013), studies have shown that more than 80% of first time visit to a website are navigated from a web search engine' result page and most web search engine users do not usually go beyond the first result page of a search engine. Hence, the navigational links available from a good web search engine, it is like having a guide book and a road map to your destination (Jadhav et al., 2011). It should be added that having your website displayed at the top of a web search engine's result page is very important to the flow of visitors/customers to the website (Manral & Hossain, 2012), which is why search engine's optimisation is important in website development. Moreover, it is the navigational query model/types that displays navigational links in the search engines result page leading to the particular website the user has in mind. Broadly speaking, there are three types of query models/types or query intent in web search engine's studies, these are: informational, navigational and transactional (Gabbert, 2018; Kang, 2005; Anthony, 2016) asserted that the categories of these query types were formally issued in 2002 in a peer-reviewed journal by Andrei Broder who worked for Altavista web search engines at the time. Furthermore, Google categorised these queries as: Know (Informational), Go (Navigational), and Do (Transactional), respectively. The web search engine works accordingly to each query model in order to provide appropriate answers to users. If an informational query is submitted into a web search engine, it is expected to provide informational results (resources), likewise for navigational and transactional queries. The studies focus on the navigational query model.
Navigational queries aim at locating a particular web page which might, or might not be known to the user. According to Gabbert (2018) and Anthony (2016), navigational query models are search queries entered with the aim of finding or locating a specific website or webpage on the Internet. For example, users may enter 'IFLA' into the search engine's text box to find the website of IFLA (International Federation of Library Association), rather than entering the IFLA's web address directly into the web browser's address bar. Queries such as “Where is the site of Nigerian Library Association?” or “Nigerian Library Association” are navigational queries, their goal is finding the website of Nigerian Library Association (Kang, 2005). Furthermore, navigational queries have very clear purpose – the user has an exact site in mind and if it is not that site, then the results are not relevant to his needs. University of West Georgia (2018) further asserted that navigational queries are used with the primary intent of surfing directly to a specific website. Apart from satisfying users' information needs, web search engines are navigational tools used to traverse the web, hence, they can take users to specific Uniform Resource Locators (URLs) or to aid in browsing (Jansen, Booth & Spink, 2007).
It should be added that navigational queries are quite easy to identify, especially those queries containing portions of URLs or even complete URLs. Company and organizational names can also be classified under navigation queries, assuming that the user intended to go to the website of that company or organization. In addition, most navigation queries are short in length and occurred at the beginning of the user session. Jansenet al. (2007) outlined some of the characteristics of navigational queries to include: queries containing company/business/organization/people names, queries containing domains suffixes, queries with “web” as the source, queries length (i.e., number of terms in query) less than three, though not always, searcher does not go beyond the first results page.
Anthony (2016) presented a model that captures the informational, navigational, and transactional query types as shown in Figure 1.
The categorisation of the three query models in Figure 1 allow users to understand query intent, and determine the appropriate result page that would be retrieved for each categor y of query used. More importantly, the transactional and informational queries in Figure 1 are segmented into two: the Web and the Intelligent Personal Assistant (IPA), whereas the navigational query is unsegmented. This is because it does not really change in a very significant way (since the user is just looking for certain destination) (Anthony, 2016). When considering the Intelligent Personal Assistant (IPA), is where a vast amount of 'Know-Simple' queries exist that will lead the user directly to the answer to his question. The 'Know-Simple' queries are for short, relatively factual answers to searchers; for example: How old is Goodluck Jonathan? or what year did Nigeria got her independence? Search engines like Google can answer from their own Intelligent Personal Assistant (IPA), as it is something that changes rarely. However, there are other search queries that fall into the Informational/Intelligent Personal Assistant (IPA) category; for example: Are computer accessories cheaper in Lagos? This kind of queries where different users may want different types of information are not considered 'Know- Simple; queries. (Anthony, 2016).
Figure 1. Web Search Engines' Query Models (Source: Anthony, 2016)
The bedrock of library and information services is in its provision of adequate information resources to meet the needs of its clientele. As such, all the services offered in the library are intended to facilitate access and retrieval of information resources by the library clientele through the appropriate use of information retrieval tools installed by the library. The role of the library and information professional has been expanded beyond preservation and maintenance of resource to include appropriate use of technology such as computer systems with appropriate software installed (like web browsers, e-book readers, and electronic databases) to facilitate information searching and retrieval within the library and on the Internet (Haliso & Aina, 2012). Today, links to various educational sites, web search engines, and electronic databases are provided to library clientele to access and download, e-books, e-journals/articles, and various other electronic resources available online. Some are interdisciplinary while others are subject specific.
Although web search engines are generally known for searching and retrieving information online, the usefulness of a web search engine depends on the relevance of the results it gives back (Jadhav et al., 2011). Other parameters for judging the usefulness of web search engines include, the appropriate hyperlinks(s) leading to the content of the search results or location to a particular site – navigational queries. It has been observed that some of the 'useful' links provided by web search engines lead to a dead-end, that is, either the Uniform Resource Locator (URL) is no longer available (page not found error) or the URL will redirect you to some unwanted documents not related to what was presented on the web Search Engine's Result Page (SERP). This often occurs when the webmaster of a website shuts down, removes or updates the original link to the wanted site/document and the web search engine fails to update/remove it from the content of its index/database to accommodate such changes (Lewandowski, 2011). Moreover, web search engines are primarily designed to be used by users and information seekers, however, if the users find it difficult to use them then the web search engines have failed in their usability performance. Web search engines are expected to be efficient, effective, and user friendly in order to satisfy the needs of the users, that is, a new user should not find it difficult to use and learn from it. Also, they should be designed in a simple (less complex) manner with rich interfaces to adequately respond to users' needs, hence the need for usability evaluation of web search engines in order to ascertain their usability performances by specified users to achieve specified goals in a specified context of use (ISO 9241-11, in Joo, Lin, & Lu, 2011).
The main objective of this study is to carry out a usability evaluation of Ask.com, Bing, Excite, Google, and Yahoo! web search engines. However, the specific objectives of the study is to:
The following research question guided the study:
This study formulated and tested the following null hypothesis at 0.05 level of significance:
Ho1: There is no significant difference in usability effectiveness between Ask.com, Bing, Excite, Google, and Yahoo! using navigational query models.
This study will be of great importance to students, educators, librarians, researchers, and web search engines' developers. The study will enlighten students, educators, librarians, and researchers on the appropriate web search engines to use when searching for relevant information or locating a particular site online to accomplish their academic engagements/ assignments. In essence, this study will reveal the best web search engines among those selected for this study, to improve the quality of information at their disposal, and by extension improve the quality of their research works. Lastly, the study will provide deeper understanding to search engines' developers/managers on the significance of usability performance of web search engines based on its navigational query model, so that they can put it into consideration when designing or upgrading their search engine.
The research design used for the study was quasi- (laboratory) experiment (Macin, 2018). The population for the study comprised of five web search engines itemized in Table 1.
Table 1. Study Population of Web Search Engines (Source: Authors’ Concept)
The rationale behind the use of these five web search engines for this study was because they are among the first 10 popular, most widely used and best search engines as reported by Dwyer (2016) and Ratcliff (2016). Furthermore, Jeyashree and Ravichandran (2013) believed that web search engines, such as Bing, Google, and Yahoo! can provide information for deep research work and give researchers unrestricted access to their hyperlinks structure plus large amounts of data regarding those links. Ask.com and Excite web search engines do the same. In addition, all the five web search engines have a percentage of the market share of web search engines worldwide as represented in Figure 2.
Figure 2. Market Share of Web Search Engines (Source: Ratcliff, 2016)
Twenty one observation participants (experimenters) were selected to conduct the experiment from master degree students in the Department of Library and Information Technology, Federal University of Technology Minna, Nigeria. The sampling techniques used were stratified and purposive sampling techniques. The rationale behind this selection was because the queries were drawn form Library and Information Services, hence the master degree students in the department will be in the better position to conduct the experiment and judge the performances of the various web search engines as the record their observations (Lewandowski, 2012).
In addition, the observation participants (experimenters) were first trained on how to conduct the usability experiment and were adviced and encouraged to avoid biases of any kind, especially with respect to frequency and previous use of web search engine(s). The computer laboratory used for the experiment was the departmental laboratory, Library and Information Science Department, School of ICT, Federal University of Technology Minna, Nigeria.
Moreover, Joo et al.’s (2011) usability evaluation model was adapted and used for the experiment, however, only effectiveness metric was implemented. The queries outlined in Table 2, were tested in all the web search engines for this study, and their usability performances were measure based on the usability effectiveness constructs indicated in Table 3.
Table 2. Library and Information Service Queries for Testing Navigational Query Model
Table 3. Usability Evaluation Metrics (Source: Jeo et al. 2011)
The research instrument (observation template) was adapted from Joo et al.’s (2011) usability evaluation model and modified to suite this research work. Consequently, the reliability of the research instruments was tested by conducting a pilot study to determine the reliability coefficient and internal consistency of the instrument. The pilot study was done using master degree students in the Department of Library and Information Science, Federal University of Agriculture Makurdi, Benue State, Nigeria. The results from the pilot study of the instrument was analysed using Cronbach Alpha Coefficient, and the value of the Alpha Coefficient was 0.979, which showed that the research instrument was excellently reliable.
What is the usability effectiveness of web search engines using navigational query model?
Table 4 presents the results of the usability effectiveness of the selected search engines using navigational query model.
Table 4. Usability Effectiveness of Web Search Engines using Navigational Query Model
The mean of the group distribution for the navigational query model is calculated using the formula:
where
f1 = frequency for SA (Strongly Agree)
f2 = frequency for A (Agree)
f3 = frequency for D (Disagree)
f4 = frequency for SD (Strongly Disagree)
N = total number of responses for a particular web search engine for the navigational query model.
The usability effectiveness of the web search engines using navigational query model is presented in Table 4 and Figure 3. The results of the usability effectiveness using navigational query model showed that Google ranked 1st, with a mean value of 3.59. Yahoo! ranked 2nd with a mean value of 3.50, followed by Ask.com with a mean value of 3.29. Bing ranked 4th with a mean value of 3.01, and lastly Excite ranked 5th with a mean value of 2.73.
Again, the results revealed that the mean responses to the usability effectiveness of the web search engines using navigational query model was highest for Strongly Agree (273), followed by Agree (227.6), Disagree (122), and lastly Strongly Disagree (7.4). This showed that for the usability evaluation constructs/metrics itemized in Table 3, from efy1 – efy6: users can complete a navigational search task using the search engines, users are successful in general in finding navigational links/resource(s) using the search engines, the search engines are useful in helping users find their navigational needs, users can achieve what they want using the search engines, the navigational links/resources obtained by users from the search engines are usually useful, and the search engines usually covers sufficient navigational links they try to explore. Moreover, in experimenting these usability effectiveness constructs using navigational query model, Google ranked 1st, followed by Yahoo!, Ask.com, Bing, and lastly Excite, for the five navigational queries drawn from library and information services.
From Table 4 and Figures 3, 4, 5, 6, 7, and 8, the responses of the distribution for navigational query model showed that Google was highly skewed to the right while Yahoo! was moderately skewed to the right. Ask.com and Excite were moderately skewed to the left while Bing was highly skewed to the left.
Figure 3. Level of Agreement with the Usability Effectiveness of Web Search Engines using Navigational Query Model
Figure 4. Skewed Graph of Ask.com for Navigational Query Model
Figure 5. Skewed Graph of Yahoo! for Navigational Query Model
Figure 6. Skewed Graph of Excite for Navigational Query Model
Figure 7. Skewed Graph of Bing for Navigational Query Model
Figure 8. Skewed Graph of Google for Navigational Query Model
Figure 9 shows the chart of mean values and standard deviation of each web search engine using navigational query model, from Strongly Agree to Strongly Disagree respectively. The results revealed that the standard deviation of all the web search engines using navigational query model are low, indicating that the data points from Strongly Agree to Strongly Disagree tend to be close to the mean. In addition, it shows that the data points are not spread out over a wider range of values. Here, Google and Yahoo! has the same standard deviation of 0.62, Ask.com has 0.79, and Bing has 0.82, while Excite has the highest, 0.78.
Figure 9. Mean and Standard Deviation of Search Engines' Navigational Query Model
Ho1: There is no significant difference in usability effectiveness between search engines using navigational query model.
The Friedman's ANOVA by rank is given by:
where:
k = number of categories/groups in the distribution
n = number of responses/values in each category/group
df= degree of freedom = k-1
R = summation of the ranks in each category/group
i = 1, 2,…k
Satisfied condition for using Friedman's ANOVA test:
Table 5 shows the responses of the distribution for navigational query model, from Strongly Agree to Strongly Disagree across the five web search engines selected for the study.
Table 5. Responses from Usability Effectiveness between Web Search Engines using Navigational Query Models
Table 6 shows the rank of the responses of all the search engines in ascending order, group by group, and the sum of ranks.
Table 6. Rank of Responses from Usability Effectiveness between Web Search Engines using Navigational Query Models
Using the Friedman's ANOVA by rank equation (2), with an alpha (α) value of 0.05, and df of 4,
Table 7 shows the results of the Friedman's ANOVA by rank test statistics.
Table 7 shows the results of the computation of Friedman's ANOVA by rank test using navigational query model, which revealed that the hypothesis was not significant (p-value = 0.986361999, k-value = 0.35 and critical value (X2) = 9.487729), at an alpha (α) value of 0.05. Hence, the null hypothesis (Ho1) is retained. The results proved that there is no significant difference in usability effectiveness between search engines using navigational query model, as reflected in their mean values except for Excite which had a lower score.
Table 7. Results of Friedman's ANOVA by Rank Test Statistics on Usability Effectiveness between Web Search Engines using Navigational Query Model
The responses of the respondents as to the usability effectiveness of the web search engines using navigational query model was presented in Table 2. The result showed that Google ranked 1st, with a mean score of 3.59. Yahoo! ranked 2nd with a mean score of 3.50, followed by Ask.com with a mean value of 3.29. Bing ranked 4th with a mean score of 3.01, and lastly Excite ranked 5th with a mean score of 2.73. This result tallies with the research of Lewandowski (2015) on evaluating the retrieval effectiveness of web search engines using a representative query sample. For the two web search engine used for the study: Google and Bing, Google rank first with an accuracy of 95.3% on finding navigational results, Bing however had an accuracy of 76.6%. Furthermore, results of the study corresponds with the work of Lewandowski (2011) on evaluating the retrieval effectiveness of search engines on navigational queries. The web search engines used for his study were Google, Yahoo, MSN, Ask, Seekport, and Exalead. His study reported that the performance of Google, Yahoo, and MSN search engines is best, with around 90% of queries answered correctly, while Ask and Exalead perform worse, but receive good scores notwithstanding. In a nutshell, web search engine's usability performance on navigational queries is of great importance because users can clearly identify queries that have returned correct results.
From the facts emanating from this study, it is clear that web search engines' usability performance with respect to their navigational query models, affects their users' visit or patronage to order website and by extension their ranking. Hence, the following recommendation was made:
This study adapted six constructs of the eighteen outlined by Joo et al. (2011) in their usability evaluation model. Future scope of this study should adapt/adopt the entire eighteen constructs accordingly. In addition, further studies of this nature should lay more emphasis on classification and identification of search engines' query types produced by each search engine's results page. That is, since navigational queries are expected to produce navigational results, the search result of each of the query submitted should be analysed to determine the percentage of navigational results and the inclusion of unwanted results (informational and transactional) in order to judge the level of accuracy of the results with respect to navigational queries – this will help to ascertain the accuracy of the search engine's query structure/ model.
From the foregoing, there is no significant difference in usability effectiveness between web search engines using navigational query models. i.e, all the web search engines performed well. Specifically, Google has the highest performance on usability effectiveness using navigational query model, as reflected in its mean score. Yahoo! ranked second in usability performance, the third is Ask.com, Bing, and lastly Excite which had a lower score.