SEARCH

SEARCH BY CITATION

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References

Usability in general and web usability in particular are complex concepts with multiple dimensions and elements/attributes. What is more, there is no comprehensive definition that takes into account the varying aspects of usability. This leads to discrepancies in policies/standards/guidelines which facilitate good usability practices. To initially address this problem, we took the first step of systematically analyzing the terms represented in formally published definitions from a number of sources in the subject areas of library & information science and computer science/information systems. Our results reveal that there are approximately 11 elements/attributes of usability and web usability addressed within these sources. The top five attributes are: learnability, effectiveness, user-related attributes, users' attitude/satisfaction, and efficiency. However, other critical usability attributes received little to no attention in publications of either discipline.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References

Better usability and Web usability (thereafter referred to as usability) benefit both the user and the organization responsible for a system/Web site's content, design, and maintenance. A clear, all-encompassing definition of usability is instrumental in helping stakeholders of a system and/or Web environment to gain a consistent understanding of this concept and for crafting well-rounded, sound usability principles, policies, standards or guidelines for system design and Web development.

To establish, maintain, and promote user-friendly interfaces and Web pages, usability principles based on clearly defined terms are essential. As a complex concept with multiple facets/dimensions, usability is not easy to define. The two most frequently cited definitions are those by the International Standards Organization (ISO, 1994) and Nielsen (1993). According to Nielsen (1993), usability of a system is multi-faceted and consists of five properties: easy to learn; efficient to use; easy to remember; low error rate; and meets user satisfaction. ISO defines usability as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use” (1994, p. 10). In addition, many other published definitions do include additional elements/attributes necessary for improved usability. Yet there is no definitive definition of usability that is consistently accepted by researchers and professionals such as information architects, Web designers, and others.

“In technical writing, clear and accurate definitions are critical” (Alred, Brusaw, & Oliu, 2000, p.155) and a concise definition is vital due to the nature of usability, which is potentially affected by technical/system and human factors. Yet the literature points to difficulties: “defining usability is complicated” (Guenther, 2003, p. 65); “the term has been used so often in so many different contexts, it is in danger of losing its precise meaning” (Pack, 2003, p. 36). Gray & Salzman (1998) noted that the term does not have a precise enough definition. The varying perspectives on the attributes of the term may help to explain why there is so much difficulty with producing functional, effective computer systems and Web sites that address user needs. This may also shed some light on why there is little written documentation in the form of policies/standards/guidelines (Chen, Germain, & Yang, 2009). A consequence with the lack of a more authoritative definition for usability is the inability to adequately design for it. Without an overarching, distinctly defined structure it impedes the ability to implement a consistently usable Web environment.

Policies, guidelines, and standards are drafted regularly in government, as well as by academic, commercial, and governing institutions. In the library and information science field, policies, guidelines, and/or standards have been created by professional organizations to promote uniformity to enable good practice. These normally include clearly articulated definitions of major terms within the document which help to provide continuity and understanding. “To define something is to clearly identify precisely its fundamental qualities” (Alred, Brusaw, & Oliu, 2003, p.142). Well-constructed definitions give readers full knowledge of a word or phrase, leaving little or no room for ambiguity. By utilizing strong, authoritative definitions within policies, guidelines, and/or standards, users of those documents will have a consistent understanding of a term's meaning and thus can develop and produce uniform outcomes. One example in library & information science is the term ‘information literacy.’ Most documents pertaining to this process, including the Association of College and Research Libraries (ACRL) competency standards for information literacy in higher education (ACRL Task Force on Information Literacy Competency Standards, 2000), use the American Library Association's definition of information literacy from American Library Association's Presidential Committee on Information Literacy's Final Report (1989). This definition is referenced in most information literacy-based policies, guidelines, and/or standards, and as such is the seminal definition for the term. This ensures consistent instruction practice and thus a uniform learning outcome.

In the efforts to help determine a comprehensive definition for usability, we initiated this study. In this research initiative, we analyzed previously published definitions of usability in the library & information science and computer science/information systems literature. Our objectives were to:

  • identify attributes within the definitions
  • compare attributes used among subject areas
  • determine the change in attribute emphasis over time

The results of this study could eventually assist researchers and practitioners in constructing a functional working definition of usability incorporating key elements/attributes critical to both the users and the information systems.

Literature Review

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References

Usability predates Web usability by over a decade. In 1984, Eason indicated that although the concept of usability had increasingly played an important role in human computer interaction, it had not been well-defined and there was no universally agreed method of measurement. Shackel (1986) presented an operational definition of usability that allows a system to be evaluated throughout the development life cycle. According to Shackel (1986), for any system to be usable, it must allow users to accomplish a range of required tasks and pass four criteria: effectiveness, learnability, flexibility, and attitude. Booth (1989) thought it difficult to specify and measure flexibility of a system and believed that being useful should be fundamental to usability, thus he modified Shackel's criteria into usefulness, effectiveness, learnability (or ease of use), and attitude (or likeability). Campbell and Aucoin (2003) explicitly stated that “usability refers to the relationships between tools and their users… is the quality of a system that makes it easy to learn, easy to use, easy to remember, error tolerant and subjectively pleasing” (p. 172). As noted earlier, Nielsen (1993), a usability expert, offered a similar operational definition in that usability of a system is multi-faceted and consists of five properties: easy to learn; efficient to use; easy to remember; low error rate with zero catastrophic error; as well as user satisfaction. The ISO (1994) views usability as the way in which a system can be used by target users to accomplish specified tasks and achieve a degree of effectiveness, efficiency, and satisfaction. In a Web environment, this means that an interface must be easy to learn, remember, and use, with few errors for its target users and the specific tasks it is designed to support (Battleson, Booth, & Weintrop, 2001). Different from the conceptualization integrated in the ISO standard, Lecerof and Paterno (1998) provided a broader definition addressing relevance of a system to users' needs, efficiency, users' subjective feelings, learnability, and a system's safety feature, such as granting users the right to undo actions that may lead to errors. From their perspective, functionality is essential to usability.

Dumas and Redish (1993) noted that a system with functions does not necessarily have usability. They asserted that “usability means that the people who use the product can do so quickly and easily to accomplish their own tasks” and focused on four main points: users, productivity, tasks, and ease of use (p. 4). Rubin (1994) echoed Eason's (1984) viewpoint acknowledging the lack of a universal usability definition and expressed that definitions with one or more of the four components outlined by Booth (1989) are generally accepted by the usability community. Applied to a Web environment, Web usability consists of learnability, throughput, and flexibility of a Web site, as well as a user's attitude towards it (Preece, Rogers, Sharp, Benyon, Holland, & Carey 1994; Sicilia & Garcia, 2003). In addition, Rubin (1994) advocated user-centered design (UCD), also known as usability engineering, which is reflected in Nielsen's (2000) Designing Web Usability, focusing on the strategy of positioning users' needs at the center of a Web design.

The notion of human-computer interaction extends to Web technology. Web usability refers to a user's experience in reading or interacting with a Web site (Brophy & Craven, 2007; Hudson, 2001). Microsoft regards usability as strategies for catching users' reactions to a system and incorporating them into the various development stages (Veldof, Prasse, & Mills, 1999). Palmer (2002) stated that usability consists of Web site consistency, ease of navigation for task performance, clarity of interaction, ease of reading, information organization, speed and layout. Krug (2006) looked at usability from the user's perspective with the need for an intuitive experience. He emphasized the development of intuitive-based Web sites so average users can easily navigate for needed information without struggle. Similarly, Dee and Allen (2006) noted that when an end-user interface is easy to use and intuitive, it conforms to usability principles. However, Head (1999) indicated that an easy-to-use interface is the face value of usability: the core value of “usability is rooted in cognitive science - the study of how people perceive and process information through learning, the use of memory, and attention” (p. 22). Some scholars upheld the same perspective and have regarded information as a central theme of Web usability and stressed that a more usable site helps its users find their needed information successfully (Keevil, 1998; Spool, Scanlon, Schroeder, Snyder, & DeAngelo, 1999).

Methodology

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References

We believe that a more usable, more successful Web site/system is achieved by designers/developers who are working with a set of consistent guidelines based on a clearly defined concept of usability. Publications emphasize authority. Thus we undertook this study to further explore published definitions of usability. Between December 2008 and April 2009, we searched two citation index databases, Scopus and Web of Science, for specific publications with usability definitions published prior to 2009. Using the 2007 ISI Journal Citation Reports, we selected 56 journal titles listed in the Information Science & Library Science category in the Social Science edition and 92 journal titles under the Computer Science, Information Systems category in the Science edition. Fifteen journals overlapped the two subject lists. We made this choice because journals listed in these reports are prominent publications which are evaluated by a set of consistent criteria.

For the searches, each journal title was entered into the “source title” field for Scopus and the “title” field for Web of Science. In addition, we combined the title search with the phrase “web usability” in the “titles, abstracts, and keywords” option in Scopus and the “topic” field in Web of Science. Searches through the two citation databases ensure that sources from the two subject areas were more fully represented in our sample. The searches retrieved a total of 440 records. Individually we looked through the 440 articles to identify formal definitions of ‘usability’ or ‘web usability.’ Thirty-six contained definitions of ‘usability’ or ‘web usability.’ From the articles that contain definitions cited from other sources, we located those cited sources and included them in the data set. In total, 63 formal definitions were compiled and constitute the core data for this work.

The formal definitions from these publications were content analyzed following Weber's (1990) standard content analysis procedures. Initially, we individually extracted the key terms from each definition and used them as our recording units. Then together we identified concept patterns among the terms and grouped similar terms into concept categories of usability attributes; for example, users, target audiences, and people were all placed in the ‘users’ category. The 63 formal definitions produced a total of 502 terms/concepts which fell under 11 categories. A majority of these attributes are identified in the literature (e.g., Jeng, 2005). We also included a less referenced attribute - interface/design, because we believe that information systems' developers should go beyond mechanical specifications, such as font size, color, etc., and focus on site navigation, for example, information architecture. In addition, we created two attributes: ‘user characteristics’ (e.g., type of user, level/experience of user, demographic information, etc.) and ‘context/purpose’ (e.g., context and environment of use, purpose of use, etc.). We added the user characteristics category because we wanted to emphasize the user-centered concept and show that it is an important element/attribute of usability specifically addressed by the formal definitions. Frequently, Web developers design systems that require prior knowledge from users; they expect users to know the system instead of designing a system that fits general users' mental model. As Head (1999) noted, the cognitive aspect is important because systems that provide cognitive cues, such as metaphors, add the feeling of familiarity to users and thus increase intuitiveness. This is especially important when a task is not just fact-finding, but in-depth research. The ‘context/purpose’ category mainly addresses the physical, social and cultural environments which play an important role in how well a system works within a particular setting (Bevan, 1997; Gonzalez, Lores, & Granollers, 2007; Hillier, 2003). This constitutes our coding scheme. We coded the entire data set separately. All differences in coding between the authors were resolved through discussions until 100% agreement was reached.

Table 1. Attributes of usability and their descriptions
Thumbnail image of

In addition to our coding, one graduate student, blind to the purposes of the work, coded 80% of the set of terms. In order to ascertain coding consistency (reliability), two popular measures of intercoder agreement, percent agreement and Cohen's (1960) Kappa, were used to compare the results of the authors and those of the graduate student. The computed percent agreement and Cohen's (1960) Kappa values were 0.76 and 0.72, respectively. These are above the often-cited threshold of 0.70 (Neuendorf, 2002). Hence, the coding procedure was deemed to be reliable. The attributes together with their descriptions are presented in Table 1.

Results and Discussions

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References

We analyzed the 63 sources of formal definitions and the terms/concepts used in framing them to identify the common elements/attributes emphasized in the literature definitions. These will eventually point toward a formulation of a holistic definition that takes critical elements/attributes into account. We found that the 63 sources of formal definitions were evenly distributed across the subject areas of library & information science, computer science/information systems, and both computer science/information systems and library & information science (i.e., journal titles that overlapped the two disciplines identified in the ISI reports) (see Figure 1).

thumbnail image

Figure 1. Number of sources of formal definitions of usability and web usability by subject area

Download figure to PowerPoint

The 63 publications were published between 1989 and 2008. With regard to the distribution of the sources' publication year, a little over half (54%) were published over the 15-year period, 1989 to 2003. We analyzed the distribution of the data in five-year intervals and found that four publications (6.3%) were published between 1989 and 1993; seven (11.1%) between 1994 and 1998; 23 (36.5%) between 1999 and 2003; and 29 (46%) between 2004 and 2008. We observed a growing trend in publications including usability definitions (see Figure 2). A comparison of the source numbers between the first 10 years (17.4%) and the second 10 years (82.6%) revealed a dramatic increase of 475%. We interpret this as an indication of researchers and authors paying closer attention to the subject of usability especially in light of the emergence of the World Wide Web, its popularity among users, and the importance of creating resources which follow usability principles.

thumbnail image

Figure 2. Number of sources with formal usability/Web usability definitions by year of publication (n=63)

Download figure to PowerPoint

Table 2. Frequency of terms used in formal definitions by usability attributes and subject area of source
Thumbnail image of

We examined the attributes of the definitions and their relationship to their respective published subject areas. Results of our content analysis (coding) of the significant terms/concepts used in the formal definitions categorized according to the subject areas of the sources (Table 2) reveal that the top five most emphasized elements/attributes of usability are learnability, effectiveness, user characteristics, users' attitude/satisfaction, and efficiency, regardless of the subject area of the sources. These five elements/attributes (about half of the 11 attributes) account for approximately 77% of the total number of concepts used in the definitions pointing to their importance for researchers and authors. However, the distribution of the top three attributes varied across the disciplines: learnability and user characteristics were mentioned more frequently in Library & Information Science (LIS) than in Computer Science (CS) and the Overlap (O). This indicates that the LIS literature focuses more on the user side of usability than CS which highlights the systems perspective. Nevertheless, while the top five attributes are highly important in usability definitions, it would be expected that the other attributes, such as context/purpose, interface/design, low error rate/error tolerance, and memorability/retainability, would also be viewed as equally important and mentioned as close to or as frequently in the literature. Our results show that many of these received little attention.

Table 3. Number of sources by usability attributes and subject area (n=63)
Thumbnail image of

A review of the top five usability attributes emphasized by both the terms/concepts in the definitions and the sources (or definitions as a whole) reveals an identical distribution pattern to the above analysis: learnability, effectiveness, user characteristics, users' attitude/satisfaction, and efficiency (see Table 3). A closer look at the data shows that the priority orders of the attributes are again slightly different for each of the three subject areas. While effectiveness is the most emphasized usability attribute in the CS and overlapping sources (computer science/information systems and library & information science); learnability is the top attribute for the LIS sources. We were surprised that some attributes, such as memorability/retainability and error rate, have not been consistently emphasized, especially in the CS literature. The ‘Other’ category included terms that did not fit into the 11 attribute categories.

In order to further ascertain the above findings, we conducted a one-way analysis of variance (one-way ANOVA) on the single factor of subject area of source mainly to test the difference between the formal definitions from sources in the three subject areas with respect to usability attributes (Table 4). Results from the one-way ANOVA revealed that overall there was no significant difference among the sources. That is, the number of terms used in formal definitions of usability were not significantly different for all three subject areas with respect to their emphasis on the various usability attributes (F (2, 33) = 0.00144, p = 0.998). These results mirror our earlier findings; irrespective of the subject area of the sources, learnability, effectiveness, user-related attributes, users' attitude/satisfaction, and efficiency remain the prime focus of formal definitions of usability.

Table 4. Descriptive statistics for the number of terms used in formal definitions by usability attributes and subject area of source
Thumbnail image of

While there were no statistically significant differences between the sources in the subject areas with respect to the numbers of concepts used in the formal definitions, this is not entirely true when factoring in publication year. As shown in Table 5, those with overlapping subject areas began addressing usability in 1989; sources that are grouped as strictly computer science/information systems started in 1997; and purely library & information science sources emerged in 1999. The overlapping group predated the others since it includes seminal works by Booth, Dumas & Redish, and Neilsen. We believed that usability related terms appeared more frequently in computer science/information systems after 1997 due to the development of e-commerce, with LIS following shortly behind.

Table 5. Frequency of terms used in formal definitions by year of publication and subject area of source
Thumbnail image of

Given the recent history of the World Wide Web and Web usability as a fully fledged area of research and practice, we looked into whether those definitions in sources published before and after the year 2000 (an arbitrary year because it is about halfway between 1989 and 2008) differ in terms of their focus with respect to usability attributes. We conducted a paired t-test for each attribute and the results reveal no statistically significant differences for all the attributes, except for the ‘Other’ category (see Table 6). The terms in the ‘Other’ category referred to content. We found it interesting that between these two periods of time, more focus was on content in the first 10 years than the latter period. Since our sample is small in size, it is difficult to make any conclusive argument, though we recommend further exploration in this area because content plays an important role in information seeking.

Table 6. Mean number of terms used in formal definitions by usability attributes and year of publication of source
Thumbnail image of

Conclusions & Recommendations

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References

The main purpose of our paper was to provide some basic yet detailed background information on formally published usability definitions in the relevant literature and present the need for a holistic definition. In addition, this effort lays the groundwork for connecting theoretical grounding of usability with its factors/elements/attributes. We hope that this will assist future researchers and practitioners in their efforts to build quality user-centered systems and Web sites based on better policies/standards/guidelines which are guided by a uniformed usability definition. This can be utilized to create a standard of measuring the usability of a system and/or Web environment.

A review of the select LIS and CS/Information Systems literature produced 63 sources which contained 502 terms. We identified attributes for usability and coded the definition terms to determine concept patterns. Of the 11 attributes, we found that the top five mentioned attributes appeared in over 60% of the definitions while the bottom five appeared in less than 25% of the definitions. Even though some of the less referenced attributes (e.g., memorability/retainability) are highlighted in seminal works, such as Nielsen's, we believe that all of the attributes are equally important and need to be included in a holistic definition, so that essential properties of usability can be fully addressed by practitioners.

While some attributes were noted more frequently in certain subject areas, there was no statistically significant difference between the CS/Information Systems and the LIS literature. Surprisingly, we found a similar result when we analyzed the data by time frame. We had expected a difference between the two decades due to the evolution of Web technology and use. A closer analysis of the timeline may produce a different outcome.

Limitations of our work include the likelihood of missing certain formally published definitions of usability. Since we utilized the ISI Journal Citation Reports to initiate our selection of sources for usability definitions, many excluded publications may include some of the more practical, rather than theoretical, sources on the topic. Including other areas besides CS/Information Systems and LIS, such as business, may present a different perspective on usability (e.g., more focus on user satisfaction).

Therefore, with respect to usability and its definition, future research needs to consider a wider range of sources and disciplines that deal with usability. Additionally, future researchers should go beyond formally published literature and investigate how practitioners view and define these concepts. Findings from such research can be compared to the results of this study and used to bridge the gap in the basic understanding and meaning of usability that exists between researchers and professionals such as system designers, information architects, and others. A common and generally accepted definition will go a long way in helping craft appropriate and applicable usability policies/standards/guidelines for developing truly user-centered information systems and Web sites

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Literature Review
  5. Methodology
  6. Results and Discussions
  7. Conclusions & Recommendations
  8. References
  • ACRL Task Force on Information Literacy Competency Standards. (2000). Information Literacy: Competency Standards for Higher Education. College & Research Libraries News, 61(3), 207215.
  • Alred, G. J.Brusaw, C. T.&Oliu, W. E.(2000). Handbook of technical writing. 6th ed. New York: Macmillan.
  • Alred, G. J.Brusaw, C. T.&Oliu, W. E.(2003). Business writer-s handbook. 7th ed. New York: Macmillan.
  • American Library Association. (1989). Presidential Committee on Information Literacy. Final Report. Chicago: American Library Association.
  • Battleson, B.Booth, A.&Weintrop, J.(2001). Usability testing of an academic library Web site: A case study. The Journal of Academic Librarianship, 27(3), 188198.
  • Bevan, N.Azuma, M.(1997). Quality in use: Incorporating human factors into the software engineering lifecycle. Proceedings of the 3rd International Software Engineering Standards Symposium and Forum (ISESS): Emerging international standards, pp. 169179. Los Alamitos, CA: IEEE Computer Society.
  • Booth, P.(1989). An introduction to human-computer interaction. Hillsdale, USA: Lawrence Erlbaum Associates Publishers.
  • Brophy, P.&Craven, J.(2007). Web accessibility. Library Trends, 55(4), 950972.
  • Campbell, K.&Aucoin, R.(2003). Value-based design of learning portals as new academic spaces, In: Jafari, A. and M.Sheehan (Eds.), Designing Portals: Opportunities and Challenges (pp. 162185). Hershey, PA : IRM Press.
  • Chen, YGermain, C. A. & Yang, H.(2009). An exploration into the practices of library Web usability in ARL academic libraries. Journal of the American Society for Information Science and Technology, 60(5), 953968.
  • Cohen, J.(1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 3746.
  • Dee, C.&Allen, M.(2006). A Survey of the Usability of Digital Reference Services on Academic Health Science Library Web Sites. The Journal of Academic Librarianship, 32(1), 6978.
  • Dumas, J. S.&Redish, J. C.(1993). A practical guide to usability testing. Norwood, N.J.: Ablex.
  • Eason (1984). Towards the experimental study of usability. Behaviour and Information Technology, 3(2), 133143.
  • Gonzalez, Lores&Granollers(2008). Enhancing usability testing through datamining techniques: A novel approach to detecting usability problem patterns for a context of use. Information and Software Technology, 50(6), 547568.
  • Gray, W. & Salzman, M.(1998). Damaged Merchandise? A review of experiments that compare usability evaluations methods. Human-Computer Interaction, 13, 203261.
  • Guenther, K.(2003). Assessing Web site usability. Online, 27(2), 6568.
  • Head, A.(1999). Web redemption and the promise of usability. Online, 23(6), 2032.
  • Hillier, M.(2003). The role of cultural context in multilingual website usability. Electronic Commerce Research and Applications, 2(1), 214.
  • Hudson, L.(2001). From theory to (virtual) reality. Library Journal, 126(11), 1215.
  • International Standards Organization (ISO). (1994). Ergonomic requirements for office work with visual display terminals. Part 11: Guidance on usability (ISO DIS 9241-11). London: International Standards Organization.
  • Institute for Scientific Information (ISI). (2008). Journal citation reports on microfiche (Science ed.). Philadelphia, PA: Institute for Scientific Information.
  • Institute for Scientific Information (ISI). (2008). Journal citation reports on microfiche (Social Science ed.). Philadelphia, PA: Institute for Scientific Information.
  • Jeng, J.(2005). What is usability in the context of the digital library and how can it be measured? Information Technology and Libraries, 24(2), 4756.
  • Keevil, B.(1998): Measuring the usability index of your Web site. In: CHI '98, Conference Proceedings on Human factors in Computing Systems. April 18-23, 1998, Los Angeles, CA. The Association for Computing Machinery, Special Interest Group on Computer Human Interaction, pp. 271277. New York, NY: ACM Press.
  • Krug, S.(2006). Don't make me think: A common sense approach to Web usability. Berkeley, CA: New Riders Publishing.
  • Lecerof, A.&Paterno, F.(1998). Automatic Support for Usability Evaluation. IEEE Transactions on Software Engineering, 24(10), 863888.
  • Neuendorf, K. A.(2002). The content analysis guidebook. Thousand Oaks, CA: Sage.
  • Nielsen, J.(1993). Usability engineering. Boston, MA: Academic Press.
  • Nielsen, J.(2000). Designing Web usability: The practice of simplicity. Indianapolis, IN: New Riders Publishing.
  • Pack, T.(2003). Fiddling with the Internet dials: Understanding usability. Online, 27(2), 3638.
  • Palmer, J. W.(2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2):151167.
  • Preece, J.Rogers, Y.Sharp, H.Benyon, D.Holland, S.&Carey, T.(1994). Human-Computer Interaction. Wokingham, UK: Addison-Wesley.
  • Rubin, J.(1994). Handbook of usability testing: How to plan, design, and conduct effective tests. New York: Wiley.
  • Shackel, B.(1986). Ergonomics in design for usability. In Harrison, M. D. and A. F. Monk (Ed.), Proceedings of the Second Conference of the British Computer Society Human Computer Interaction Specialist Group: People and Computers – Designing for Usability Cambridge. British Computer Society Human Computer Interaction Specialist Group, pp. 4464. New York: Cambridge University Press.
  • Sicilia, M. A.&Garcia, E.(2003). Modelling interacting Web usability criteria through fuzzy measures. Lecture Notes in Computer Science, 2722, 6988.
  • Spool, J. M.Scanlon, T.Schroeder, W.Snyder, C.&DeAngelo, T.(1999): Web site usability: A designer's guide. San Francisco, CA: Morgan Kaufmann Publishers.
  • Veldof, J. R.Prasse, M. J.&Mills, V. A.(1999). Chauffeured by the user: Usability in the electronic library. Journal of Library Administration, 26(3-4), 115140.
  • Weber, R. P.(1990). Basic content analysis. (2nd ed.). Newbury Park, CA.: Sage Publications.