Assessing the accessibility of Web 2.0 websites

Authors


Abstract

This study examines the accessibility of Web 2.0 websites to the visually impaired. Web accessibility standards are established to maximize the ability of those with impairments to navigate the web. Various computer tools exist to evaluate web HTML content against existing accessibility standards. Using a weighted metric-based formula called the Web Accessibility Barrier (WAB) score, this study adopted an experimental design and compared 88 randomly selected Web 2.0 websites against 88 randomly selected Web 1.0 websites. The study found that Web 2.0 websites are significantly less accessible than Web 1.0 websites. Details of the analysis, evaluation, discussion, and recommendations are included in the paper.

Introduction and Main Problem

Individuals who are visually impaired or impaired in other manners use various tools to browse web-based content. These tools rely heavily on well-designed HTML code. Web 2.0 is a popular trend in web design that takes full advantage of the social and interactive features of the web (Fetscherin & Lattemann, 2007). As Web 2.0 websites increase in popularity, it is necessary to assess their accessibility to the visually impaired.

The purpose of this study is to assess to what extent Web 2.0 websites are accessible to the visually impaired by using the Web Accessibility Barrier (WAB) score metric proposed by Parmanto and Zeng (2005). The WAB metric provides web developers with a picture of the extent to which his or her web code is compliant. Numerous research has been done on Web Accessibility or on Web 2.0 separately; however, research on the combined two are limited. From 1995 to 2007, little research used the WAB metric to measure Web 2.0 site accessibility (Thomson Scientific, 2007). As a result, there is little research examining the relationship between Web 2.0 and accessibility issues. This study, therefore, is unique in its use of the WAB score metric to compare the accessibility of Web 2.0 websites to Web 1.0 websites through a random sampling from both categories.

Background and Related Work

Web accessibility issues and measurements

The Web Content Accessibility Guidelines (WCAG), the Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C), and Section 508 legislation all seek to maximize the ability of those with impairments to navigate the web (W3C, 2007a; W3C, 2007b; Section 508, 2007). The WAI's purpose is to create strategies, resources, and guidelines to make the web accessible to people with disabilities (W3C, 2007a). WCAG are foundation guidelines of the W3C that explain how to make Web content accessible to the greatest extent possible for people with disabilities (W3C, 2007b). Section 508 is an extension of the 1998 Rehabilitation Act enacted by Congress to:

  • 1.Eliminate barriers in information technology
  • 2.Ensure availability of new opportunities for people with disabilities, and
  • 3.Encourage development of information technologies that will help achieve the goals of accessibility.

Section 508 applies to all federal agencies that develop, procure, maintain, or use electronic and information technologies (Section 508, 2007). Ideally, adherence to these standards and mandates enables accessibility tools to perform well. Web code compliance with WCAG 1.0, ideally will reduce the obstacles of those who use screen readers for accessibility purposes. This is due to the fact that WCAG 1.0 establishes best practices for web coding standards (W3C, 2007a; W3C, 2007b). WCAG 2.0 is now under development to accommodate the rapid development of newer web technologies (W3C, 2007b). These standards further are made by experts in the field (W3C, 2007a).

Several automated tools are available to assess compliance with WCAG 1.0 guidelines and WCAG 2.0 is presently under development (W3C, 2007b). Such accessibility tools include readers like the JAWS® screen reader made by Freedom Scientific and ZoomText Magnifier/Reader from Ai Squared (Theofanos and Redishm, 2003). Accordingly, it is often beneficial to evaluate a website's compliance with these standards by using accessibility evaluation software tools. Several accessibility evaluation software tools, such as Hi Software's Cynthia Validator and Watchfire's Bobby validator, exist to assess the adherence of HTML code to WCAG 1.0 standards. The output of such tools is very thorough and meaningful; however, these tools often give results on a pass/fail basis as they relate to the WCAG 1.0 checkpoints. Understandably, these tools tend to fail an entire page when there is even one violation of WCAG 1.0 guidelines. With the binary pass/fail results of such tools alone, however, it is difficult to gauge a website's overall compliance with WCAG 1.0 standards. Is the website for instance, 90% compliant, or 25% compliant? Accordingly, it would be beneficial to use a quantitative metric to assess web accessibility, and to obtain results that are more meaningful. Furthermore, a quantitative metric gives unique insight into the accessibility of Web 2.0 websites by providing a numerical score that can be used for comparison purposes.

Web Accessibility Barrier (WAB) Score

The Web Accessibility Barrier (WAB) score metric was proposed by Parmanto and Zeng (2005). It is a method that enables one to quantitatively identifying accessibility trends across the 2 studied groups: Web 1.0 websites and Web 2.0 websites. The WAB score formula tests 25 WCAG 1.0 criteria that can be evaluated automatically and is defined as follows:

equation image

Where p is the total pages of the website, v is the total violations of a web page, nv is the number of violations overall, Nv is the number of potential violations, wv is a weight value in inverse proportion to WCAG priority level, and Np is the total number of pages checked.

A 0 WABScore indicates a website that passes all 25 checkpoints. Any number above zero indicates a site moving further away from accessibility criteria. For wv, the inverse weight was applied for each error in Priority I, II, or III status. Priority I errors, for instance received a 3 weight, and Priority III errors received a 1 weight (Parmanto and Zeng, 2005). The advantage of the weighted average is that it does not fail an entire webpage due to one error. Errors are looked at as it relates to the actual errors and potential errors, thus weighing them in what seems to be a more fair metric. In other words, one has a more complete picture of the extent to which one's webpage is compliant.

Web 1.0 and Web 2.0 Website Characteristics

Although there are different arguments regarding the definitions and differences between Web 1.0 and Web 2.0 websites, several characteristics help generally distinguish Web 1.0 websites from Web 2.0 websites. A Web 1.0 website is characterized by simply html-driven, usually static text and flash (Fletscherin & Lattemann, 2007). Web 2.0 websites characteristically contain elements that allow users to contribute to the content. Users typically actively participate in and contribute to the content of such websites. Examples of Web 2.0 include Weblogs, Wikis (i.e. Wikipedia.org), Podcasts, RSS feeds, and other popular social networking and tagging websites such as MySpace.com, Facebook.com, Flickr.com, and the like (Fetscherin & Lattemann, 2007). Static html driven Web 1.0 websites usually do not contain a component that allows users to contribute to the website's content. The lack of this user contribution component is the major factor that distinguishes Web 1.0 websites from Web 2.0 websites. A blog website for instance, is different from a static informational website that does not allow users to post content.

The Study

Research Questions

This study examines accessibility of Web 2.0 websites and aims to answer the following questions:

  • 1.Does Web 2.0 website affiliation present challenges in web navigation for the visually impaired?
  • 2.Is the HTML code in common Web 2.0 applications compliant with web accessibility standards?
  • 3.Is there a correlation between Web 2.0 affiliation and navigability of a website?

Methodology

Since early websites were all static websites with no user contribution component, they can be categorized as Web 1.0. As such, there is a large pool of Web 1.0 websites from which to choose. For the purposes of this work, it was necessary to select a manageable sample of Web 1.0 websites. After analysis of several Web 1.0 groups, university websites emerged as a legitimate Web 1.0 source. Like federal agency websites, university websites tended to adhere well to Section 508 standards in their design and are more highly regulated than other Web 1.0 websites due to state guidelines and individual university guidelines (Parmanto and Zeng, 2005). Also, university websites tended to be slower in adopting Web 2.0 technologies when compared to other Web 1.0 website groups. The limited use of Web 2.0 technologies on university websites at the time of this work, and the perceived adherence of university websites to Section 508 regulations, made such websites an ideal Web 1.0 source.

In this study, a random sample of 88 U.S. university homepages is selected to form the category of Web 1.0 websites. The university list is based on the U.S. News and World Report (2007), which listed top schools for 2008. The extent of overall accessibility of this sample of top universities will be necessary for comparison purposes. In addition, a random sample of 88 Web 2.0 sites is selected from E-Consultant (2007) to form the category of Web 2.0 websites. E-Consultant (2007) listed various Web 2.0 websites by category. This particular website was selected due to the extensive work of categorization that was done on the part of the website authors. Table 1 summarizes the sampling:

Table 1. 
original image

A-priori analysis with the GPOWER tool for a t-test experiment comparing the two groups is conducted. Posttest Only Control Group experimental design with Web 2.0 affiliation as the treatment is utilized, with the WAB score as the method of observation. A controlled experimental design is used to minimize threats to the internal and external validity (Campbell and Stanely, 1963, p. 8). Accordingly, Posttest Only Control Group true experimental design as defined by Campbell and Stanley is adopted (1963, p. 8). T-test statistical analysis is conducted for an experiment comparing the average WAB scores of sample Web 2.0 websites to the WAB scores of the sample Web 1.0 websites. Table 2 summarizes the experimental design of the study:

Table 2. 
original image

It is important to note that web pages are constantly being changed. Accordingly, data should be looked at as a snap shot in time of a webpage as it appeared at the time of this work's evaluation. Use of High Software Accessibility Evaluator was essential to helping obtain the statistics for each website in this work. The High Software tool evaluates HTML of the requested URL, and provides output data that can be evaluated via the Web accessibility barrier method, Web Content Accessibility Guidelines, or other methods. Campbell and Stanley (1963, p. 26) states that a T-Test is optimal for this Posttest Only Control Group true experimental design that has been utilized in this work. Thus, after the WAB score was obtained for each of the websites in the experiment, a T-test was conducted. Of the 88 Web 1.0 websites randomly generated, one website was assigned the wrong URL at the time of data collection in November 2007, and three Web 2.0 websites no longer existed at the time of data collection. This is the equivalent of mortality in human-subjects research and thus accounts for the four missing WAB scores in TABLE 3 below:

Table 3. 
original image

Table 4 shows the mean WAB Scores obtained for the sample Web 1.0 websites and the sample Web 2.0 websites in this work.

Table 4. 
original image

Results

Figure 1.

 

Figure 1 demonstrates that WAB scores of Web 1.0 websites reside in a similar cluster, whereas WAB scores are widely dispersed among Web 2.0 websites.

The following means plot (Figure 2) tells the best story of the significance of the results:

Figure 2.

 

Only 10 out of the 176 or 5.68% of the websites in this work achieved a WAB Score of 0. The top score exceeded 150,000. The mean WAB score of homepages of the representative Web 1.0 websites is significantly lower than the mean Web 2.0 score of homepages of the representative sample of Web 2.0 websites. By WAB score standards, Web 1.0 websites are more accessible than Web 2.0 websites. Illustratively, Web 2.0 websites are on average less accessible than there Web 1.0 counterparts.

The significant difference between the average mean of Web 1.0 website homepages and Web 2.0 website homepages, suggests that Web 2.0 websites on average are less accessible than there Web 1.0 counterparts. In fact, Web 2.0 websites homepages on average were 15 times less accessible than their Web 1.0 counterparts. This further suggests that there is a positive correlation between accessibility compliance and site navigability for the visually impaired. This is due to the fact that Web 2.0 websites tended to violate more WCAG 1.0 checkpoints than their Web 1.0 counterparts. These results have great promise for future exploration on these matters.

The T- test cannot assume equal variances due to the Sig of.000 at the.05 level of significance. However, our T-Value is char?2.68 with 85.029 degrees of freedom, according to the Welch test, which assumes unequal variances. Under the conditions of the Welch test, there is significance between the groups since.009 is less than the.05 level of significance. This indicates that there is a negative correlation between Web 2.0 affiliation and site accessibility. Table 5 shows the t-test results:

Table 5. 
original image

It is important to note that the results are not significant when assuming Equal Variances in T-tests (the stronger) test. However, they are significant by the Welch test, which assume unequal variances. In spite of the t-test information, the WAB score results alone are a very important contribution.

Analysis and Discussion

In general, Web 1.0 websites were on average more accessible than their Web 2.0 counterparts. This may be due to the fact that the university Web 1.0 websites are in highly regulated environments. Public universities, for instance, generally must adhere to Section 508 standards. This, among other things, made university sites an optimal control group instead the other Web 1.0 websites groups available. Web 2.0 websites generally failed automatic accessibility checkpoints due to their large use of web-based images without alternative text. Implementing several images without alternative text was the primary violation that made Web 2.0 websites less accessible than their Web 1.0 counterparts.

The large WAB score difference between Web 1.0 and Web 2.0 alone is an indication that Web 2.0 websites are on average more inaccessible than their Web 1.0 counterparts. Web 1.0 and 2.0 websites on average had a similar amount of homepage images. Web 1.0 sites averaged 22.18 homepage images and Web 2.0 averaged 23.27 homepage images. However, Web 2.0 websites on average had 2.3 times more violations related to images than their Web 1.0 counterparts. This partially explains why WAB scores for Web 2.0 sites where on average greater than their Web 1.0 counterparts. Priority I errors get the largest violation weight of "3" thus Priority I errors have a large impact on overall WAB Score. Web 2.0 websites had more link phrase violations. Therefore, this also affects accessibility of Web 2.0 websites. Priority II Errors get a weight of "2" in the WAB Score, thus, these errors also have a fairly large impact on overall WAB score. Web 2.0 websites had more links on average, thus more potential for link violations.

Limitations of the Study

In order to keep the scope manageable, this work only evaluates the home page of each of the mentioned websites. According to Hackett and Parmanto's (2009) study testing the homepage only is not enough, therefore future studies should look into each website more comprehensively. Any future study should take into consider the dynamic and evolving nature of the web.

Conclusion

This work sought to address several questions as it relates to Web 2.0 website accessibility. To the questions “Does Web 2.0 website affiliation present challenges in web navigation for the visually impaired?” and “Is there a correlation between Web 2.0 affiliation and navigability of a website?”, analysis revealed higher on average WAB scores on Web 2.0 website homepages compared to Web 1.0 website homepages. On average this suggests that more errors related to accessibility standards, thus more potential obstacles for the visually impaired users on Web 2.0 websites.

To address the question, “Is the HTML code in common Web 2.0 applications compliant with web accessibility standards?”, only 10 out of the 176 or 5.68% of the websites in this work achieved a WAB Score of 0. The top score exceeded 150,000. The mean WAB score of the representative Web 1.0 homepages is significantly lower than the mean score of the representative Web 2.0 homepages. There is room for improvement in both Web 1.0 websites and Web 2.0 websites, however, Web 2.0 websites have a greater distance to travel in order to achieve better accessibility.

Developers can use accessibility tools such as Hi Software's Cynthia Validator and the WAB Score metric together. The use of the two tools does not have to be mutually exclusive. This work simply proposes that the WAB score helps provide a more complete picture of the extent to which one's webpage is accessible. Web 2.0 is a trend that will continue to gain in popularity. Understanding issues of accessibility as it relates to Web 2.0 will benefit technical audiences such as web developers, academics audiences such educators and students, and general web users audiences. From a website developer's standpoint, the WAB Score of Parmanto and Zeng (2005) offers a numerical gauge of the extent to which one's work is accessible. The further away a developer's score is from 0, the more work he or she must do to bring the webpage up to better compliance. As research issues on Web 2.0 become more popular, educators will be able to teach methods to students that enable them to design more accessible web-based content while also taking full advantage of the benefits of the technologies. By following accessibility standards, both visually impaired users, and general users can take advantage of Web 2.0 websites.

Acknowledgements

The authors would like to acknowledge the work of the individuals of the World Wide Web Consortium (W3C) who contributed to the Web Content Accessibility Guidelines (WCAG) 1.0 standards. The authors are also grateful to the University of North Texas College of Information.