Putting Web Site Quality and Accessibility into Context - Quality Factors

What we know is that organizations are poor at setting targets for success. However, given a set of factors, what do they consider to be of importance in terms of quality? In the survey conducted we provided a set of factors derived from Cutter Senior Consultant Robert Thomsett. He looked to the IEE and ISO 9000 standards, which define quality as “fit for purpose.” The phrase “fit for purpose” ties directly back to my argument that quality is based on the targets set for the Website. That’s what will define if the Web site is fit for purpose and therefore of high quality. The quality factors used in the survey are adapted from those set out by Thomsett, which are focused on software projects. They have been adjusted to be Web-specific.

The factors are as follows.

Conformity. The Web site adheres to industry standards and best practice (e.g., W3C coding and accessibility standards).

Usability. The Web site is easy to use and understand from the client’s perspective.

Efficiency. The Web site uses the people, business process, hardware, and software efficiently.

Maintainability. The Web site is easy to maintain and support.

Reliability. The Web site performs reliably and is free of errors.

Performance. The Web site works well under load.

Portability. The Web site works properly in different environments (e.g., low bandwidth, with different browsers and operating systems).

Security. The Web site is secure from hacking.

The first thing to note about these factors is that not all of them are relevant to every project. Some Web sites simply do not need to worry about performance as they are never likely to be under a high load and will not need to go through extensive load testing. Some Web sites will only get a small number of hits because they have a niche target market so worrying about load is of little value. Also, it may not matter if the Web site does not work in all operating systems (i.e., portability). It might be that the particular Web site is only going to be accessed using IE 7/XP because we know the client base is a corporation that has a standard operating environment so we don’t have to worry about users accessing the site using Safari on Mac OS X.

What is interesting though is that when we asked which of these factors were important in considering a Web site to be “high quality,” the results show that they are all important (see Graph 12).

Graph 12 - What is high quality?

The results indicate that people think a Web site should either “fairly well” or “precisely” fulfill all the factors. The factors that stood out most were usability, reliability, and security. What this tells us is that the feeling in the industry is that first and foremost a Web site should be useable, reliable, and secure. In order of importance, people said the following factors should be present:

  1. Usability

  2. Reliability

  3. Security

  4. Efficiency

  5. Performance

  6. Maintainability

  7. Conformity

  8. Portability

So we know that people put usability ahead of pretty much everything, not by much, but it’s still at the top. This makes sense: if a Web site is not useable, it doesn’t really matter if the content is great, if it follows XHTML and CSS standards, or if it is extremely reliable. First and foremost a Web site should be easy to use. People will tolerate poor usability only so long until something that is easy to use comes along, and then they will switch. It stands to reason that ease of use is paramount.

Graph 1 - what matters on your website?

But what about security? A site that is easy to use but has online transactions needs to be secure otherwise there is a risk of financial loss. Does that mean as long as a Web site is useable, we cannot worry about the other factors? Would you prefer a Web site that is a little harder to use if you know it makes it secure? That would mean in the context of sites that manage financial transactions, usability could be less important than security. There is no black-and-white checklist from which we can tick off items to say a site is of high quality; what matters is the quality factors that have the greatest impact in achieving the purpose of the Web site.

Which Factors Apply to Your Site?

It’s one thing to say a set of quality factors should apply to a site, it’s another to actually make the effort to ensure those factors are present and tested to prove they are in place. In my experience, quality factors are rarely considered up front. What the survey results show is a sad reflection of my experience. Not a single factor is present in all Web sites even though, as pointed out above, all factors are thought to be important for the Web site to be considered of high quality. Overall, does this matter? Well, if we go by what the survey results show, then, yes, it’s a problem. We may claim that certain factors should be present, but we fail to actually implement them. The most important factor, usability, was only present in 53% of Web sites (see Graph 3). This tells us that although we may say something is important, we don’t act that way. We talk the talk, but only half of us will walk the walk.

Graph 3 - What quality factors does your site have?