Archive for the ‘Best Practices’ Category

6 Metrics for Managing UI Design

August 18th, 2008 by Russell Wilson

As part of a recent management summit at my company, we were asked to fill out an RMPT matrix for our departments (I head up Product Design).  An RMPT matrix consists of (R)esponsibilities, (M)etrics, (P)rocesses, and (T)ools.  I have been intending to develop better metrics for both measuring and guiding our design efforts, and this exercise served as a catalyst to get me started.  Bear in mind that metrics help you focus your efforts and measure your progress, but you are also held accountable to them.

For (R)esponsibilities I specified the following:
1) Improve our products & innovate
2) Provide the UI design for new features/functions/products
3) Approve any UI design work done outside of Product Design
4) Validate our UI designs and explore user needs through user testing

Given those responsibilities (and that’s important because your metrics are linked to them) I then came up with the following metrics and met with my team who helped to refine them:

For a given period (e.g. a business quarter):
1) Number of layouts delivered
2) Number of interactive prototypes created
3) Percentage of product design requests completed by commit date
4) Number of users tested
5) Number of product improvements made
6) Number of products insights documented

Discussion:

1) We struggled with what to call this and how to word it specifically.  We started with “Number of designs delivered”, moved to “Number of screens delivered” and settled on “Number of layouts delivered.”  We chose layouts because designs seemed a little too broad and didn’t really give any indication of what we would actually be delivering.
2) We prototype when feasible and want to both push this effort as well as measure it.  Again we struggled with the wording (i.e. “prototypes” vs. “designs”, etc.).  We wanted to make it clear that we are measuring the number of interactive, non-static prototypes.
3) We recently implemented a Product Design Request System where we track the growing volume of design requests and deliverables to development and product management.  We are holding ourselves accountable to deliver on these requests by our committed due date.
4) We test designs on users as much as possible and want to both measure this and push ourselves to do this as much as possible.
5) Here’s where it gets a little interesting.  This one is admittedly qualitative and may need further refinement.  With that said, it is easy for us to remain reactive and work day to day by responding to requests from product management and development without ever taking the initiative to improve our products by combining our research with our unique expertise in usability and design.  This metric is meant to drive us to be proactive and take initiative to improve our products and to track those improvements over time to further illustrate our value.  To make this metric more objective I plan to tie in efficiency measurements and satisfaction scores (SUS).
6) With our considerable user “face time” we are in a unique position to collect “insights” into product shortcomings, needs, and opportunities.  We often take copious notes when doing user studies only to leave the notes sleeping silently in our Moleskines.  We meet regularly with SME’s (subject matter experts), developers, users, and various other stakeholders.  This metric is intended to remind us to document, publish, and take action on the “insights” that we collect.

My reason for publishing these metrics is to get feedback and to learn what metrics other designers work with.  Let me know if you think an important metric is missing or if you disagree with these metrics and why.


Submit to Reddit

Can you name which design or usability principles this violates?

August 11th, 2008 by Russell Wilson
Bad Door Sign

The bottom line is a series of arrows pointing to the left door.
(apologies for the poor quality photograph)


Submit to Reddit

Consistency and Conceptual Integrity

August 21st, 2007 by Russell Wilson

If consistency is your goal I doubt you will ever produce world-class designs.

The importance of consistency, and what consistency really means with regard to software design, has to be one of the most misunderstood and misapplied principles. I am constantly tortured with “well, Microsoft does it like this – why don’t we just copy that? It would be consistent with what users expect.”, and “let’s make this control look like this other one to be consistent”.

With this modus operandi, how can you ever expect to innovate?

Consistency is not about copying existing designs to shortcut necessary creative work (Note: I’m not talking about looking to other designs for inspiration). True, this can be used effectively, and may even be a good idea in some cases – there are many great designs out there worthy of mimicry! But it should not be a standard practice. Should every car look like a Honda? Should every tool in your garage have the same handle?

Should form follow function, or form follow form?

Maybe we should create a standards committee that specifies how all buttons should look and behave; then when someone develops a new software application, they can go to a website and download all the controls they need to build it. Over time all software products will look and behave exactly the same and our users will be much happier, right? I don’t think so.

And even more importantly, consistency does not mean sacrificing usability for the sake of code reuse! Again, I am a practical designer; I understand the various business and engineering considerations (time to market, cost, etc.). I’m not advocating reinventing the wheel. But I certainly don’t think that a bicycle, motorcycle, van, or high-performance sports car should all share the same wheel design either! Code reuse should be leveraged whenever possible but it has to be weighed against other considerations such as basic usability, existing metaphors and mental models, etc.

So, how can one apply consistency correctly? By thinking in terms of “conceptual integrity,” a slightly more abstract concept coined by Frederick Brooks in The Mythical Man-Month. CI means that the software system as a whole should reflect one vision and should fit together seamlessly and flow naturally. This will require consistency throughout the system, but it will not be the driving force. The usage of the system is the first consideration and consistency is applied to support this.

Form follows function and then goes through the consistency/best-practices and usability testing ovens to create the final product.


Submit to Reddit

Challenges to world class software design

August 8th, 2007 by Russell Wilson

Every job has its challenges. As a programmer, I spent long hours fixing bugs and finding workarounds. As an engineering executive, I dealt with resources, schedules, and politics. And as an entrepreneur, I struggled to find customers and generate revenue.

But designing software is tough. I’m not talking about the creative work – that’s our passion and we love it. I’m referring to the “tax” we pay for doing what we love.

So what makes up this tax? What challenges am I referring to?

1) Everyone thinks they are designers

Developers, product managers, sales, and even customers, can’t resist their own needs to create or invent by suggesting ways to change an interface or add capabilities – “let’s just add a drop-down to the top…”. Either in the form of “design on the spot” during a product meeting or customer visit, or a developer going ahead and “fixing the problem” without waiting for input from design, it happens often, and reflects perceptions and lack of understanding of the design role and expertise. It can also reflect poor adherence to process, or lack thereof, and a need for cross-department executive sponsorship and continued support. (I am not suggesting that no one can give input to the design process; many of our best designs are the result of collaborative efforts with product management and sales.)

2) Design is a nice to have

I’m always perplexed by this one, but many bottom-line executives still perceive good design as a nice to have. No, not at Apple or Intuit (I see more design-related job postings from Intuit than any other company – it makes me wonder if they just want to interview everyone they can, or if Intuit has a Wonka factory somewhere full of designers dressed like ump lumpas, all building accounting applications). Apple and others have helped to move this forward, but I still run into the skeptical eye from time to time. Another manifestation of this is when a new product or concept is attempted and the strategy is to “get something working, find some buyers, and then make it better.”

3) Who makes the final decision?

The Biltmore Estate is a remarkable architectural achievement, and is commonly considered the result of a single vision where the overall design was driven and conceived by one person. Frederick Brooks states that “conceptual integrity” is the single most important factor in the development of successful software applications. But often, with software design, there are many stakeholders, business and marketing agendas, and the need to create something as soon as possible. The nature of software design and development within high-tech companies doesn’t seem to lend itself to the purity or grandeur of a project like the Biltmore where the “genius” is given free reign to produce a work of art. It’s difficult to find the right balance between art and business in software design, and this is evident in the careful politicking among the design executive and various stakeholders to decide what gets built and what doesn’t.

4) The difficulty in justifying designs to critics

True, the best justification comes from users, and you can get that (to some degree) from usability testing — assuming you have the time and resources to conduct formative testing. But what about colors? What about visual treatments that are more subjective? What do you do when the product manager says “I hate that blue — why do we have to use that blue?” (I recently emailed a color wheel to a large group of employees at our company - that was a big mistake.) There are many cases where I wish I could just say “just do it that way, trust me!” My boss suggested that I respond with a standard “thank you for your input” for things such as this.

Well, those are some key ones, but I’m sure I’ve missed several. I invite anyone who reads this to submit their own design challenges. This is a work in progress and I intend to update it with the best ways to address these challenges.


Submit to Reddit

The same, or different, but not similar!

August 3rd, 2007 by Russell Wilson

Inspired by a post from Michael Tuminello in response to a question I had on the IxDA list, I propose this rule of thumb: “The same, or different, but not similar!

The concept that Michael discussed (and I have said in the past in different terms), is that when designing, controls/patterns for similar functions across products (or even within the same product) should either be exactly the same (consistent) or vastly different. But they should never be “similar”. Being similar will result in many more usability issues than being entirely different where the user does not have certain expectations for behavior.

Selecting reporting time frames should be consistent across a suite of reporting products. If one product has vastly different constraints on time selection, requiring a different time selection mechanism, it should be obvious to the user that they are working with a completely different control with different behavior.


Submit to Reddit

Usability testing at conferences

July 29th, 2007 by Russell Wilson
I just returned from the Cisco Networkers 2007 conference in Anaheim, California where we tested various designs in two of our products on 45 network engineers. Testing users at the right conferences has become very successful for us. First, it is very easy to recruit participants. Second, the conference attendees happen to be our target users!

We reserved a conference-floor meeting room (which was included in the cost of our company’s booth) for the actual testing. We rented two monitors and had one power strip. Internet connectivity was not necessary for our testing, but we could have used wireless (for free) if we needed to. We shipped two servers and two desktops, with mice and keyboards, and an 8-port switch. Each of the desktops had Morae Recorder installed as well as an in-house Adobe AIR (formerly Apollo) application I wrote to allow users to pick descriptive words they would associate with the products after the test (thanks to Microsoft for the idea). We had everything up and running within an hour of carrying the boxes to the testing room. This setup gave us the capability to test two participants at once which increased our efficiency and allowed us to conduct more tests during open hours.

The tests were structured to take no more than 30 minutes, and would consist of several tasks with participant-tester interaction, followed by a SUS satisfaction survey, and the Apollo/AIR reaction-card application. In past testing sessions I would give participants tasks and then silently observe with very little interaction, but this time I decided to make the task-portion more interactive and I’m much happier with the results. At the risk of disturbing their natural problem-solving flow, the feedback we recorded provided us with excellent insight into positives and negatives with the designs: what was working and what wasn’t.

When it came time to recruit volunteers, I asked all of our booth presenters to add a slide to their presentations about the usability testing we were doing at the conference. I also offered anyone who would participate a $50 American Express gift card. I wondered whether it was too much, but surprisingly, it was still challenging to get people to sign up. Ultimately we filled every available slot we had, but not without some work (I recruited 3 people while riding the buses back and forth to the convention center).

All in all, it was a great success and provided us with valuable feedback and insight that we will take back to R&D and consider over weeks to come. Several small changes will be made resulting in a much better product prior to release. And despite all the positives I’ve discussed, there were some drawbacks to testing at a conference. Namely the noise, even inside the meeting rooms, as well as the general excitement level (it’s not exactly a “relaxed” setting). But at least for us, the positives far outweigh the drawbacks. And the final price? Here it is (rounded up)..


Submit to Reddit

Rant: Just say NO to tabs within tabs!

July 13th, 2007 by Russell Wilson

There are bad navigation choices, and then there is “tabs within tabs” or “TWIT”.

I can’t think of a less inspired solution to hierarchical navigation, and yet I encounter it all the time (guilty omitted). What exactly do I mean by TWIT?

Avoid TWIT at all costs!! In a future post I will describe some basic alternatives.


Submit to Reddit

Microsoft’s Inductive User Interface

July 11th, 2007 by Russell Wilson

Interesting. Although I’m not sure what they think they’ve invented.
Most (if not all) of this has been around for some time. - Russ

Microsoft’s Inductive User Interface

MSDN (the Microsoft Developers Network) has a short introduction to a relatively new trend in the way Microsoft thinks about Interface Design.

Inductive User Interface (IUI for short) is a term that describes the collection of methods and guidelines for designing interfaces that, according to Microsoft, are easier to follow than the current generation of software products are.

According to Microsoft, IUI gained traction as a design process as a result of the research they’ve done on actual users performing tasks on their products. In short, they found that a number of important assumptions that are commonly made by User Experience practitioners are incorrect. They found that, contrary to the commonly held notion, most users are unable to successfully perform even basic computer tasks. The article stated 3 key reasons as to why they have concluded that software is hard to use:

* User’s don’t understand the software’s conceptual model. From the original article:

“The interface design for most current software products assumes that users will understand a conceptual model that the designers carefully crafted. Unfortunately, most users don’t seem to ever acquire a mental model that is thorough and accurate enough to guide their navigation. These users aren’t dumb - they are just very busy and overloaded with information. They do not have the time, energy, or desire to wonder about a conceptual model for their software.”

* Even expert users never master common interface tasks. From the original article:

Designers know that new users may have trouble at first, but expect these problems to vanish as users learn common tasks. Usability data indicates this often doesn’t happen. In one study, researchers set up automated equipment to videotape users at home. The tapes showed that users focusing on the task at hand do not necessarily notice the procedure they are following and do not learn from the experience. The next time users perform the same operation; they may stumble through it in exactly the same way.

* Every piece of functionality on a screen takes effort to figure out how to use. From the article:

Most software products are designed for (the few) users who understand its conceptual model and have mastered common procedures. For the majority of customers, each feature or procedure is a frustrating, unwanted puzzle. Users might assume these puzzles are an unavoidable cost of using computers, but they would certainly be happier without this burden.

Most current software GUI’s aren’t addressing these problems. Instead, assuming the user’s (1) are familiar with standard Interface controls (2) have the time or the desire to learn the software’s conceptual model (3) Are willing to put up with a steep learning curve for additional functionality rather than use a more basic, yet simpler product.

As a result is what Microsoft calls the Deductive User Interface (see image). An Inductive User Interface is one whose screens require the user to figure out what can be done, and how to do it. The more time spent trying to figure out what can be done, the less energy and patience the user has left to actually perform them.

Microsoft’s solution is to design interfaces that induce, or lead, the user through one task at a time. As such, the computer screen should act not unlike an expert standing over the user’s shoulder, directing them through one screen at a time. The four essential ingredients to designing an IUI are:

1. Focus each screen on a single task.
Don’t try to accommodate multiple distinct and possibly unrelated tasks onto one screen. This will potentially overwhelm the typical user, in order to satisfy the expert, or speed user.

2. State the task.
Part of identifying a task, is stating it, clearly. This sounds elementary, but there is actually a lot of literature on the advantages of compact, even terse language in interface design. IUI screen title should use natural language and state the exact task at hand, using verb / object phrases. The example Microsoft gives in the article is from it’s redesign of Microsoft Money: One of the original screen title’s was this too general “Account Details”, whereas the redesigned screen title was “Change account setup”-much clearer.

3. Make the screen’s contents suit the task.
Once users have read the screen title they will proceed directly below, to the contents of the screen, IUI’s make that transition effortless, as the tasks associated with the screen are intuitive and natural, corresponding directly with the title (or primary task).

4. Offer links to secondary tasks.
Unlike the Wizard, that ubiquitous and often controversial feature of many a Microsoft product, IUI’s aren’t meant to be modal, and according to Microsoft, aren’t intended to impede the expert user. Adding links to secondary tasks allows the user some flexibility in the way he/she goes about performing their tasks.

Criticisms:
The web is full of interfaces that exhibit many of these same characteristics for a few reasons:

* Relatively slow reaction times as commands are frequently sent over the internet and processed remotely

* Products need simple interfaces so as to flatten learning curves, to thwart the relatively quick abandonment resulting from the democratic, highly competitive nature of the web

Microsoft calls the IUI design process an extension of the Web -Style Interface, and a few bloggers have commented that there isn’t really much new here. Just a rehashing of tried and true design practices applied to the desktop model.

Additionally, there has been much discussion in the blogshpere on the relative merits and disadvantages of IUI. Most comments have lamented IUI’s similarities with Wizards, which have been rightly blamed for dumbing down the population of computer users by unnecessarily shielding them from any complexity, and preventing them from learning.


Submit to Reddit