"The Hiring Problem"

1
"The Hiring Problem" Academic HCI vs. the Real
World of Practice

• Lynn Cherny
• (a talk given at UW in 2004, while I was at Adobe)

2
Contents

• My background
• Industry Today
• Usability and Design
• We need HCI Executives!
• The Professionals Contractors vs. In-House folks
• Interviewing for HCI folks
• Example job ad and interview contents
• But is it the real job or the advertised job?
• HCI in Education and Industry Preparation
• Please, Teach the Design/HCI Students A Big
  List!!
• And Teach
• Your Research Students
• Possible Lessons for Faculty

3
My background

• HCI Research at ATT Labs
• Excite, Senior UI designer
• TiVo, UI designer then manager
• Axance (consulting), Dir of Methodology
• Adobe, Senior UI Designer for cross-product
  issues
• The Mathworks (current), Principal Usability
  Specialist

4
Industry Today Usability vs. Design

• Usability Engineering Evaluation, field work,
  lab work
• ? often highly-educated voices stuck in the lab
  ghetto.
• Interface Designers A variety of backgrounds,
  doing a lot of different jobs not all from HCI.
  
• Strongs NSF report (1995) researchers, research
  professionals, converts from soft disciplines,
  ergonomists, converts from computer disciplines,
  graphic designers

5
Industry Today We Need Designers and HCI
Executives

• The rise in the dependence of HCI on usability
  labs is basically a regression. Design is where
  the action is.
• -- Stu Card, CHI 2002
• We do not create anything of substance we are
  critics. The Design profession flourishes because
  they do things, they create. We must become
  designers. Don Norman, CHI 2002
• If you want to be the low status, low on the
  totem pole person in your company, then yeah,
  rejoice in the fact that you are hiring user
  testers. User testing is not where the action is.
  The action is with those people who decide what
  product to build in the first place. That isn't
  the user tester community, but it should be the
  CHI community. Don Norman, interactions (2000)
• ? Design skills will help break the glass
  ceiling on promotion for HCI professionals. We
  want more designers (and executives) coming from
  solid HCI backgrounds.

6
Industry Today Who are the Professionals?

• Mantei and Hewett provided a high-level analysis
  into the professional categories at work in HCI
• researcher,
• profesionally oriented researcher,
• research oriented professional,
• professional
• What I see
• Consultants The senior folks you see and hear
  from most (from CHI, UPA, etc.)
• In-House permanent staff (the professional
  category)
• Go to fewer conferences
• Are much younger and very often somewhat
  alienated from the profession nothing new,
  heard it all before, nothing will benefit me
  there
• Dont belong to ACM/Sigchi, dont read the
  articles (or write)
• Dont know the theory, quite often
• Do far less original design work than youd
  expect!
• ? Lets assume these folks are your students

7
Interviewing Example Job Ad

• Responsibilities Works with multidisciplinary
  software development teams to help understand
  user needs, specify usability requirements, and
  verify requirements and needs are met. Conducts
  user research, including contextual inquiry.
  Facilitates user-based requirements discovery and
  user-centered design. Plans and conducts lab and
  field usability tests. Analyzes, reports, and
  presents results and recommendations to
  development team and product management.
• Qualifications Experience 4-Year degree or
  Graduate degree in Computer Science, Software
  Engineering, Cognitive Psychology, Human Factors,
  or related field. Practical knowledge and
  experience in usability engineering principles
  and methods and familiarity with various
  prototyping tools and techniques. Strong
  communication, interpersonal, and organizational
  skills. Familiarity with chemistry and the
  pharmaceutical industry helpful.

8
Interviewing Hiring a UI Designer

• Traditional questions describe a difficult team
  interaction around a design and what you did to
  resolve it
• Portfolio work review (how much is theirs?)
• Design critique
• Design problem whiteboard
• General vocabulary assessment UCD,
  prototype, flowchart, usability,
  iterative
• ? It is famously difficult to hire good
  designers. The question is, do we really need
  to worry about this?

9
Interviewing But is it for the Advertised Job or
the Real Job?

• Evangelism often comes up in ads/interviews a
  warning sign of dysfunction
• Note that many designers portfolios dont show
  their work but are they really trying to
  mislead you?
• Not everyone knows what the real job consists of
  (at hiring and performance eval time)
• We interview in a self-deluded state how we
  want it to be (or to evolve) vs. how it really
  is hence we hire on the wrong criteria
  sometimes.
• Exactly how important is design talent for
  success? Possibly less so than a complete skill
  set for getting design DONE.
• ? Case Study Example Hiring Sally, a good
  designer

10
HCI and Education for Practice

• SigCHI was 68 people in 1982, over 6000 in 2002
  well-known conference rejection rate ? and many
  satellite conferences
• Much interest internally in status of the
  profession and HCI education
• Strongs NSF Report (1995),
• Hewett, T. T. et al Curricula in HCI (1992),
• Perlman Gasens The ACM SIGCHI HCI Education
  Survey (1993), etc.
• multiple panels at CHI

11
HCI and Education Industry Preparation? A known
problem.

• Its difficult to address needs of industry in
  HCI curricula because
• HCI itself is expanding, parts are ill-defined,
  and we dont have a good understanding of what
  some topics have to offer industry,
• Industry is a collective term for a wide range
  of different activities done by people whose
  needs vary considerably
• When you talk to people in industry they usually
  dont know their own needs and they certainly
  dont know what their colleagues needs are, many
  dont even know what most of their colleagues do.
• --Jenny Preece, CHI 94 on Is HCI Education
  Getting a Passing Grade from Industry?

12
HCI and Education Supposed Activities of
Practice (Strongs NSF Report, 1995)
13
HCI and Education Reality of Practice

• From Strongs NSF Report in interactions (1995)
• Many HCI practitioners experience identity
  conflicts within the first few years of entering
  an applied position. In general, they must
  reorient their views of how they work, what
  questions they ask, and what about their work is
  valued.
• Can we improve their experience by better
  preparation?
• (Internships arent sufficient.)

14
Teach them The Language of Business Marketing
instead of HCI UCD

• Don Why do you think marketing has so much of
  a role determining the products? It's not because
  they're brighter, and it's not because they have
  any more truths. It's because they know how to
  play the game better. What I suggest is that it's
  time we learn how to play the game.
• Janice The advantage that marketing has is that
  it's a well-known concept. Executives have
  typically gone to business school where marketing
  is well-known. One of the problems that we have
  is that we don't have usability engineering and
  HCI in most curricula for business and
  engineering.
• Don One reason we don't is that we don't talk
  the language of business.

Don Norman and Janice Rohn, Interactions, Volume
7, Number 3 (2000), Pages 36-60
15
Teach Them Product Design instead of Interface
Design, If You Can!

• Think about the big picture building and selling
  a product, not doing just the UI design
• From market research, write a business plan
• Identify the crucial factors to distinguish your
  product in the marketplace
• ? Consider that usability alone doesnt motivate
  intent to purchase. Study some business cases and
  learn what makes something sell this will give
  perspective and credibility and reduce their
  anxiety tremendously!

16
Teach Them Usability Methods

• Produce designers who know usability evaluation
  methods
• Weve won the battle of the usability evaluation
  at most companies this is a needed job skill
  now.
• But teach them to make strong recommendations
  from user problems, that are credible, grounded,
  realistic.
• All field methods, surveys, focus groups,
  usability test design, basic statistics,
  interviewing, content analysis
• Researchers who want to make an impact on
  product definition should learn how to disguise
  themselves as designers till they can achieve the
  impact they want.

17
Teach them How to Determine Appropriate Study
Methods

• Propose the appropriate diagnosis method to get
  at useful data for a design problem
• What usability technique will get useful data?
• What type of data will be wanted/understood?
• What arguments are needed for it?
• Guard against the naive proposals Lets just
  ask users on the beta list.
• ? Consulting is therapy for dysfunctional
  companies.

18
Teach Them To Characterize their Customers

• Marketing thinks in segments its our job to
  define and identify who will really be using the
  products.
• Be able to get the data, or make a reasonable
  guess and review with team. The design should be
  tied to the user/customer profiles, this will
  save much pain!
• Dont go through the motions if its not in a
  form thats obviously useful, youve done
  something wrong.

19
Teach them To Identify and Characterize Problems

• To identify and summarize the key problems to
  solve in any domain!
• By what criteria
• According to what evidence
• Identify the causes of the problem that require
  action break it down usefully.
• ? Stated concisely
• Inconsistent color is a user complaint but
  the operational problem is not in the same terms.
• ? Skill that requires strong analysis and
  abstraction skills, backed up with excellent
  communication ability.
• ? Teach them how to THINK through an issue!

20
Teach them To Review and Critique Designs

• To critique designs and find improvements, both
  minor and major.
• ? Most practical UI design is incremental
  improvement and new work is always relative to
  existing products in some way. Dont skip this
  step, it seeds new ideas.
• ? Design test at interviews often checks on this

21
Teach Them Different Kinds of Design

• Expose them to many kinds of design visual,
  industrial, IA (web), interaction,
  object-oriented software design, patterns,
  functional specification, chart/table, document
  layout, experimental design
• They all have different types of deliverables
  consider a project with several of them, and make
  them distinguish their contributions and
  boundaries/limits
• Not everyone is equally good at each, but
  learning limits now is better.
• ? Try brainstorming and use methods to unlock
  creativity

22
Teach them Design for Multiple Devices

• Consider design for
• Mobile devices
• Desktop apps
• Social/community
• Web
• TV
• Physical objects
• Games (several types)

23
Teach Them To do Fast Low-Fi Design

• To produce fast preliminary designs multiple
  concepts per problem
• With callouts/text explaining concisely the
  differences
• Produce multiple formats of low fidelity
  workflow diagrams, task breakdowns, wireframes,
  sketches
• ? Accept that others will help with this and
  thats useful, not a threat.

24
Teach Them To Prepare Multiple Deliverables

• To prepare and explain value of all typical
  design stage deliverables
• Workflows, flowcharts, wireframes, storyboards,
  high fidelity mockups
• Write specifications in increasing detail,
  focusing on the important details at each stage

25
Teach them To Recognize the Value of Multiple
Levels of Abstraction

• Describe the users mental model and how it
  relates to the UI in taskflow/vocabulary
• Flowcharts of the UI experience vs. the
  storyboard taskflow vs. the data flow

26
Teach them To Present Their Design Work

• Make them present their designs to diverse
  groups business, engineering
• With appropriate justification of thought process
• With strategic defenses prepared
• Orally, but with supporting materials
• Make them respond to naive questions and
  criticisms of any type
• ? Consider asking them to present someone elses
  work it will strengthen the arguments, and
  require the author to distill their points and
  this wont provoke so defensive a response to
  critique from the naïve.

27
Teach them To Find and Interpret Previous
Research

• Research literature can be useful, but requires
  interpretation, summarization, digestion. If
  they cant do this, the gulf between practice and
  the academy will grow.
• Consider book reports oral recaps to audiences
  who are naïve make it interesting and relevant
  to them!
• Sometimes previous work is not reviewed
  research dont forget about online white
  papers, etc.
• Identify ways to work research strategically and
  tactically into design work
• Make them use the ACM digital library!!

28
Teach them To Relate In-House Research to Design

• To summarize the important conclusions from
  research/evaluations and how they relate to their
  design decisions
• Throughout the design process, at all stages!
• This will be a cover-your-ass strategy (for HCI
  as a profession) but will also pay off in a
  better design, if were really doing the right
  things.
• Gap between field research and design is still
  huge! Be an advocate for usability methods!

29
Teach them Justify without Defensiveness

• They must justify their work gracefully.
• Everyone thinks shes a designer every decision
  will be questioned no matter how minor.
• ? Note that bad designers cant necessarily
  recognize good designs. (The question is not
  will this happen, its how insane will it make
  them.)

30
Teach them Software Development Lifecycle Models

• To pay attention to engineering lifecycle
  concerns, and learn how to argue for and adapt
  UCD to work with them (Rapid Development, Agile
  and XP, RUP, boxcar)
• Strong (1995) HCI practice can help to inform
  and improve life cycle activities. Within such a
  life cycle, HCI emphasizes iteration and concrete
  communication. HCI research has demonstrated that
  straightforward activity categories-such as
  analysis, design, and evaluation are not
  separable in practice. HCI practitioners can
  help show the weaknesses of the waterfall model,
  but not without risk. The risk is that the
  somewhat marginalized field of HCI practice may
  lose some of its credibility and influence if it
  is perceived within organizations as opposing
  their received wisdom.

31
Teach Them How to Interview

• Have them interview each other for jobs, and ask
  each other portfolio questions challenge each
  other with design tests
• Review resumes with them
• Their peers will be hiring them eventually!

32
Teach them To Drive Projects

• They should assume that nothing will happen
  between high level marketing requirements and
  engineering implementation theyre the
  connector
• Organize meetings, identify stakeholders,
  investigate technical issues, do research if
  needed, design and get it reviewed.
• ? But drive decisions!

Design
MRD/Biz plan
Implementation
33
Teach them Most Design Happens in Committees

• The challenge is how to have an influence over
  the committee how to prepare the right
  materials and arguments.
• ? Come in having thought about it harder than
  anyone else and expecting to present ideas in
  some concise form.

34
Teach Them Public Speaking

• Require presentations confidence, poise,
  clarity, extemporaneous speech
• ? Start with article/book reports, move on to
  design presentations

35
Teach them To Study Their Own Organization

• They must be ethnographers of the workplace, to
  understand the best methods of success and tailor
  their message for their audience.
• How do people get promoted? What gets rewarded?
  Whats teamwork? Whos in charge?
• How can you document the impact of your own
  contribution?

36
Teach them To be Solid Experts but not Shrill
Evangelists

• Teach them to justify in plain language, but not
  to be pushy. Evangelism is dangerous and
  marginalizes over the long term.
• Siegel and Dray (interactions, Aug 2003)
  Actively demonstrate your identification with
  concerns that exceed your discipline and with the
  companys larger goals. Whenever possible, speak
  a common language rather than the language of
  your discipline. All your recommendations should
  be clearly supported not by reference to the
  principles or belief system of your discipline,
  but by clear evidence of what is best for the
  company in a practical way. Consider reframing
  what UCD is about, in ways that show its links to
  larger, shared concernsdue diligence, product
  planning, quality, reduced risk, efficiency, and
  innovationand to marketing and business goals
  like customer satisfaction and retention, and,
  ultimately, profitability.

37
Teach them Organizational Planning

• Make them plan for team projects
• How will the team resolve debates?
• How will the team prioritize features vs.
  schedule and resources?
• Track action items and evolution of work?
• How do you plan to estimate work across team and
  disciplines? Scope work?

38
Teach them To Know and Evaluate Books on Design

• Engineers regularly ask for references on how to
  do design. (See Jeff Johnsons panel at CHI 2002
  on books on design)
• Consider Coopers books, Johnsons, Mayhew,
  Mullet and Sano, Norman, Tufte, etc.
• Books on requirements analysis that include
  usability research are useful recommendations too

39
Teach them The Difference Between Peers and
Resources
There are two kinds of people in
organizationsthere are peers and there are
resources. Resources are like usability
consultantswe go out and we hire them. We'll
hire a consultant or we'll have a little section
that does usability and think of it as a service
organization. We call upon them when we need them
to do their thing, and then we go off and do the
important stuff. That's very different than
peers, where a peer is somebody I talk to and
discuss my problems with, and who helps to decide
upon the course of action. As you get higher and
higher in the organization, this becomes more of
an issue. The executive staff talks to the
executive staff, and they have beneath them all
this organization, which are their resources that
they deploy. But the big decisions are being made
among peers. And it's really important to advance
in the world to be thought of as peers. Our
usability labs are resources that we call in when
we think we have some trouble with usability. We
go and spend a few hours and we worry about the
budget. But we don't take it seriously, we only
take peers seriously. And you only get to be
peers if you speak the right language and if
you're making a contribution at the level the
company cares about...which is profitability.
--Don Norman,
interactions (2000)
40
And Teach Your Research Students

• Ben Shneiderman, CHI 2002 We must recognize
  that nothing is so practical as a good theory and
  that theory thrives when challenged by practice.
  Our goals should include development of
  predictive, explanatory, and generative theories
  that systematically support the next generation
  of innovations.
• ? Express the implications of your research
  clearly for your colleagues in practice or it
  will have limited impact.

41
Possible Lessons for Faculty

• Stay in touch with practice, not just via
  consulting and conferences
• Build departmental bridges, to business schools
  in particular
• Consider offering co-taught Product Innovation
  and Product Management courses to
  cross-pollinate
• Encourage good software engineering education and
  practice as well concern for architecture,
  teamwork, documentation, test plans
• Evaluate and improve your own teaching with
  feedback from former students in professional
  practice

42
Some Takeaways

• We have to produce Supermen, Know-it-alls
  obviously difficult.
• Not everyone will be a good designer, but this
  may be less important than many of the other
  skills used on the job
• You cant teach everything in a pinch,
  sacrifice some of the theory (move to advanced
  classes) and instead teach how to do
  research/where to find it, and how to read it.
  Plus the practical items I described!
• Usability engineering (of the evaluation sort) is
  clearly a valuable and important skillset, BUT
• ? By getting more design into this practice, the
  door to true and deep impact will open wide.

43
Thanks for listening!

• Lynn Cherny
• arnicas_at_gmail.com
• www.ghostweather.com