Archive
Poka-yoke

Education, forcing functions and understanding

By: Dan
21-01-2007
Control, Design, Design with Intent, Education, Embedding code, Forcing functions, Good design, Hidden persuaders, Innovation, Interaction design, Mathematics, Mistake-proofing, Poka-yoke, Security, Techniques of persuasion, Teenagers, Textbooks, University, Usability, User Psychology

Engineering Mathematics, by K Stroud

Mr Person at Text Savvy looks at an example of ‘Guided Practice’ in a maths textbook – the ‘guidance’ actually requiring attention from the teacher before the students can move on to working independently – and asks whether some type of architecture of control (a forcing function perhaps) would improve the situation, by making sure (to some extent) that each student understood what’s going on before being able to continue:

Image from Text Savvy
Image from Text Savvy
Is there room here for an architecture of control, which can make Guided Practice live up to its name?

This is a very interesting problem. Of course, learning software could prevent the student moving to the next screen until the correct answer is entered in a box. This must have been done hundreds of times in educational software, perhaps combined with tooltips (or the equivalent) that explain what the error is, or how to think differently to solve it – something like the following (I’ve just mocked this up, apologies for the hideous design):

Greyed-out Next button as a forcing function

The ‘Next’ button is greyed out to prevent the student advancing to the next problem until this one is correctly solved, and the deformed speech bubble thing gives a hint on how to think about correcting the error.

But just as a teacher doesn’t know absolutely if a student has really worked out the answer for him/herself, or copied it from another student, or guessed it, so the software doesn’t ‘know’ that the student has really solved the problem in the ‘correct’ way. (Certainly in my mock-up above, it wouldn’t be too difficult to guess the answer without having any understanding of the principle involved. We might say, “Well, implement a ’3 wrong answers and you’re out’ policy to stop guessing,” but how does that actually help the student learn? I’ll return to this point later.)

Blind spots in understanding

I think that brings us to something which, frankly, worried me a lot when I was a kid, and still intrigues (and scares) me today: no-one can ever really know how (or how well) someone else ‘understands’ something.

What do I mean by that?

I think we all, if we’re honest, will admit to having areas of knowledge / expertise / understanding on which we’re woolly, ignorant, or with which we are not fully at ease. Sometimes the lack of knowledge actually scares us; other times it’s merely embarrassing.

For many people, maths (anything beyond simple arithmetic) is something to be feared. For others, it’s practical stuff such as car maintenance, household wiring, and so on. Medicine and medical stuff worries me, because I have never made the effort to learn enough about it, and it’s something that could affect me in a major way; equally, I’m pretty ignorant of a lot of literature, poetry and fine art, but that’s embarrassing rather than worrying.

Think for yourself: which areas of knowledge are outside your domain, and does your lack of understanding scare/intimidate you, or just embarrass you? Or don’t you mind either way?

Bringing this back to education, think back to exams, tests and other assessments you’ve taken in your life. How much did you “get away with”? Be honest. How many aspects did you fail to understand, yet still get away without confronting? In some universities in the UK, for instance, the pass mark for exams and courses is 40%. That may be an extreme, and it doesn’t necessarily follow that some students actually fail to understand 60% of what they’re taught and still pass, but it does mean that a lot of people are ‘qualified’ without fully understanding aspects of their own subject.

What’s also important is that even if everyone in the class got, say, 75% right, that 75% understanding would be different for each person: if we had four questions, A, B, C and D, some people would get A, B, and C right and D wrong; others A, B, D right and C wrong, and so on. Overall, the ‘understanding in common’ among a sample of students would be nowhere near 75%. It might, in fact, be small. And even if two students have both got the same answer right, they may ‘understand’ the issue differently, and may not be able to understand how the other one understands it. How does a teacher cope with this? How can a textbook handle it? How should assessors handle it?

I’ll admit something here. I never ‘liked’ algebraic factorisation when I was doing GCSE (age 14-15) A-level (16-17) or engineering degree level maths – I could work out that, say, (2x² + 2)(3x + 5)(x – 1) = 6x^4 + 4x³ – 4x² + 4x – 10 (I think! I don’t think there’s an HTML character code for a superscript 4, sorry), but there’s no way I could have done that in reverse, extracting the factors (2x² + 2)(3x + 5)(x – 1) from the expanded expression, other than by laborious trial and error. Something in my mathematical understanding made me ‘unable’ to do this, but I still got away with it, and other than meaning I wasted a bit more time in exams, I don’t think this blind spot affected me too much.

OK, that’s an excessively boring example, but there must be many much, much worse examples where an understanding blind spot has actually adversely affected a situation, or the competence of a whole company or project. Just reading sites such as Ben Goldacre’s Bad Science (where some shocking scientific misunderstandings and nonsense are highlighted) or even SharkTank (where some dreadful IT misunderstandings, often by management, are chronicled) or any number of other collections of failures, shows very clearly that there are a lot of people in influential positions, with great power and resources at their fingertips, who have significant knowledge and understanding blind spots even within domains with which they are supposedly professionally involved.

Forcing functions in textbooks

Back to education again, then: assuming that we agree that incompetence is bad, then gaps in understanding are important to resolve, or at least to investigate. How well can a teaching system or textbook be designed to make sure students really understand what they’re doing?

Putting mistake-proofing (poka-yoke) or forcing functions into conventional paper textbooks is much harder than doing it in software, but there are ways of doing it. A few years ago, I remember coming across a couple of late-1960s SI Metric training manuals which claimed to be able to “convert” the way the reader thought (i.e. Imperial to SI) through a “unique” method, which was quoted on the cover (in rather direct language) as something like “You make a mistake: you are CORRECTED. You fail to grasp a fundamental concept: you CANNOT proceed.” The way this was accomplished was simply by, similarly to (but not the same as) the classic Choose Your Own Adventure method, having multiple routes through the book, with the ‘page numbers’ being a three digit code generated by the student based on the answers to the questions on the current page. I’ve tried to mock up (from distant memory) the top and bottom sections of a typical page:

Mock-up of a 1960s 'guided learning' textbook

In effect, the instructions routed the student back and forth through the book based on the level of understanding demonstrated by answering the questions: a kind of flow chart or algorithm implemented in a paperback book, and with little incentive to ‘cheat’ since it was not obvious how far through the book one was. (Of course, the ‘length’ of the book would differ for different students depending on how well they did in the exercises they did.) There were no answers to look up: proceeding to whatever next stage was appropriate would show the student whether he/she had understood the concept correctly.

When I can find the books again (along with a lot of my old books, I don’t have them with me where I’m living at present), I will certainly post up some real images on the blog, and explain the system further. (It’s frustrating me now as I type this early on a Sunday morning that I can’t remember the name of the publisher: there may well already be an enthusiasts’ website devoted to them. Of course, I can remember the cover design pretty well, with wide sans-serif capital letters on striped blue/white and murky green/white backgrounds; I guess that’s why I’m a designer!)

A weaker way of achieving a ‘mistake-proofing’ effect is to use the output of one page (the result of the calculation) as the input of the next page’s calculation, wherever possible, and confirm it at that point so that the student’s understanding at each stage is either confirmed or shown to be erroneous. So long as the student has to display his/her working, there is little opportunity to ‘cheat’ by turning the page to get the answer. No marks would be awarded for the actual answer; only for the working to reach it, and a student who just cannot understand what’s going wrong with one part of the exercise can go on to the next part with the starting value already known. This would also make marking the exercise much quicker for the teacher, since he or she does not have to follow through the entire working with incorrect values as often happens where a student has got a wrong value very early on in a major series of calculations (I’ve been that student; I had a very patient lecturer once who worked through an 18-side set of my calculations about a belt-driven lawnmower which all had wrong values, based on something I got wrong on the first page.)

Overall, the field of ‘control’ as a way of checking (or assisting) understanding is clearly worth much further consideration. Perhaps there are better ways of recognising users’ blind spots and helping resolve them before problems occur which depend on that knowledge. I’m sure I’ll have more to say too, at a later point, on the issue of widespread ignorance of certain subjects, and gaps in understanding and their effects; it would be interesting to hear readers’ thoughts, though.

Footnote: Security comparison

We saw earlier that there seems to be little point in educational software limiting the number of guesses a student can have at the answer, at least when the student isn’t allowed to proceed until the correct answer is entered. I’m not saying any credit should be awarded for simply guessing (it probably shouldn’t), just that deliberately restricting progress isn’t usually desirable in education. But it is in security: indeed that’s what most password and PIN implementations use. Regular readers of the blog will know that the work of security researchers such as Bruce Schneier, Ross Anderson, Ed Felten and Alex Halderman is frequently mentioned, often in relation to digital rights management, but looking at forcing functions in an educational context also shows how relevant security research is to other areas of design. Security techniques say “don’t let that happen until this has happened”; so do many architectures of control.

Read More

Shaping behaviour: Part 1

By: Dan
09-11-2006
Architecture & urbanism, Bad design, Business model, Circumvention, Civil rights, Consumer rights, Control, Design, Design philosophy, Design with Intent, Designed to be unpleasant, Designed to injure, Distasteful corollary, Do artifacts have politics?, Embedding code, Entropy, Erosion of liberty, Exclusion, External Control, Forcing functions, Foucault, Future, Good design, Gravy train, Health and safety, Hidden persuaders, Indoctrination, Interaction design, Intrusive technology, Mistake-proofing, Norms, Philosophy of control, Poka-yoke, Political design, Product design, Propaganda, Restriction, Site Announcements, Skateboarding, Sneaky, Social engineering, Techniques of persuasion, Technology, Technology policy, User experience, User Psychology

A couple of months ago I posted about the ‘shaping behaviour’ research of RED, part of the UK Design Council. At the time I noted in passing a classification of design approaches for shaping behaviour, mentioned by RED’s Chris Vanstone: “stick*, carrot or speedometer.” It’s worth looking further at this classification and how it relates to the spectrum of control, especially in a technology context:

Yes, it's a stick (well, a branch), next to a PCB

Stick

If we define ‘stick’ as ‘punishing the user for attempted deviation from prescribed behaviour’, then many of the architectures of control we’ve examined on this site demonstrate the stick approach. They’re not explicitly ‘technologies of punishment’ in Foucault‘s phrase, but rather a form of structural punishment. The thinking seems to be (for example):

  • If you try to sleep on this bench, you will be uncomfortable (and hence won’t do it again)
  • If you try to copy a DVD, your copy will be degraded and your time and blank DVD wasted (and hence you won’t do it again, or will buy another authorised original)
  • If you try to view our website using a competitor’s browser, your experience will be broken (and hence you’ll switch to our browser)
  • If you try to skateboard here, your board will be damaged and you will be maimed (and hence you won’t do it again)

    • …and so on. There are numerous other examples from software and urban planning, especially.

    The thing is, though, for each of those ‘sticks’, a large percentage of people will not be obedient in the face of the ‘punishment’. They’ll try to find a way round it: a way of achieving their original objective but avoiding the punishment. They’ll search for what others in similar situations have done (e.g. DeCSS in the DVD example) or ask among friends until they find someone with the required expertise or who knows about an alternative. They may even actively destroy the ‘stick’ that punishes them. In some cases they might not even understand that they’re being punished, simply seeing ‘the system’ as beyond their comprehension or stacked against them.

    Equally, there isn’t always a rational strategy behind the ‘stick’ in the first place. The anti-homeless bench doesn’t ‘solve’ the ‘problem of homelessness’. It just punishes those who try to lie down on it without offering an alternative. It’s punishment with no attempt at resolving the problem.

    If a stick does get people to change their behaviour in the intended way, it will be accompanied by resentment, anger and dissatisfaction. It may only be fear of the consequences which prevent actual rebellion. In short: using sticks to change people’s behaviour is not a good idea.

    Carrots: image from image.frame
    Image from image.frame

    Carrot

    A ‘carrot’ means offering users an incentive to change their behaviour. This moves away from actual control to something closer to some aspects of captology – making a persuasive case for behaviour change through demonstrating its benefits rather than punishing those who disobey.

    To some extent, control and incentives may be incompatible. Taking away functionality from users then showing them how they can get it back (usually by paying something) might be a classic combined “carrot and stick” technique, but it’s also bordering on a protection racket, and it doesn’t fool many people.

    However, can control be used in conjunction with genuine incentives to serve the agendas of both sides? Electric lights that turn off automatically if no-one’s in the room take some control away from the user, but also offer benefits to both the user (lower electricity bills) and society as a whole (less energy used). But if they turn off automatically, is there actually any incentive for the user to change his or her behaviour? If we’re always spoon-fed, will we ever learn?

    Perhaps mistake-proofing measures or forcing functions which allow a user to increase his or her productivity or safety, in return for giving up some ‘control’ – which may not be highly valued anyway – fit the definition best. If I’m working in a factory painting coachlines on hand-built bicycles, a steady guide arm that damps my arm vibrations – but only if I also take care as well – takes some control away from me, but also prevents me making mistakes, allowing me to paint more coachlines per hour, more accurately. It also helps my employer.

    But that’s a very weak degree of control. Unless anyone can come up with any counter-examples, I would suggest that providing real incentives for users to change their behaviour is fundamentally a very different approach to the ‘control mindset’ (unless you are trying to trick people by offering false incentives, or by understating what they could lose by changing their behaviour).

    I’ll get round to speedometers in a future post, since this approach is worthy of a deeper treatment.

    *The phrase “carrot and stick” seems now universally to imply “offering incentives with one hand and punishment with the other” (though not necessarily at the same time), rather than the “carrot dangling from a stick, just out of reach” meaning (i.e. “motivating people to perform with incentives which will never be fulfilled”) which I first assumed it to have when I heard the phrase as a kid (I’m not the only one with this issue). In this post, I’ll use “stick” to mean “punishment”.

    Read More

    Preventing baggage trolleys going down the escalator

    By: Dan
    28-10-2006
    Architecture, Architecture & urbanism, Bad design, Britain, Design, Design engineering, Design philosophy, Design with Intent, Engineering, Engineering design, Forcing functions, Health and safety, Interaction design, Poka-yoke, Restriction, Site Announcements, Spatial, Traffic calming, User experience

    Barriers at Heathrow

    These ‘pinch point’ barriers at London’s Heathrow Airport prevent the baggage trolleys from the Bus Station being taken down the escalators which lead to Terminals 1, 2 and 3. Mistake-proofing (for safety reasons: a trolley down the escalator would be dangerous) but also unnecessary if the airport had been designed differently from the start. Is forcing users to load baggage on and off multiple trolleys whenever their path descends or ascends really desirable? A lift (elevator) may be available, but how many people – and their trolleys – can fit in it at once?

    An inclined travelator (as used elsewhere at Heathrow) would be a better solution.

    Read More

    Product psychology to discourage anti-social behaviour

    By: Dan
    20-10-2006
    Design, Design philosophy, Design with Intent, Designers, Do artifacts have politics?, Forcing functions, Future, Good design, Health and safety, Hidden persuaders, Innovation, Interaction design, Mistake-proofing, Philosophy of control, Poka-yoke, Political design, Product design, Site Announcements, Sneaky, Techniques of persuasion, User Psychology

    Yes, that's a bottle of Bucky in the background, along with Waitrose Tonic Water (no aspartame) and some Mauby Bark syrup

    From McGazz (who also has some great music to listen to on his website):

    “As I was getting myself a cup of tea in work this morning, I overheard a colleague talking about a problem at the tanning salon his wife runs. Each cubicle has a bin in it, and a regular customer has apparently taken to vomiting and urinating in it (the guy reckoned the tannee in question might be bulimic).

    I suggested he get round the problem by using wire mesh bins. While he was chuffed with this idea, I’m slightly worried that I managed to devise an ‘architecture of control’ after only a few seconds thought. I must have authoritarian tendencies…”

    This is a clever, non-invasive, psychological deterrent to the undesirable behaviour. I wouldn’t call it authoritarian: it’s guiding behaviour without outright control. This is good design.

    The closest parallel example I can think of is the use of cone-shaped paper cups for water-coolers (see image below left): besides being simpler & cheaper to make than flat-bottomed cups, people (generally) have a much lower tendency to leave them lying around once they’re empty. The psychological resistance to leaving the cup on its side (since it can’t stand up on its own) on the table, in case that last drip of water leaks out, is – oddly perhaps – fairly high. Especially when in company, people just don’t do it, whereas they’ll happily leave empty coffee cups and screwed-up cake wrappers on the tables. (I spent a lot of time in the Judge Business School, in Cambridge, where the Common Room – below right – had a water cooler using cone cups. It was rare to find them left on the tables, but common to find other litter.)

    Cone cup compared to normal flat-bottomed cups  Common room, Judge Business School

    How could this type of design thinking be used in more situations to guide people into better behaviour? Littering seems an obvious theme to target, but also perhaps energy waste? Can devices which show us our energy usage in real time, such as the Wattson, really change people’s behaviour, or is it better to embarrass them into change? Roadside CO2-readers which flash up to you (and other drivers) just how much damage you’re doing to the environment?

    Read More

    The Tell-Tale Part

    By: Dan
    17-10-2006
    Bad design, Business model, Cargo cult, Cell phones, Consumer rights, Design, Design engineering, Design philosophy, Design with Intent, Designed to be unpleasant, Designers, Do artifacts have politics?, Embedding code, Engineering, Engineering design, Fightback Devices, Freedom to tinker, Good design, Interaction design, Intrusive technology, Mistake-proofing, Mobile phones, Panopticon, Philosophy of control, Poka-yoke, Product design, Rent-seeking, Site Announcements, Sneaky, User experience, Worldwide, Your property

    Open the case of your mobile (cell) phone. Do you see a round white sticker, similar to that in the first photo below?

    Water damage sticker

    This is a water damage sticker, which changes colour if moisture gets into this bit of the phone, and will be used to void your warranty if your phone stops working for any reason.

    A single droplet of water placed on the sticker turns it bright red (in the case of my phone, anyway):

    Water damage sticker

    WikiHow’s ‘How to save a wet cell phone’ (found via Consumerist) recommends that you:

    “Place a piece of satin finish scotch tape over your water damage sticker before you drop your cell phone in the water to prevent the water damage sticker from voiding your warranty… Remove the tape if you ever have to return your phone for repairs or warranty.”

    Now, it’s a clever idea on the part of the phone companies, and presumably water-damaged phones being returned under warranty were enough of a problem to make such stickers ‘necessary’.

    However, we all know that in practice, any non-working phone where the sticker has changed colour will be immediately classified as ‘water-damaged’ and the customer’s rights voided, even if the actual phone was independently defective.

    As a designer, I would much prefer to look at the problem as “How can we improve the sealing of phones so that water ingress is no longer a major problem?” than “How can we design something to cover our backs and shift all the blame onto the user for our design fault?”

    But maybe I’m naïve.

    P.S. My Motorola, shown above, began to work intermittently just a month after the warranty expired, completely unrelated to any water issues, hence I don’t mind getting the sticker wet.

    P.S. Hi, visitors from Nokia. Please note, my intention wasn’t to have a go at phone designers (or the engineering teams); and your phones seem superior on the water-protection front anyway. It’s just a commentary on the mindset which says “it’s easier/cheaper to catch users out than it is to solve the problem.”

    Read More

    Using trees to encourage safer driving

    By: Dan
    12-10-2006
    Architecture, Architecture & urbanism, Control, Design, Design engineering, Design philosophy, Design with Intent, Designers, Embedding code, Engineering, Engineering design, Forcing functions, Good design, Health and safety, Interaction design, Law, Mistake-proofing, Motoring, Philosophy of control, Poka-yoke, Political design, Public money, Regulation, Site Announcements, Speed control, Speeding, Techniques of persuasion, Traffic calming, Urban, User experience, User Psychology

    Image from New Urban News, by Eric Dumbaugh
    Image from New Urban News, by Eric Dumbaugh

    Ryan G Coleman kindly sent me a link to this very interesting New Urban News story, ‘Research: trees make streets safer, not deadlier’. The gist is that roads planted with trees cause drivers to put themselves in state of greater alertness, which makes them generally more cautious about driving and generally slow down:

    “Proposals for planting rows of trees along the roads — a traditional technique for shaping pleasing public spaces — are often opposed by transportation engineers, who contend that a wide travel corridor, free of obstacles, is needed to protect the lives of errant motorists…

    [However], Eric Dumbaugh, an assistant professor of transportation at Texas A&M… looked at accident records and found that, on the contrary, wide-open corridors encourage motorists to speed, bringing on more crashes. By contrast, tree-lined roadways cause motorists to slow down and drive more carefully, Dumbaugh says.

    Dumbaugh examined crash statistics and found that tree-lined streets experience fewer accidents than do “forgiving roadsides” — those that have been kept free of large, inflexible objects. He points to “a growing body of evidence suggesting that the inclusion of trees and other streetscape features in the roadside environment may actually reduce crashes and injuries on urban roadways”…

    Dan Burden, senior urban designer for Glatting Jackson and Walkable Communities Inc. in Orlando, notes that there is research showing that “motorists need and benefit from tall vertical roadside features such as trees or buildings in order to properly gauge their speed.”

    The article goes on to mention the ‘Shared Space‘ work of Hans Monderman, Ben Hamilton-Baillie and others, which includes removing road markings as part of a wider scheme to change the perceived emphasis of an environment and, again, put drivers into a state of greater awareness. From the BBC article on the ‘naked road’ experiment in Seend, Wiltshire:

    “Motoring psychologists and urban planners seem to agree that, overall, “naked roads” appear to have a positive effect on motorists…

    “This approach draws on behavioural psychology involving the way drivers respond to their surroundings,” [Ben Hamilton-Baillie] says. “It removes the sense of security provided by barriers – such as kerbs, and traffic lights. Instead of relying on the street system for security, drivers are forced to use their reactions.”

    According to Mr Hamilton-Baillie, the removal of a psychological safety net encourages drivers to exercise caution and restraint. He believes that the lack of clear markings encourages drivers to slow down and mingle with pedestrians, forcing them to make eye contact with one another.”

    Why are these techniques so much better than this kind of thing?

    As so often, I feel it’s better to put users of a system into a state of mind where they are actively, intelligently thinking about what’s going on, and how they can respond to dangers or risks in the environment, than to remove that option for awareness or action planning, and deliberately force them into a state of ignorance of the risks ahead just to compel them to slow down. The driver in the tree-lined or Shared Space road situation can read the road ahead, and adjust his or her behaviour based on the risks that are perceived, whereas just blocking drivers’ vision so they can’t read road hazards ahead and must therefore actually come to a stop, does much less to help safety, and instead merely causes frustration.

    Read More

    About me
    Publications
    My PhD
    Empower project / CarbonCulture


    Design with Intent toolkit
    Design with Intent survey


    Freelance consultancy

    @danlockton

    Upcoming talks & events

    I'm speaking or running workshops at:
    More... (Lanyrd)

    Some of my previous presentations

    Comments & trackbacks

    Blog 2005-date

    Architectures of Control project (2004-5)

    Subscribe (RSS)



    Written by Dan Lockton, 2004-12

    Theme is Blogum by Wpshower

    Powered by WordPress