It is remarkable… how often thinking for oneself will lead us to conclusions written about before we were born.
From a post by Vera Bass, ‘Teaching requires learning’, 6th November 2006.
Many people have probably also said this, but that’s the point, pretty much.
Read More
Mr Person at Text Savvy looks at an example of ‘Guided Practice’ in a maths textbook – the ‘guidance’ actually requiring attention from the teacher before the students can move on to working independently – and asks whether some type of architecture of control (a forcing function perhaps) would improve the situation, by making sure (to some extent) that each student understood what’s going on before being able to continue:
Image from Text Savvy
Is there room here for an architecture of control, which can make Guided Practice live up to its name?
This is a very interesting problem. Of course, learning software could prevent the student moving to the next screen until the correct answer is entered in a box. This must have been done hundreds of times in educational software, perhaps combined with tooltips (or the equivalent) that explain what the error is, or how to think differently to solve it – something like the following (I’ve just mocked this up, apologies for the hideous design):
The ‘Next’ button is greyed out to prevent the student advancing to the next problem until this one is correctly solved, and the deformed speech bubble thing gives a hint on how to think about correcting the error.
But just as a teacher doesn’t know absolutely if a student has really worked out the answer for him/herself, or copied it from another student, or guessed it, so the software doesn’t ‘know’ that the student has really solved the problem in the ‘correct’ way. (Certainly in my mock-up above, it wouldn’t be too difficult to guess the answer without having any understanding of the principle involved. We might say, “Well, implement a ’3 wrong answers and you’re out’ policy to stop guessing,” but how does that actually help the student learn? I’ll return to this point later.)
Blind spots in understanding
I think that brings us to something which, frankly, worried me a lot when I was a kid, and still intrigues (and scares) me today: no-one can ever really know how (or how well) someone else ‘understands’ something.
What do I mean by that?
I think we all, if we’re honest, will admit to having areas of knowledge / expertise / understanding on which we’re woolly, ignorant, or with which we are not fully at ease. Sometimes the lack of knowledge actually scares us; other times it’s merely embarrassing.
For many people, maths (anything beyond simple arithmetic) is something to be feared. For others, it’s practical stuff such as car maintenance, household wiring, and so on. Medicine and medical stuff worries me, because I have never made the effort to learn enough about it, and it’s something that could affect me in a major way; equally, I’m pretty ignorant of a lot of literature, poetry and fine art, but that’s embarrassing rather than worrying.
Think for yourself: which areas of knowledge are outside your domain, and does your lack of understanding scare/intimidate you, or just embarrass you? Or don’t you mind either way?
Bringing this back to education, think back to exams, tests and other assessments you’ve taken in your life. How much did you “get away with”? Be honest. How many aspects did you fail to understand, yet still get away without confronting? In some universities in the UK, for instance, the pass mark for exams and courses is 40%. That may be an extreme, and it doesn’t necessarily follow that some students actually fail to understand 60% of what they’re taught and still pass, but it does mean that a lot of people are ‘qualified’ without fully understanding aspects of their own subject.
What’s also important is that even if everyone in the class got, say, 75% right, that 75% understanding would be different for each person: if we had four questions, A, B, C and D, some people would get A, B, and C right and D wrong; others A, B, D right and C wrong, and so on. Overall, the ‘understanding in common’ among a sample of students would be nowhere near 75%. It might, in fact, be small. And even if two students have both got the same answer right, they may ‘understand’ the issue differently, and may not be able to understand how the other one understands it. How does a teacher cope with this? How can a textbook handle it? How should assessors handle it?
I’ll admit something here. I never ‘liked’ algebraic factorisation when I was doing GCSE (age 14-15) A-level (16-17) or engineering degree level maths – I could work out that, say, (2x² + 2)(3x + 5)(x – 1) = 6x^4 + 4x³ – 4x² + 4x – 10 (I think! I don’t think there’s an HTML character code for a superscript 4, sorry), but there’s no way I could have done that in reverse, extracting the factors (2x² + 2)(3x + 5)(x – 1) from the expanded expression, other than by laborious trial and error. Something in my mathematical understanding made me ‘unable’ to do this, but I still got away with it, and other than meaning I wasted a bit more time in exams, I don’t think this blind spot affected me too much.
OK, that’s an excessively boring example, but there must be many much, much worse examples where an understanding blind spot has actually adversely affected a situation, or the competence of a whole company or project. Just reading sites such as Ben Goldacre’s Bad Science (where some shocking scientific misunderstandings and nonsense are highlighted) or even SharkTank (where some dreadful IT misunderstandings, often by management, are chronicled) or any number of other collections of failures, shows very clearly that there are a lot of people in influential positions, with great power and resources at their fingertips, who have significant knowledge and understanding blind spots even within domains with which they are supposedly professionally involved.
Forcing functions in textbooks
Back to education again, then: assuming that we agree that incompetence is bad, then gaps in understanding are important to resolve, or at least to investigate. How well can a teaching system or textbook be designed to make sure students really understand what they’re doing?
Putting mistake-proofing (poka-yoke) or forcing functions into conventional paper textbooks is much harder than doing it in software, but there are ways of doing it. A few years ago, I remember coming across a couple of late-1960s SI Metric training manuals which claimed to be able to “convert” the way the reader thought (i.e. Imperial to SI) through a “unique” method, which was quoted on the cover (in rather direct language) as something like “You make a mistake: you are CORRECTED. You fail to grasp a fundamental concept: you CANNOT proceed.” The way this was accomplished was simply by, similarly to (but not the same as) the classic Choose Your Own Adventure method, having multiple routes through the book, with the ‘page numbers’ being a three digit code generated by the student based on the answers to the questions on the current page. I’ve tried to mock up (from distant memory) the top and bottom sections of a typical page:
In effect, the instructions routed the student back and forth through the book based on the level of understanding demonstrated by answering the questions: a kind of flow chart or algorithm implemented in a paperback book, and with little incentive to ‘cheat’ since it was not obvious how far through the book one was. (Of course, the ‘length’ of the book would differ for different students depending on how well they did in the exercises they did.) There were no answers to look up: proceeding to whatever next stage was appropriate would show the student whether he/she had understood the concept correctly.
When I can find the books again (along with a lot of my old books, I don’t have them with me where I’m living at present), I will certainly post up some real images on the blog, and explain the system further. (It’s frustrating me now as I type this early on a Sunday morning that I can’t remember the name of the publisher: there may well already be an enthusiasts’ website devoted to them. Of course, I can remember the cover design pretty well, with wide sans-serif capital letters on striped blue/white and murky green/white backgrounds; I guess that’s why I’m a designer!)
A weaker way of achieving a ‘mistake-proofing’ effect is to use the output of one page (the result of the calculation) as the input of the next page’s calculation, wherever possible, and confirm it at that point so that the student’s understanding at each stage is either confirmed or shown to be erroneous. So long as the student has to display his/her working, there is little opportunity to ‘cheat’ by turning the page to get the answer. No marks would be awarded for the actual answer; only for the working to reach it, and a student who just cannot understand what’s going wrong with one part of the exercise can go on to the next part with the starting value already known. This would also make marking the exercise much quicker for the teacher, since he or she does not have to follow through the entire working with incorrect values as often happens where a student has got a wrong value very early on in a major series of calculations (I’ve been that student; I had a very patient lecturer once who worked through an 18-side set of my calculations about a belt-driven lawnmower which all had wrong values, based on something I got wrong on the first page.)
Overall, the field of ‘control’ as a way of checking (or assisting) understanding is clearly worth much further consideration. Perhaps there are better ways of recognising users’ blind spots and helping resolve them before problems occur which depend on that knowledge. I’m sure I’ll have more to say too, at a later point, on the issue of widespread ignorance of certain subjects, and gaps in understanding and their effects; it would be interesting to hear readers’ thoughts, though.
Footnote: Security comparison
We saw earlier that there seems to be little point in educational software limiting the number of guesses a student can have at the answer, at least when the student isn’t allowed to proceed until the correct answer is entered. I’m not saying any credit should be awarded for simply guessing (it probably shouldn’t), just that deliberately restricting progress isn’t usually desirable in education. But it is in security: indeed that’s what most password and PIN implementations use. Regular readers of the blog will know that the work of security researchers such as Bruce Schneier, Ross Anderson, Ed Felten and Alex Halderman is frequently mentioned, often in relation to digital rights management, but looking at forcing functions in an educational context also shows how relevant security research is to other areas of design. Security techniques say “don’t let that happen until this has happened”; so do many architectures of control.
Read More
Speedometer, rev counter and fuel and temperature gauges on the dashboard of my 1992 Reliant Scimitar SST. Photo taken on B1098 alongside Sixteen Foot Drain, Isle of Ely, England.
In part 1 of ‘Shaping behaviour’, we took a look at ‘sticks and carrots’ as approaches for shaping (or changing) people’s behaviour. It’s especially worth reading and thinking about the comments on that post as there are some very thoughtful analyses which go beyond my rather cursory treatment. ‘Shaping behaviour’ is a vast field, encompassing pretty much all of politics, advertising and marketing alongside much of religion, education, psychology (and psychiatry?), product and graphic design.
The ‘sticks, carrots and speedometers’ classification was originally mentioned to me as a possible method by Chris Vanstone, of the UK Design Council’s former research arm, RED. The idea is that you can get people to change their behaviour by persuading (or forcing) them with ‘sticks’ (punishment/disincentives), ‘carrots’ (rewards) or ‘speedometers’ (showing them the results of their actions, how they’re doing, or how well they could be doing if they changed their behaviour). Having looked at sticks and carrots – and found the classification rather limiting – let’s take a look at speedometers.
Some gauges provide information which directly relates to a user’s actions at that time. An actual speedometer or rev counter allows the user to determine what effect his or her actions are having on a vehicle, and take corrective action if the information displayed is outside the ‘correct’ range (of course there are other factors, such as the resistance to motion from drag or going uphill, and if one can hear the engine, a rev counter’s perhaps not really necessary, but I digress). Other gauges, such as fuel or temperature gauges (see photo at top) show us information over which we can’t have so much direct influence (or, in the case of a clock, say, no influence…) but about which we need to take action if certain levels are reached. Certainly, we change our behaviour as a result of taking in the information displayed. Usually. And the speedometer can of course be a metaphor for other methods of feedback or information displays – which I’ll get to later on.
Energy use
Sticking with physical gauges for the moment, in recent times there’s been a lot of design effort put into devices which monitor and display our energy or fuel use, with the hope that they’ll persuade us to change our behaviour, or bring to our attention which devices (e.g. in a home) are more power-hungry than others in an immediately persuasive way. The Design Council’s Future Currents project, which investigated a range of interesting techniques and design approaches, put the idea well:
Energy is invisible, which makes it difficult to control. We can give people the tools to monitor their own energy use. Studies show that if people can see what they’re using, they use up to 15% less energy.
An anecdote in Kalle Lasn’s Design Anarchy claims an even larger reduction:
The manager of a housing co-op was increasingly frustrated with her tenants. No matter how much she reminded and badgered them… the tenants would not, could not reduce their energy consumption. Finally she hit an idea. What would happen, she wondered, if the electricity meters were moved from the basement to a conspicuous spot right beside the front door, so that each time the tenants left or entered their home, they could see how fast their meter was whirring? The meters were moved. Lo and behold, within a few weeks electricity consumption fell 30 percent.
(It’s not clear whether there were individual meters so tenants could see each other’s consumption – that kind of “control by embarrassment“, or social pressure, may be effective in this free-rider or unequal contribution situation.)
Above left: Wattbox by Gary Lockton, Brunel University, 1992, a simple unit which displayed the cost of electricity being used as well as estimated bills; Above right: ‘You make waste visible’ from Kalle Lasn’s Design Anarchy; Centre left: Wattson, from DIYKyoto; Centre right: An example ‘greenness gauge’ from the Design Council’s Future Currents project; Bottom left: Static! Flower Lamp ‘blooms’ when a household has reduced its power consumption for a period; Bottom right: Static! Power Aware Cord glows with an intensity related to the power being used. First image courtesy of Paul Turnock; other images from the websites linked.
The convergence of new monitoring and connectivity technologies such as home wireless networks and RFID, with the pressure to scrutinise our environmental impact, has meant that there are more opportunities for potentially persuasive, interesting ways of approaching this area. Tom Coates has some good thoughts on this, and the relation to continuous monitoring of other parts of our (and others’) lives, and how fascinating it can be. Wattson (thanks to both Richard Reynolds and Michelle Douglas for originally bringing this to my attention) takes an especially ‘designer’ approach, becoming a coffee-table talking point as well as showing (in different display modes) the power currently being used, the costs, and, via a coloured glow projected onto the table below, a non-numerical indication of the intensity of power usage. Similarly playful methods are used in some of the Static! projects from Stockholm’s Interactive Institute – perhaps, in fact, when the ‘event’ which occurs as the ‘speedometer’ registers more desirable values is exciting in itself, the technique is closer to a ‘carrot’ than a speedometer.
Left: The Energy Label, required on certain products/packaging in the EU; Right: A typical mess of adaptors powering home electronic equipment. Here we have a scanner, a power drill charger, a printer (plug hidden), a battery charger and a cutting plotter. How easy is it for a consumer to audit the power usage of this kind of mess?
The related debate over standby buttons on home electrical equipment which I covered briefly in July last year, brought home an important point to me, as someone who’s worked on quite a few consumer electronic products powered from adaptors: many users think that if a red LED is on when the product is ‘off’, that little LED is all that’s being powered. That’s quite an important issue when it comes to consumers having a better understanding of their home energy use.
When seeing the Wattson and Future Currents projects for the first time, I was tempted to say “well, why don’t people just look at the power ratings on the appliances they buy?” but soon realised that that’s a pretty entrenched engineering mindset rearing itself in my mind. People don’t want to have to look on a label on the back of the product. They mostly don’t think about energy use when buying products. Even the use of ‘green’ labelling on the front of products (e.g. the EU label shown above) doesn’t hit home the actual monetary costs of different devices over typical usage periods. In this sense, monitoring devices which really get the user interested in using products more efficiently do seem to be very much worth it, even when they themselves use more power than strictly ‘necessary’.
(There are a few points I’d like to make about home lighting and ‘energy saving’ light bulbs, especially since some aspects of the recent blogosphere commentary made me think a little further, but they can wait for another day…)
Economy gauges
Left: A traditional analogue vacuum gauge showing ‘fuel economy’. Image from brochure for Reliant Rialto 2, 1984; Right: Toyota’s Eco Drive meter from the Camry – image from HybridCars.com. As an aside, I have no idea how 35-40 mpg can be considered ‘excellent’! What year is this?
Moving away from home electricity consumption, the increased prevalence of electronic in-car trip computers, usually built-in, has meant that second-by-second fuel economy read-outs are much more common, and can again inspire a kind of self-challenge to maximise economy while driving. As the miles-per-gallon (or perhaps L/100 km) figure drops (or increases) with every blip on the accelerator or rapid acceleration from the traffic lights, drivers really can train themselves to change their behaviour (indeed, I know a couple of people who are constantly shifting their gaze from the road ahead down to, alternately, the speedometer and the miles per gallon figure, to see “how well they are doing”, which is not necessarily ideal). Economy gauges in cars are nothing new – vacuum gauges were quite a popular home-fit accessory at one time, but they generally did not directly relate to the fuel consumption per distance travelled, merely the vacuum in the inlet manifold, hence the amount of fuel-air mixture being drawn through, whether or not the car were moving.
An alternative type of economy gauge was that once used by Volvo and other manufacturers, which compared the engine’s rpm (or the gearbox rpm?) to the gear selected (manual only, I presume) and illuminated a gearstick icon when the driver was in the ‘wrong’ gear, i.e. driving at less than optimum efficiency. Even more simply, some car companies used to mark the ‘gearchange points’ on the speedometer with dots at certain speeds – assuming the driver could not tell from the engine note that the gear engaged was too high or low, the dots would at least give some indication, though of course different driving conditions and loads would make the dots’ positions guidelines rather than absolutes. (I do have photographs of both these designs, somewhere, but will have to post them at some point in the future.)
Speedometers and control
Certainly, then, physical speedometers and gauges can have an effect on users’ behaviour and can encourage people to change; technology seems to be making this easier and more interesting and engaging. There are so many opportunities; already in some countries, there are roadside speed displays to make motorists aware of their speed (which present a fun challenge for drivers, or indeed cyclists, wanting to see what they can achieve) – how long before we have roadside CO2 monitoring (with displays)?
But are any of these ‘architectures of control’?
In the sense that they are methods of persuasion rather than methods of restriction or enforcement, they are on one side of a line with rigid control on the other, but when we look at techniques such as the “control by embarrassment“, or social pressure mentioned earlier, we can see that there is some kind of continuum related to how the information displayed by the speedometer (of whatever form) is used: if only you can see your personal energy usage habits within a house, you can make the choice whether or not to change your behaviour, but if the rest of your household can also see your habits, and see that you’re costing them unnecessary money, the pressure on you to change is much greater.
That, I think, is where the ‘control’ element comes in. Say that every household’s yearly carbon emissions (however this were to be calculated) were monitored. If the information were available to the householders, it may give them food for thought, and may inspire changing behaviour. If the information were available to the government, it may lead to taxation, and may lead to changing behaviour. If the information were legally required to be displayed on an illuminated sign outside the house, so neighbours could see who was “getting away with more carbon emissions”, it may (perhaps) lead to people changing behaviour too, or risk recriminations from the community, possibly worse than just social embarrassment. This last case is pretty much speedometer + blackmail, and I would say that that crosses the line to become control. If you want to fit in, and not be censured by others, you have to conform. That is an architecture of control, very much so, and hence we can see that speedometers, as with many other possible design elements, can be used as part of systems of control, but are not in themselves necessarily political. It’s the way they’re used that makes them, possibly, controversial.
The speedometer metaphor
Metaphorically, of course, a speedometer can be any method of making users aware of their behaviour, or the link between their behaviour and some other effect. Many of the examples studied and created by Stanford’s Captology / Persuasive Technology lab fall into this area, offering users feedback on their actions, or encouraging them to behave in a certain way (e.g. giving up smoking) through highlighting causal relationships.
But isn’t this, to some extent, what all persuasion is about, if we allow our ‘speedometer’ to have, in some situations, only two values (on/’good’ vs off/’bad’)? Everything ‘persuasive’, from advertising campaigns to counselling, is about saying “A is happening/not happening because you’re doing/not doing B; it will be better/stop happening if you stop/start doing C.” A speedometer is saying “You’re doing OK because this is the result of your actions” or “Look at the results of your actions – you need to change what you’re doing!”
Is it true, then to say that any situation where one entity (person/animal/plant) is trying to change the behaviour of another entity is resolved either by control (forcing the change in behaviour) or persuasion (inspiring the change in behaviour), or a combination of the two (e.g. by tricking the entity into changing behaviour)?
Or is that too simplistic?
Read More
Image from News Sniffer
News Sniffer‘s Revisionista monitors alterations to published news stories from a variety of sources by comparing RSS feeds, sometimes revealing subsequently redacted information or changes of opinion (e.g. note the removed phrase in the first paragraph of this story about Cuba). While many of the changes are simply re-wordings for clarity or to correct grammatical errors, there are certainly also some instances of more substantial revisions – see the ‘recommended’ list.
Perhaps more revealing is News Sniffer’s Watch Your Mouth, which shows the reactively moderated comments removed from the BBC’s ‘Have Your Say’ threads. I’ve been reading this for a while – in fact I think I might have been one of the first subscribers via Bloglines – and am still amazed by just how many comments are removed by the BBC’s moderators, often making points which, though maybe controversial, are very much the voice of the common man and woman. Some are offensive, yes; others are genuine expressions of frustration or even first-hand annotations to or clarifications of aspects of the story above. Many are critical of the BBC, including those criticising the moderators for censorship of the very comments under dicsussion.
Read More
From McGazz (who also has some great music to listen to on his website):
“As I was getting myself a cup of tea in work this morning, I overheard a colleague talking about a problem at the tanning salon his wife runs. Each cubicle has a bin in it, and a regular customer has apparently taken to vomiting and urinating in it (the guy reckoned the tannee in question might be bulimic).
I suggested he get round the problem by using wire mesh bins. While he was chuffed with this idea, I’m slightly worried that I managed to devise an ‘architecture of control’ after only a few seconds thought. I must have authoritarian tendencies…”
This is a clever, non-invasive, psychological deterrent to the undesirable behaviour. I wouldn’t call it authoritarian: it’s guiding behaviour without outright control. This is good design.
The closest parallel example I can think of is the use of cone-shaped paper cups for water-coolers (see image below left): besides being simpler & cheaper to make than flat-bottomed cups, people (generally) have a much lower tendency to leave them lying around once they’re empty. The psychological resistance to leaving the cup on its side (since it can’t stand up on its own) on the table, in case that last drip of water leaks out, is – oddly perhaps – fairly high. Especially when in company, people just don’t do it, whereas they’ll happily leave empty coffee cups and screwed-up cake wrappers on the tables. (I spent a lot of time in the Judge Business School, in Cambridge, where the Common Room – below right – had a water cooler using cone cups. It was rare to find them left on the tables, but common to find other litter.)
How could this type of design thinking be used in more situations to guide people into better behaviour? Littering seems an obvious theme to target, but also perhaps energy waste? Can devices which show us our energy usage in real time, such as the Wattson, really change people’s behaviour, or is it better to embarrass them into change? Roadside CO2-readers which flash up to you (and other drivers) just how much damage you’re doing to the environment?
Read More
Niblettes tackles the issue of designers and control, specifically, how much the user’s experience and methods of using a product or service should be defined by the designer. The conclusion – paralleling a theme in a marketing speech by Procter & Gamble’s Alan G Lafley – is that designers must start to think in terms of relinquishing control:
“As the things we design become more interactive, more self-determining, more deeply integrated with their users, the less control designers will exercise over the final artifact. This is not only inevitable, it’s good. And it demands that we work and think in ways earlier generations of designers did not and could not.”
Niblettes’ earlier post, comparing designers and authors, also makes an interesting point:
“The relationship between the designer and the user strikes me as very similar to the relationship between the author and the reader. And it has long been understood in literature that the story belongs to the reader and his or her interpretation of it. Once written the author relinquishes all control.
Although I have absolutely no empirical evidence for this, it seem like some of the most successful authors write with this in mind—they write to relinquish control. Designers on the other hand still seem to be greedy for ever more control.”
Designing for users rather than against them (or in spite of them) ought, of course, to be a given. But since the designer is working for his or her employer, that company’s priorities are really what determines how a product develops.
If the company recognises that treating users well, empowering them to do more with its products, is to its own ultimate benefit, then all is well, but if it serves the company better (in the short term) to force users into tightly controlled behaviour models, then, unfortunately, that’s how the products are going to be designed.
Authors usually have a different relationship with their publishers than designers and engineers do with the companies which pay their wages. for example, I can’t imagine publishers very often compel authors to leave cliffhangers at the end of novels “in order to force readers to buy the sequel”, yet that’s the razor-blade model mentality evident in so many consumer products.
This is an area worthy of much further comment & discussion: as someone who’s been developing, slowly, a ‘philosophy’ for my design work (which isn’t yet ready to unleash on this blog!), I’ll be keeping a close eye on Niblettes’ and others’ thoughts on this.
Read More
Comments & trackbacks