Is there wisdom in crowds and is group decision-making effective? Fraser, Nick and Peter reach a swift and conclusive judgement.

• https://www.flickr.com/photos/whitehouse/48937685671

Things mentioned in this podcast:

• How group dynamics affect decisions
• ‘Officially official’
• Group decision-making
• The Expert

For more information on Aleph Insights visit our website https://alephinsights.com or to get in touch about our podcast email [email protected]

Hello and welcome to the Cognitive Engineering Podcast produced by me, Fraser McGruer, for Aleph Insights. In this series of podcasts, we take a look at interesting topics and discuss what we think they tell us about analysis and decision-making. I'm here with Peter Coghill and Nick Hare of Aleph Insights, and this week we're discussing which is better, decision by committee or decision by an individual. And usually, due to the lockdown, we're recording this remotely. Yeah, so recently, you guys contacted me because you've got a new project underway and for which you're having a dedicated logo made. And rather kindly, you brought me into the loop just to get my view on which logo do you think we should go with.

Only so we could do the exact opposite of what you suggested.

Yeah, and actually, I don't know what the result of that was yet. But for me, when I looked at the logos, there's about six or seven, something like that to choose from. And straight away, there was one that I felt was head and shoulders above the rest. And I said, look, forget... So I think I emailed that back to Nick, saying, look, I've taken a look, this is really simple. Because in classic sort of Aleph style, you'd wanted me to sort of...

There were five different Alephs, and we wanted somebody to think about it. And in classic Fraser style, he shot from the hip.

Yeah, we designed an elaborate assessment framework involving several different categories which you could rate.

It was like two universes colliding, really. And so, you know, yeah, I didn't want to go through this whole sort of giving numbers to stuff and rating it on this versus that. And going through all the five of them, when I could see this straight away, there was one, look, this is the one, clearly it's the best. And I said to Nick, look, there's one that I like, can I do this? And he said, no. You need to make sure, because we need to make sure that it's sort of in, you know, there's a coherent... We need to make sure they're standardized, how this gets decided on which one we have. Which I was outraged. And so, I had to sort of rather regressionally go through all the different ones and give a one to this, give a two to that, five, you know, which I felt was pointless. Because it was just...

You'd already decided, and it was a lot of work trying to make your scores fit your opinion.

Yeah, exactly. I was making the tail wag the dog, exactly that. And so, that was the result. And I was just sort of sitting there going, razzle snazzle, like that. Just thinking, well, clearly an individual decision in some cases, an individual decision making process is way better than a committed decision. And here we are now. So, I guess the first one is, did you decide which logo to go with?

No, we still haven't polled as many people as we'd like. So, you just happen to be sort of there. So, I thought, oh, I'll ask you first. But yeah, no, we haven't really rolled out the assessment yet. No, I mean, you know, but yeah, I guess, I mean, you know, this, the thing is, you see, the theory is why groups ought to be better is because they have a bigger collective memory. So, they have a bigger pool of information to draw on. And if they share it, then they're just more likely to form the correct belief about things. Right. So, and I mean, with things like the logo, I mean, it's almost like directly going to the source because we want people to look at it and go, that looks good. So, you know, if the people we ask say, yeah, that looks good, then that, you know, you think, well, that's answering the question, really. But I think when you think of collective decision making, it's probably more, you're thinking more along the lines of designing a logo, for example, where you get sort of six people around the table, and you all sit there going, oh, I think this one should be a bit squigglier, or we need some more purple there. Or we need, you know, we need this to have a square on it. And then we need to have, and that's the sort of the caricature for decision by committee.

We need parallel lines that all cross.

Is that sort of sense of, you know, if I were to say this car looks like it was designed by a committee, you'd know what I meant. It would look like, you know, 15 different people had had an input into it. And there was no coherent design over the whole thing. So, you know, on one hand, decisions are being made by groups all the time. Right up to the very top, I mean, obviously, in the UK, you know, decisions are made by the cabinets, and they take collective responsibility. So they all pretend that they all agreed with everything that the cabinet decides to do. And that's the norm. Decision making by boards is the norm in the world of business. It's very rare to get someone who's, you know, invested with all the power to make decisions. And yet, at the same time, we all take the mickey out of, you know, how bad groups aren't making decisions. So, yeah, the question really is, are groups better or worse? And under what circumstances? And how can you, if you have a group that makes decisions, how can you make sure that it's going to be more likely to be the good kind rather than bad kind?

Yeah. Okay. Yeah. I think the question is, why don't you just go with my bloody selection for the logo? I think that's really the question. But anyway, Peter, anything to say at this point?

Yeah. Well, this goes really to the core of Aleph Insights and what we do and why we exist. And as a systems engineer, I find the sort of concept of decision design quite an interesting one. And I think the sort of the much maligned decision by a committee is really a colloquial shorthand for where a decision making process has been poorly designed or not designed at all. Just be a historic kind of evolution of how decisions are collectively made by a group of people. Because, as Nick said, in theory, the group decision should be better. But it falls foul of a number of things, various biases, individual biases, but also collective biases. It falls foul of people not sharing information when they could. It falls foul of not making use of people's particular skills. All sorts of things well researched or well understood. And it really depends on a number of factors about the nature of the decisions that are going to be made. So, for example, the decisions that a county council make routinely, day by day, are quite different from the decisions that a military commander has to make. Quite different, again, from the decisions that the CEO of a large corporation have to make. They're very different sets of problems. But there are common factors between those which you can use to design how that governance structure should work and should operate in order to make it most efficient and most applicable to the problem set that it faces.

Well, I think there's clearly, when you're designing such a thing as a decision making structure, a trade off to be made between efficiency and quality, in the sense that we know that decisions through committee is slow. And that's one of the reasons why people punt difficult topics into committees. Because it takes time to share information. It takes time to analyze it, compute it as a group. And then to sort of agree on a common view. And that's why the military don't do that. Because they actually, a lot of the time, you're willing to accept a worse decision because you need a decision now.

I would slightly disagree with that. Although they don't discuss all decisions in the military, what they do do is they exercise mission command so that decisions are devolved to the lowest possible level to the person who is closest to the problem. But decisions are collectively made because there's constant information sharing between different parts of a military organization. But if a decision has to be made, a tactical decision has to be made, it's made by the tactical commander who's there at the time. And they're given the support of their superiors. There's always somebody higher up who's got nothing necessarily to do the decision, but they've empowered their subordinates to make that decision. So it's not like it's not a sort of classic Wehrmacht kind of decision making process, but everything goes on.

The Wehrmacht invented mission command.

No, they were highly centralized and they failed to respond as quickly as they should have been able to. I'm thinking First World War kind of.

Oh, OK, fine. I'm out of my depth here with this military history.

They were far too centralized and everything had to come from the top. Much like the Soviet Union as well. But modern mission command is a very different animal in that it's highly devolved. So that model is applicable to business. So if you're corporate directors who have different seats, you've got a finance director and you've got the HR director and you've got the technical director, they are empowered to make decisions within their own brief. But as soon as a decision meets a certain criteria, then it becomes the board for a collective decision because it's too big or it's too broad for one director to make. But effective boards, they have power over their own column below them.

Out of interest and something I've often wondered about. You started out at the beginning talking about a cabinet, Nick, and I've often wondered and you two sort of come from government. I've often wondered to what extent a minister has leeway within his or her brief. And I've never quite understood. So if we take, for example, the Home Secretary, to what extent can they... How does decision making work there?

But also, presumably, I mean, although, you know, we talked at the beginning about my preference in that individual case for an individual decision, you know, I think Peter's point that there's all sorts of different kinds of decisions and different sort of circumstances. So it's not a sort of, you know, a one model fits all. But nonetheless, broadly speaking, you know, decision by a group should be better. And you've talked about some of the pitfalls there. So how do we avoid that, if we still think that's the better way to go?

I mean, I suspect, without having any data, and this is pure speculation, that where decisions are badly made by groups of people like the Bay of Pigs, etc. That's where either a deliberately engineered decision making system has been badly engineered, or no deliberate engineering has occurred in the decision making system. And it's just a group of people having a chat and come out with some kind of consensus. Or there is a well engineered system that has been deliberately or inadvertently subverted in some way. So, for example, the higher echelons of American decision making generally, by and large, pretty good. They run a very powerful country, and they seem to get it right most of the time, not withstanding current administration. But their decision was probably a political one to go ahead with the invasion, overruling much of what was being suggested by their commanders and intelligence, etc.

Well, I don't know. I think you're being a bit probably a bit optimistic. I mean, I suspect that most people were just sort of going along with it, I think. I don't think I mean, even though there were dissenting voices, by and large, I think most people felt like it wasn't in their power to do anything about it.

Well, okay. In that case, I would chalk that up to a badly designed decision making system then. But my point really is that you can engineer, you can engineer the decision making system to try to minimize the effect of these biases that cause the bad decision making if just left unchecked.

Yeah, I think that like the number one piece of advice would be if you have some collective decision making, think about how it works. You know, think about what you're actually doing as a group.

And if it helps, I brainstormed some of the factors which I think would be important. Right, go. So, time sensitivity. So, some decisions can be taken slow burn, some are very immediate. So, think tactical military decisions, you have to decide within seconds what you're going to do about the enemy shooting at you. Whereas decisions about- Shoot back. Yeah, decisions about- No, don't do that. For crying out loud, you'll only make them angry. Hold on, let's have a vote.

Let's take a vote.

Yeah, marketing decisions about which supply you go with could take months or years to go through. Cost and scale. Yeah, but what does that mean then?

So, let's say we had to make decisions quickly. What would that mean for our decision making apparatus?

So, if you had to make decisions quickly, you probably need one person to make the decision. And they don't need, they need to avoid having lots of information flowing in. They just need to make a snap decision. Whereas time sensitivity gives you more options to design how that information feeds in, how the options are created and then discussed and then selected from if you have more time. Cost and scale. So, if you've got lots of small decisions to make that don't cost much if you get that wrong, that's a different kind of problem. If you've got smaller number of very large decisions, you need to obviously need more information and more deliberation about the bigger decisions. Information availability. Sometimes you have lots of information about your decisions. So, think about, again, marketing of stuff on websites. You've got loads of data about who's coming to the websites, when they come, what times, what kinds of devices they use. All of that can inform how you go about engineering your marketing campaign. Some decisions, like policy decisions in government, if they're bringing about a new big piece of policy, there may be no precedent whatsoever for that kind of thing happening. So, there's much less information availability. Repetition and feedback. How often is that kind of decision made? And if it is repeated, can you get feedback as to how well the decision, how good the outcome was of that decision? That will change the way you design your decision making. Technical specialties. Are they inherently technical things about the decision? Do you need specialists to be a key part of the sort of scenario generation, the option selection and the option identification and the option selection? And finally, accountability. Are the decisions you're making, does it require legally as well as sort of morally and socially? Does it need somebody to be accountable in some way for that decision? In which case, you need, legally, you need one person who's going to take the fall for bad decisions. And that will affect how you design your board.

Look, just talking about board. I mean, I can see what are the problems here with all the stuff that you're talking about here, Peter. On one level, I can see it totally makes sense. On the other, when you start talking about all the stuff, for me, it goes in one ear and out the other. It's just really, I can't be bothered with all that. Let's just make a bloody decision. You know, even if it's like about supply chain stuff that you talked about that can take years, it's just, oh, Christ, I can't be bothered with all that. And which makes me think, first of all,

There's not really a big surprise that Fraser is not a captain of industry.

Well, this is exactly the point I was going to make, because I think you're right and I'm wrong, actually. And so I think two things. One thing that you're going to come up against with this sort of thing is people like me who are just going to put a spanner in the works or become disengaged or whatever it might be. But also, that's why bad decisions get made is because of people like me. And I'm just forever going to be, you know, whether it's me or my organization, some, you know, peripatetic decision wrongmaker losing out on life.

I'm going to I'm going to know I'm going to I'm going to disagree with that. I don't know. In your case, I do not. I'm not informed enough about you. But I mean, the point about group decisions is they're different. Right. They are different on average to the decisions that any individual within them will make. Right. Now, if you've got a reason to believe that you're better at making decisions than people in that group, then your decision is going to be on average better than the decision of the group. I mean, it must be. It's like if you were really, really good. You know, you know, the whole sort of argument about about the wisdom of crowds and guessing the weight of an ox and so on. But if you are really good at estimating the weight of oxes, then you you should just go with that. And I know I'm not I'm not, you know. Yeah, I think I know what I was when I sort of won my first jobs as a manager. I was very keen on getting consensus for things. I was I was head of a team called Strategic Futures at one point. We wanted to change. I felt that the name was misleading. We wanted to change it. And I had a fairly clear idea that we should change it to Futures and Analytical Methods, which is a much better description of what we did. So, you know, I got all the team in the room. It's like, what do we think we should change the name to? And, you know, and then at one point I said, what do you think of this Futures and Analytical Methods thing? And yeah, it seems all right. And then one of my team who I who I actually had a lot a great deal of respect for said, Nick, just tell us, tell us what you're going to change your name to. There's no point. No, you're right, actually. Yeah, we should just change it to that. Let's go with it. The problem is, of course, that most people aren't very well informed about how good they are at stuff. And, you know, the well-known Dunning-Kruger effect means that if anything, people who are really bad at stuff are more likely to think they're good at it than people who are kind of OK at things. So I don't really know what to advise other than to observe that, you know, if you are better at making decisions than the people on your team, then you ought to be making them and not the people on your team. But the problem is that you probably aren't very good at working out whether that's true. Exactly. That's the downside.

Yeah. So, yeah, it's hard to operationalize that advice. Yeah. And one of the things you were saying there, actually, and as you were saying, it made me realize that something about myself, which I've worked in big companies, I've worked in small companies, and I'm much for working in smaller companies, just because the decision making is just much more fluid.

Well, it's much more discursive. I mean, I'm fairly new to being in a small company. And it's the way things are done is much more discursive. Because there's so few of you, you need to make a decision, even relatively minor ones. You get everyone on the hangout together and say, what do we think of this? What do we think the best way of going forward is? But that doesn't scale. You have to engineer it differently when you get bigger. You simply can't involve everybody in every decision as you get bigger. And I think that's why you like it, because you like talking to people, whereas I actively avoid it wherever I can.

Yeah, yeah, yeah, yeah. Which isn't hard, because everyone else is avoiding talking to him.

I'm the perfect, yeah. Just a quick question. Has there ever been a time when you've been in a particularly interesting situation or particularly important situation where you're making a decision either individually or as a group?

Any of you? Yes.

Sorry. Far away, Peter.

When I was a civil servant, I spent an operational tour attached to a headquarters in Afghanistan. So I got to witness firsthand how a brigade level headquarters operates, which is quite interesting. As you can imagine, a campaign like Afghanistan is very complicated. There's a lot going on. Aside from the military things that are going on, all of the stuff around to support that military operation is complicated. But then also all the stuff that's helping engage with local government, engage with national government, international government. All of that stuff is really there's a lot going on. So these headquarters are understandably quite enormous. There's lots and lots of people working 24 hours a day on managing and running these campaigns. So being one small cog in that large machine was a very interesting experience.

That sounds fascinating, actually. Yeah, yeah.

Nick?

Well, one intriguing insight, I suppose, into a very common category of collective decision making was when I was on a jury. So I got called up for jury service twice. The first time was a total bust because I turned up for about three days and eventually was sent home because they didn't need me, which was sad. But a few years ago, I was a juror in Wood Green. And that was absolutely fascinating because, you know, it really it really was the opposite of restoring your faith in human nature. It was the opposite of that. Undermining. Thank God I'm here, because otherwise this would be a shit show. And the first, luckily, I'd say about two or three people there who I would have trusted to make any kind of decision about anything. And the rest, I mean, you know, it was a combination of people who were entirely driven by intuition and anecdote and very difficult to keep focused on the topic in hand. It was all very sort of, oh, you know, I knew someone once who did that. And this happened and that happened. And you think, OK, but what about is this person guilty or not? Yeah, but I had a good argument that day when I said that. Yeah. So naturally, I was I was the foreman for one of these cases. And, you know, kind of boiled down to essentially the three sort of sticks in the mud who were who were clearly wanted to say, wanted to give a particular decision because of reasons that really didn't have very much to do with the facts of the matter. And eventually, you know, over the course of actually about about a day and a half, we finally ground them down and got them. You know, we got a majority verdict in the end, 10 to 2. And the judge accepted that. And but and yeah, but but, you know, there were there were people, I mean, people on there who I don't none of them were malicious. I mean, there wasn't anyone nasty on there. No one was motivated by sort of, you know, hatred or racism or anything like that. It was it was literally just sort of they were just using heuristics to form views rather than really trying to think about it, you know. And and I yeah, there was there's one guy who I really liked, who was a guy from Lincolnshire who at one point he said, well, I think that confuses me. What I want to know is when you said something there, I I'm totally convinced by it. And then but then when he said something over there, I thought, actually, he's right. I don't know what to think. I was like, actually, if they were all like him, it would have been much easier.

Whoever speaks last. Yeah, yeah. Yeah. Yeah, I will. I like that. Unfortunately, I don't have anything remotely close to either of those. But all it does is remind me of one of my favorite episodes of Happy Days. Is that what it was called? Happy Days these days? You know, with the Fonz? Yeah. Happy Days. Yeah. Yeah. Well, he's a member of the jury and everyone's against this person who's up for who's cruising as a kind of pickpocket come shoplifter type person who's going by. This is the accused going by on a motorbike and nicks a purse off a lady. And everyone's going, yeah, we need to send this person to jail. Except one person who's just not sure. It's not just it's not right. And of course, that's the Fonz. And he works out that actually, no, that this person must be innocent because the motorbike they were riding, the throttle was on the left because it was a British made motorbike. The throttle was on the left, not the right. So therefore, he couldn't have used the hand that they said. And then so to great, you know, glorious sort of applause, Fonz sort of gets the guy off the motorbike.

It's kind of the 12 angry men scenario. Yeah, exactly. Yeah, because your juries aren't actually explicitly not meant to do that. Right. They're not meant to do their own investigation in the jury. Now, having said that, there's there's no there really aren't any rules. I mean, you know, there's no no one checks what they're doing. And it's a bit of a black box, a jury room. But they do tell you, you know, don't you're not you really shouldn't discuss things that you happen to know about. Or, you know, so things like that will be explicitly if another member of the jury had decided to complain about that, they'd have been entitled to do that. I'm assuming it's the same in the US. Yeah. Yeah. You can't bring your own evidence to to, you know, a jury decision.

If it wasn't mentioned in court, it didn't happen. So maybe the Fonz should have been a lawyer and not a juror. Yeah. OK, look, let's wrap up there before we do. And on finishing, can I can I can anyone second me in this? Can we wrap up? Yeah, sure. Yeah. OK. And can we let's take a vote on that. I vote we wrap up. Yeah. Peter. All those in favor say Aye. Aye. Aye. Aye. Aye. Aye. Aye. OK. All right. Let's stop there. Thank you, as always, for listening to the Cognitive Engineering Podcast. I'm Fraser McGruer, being here with Peter Coghill and Nick Hare of Aleph Insights. Until next time, goodbye.