From Data Dependence to Informed Insight

Is it possible that your fixation on numbers is leading you astray in research or supply chain management? On this episode of the Chain Reaction Podcast, we promise to unravel the complex web of quantitative bias, guided by the illuminating insights from Linda Chang of the Toyota Research Institute. We explore the many faces of bias—selection, information, confounding, publication, and observer bias—and how they can stealthily manipulate research outcomes. With Chang's pivotal paper as our compass, we delve into the world of quantification fixation and its implications on decision-making. Our discussion sheds light on the vital importance of recognizing and mitigating these biases to maintain integrity in research findings.

As we transition from research to the realm of supply chain management, the conversation takes a reflective turn. We examine the potential pitfalls of an over-reliance on quantitative data and the art of balancing it with qualitative insights. Supplier reliability and communication practices become key players in our analysis as we advocate for a holistic evaluation of supply chain performance. By weaving in methodologies like Lean and Six Sigma, we illuminate how a balanced approach not only enhances decision-making but also secures a competitive advantage in today’s dynamic business environment. For those eager to explore further, our episode is peppered with recommended readings on sustainable supply chain management and multi-criteria decision-making methods. Join us for a journey that promises to challenge your thinking and refine your approach to both research and supply chain strategies.

Further Reading at From Data Dependence to Informed Insight – TONY HINES BLOG

Bai, C., Dhavale, D. G., & Sarkis, J. (2020). Sustainable supply chain management and multi-criteria decision-making methods: A systematic review. Journal of Cleaner Production, 256, 120334. https://doi.org/10.1016/j.jclepro.2020.120334

Coulter, A., & Kelly, S. (2018).

You can follow Chain Reaction on LinkedIn, Twitter and Facebook

Chapters

• 0:02 Quantitative Bias in Research
• 6:33 Balancing Quantification Bias in Supply Chains

Transcript

Tony Hines: 0:02

Hello, tony Hines. Here You're listening to the Chain Reaction Podcast, all about supply chain advantage, and we've got some great things coming along on the episode today. So have a listen, stick around, stay tuned and stay informed. Training Reaction. Now I came across an interesting paper by Linda Chang from the Toyota Research Institute, based in Palo Alto, and I think what's interesting about the paper is that it talks about quantitative bias. Is that it talks about quantitative bias? Now, before we begin on this particular summary, I want to talk about the different types of bias that can be introduced into any piece of research.

Tony Hines: 1:00

Quantitative bias refers to systematic errors that can distort the results of quantitative research. The biases can occur at any stage of the research process, and that includes the study design, the data collection, the data analysis and the interpretation that's made. Understanding and addressing quantitative bias is crucial for ensuring the validity and reliability of any research findings. Now, I was head of a doctoral program at a university for a number of years, and so I've come across this on a number of occasions as researchers begin to develop their ideas and their own notions of the importance of the work that they're doing, and one of the things that quite often occurs is the selection bias, and that occurs when a sample isn't representative of the population and can lead to a skewed result, and that's because the sample selection has not been done in a scientific way. Often people will go for data that supports their argument rather than data which could contradict it as well. Then there's information bias, and that arises from inaccuracies in the measurement or classification of variables, and that can happen due to poor in the measurement or classification of variables, and that can happen due to poor data collection methods or participant recall issues. So, for example, if you conduct interviews with people and you ask some questions about your particular topic, people have selective memories and they will give you answers from their selective memory rather than what may actually have happened.

Tony Hines: 2:28

There's confounding bias, and that happens when an extraneous variable influences both the independent and dependent variables, creating some kind of false association. Let's give an example of confounding bias just to make it clear. So if you conduct a study to determine if a new diet pill helps with weight loss, you might get two groups one that takes the diet pill and the other that doesn't the control group. If the pill includes more people who also engage in regular exercise compared to the control group, the results might show significant weight loss in the pill group. However, this weight loss may not be only due to the diet pill. It could be largely influenced by the regular exercise that those participants are doing. So in this case, exercise is a confounding variable. It's relative to both the independent variable taking the diet pill and the dependent variable, weight loss. And the confounding bias would lead you to incorrectly attribute the weight loss to the diet pill, when exercise might be a major contributing factor. So to overcome or to mitigate this confounding bias, we use techniques like random selection, matching or statistical controls to ensure that confounding variables are equally distributed across the study groups.

Tony Hines: 3:43

Then there's publication bias. That occurs when studies with positive results are more likely to be published than those with negative or inconclusive results. Editors have a penchant for decisive research and they will look at a piece of research and say what's the contribution here? How is this changing things? And if you come to an inconclusive conclusion and you say, well, on the one hand it could be this and on the other it could be that, but there's no definitive outcome and the hypothesis is not supported, they're far less likely to publish that than to publish something that says this is an exciting piece of research and the conclusion is, from the hypothesis and the data that we looked at, that A causes B. And then of course, there's observer bias results from researchers' expectations influencing the outcome of the study. So the researcher, remember, came to the topic because they were interested in it and quite often they're searching for evidence that will support their own hypothesis or their own argument, and that's to be avoided too. So, with those things armed, consider what Linda Chang said in a notable paper on quantitative bias. Its title is Does Counting Change what Counts?

Tony Hines: 5:01

Quantification Fixation Biases in Decision Making, published in the Proceedings of the National Academy of Sciences in 2024, and the paper explores how the fixation of quantification can influence decision-making processes. Here's a summary of the paper in case you're not going to go ahead and read it. The paper explored how people made decisions and how they're influenced by numeric metrics. The author conducted 21 experiments involving managerial policy and consumer decisions to investigate whether people make different choices when some dimensions of a choice are quantified and others are not. They found that participants systematically shifted their preferences toward options that dominated on quantified dimensions, and that's the phenomenon that's called quantification fixation. People tend to favor options that perform better on quantified dimensions, even if other qualitative aspects are equally or more important. Financial consequences this bias can have real-world financial consequences, such as hiring decisions and charitable donations. The authors suggest that numeric information is more fluent and easier to process than non-numeric information, which leads to the bias, and the study highlighted the importance of being aware of how quantification can influence judgments and decisions, suggesting that when we're heavily reliant on numbers, we may inadvertently change what we consider to be important.

Tony Hines: 6:33

Now, as I read this, I was cast back over 30 years when I was an undergraduate student in economics and thinking of how we looked at economics at that time and there was a trend to look at positive economics and so what we were actually doing. We were looking at data about what had happened to determine how we interpreted what was happening in economics and, of course, in a sense, there's a quantification bias built into that approach, because you're looking at the mirror of what's happened. It's a reflection of the past. You're looking into a mirror, seeing the reflection of the data that confirms what happened and that's a kind of confirmation bias in a sense. And at the time there was interest in normative economics, looking at what ought to be possible. Now this is important because in managing any business, we quite often look at the data and search for explanations of what's happened. And that can be extremely important and extremely helpful. And, you will know, in supply chains and supply chain studies, it's absolutely essential, it's paramount. We look at what has happened to understand a problem, but what it doesn't tell us necessarily in our interpretations is how we move forward to a world in which we want to be, in other words, what we want to change, and often that can be more important than the reflection of what actually happened. So, like everything in life, it's a balance. You need both. You need the numbers to tell you the existing position, and then you need supplementary information from other sources and a vision of what could be possible in the future. And taking the two together well, you hope for a successful outcome. Now let's square the circle.

Tony Hines: 8:25

You might be wondering why I've spent some time looking at quantification bias. Well, when we look at decision making in supply chains, we're often focused on optimising our supply chain management effort. In today's competitive landscape, effective supply chain management is critical for success. Insight from research on decision-making and bias, such as Linda Chang's work on quantification fixation, provides valuable lessons for supply chain professionals and here are five key takeaways and the specific application in supply chain management. So the first one balancing quantification and qualitative insights.

Tony Hines: 9:06

Quantification fixation, as highlighted in the earlier discussion, refers to the over-reliance on numerical metrics at the expense of qualitative factors In supply chain management. This can lead to skewed decisions if we focus solely on the metrics, like delivery times or inventory levels. Of course we recognize they're important, we need to know them, but it's not the total picture. When evaluating suppliers, it's essential to balance quantitative data, such as on-time delivery rates, cost, with qualitative insights such as supplier reliability and relationship quality, for example. While a supplier may have excellent on-time delivery rates, their communication practices and responsiveness to issues could be equally important for long-term collaboration. So it's like the horse and carriage you can't have one without the other.

Tony Hines: 9:59

Confounding bias in supply chain decisions needs to be understood. Understanding biases such as confining biases is critical In a supply chain context. This means recognising that certain variables might obscure true relationships. For example, a supplier's performance might appear better due to external factors such as a favourable market. If you have favourable market conditions, then you might associate the success of the business not with those favourable market conditions but with something that you did, and that might be wrong. So we need to be aware that we could be wrong in our assessment or interpretation and to overcome that. We use techniques such as randomisation and cross-functional reviews to identify and mitigate bias when analysing supplier performance. Consider both direct metrics, such as delivery, and confounding factors such as market demand variations, and this way, looking at the more holistic picture, we can make more informed decisions.

Tony Hines: 11:01

Just as iterative improvements in research methodology lead to better outcomes, continuous optimisation in supply chain processes can improve efficiency and effectiveness. By regularly reviewing and refining processes, we ensure that they are aligned with business goals and market demands. Implement continuous improvement programs like Lean and Six Sigma to regularly assess and improve supply chain processes, taking a Kaizen approach. Conduct periodic reviews of inventory management, transportation, logistics and demand forecasting, with the aim of identifying areas for improvement and implement corrective actions. Decision-making processes can be influenced by biases, leading to suboptimal choices In supply chain management.

Tony Hines: 11:49

Adopting balanced decision-making frameworks that incorporate both data-driven insights and expert judgment can lead to better outcomes. Use decision support systems and balanced scorecards to guide supply chain decisions. These tools can integrate quantitative data, such as performance metrics, with qualitative assessments stakeholder feedback, for example to provide a holistic view of the supply chain. Combining quantitative metrics with qualitative insights ensures a comprehensive approach to supply chain evaluation. This balanced perspective helps in making decisions that consider both measurable performance and qualitative factors that impact long-term success. So we can't take the humans out of the decision-making process. We can't be reliant on the numbers alone, and numbers are interpreted, of course, by humans, so the human is always involved in these processes. When assessing supply chain performance, combine metrics like order fulfillment rates and inventory turnover with qualitative insights from customer feedback and supplier audits. This approach ensures a more nuanced understanding of supply chain dynamics and helps identify areas of strategic improvement. And helps identify areas of strategic improvement. So, in conclusion, by integrating insights from decision-making research, supply chain managers can enhance their decision-making processes, mitigate biases and achieve a more balanced and effective supply chain. Continuous improvement and a holistic approach to evaluation are the key to staying competitive in today's dynamic business environment.

Tony Hines: 13:38

Now, if you're interested in this particular topic and you'd like to follow up, I've got some other reading that you can probably take a look at. Of course, there's Linda Chang's article, but there's also earlier work, and it might have struck a chord with you that this type of work into decision-making was something that lots of other management researchers have looked into. The first article I'm going to recommend is Sustainable Supply Chain Management and Multi-Criteria Decision-Making Methods. It's a systematic review. The paper reviews multicriteria decision-making methods applied in sustainable supply chain management and it provides insights into how these methods can be used to address various challenges in supply chains. That paper was written by BAI, dot, vale and Sarkis in 2020, and it was published in the Journal of Cleaner Production number 256. I'll put all the details in the notes so I won't bother reading them all out. I'll just read the authors as I read the papers.

Tony Hines: 14:43

The second paper is another systematic review, and these are often very good systematic reviews for understanding all the previous literature that's been involved in bringing us to the current point in understanding the topic, and this is a systematic review of strategic supply chain challenges and teaching strategies. This review discusses strategic supply chain challenges and how they are taught within university circles supply chain challenges and how they are taught within university circles. It highlights the importance of using real-world scenarios and simulations to teach supply chain concepts and I think that's important to understand the use of these real-world cases and simulations. It's something I've always advocated. Looking at these practical applications. That was written by Coulter and Kelly in 2018. Full reference in the note.

Tony Hines: 15:32

The third article is Decision Theory and Sustainable Supply Chain Management. It's a literature review, again, not a systematic review this time, but a literature review. This article examines the use of decision theory concepts in sustainable supply chain management research. It provides a comprehensive overview of decision theory concepts in sustainable supply chain management research. It provides a comprehensive overview of decision-making processes and applications in supply chains. So, again, very useful paper to get a grounded understanding of what's happening. And that was written by Garcia and Carovas in 2019, decision Theory and Sustainable Supply Chain Management a literature review, and it's published in Business Strategy and the Environment.

Tony Hines: 16:12

Then, of course, there's the work of Daniel Kahneman and Tversky looking at cognitive biases and their impact on the supply chain inventory decisions Theory and Example, and that was published in Operations Research. So you might already have come across Daniel Kahneman's work in Thinking Fast, thinking Slow, but here's one specifically focused on supply chains. The paper explores the impact of cognitive biases on supply chain inventory decisions and provides an example to demonstrate these effects. And the fifth and final article I'm going to recommend is a conference paper. It's called the Influence of Cognitive Biases on Supply Chain Risk Management in the Context of Digitalization Projects. It discusses how cognitive biases affect supply chain risk management in relation to those digitalization projects. Yeah, and we all know about those. Very interesting topic and very timely. That was published in 2021. It's by Smith Chan and it was published in the International Conference on Engineering, technology and Innovation.

Tony Hines: 17:18

So there we are. There's five papers you can follow up on if you wish, details in the notes. I hope you've enjoyed the episode and hope you found out something you didn't know before, and I hope it inspires you to follow up and think more broadly about how quantification bias and other biases can in fact, determine how you view the world of supply chains. And it might be something you want to just step back, take notes of and think am I being biased in the way I'm conducting my understanding of this supply chain project or this piece of work, and do I need to think more broadly and should I be looking more widely, maybe in terms of the data set I'm employing, and how can I avoid those biases that I've just learned about? So there we are. That's it. I'm Tony Hines. I'm signing off. I'll see you next time in the Chain Reaction Podcast. Thanks for stopping by. Bye, for now. You've been listening to the chain reaction podcast written, presented and produced by tony hines.