The Cause and Effect of Better Decision-Making

How scientific evidence beats gut feelings when it comes to decision-making for your company.

How does a manager know which of two potential decisions will be most conducive to achieving her goals? As firms seek innovative ways to gain an edge, leaders are faced with tough, often expensive decisions, over which new programme or policy, to introduce. A new incentive system? A new marketing approach? Not being able to go down two roads at once, the choice they make will have a significant impact on the success of their corporation into the future. Given that choices are costly, wouldn’t it be great to experiment with programmes, and have a clear sense of not just the costs but also the benefits of each, before committing to one?

Randomised Control Trials (RCTs) are the standard methodology used by medical and pharmaceutical companies in clinical trials and, in recent years, the gold standard for quality research in social sciences. The trials have been used to develop evidence-based recommendations for governments and social organisations and have made a significant contribution to the identification and implementation of impactful poverty-reduction policies in areas such as skills training, agriculture and microfinance.

Despite their success in these areas, and their potential to improve decision-making, RCTs are not as frequently used in the corporate world. While internet-based firms routinely use A/B testing (which is similar to the RCT methodology) to evaluate different versions of their web-pages for marketing purposes, few use it to decide more broadly on corporate decisions.

INSEAD seeks to contribute to changing this: we are launching the INSEAD Randomised Controlled Trial Lab at the school’s Abu Dhabi campus. Funded by the Abu Dhabi Educational Council, the Lab will provide infrastructure and support to researchers using RCTs to conduct field experiments with organisations aimed at driving competitiveness, growth and productivity.

These may interest you:

Eliminating selection bias from decision-making

Just as big data opened the way for firms to use information to better understand their companies and enhance growth, RCTs hold the key to better policy design. Testing new policies or products using this methodology gives more conclusive results linking cause and effect.

Take the example of training employees. Organisations invest significant sums in courses that often appear to have a positive impact on performance. However if performance is measured on staff selected for the training, it may be the case that only the most productive employees were selected by managers to take the training, or that the best employees nominated themselves for the programme. If we only look at the productivity of the trained employees compared to employees who did not participate in the programme, we might overstate the impact of the training. Alternatively a training plan may appear to have failed simply because it was applied to a less productive group of employees who, even if they increased their productivity following the training, still appear less productive than the average worker (because they had a higher gap to bridge). In other words, programmes that look very effective on closer inspection may not be; and programmes that look like they destroy value may actually be creating value.

More broadly, when working off observational data, you can never be sure if you are observing the true causal effect because the targeted group is not random. This is where RCTs come in: a well-designed RCT will implement the target programme on randomly selected employees within the firm, and compare the outcome with a randomly selected group of non-participants. This effectively eliminates any possible selection bias. The results of the trial will not only provide the company with more accurate details about the impact of the proposed programme and the value it will bring compared to the cost of implementation, but how it should be administered, who should be targeted, which employees will gain the most from it, etc.
RCTs like this can consist of an intervention on the firm as a whole, on different work teams or employees, or on groups of small business owners or entrepreneurs. On a higher level, RCTs can help government departments and organisations avoid making bad (often very expensive) policy decisions or decision-making that affect entire communities.

The key advantage of RCTs for decision-making is that it allows for small scale experimentation on the benefits of a programme before the programme is rolled out to the organisation. As such it can allow for more innovation and risk-taking because the risks are controlled.

Measuring impact: Development Policies

RCTs are used de rigueur by many non-profit and social organisations to design development policy. The World Bank runs RCTs on many of its programmes as do regional bodies like the Inter-American Development Bank. Even governments deploy them, with, for example, Indonesia using the method to test whether identity cards would improve the delivery of subsidised rice to the poor, (they did).

The Abdul Latif Jameel Poverty Action Lab (J-PAL) formed in 2003, was the first organisation to scale up the use of RCTs outside the medical industry. In partnership with governments, foundations, international development organisations, and NGOs, it uses scientific evidence to identify the most cost-effective programmes to reduce poverty, by conducting RCTs on subjects ranging from education to agriculture, and microfinance to malaria prevention. The trials have tested diverse hypotheses such as the effectiveness of consultancy firms on SMEs in Mexico and whether a lottery prize could incentivise safer sexual behaviour in Rwanda to reduce HIV.

Several important results have come up from these efforts, such as the fact that most business skills training for small entrepreneurs do not have the long-lasting impact on growth that we hoped and, perhaps even more dramatically, that contrary to expectations, microfinance institutions have little effect on permanently raising the living standards of those with access to it.

RCTs and corporations

Scholars in management science and economics are now increasingly adopting this methodology to study the effect of corporate and government decisions on outcomes such as employee retention, career development, small firm growth and adoption of behavioral change.

Of course, RCTs, one of several tools available to us, are useful in certain (but not all) settings and will not always be able to answer every question one might have. But whereas the cost of running randomised control trials was once a prohibitive factor, today with the development of IT, the increasing use of data by firms and the growing knowledge of RCT methods, trials are more accessible. And companies that embrace this methodology, and are not afraid to experiment with new policies, will find the benefits are huge.

At the INSEAD RCT Lab, we hope to bridge business practice and science by collaborating with firms to run RCTs that will help provide hard evidence on the drivers of productivity and growth. The research produced will also help organisations to make better decisions for their company by giving a clearer understanding of the full impact of their choices.

We are beginning to engage with (and indeed, are still looking for) organisations that would like to partner with us in this effort. Given the links between corporate and community well-being, we believe the dissemination of evidence-based management recommendations to corporate leaders will help stimulate growth, productivity and competitiveness not just to firms but to whole economies.

Check out the video below on decision-making and how to execute it out! It could be beneficial to you and your company.