Case studies and Natural experiments
Natural experiments
Out of a diverse rooting in discussions about complexity, system thinking and the need to understand specific contexts more deeply, the classic experimental setting did at some point become more and more challenged. What emerged out of the development of field experiments was an almost exact opposite trend considering the reduction of complexity. What do we learn from singular cases? How do we deal with cases that are of pronounced importance, yet cannot be replicated? And what can be inferred from the design of such case studies? A famous example from ethnographic studies is the Easter Island. Why did the people there channel much of their resources into building gigantic statues, thereby bringing their society to the brink of collapse? While this is a surely intriguing question, there are no replicates of the Easter Islands. This is at a first glance a very specific and singular problem, yet it is often considered to be an important example on how unsustainable behaviour led to a collapse of a while civilisation. Such settings are referred to as Natural Experiments. From a certain perspective, our whole planet is a Natural Experiment, and it is also from a statistical perspective a problem that we do not have any replicates, besides other ramifications and unclarity that derives such single case studies, which are however often increasingly relevant on a smaller scale as well. With a rise in qualitative methods both in diversity and abundance, and an urge for understanding even complex systems and cases, there is clearly a demand for the integration of knowledge from Natural Experiments. From a statistical point of view, such cases are difficult and challenging due to a lack of being reproducible, yet the knowledge can still be relevant, plausible and valid. To this end, I proclaim the concept of the niche in order to illustrate and conceptualise how single cases can still contribute to the production and canon of knowledge.
For example the financial crisis from 2007, where many patterns where comparable to previous crisis, but other factors were different. Hence this crisis is comparable to many previous factors and patterns regarding some layers of information, but also novel and not transferable regarding other dynamics. We did however understand based on the single case of this financial crisis that certain constructs in our financial systems are corrupt if not broken. The contribution to develop the financial world further is hence undeniable, even so far that many people agree that the changes that were being made are certainly not enough.
Another prominent example of a single case or phenomena is the Covid pandemic that emerges further while I am writing these lines. While much was learned from previous pandemics, this pandemic is different, evolves different, and creates different ramifications. The impact of our societies and the opportunity to learn from this pandemic is however undeniable. While classical experiments evolve knowledge like pawns in a chess game, moving forward step by step, a crisis such as the Covid pandemic is more like the horse in a chess game, jumping over larger gaps, being less predictable, and certainly harder to master. The evolution of knowledge in an interconnected world often demands a rather singular approach as a starting point. This is especially important in normative sciences, where for instance in conservation biology many researchers approach solutions through singular case studies. Hence the solution orientated agenda of sustainability science emerged to take this into account, and further.
To this end, real world experiments are the latest development in the diversification of the arena of experiments. These types of experiments are currently widely explored in the literature, yet I do not recognise a coherent understanding of what real-world experiments are to date in the available literature. These experiments can however be seen as a continuation of the trend of natural experiments, where a solution orientated agenda tries to generate one or several interventions, the effects of which are tested often within singular cases, but the evaluation criteria are clear before the study was conducted. Most studies to date have defined this with vigour; nevertheless, the development of real-world experiments is only starting to emerge. Since this is only partly relevant for statistics, is has several ramifications for statistics that I will elaborate in in this section.
