Questioning the status quo in methods

Note: The German version of this entry can be found here: Questioning the status quo in methods (German).

In short: This entry discusses why the status quo of scientific methods is insufficient to solve the problems of our time.

Questioning the status quo

Building on the general recognition that current methods are not enough to approximate knowledge towards the solution needed for the wicked problems which we face, the questions arises how we can meaningfully question the status quo in methods, and do this in a way that moves us forward. Too many enthusiastic questions on the status quo in the realms of scientific methods have been proposed without a clear understanding what is actually the problem, or better, which knowledge we are lacking. The current state of many lines of thinking urged us to move out of the dimensions of a normal science in the sense of Kuhn. However, calling out a revolution does not mean that all problems are automatically solved. On the contrary: concerning many aspects, we are still in a state of explicitly not knowing how to solve the wicked problems we face. Let us look at three examples.

Climate Change

Climate change is a complex phenomenon: at first, the realisation that human-induced climate change is happening was slowly emerging out of diverse scientific data sources and research approaches. While today, the majority of society has accepted that human-induced climate change is real, and that it is our responsibility to react and counteract, there is no general agreement on a best way to do so. Instead, there are calls for a globally orchestrated policy-response, which takes place are on a totally different scale compared to local adaptation. Most importantly, how can we convince citizens in countries which lead in terms of a negative contribution to climate change for decades to not only change their behaviour, but actually to contribute to reversing its catastrophic effects? This is the current frontier in research, and many promising suggestions and strategies are currently being investigated. However, since we may only know in retrospect how this problem was at least approachable if not completely solvable, after all, we need to further diversify our research approaches, and also consider the urgency of the problem. A 20-year longitudinal study won't do the trick - we need suggestions as soon as possible. The urgency and the wickedness of this problem showcase the need for novel methods to contribute to the approximation of solutions.

Actor participation

Research about normative challenges - as well as research about joined problem framing between actors and researchers - has been on the rise. More and more studies engage in the new contract between science and society. However, the roles and power relations of different actors within a system are deeply contextual, and so far the knowledge of such studies did not yet saturate into a more general understanding on how such study setting can be approached. While blueprints already exist and there is a growing understanding of relevant concepts, such as social learning, actor participation is still something that did not find its way into a broad diversity of textbooks, and available approaches are far from unified. Most of the normal sciences actually dispute whether the recognition of actor knowledge is actually contributing to scientific progress, and instead keep judging the difference between applied science and their own science. What is more, actor participation is approached from all sorts of disciplinary backgrounds. This increases diversity in terms of methods, but ultimately leads to more confusion because the different approaches rooted in diverse disciplines are often pitched as being either superior or inferior to each other. While it is clear that different methods have different values in a specific context, a comparison of different methods which allow for actor participation is widely lacking to date. However, only fair comparisons of diverse approaches may allow for a claim of which methodological approach has a higher validity in a respective context.

Sustainable consumption

The question how we shift our consumption towards being more sustainable is another thriving debate within sustainability science and beyond. While there is research focusing on global trade and its inequalities, there is equally research on individual behaviour and the motivations of consumers. Understanding behavior - and even more so - driving behaviour change in terms of sustainable consumption is to date a diverse field, with methodological roots in psychology, social science and many other domains. On the other hand, global supply chains and trade arrangements are part of totally different fields in sciences and these two scales are hardly matched. There is a clear gap between research focusing on supply and research focussing on demand. From a methodological standpoint, integrating the global and the individual scale, and supply and demand, already poses a very complex challenge, showcasing how a link between these diverse line of thinking will preoccupy research for the foreseeable future. Atomising challenges into smaller chunks that represent parts of the picture follows a long tradition in science, yet integrating these diverse approaches will be vital in order to take the whole depth and width of the challenges into account.

Three pathways of methodological innovation

We can thus conclude that urgency, wickedness, normativity, context, scale integration and many other challenges are currently recognised in the community of sustainability science researchers, and deserve more focus in order to generate solutions. This non-exhaustive list of problems already showcases that this is more easily proposed than actually done. Within sustainability science and beyond, there is almost an obsession to question the status quo, and to proclaim what methods should do. It sounds so appealing to "dance with the system" - a sentence borrowed from Donella Meadows (I always asked myself what that actually means). But how do we take the normative burden off of all the proposals for transformation and take a first step as individual researchers? Here, I propose three perspectives on how innovation in terms of scientific methods could be more concretely approached.

1) Invention of new methods Inventing new methods surely sounds exciting. There is a whole world of knowledge that awaits us, and new methods might be potentially able to unlock this new knowledge. Despite the spirit that motivates many researchers to this end, I would like to highlight that in the past, the invention of new methods often came out of a clear recognition of a lack of methods in order to solve a specific problems. The proposals of Interviews as a scientific method, or Grounded Theory, were rooted in the recognition of a lack of a suitable methodological approach for a specific problem. Take Open Interviews as an example, which allowed for an inductive recognition of the perceptions of individuals. This type of knowledge as such did not exist before, and the proposal of these new methods allowed for a gap to be closed. In order to close this gap, I believe that this gap needed to be recognized, which was rooted in the recognition of clear knowledge and experience with methods that were already there.

Take Fischers Analysis of Variance as another example, which basically took the implementation of systematic experimentation onto a completely new level. Fischer saw existing research at the agricultural research center he was working at as a statistician, and he saw a clear lack of a systematic method that allowed to generate knowledge in terms of pattern recognition based on the testing of a predominated hypothesis. Fischer knew the state of the art, and he recognised a clear gap in the canon of existing methods. What is more, he had experience in the already existing methods, which was a precondition for his formulation of the Analysis of Variance. The invention of new methods is thus quite often a gradual process in a continuous development, although historians of science often reduce it to one point in time. To this end, I propose to recognise more of a continuum in which such new methods are being proposed. Such innovative settings typically build on experience regarding existing methods, and through this a recognition of a gap within the already existing methods, or better even, of the knowledge which these methods can produce.

2) Relocation of existing methods Methodological innovation often build on existing methods, if these are used in a different setting or context. For instance, interviews were applied in the world value survey at a global scale, allowing for a global comparison of people's values based on tens of thousands of interviews. The method of structured interviews existed long before that, and one could argue that such larger surveys even predated the 1960 proclamation of Strauss and Glaser regarding interviews. However, a global comparison was clearly unique. Another example is how the Analysis of Variance was once widely reserved for deductive experiments, until the rise of data science and the availability of more data though the Internet and other sources led to the application of the Analysis of Variance in purely inductive settings. Another example would be the diverse applications of text analysis, which were once postulated for very specific setting, but are today applied in all sort of branches of science. The relocation of a method basically utilises an existing method in a different setting of context than it was original intended to be used. Again, this does not only imply that experience with the existing method is a necessary precondition for its relocation, but the real innovation potential comes out of the recognition of a gap, and that the relocation of the existing method may close this gap by creating new knowledge.

3) Recombination of existing methods Another possibility of creating innovation in methodology is through the recombination of existing methods. For example, with the rise of data becoming available through surveys, statistics allowed for the utilisation of diverse methods to analyse data coming out of surveys, and later also from structured interviews. Such innovation is rather gradual, and less recognised by the scientific community. Another example would be the utilisation of statistical methods for coded items from a Grounded Theory approach. While originally this approach focused on qualitative research, there are now examples how coded information is compiled in a table which is then statistically analysed. Such innovation is often a violation of the original rules or norms that were proclaimed with a method, and thus have also attracted controversy and criticism in the past. Nevertheless, recombining diverse methods builds on a deep understanding and even experience of a respective method, showcasing yet again how researchers created new knowledge by building on already established methodological approaches.

Some remarks on innovation and experience

These three proposed possibilities of methodological innovation are clearly building on Design Thinking, as the development of methodological applications can be understood as a design problem. Creating a methodological design is necessarily a commitment that may be criticised as static or even downright dogmatic by some researchers, yet the here proposed approaches for innovation should rather be seen as a starting point than as a new dogma. Instead, I propose a counterpoint to a hype for methodological innovation that may be appealing to many, yet makes it very hard for researchers to start with their specific research in my experience. There is a tendency in sustainability science to reject all methodological dogmas in order to be unbiased and more creative. While this sounds appealing in theory, I would counterargue in the spirit of critical realism: some observable realities can be accessed by existing methods, and some new activated knowledge that we try to unravel may demand methodological innovation. All this implies that - following Roy Baskhar - we should clearly reject any claim of any knowledge being perfect, independent of whether it is based on old or new methodologies. Knowledge can and most certainly will change.

However, the urge for novel methodologies is so strong right now that experience in existing methodologies is rejected altogether. I consider this to be a dangerous new dogma, and kindly ask future researchers to consider experience in empirical methodologies as an important basis for innovation. As I have clearly shown above, there can be more than just theoretical considerations why methodological innovation can build on experience in existing methodologies. Such experience should always include a recognition of its limitations, and in the long run there might be a greater recognition of new design approaches that shall drive methodological innovation.

The interconnectedness of different forms of knowledge; reflexivity of researchers and other actors; the recognition of complexity in systems; and the necessity of power relations in knowledge production are prominent examples of current challenges that methodologies as of yet have not solved. We are probably still far away from the creation of a solidified canon of knowledge to solve these riddles. One could argue that we will never solve these methodological problems altogether, but that the joined action and interaction while working on these problems will be the actual solution, and that a real solution as with past methods of normal science shall never be reached. This is unclear to date, yet we need to become aware that when we question the status quo of knowledge production and methodologies, we may not be looking for goals, but more for a path. There are several examples in sustainability science that follow this path, such as transdisciplinarity and system thinking. However, according to critical realism, it may be good to solidify innovative and novel methodologies to a point that enables as to take meaningful actions based on the knowledge we create.

---

The author of this entry is Henrik von Wehrden.