Mixed Methods
Note: The German version of this entry can be found here: Mixed Methods (German).
"Mixed Methods refers in the broadest sense to the combination of elements of a qualitative and a quantitative research approach within one investigation or several investigations related to each other. The combination can refer to the underlying scientific theoretical position and the research question, to the methods of data collection or analysis, or to the procedures of interpretation and quality assurance" (Schreier & Odag, p.263, definition based on Johnson, Onwuegbuzie & Turner 2007, p.123).
Mixed methods have become almost like a standard reply to any given problem that researchers define to be outside of the domain of normal science. These days, mixed methods are almost like a mantra, a testimony of beliefs, a confirmation of openness and the recognition of diversity. One could speak of a confusion concerning mixed methods, where researchers speak about ontology, when they should actually speak about epistemology; the talk of mixed methods is drifting into a category mistake that then becomes a categorical mistake. Mixed methods are one of the moon shots of modern science, they are proclaimed, envisioned and continuously highlighted, but the question is now: How do we get to the moon of mixed methods?
Contents
1) Knowledge integration - epistemological problems
The first set of challenges when trying to bring mixed methods into reality are epistemological problems. Among the most profane yet widely unsolved questions is the integration of different data formats. While we can code data into qualitative and qualitative information into tables, this can hardly do justice to the diversity of knowledge within science. Even the integration of data within tables poses many unsolved problems. For instance, different methods can have not only a different understanding but also diverging indications of validity and plausibility. While these are pivotal in the realms of quantitative data but also in logic, validity and plausibility may be totally wrong criteria regarding much knowledge from qualitative domains where context and transferability - or the lack thereof - are more relevant. Some forms of knowledge are important to many, while other forms of knowledge are important to few. Yet, these forms of knowledge are still important, and science often ignores diversity more strongly than we should.
While this is a trivial statement as such, it highlights how different form of discourse and knowledge production are historically rooted and more or less consolidated in schools of thought. There is an increasing tendency of research trying to create connections between diverse domains of knowledge, which is often a basis for methodological innovation. This does not only demand more efforts from researchers attempting this, but may also lead to researchers being rejected by both communities, because they mingle with 'the others'. This readily highlights the importance of a history of ideas within which we locate our epistemological identity, and within which we can clearly indicate and reflect about it. This further highlights the importance of bridging divides. Otherwise, communication becomes more difficult if not impossible. We need to understand that all parts of science look at parts of the problem, but emergence is something that is ideally built on integration between different parts of science.
2) Facilitation - the gap between the epistemological and the ontological
In order to enable successful integration and bridge the epistemological challenges with the ontological problems, facilitation becomes a key part of mixed method research. This demands first and foremost the willingness to be critical about one's own positionality and reflection concerning the critical positioning in the history of science. Otherwise, the limitations of the knowledge in the respective area of science is often ignored, leading to a wrong recognition of these limitations. Many conflicts between different disciplines and their methodological epistemologies are nothing but an unreflected rejection of the unknown. Consequently, it takes time and effort to not only locate yourself within the canon of knowledge production, but to also value and appreciate other forms of knowledge. A simple division into a better or worse highlights the shortcomings of our understanding concerning the normativity of methods.
An example of a tangible problem when facilitating diverse researchers that use different scientific methods is language. Misunderstanding is sometimes simply rooted in a lack of understanding, quite literally in the wording and jargon. In extreme cases, some disciplines even use the same methods but do not even realise this, because the individual methodological developments led to different wordings for the same details of the respective methodologies. Imagine how these challenges are growing exponentially if you even talk about different methodologies. This leads to the most important point in facilitation: trust. If everyone wanted to understand every method that every branch of science utilises, this would be at a tremendous cost of resources. Consequently, a lack of understanding can alternatively be overcome by trust, which is often rooted in joint experience. There is currently an increasing recognition of the importance of facilitation, yet this also needs to recognise the ontological challenges scientists face when attempting research built on mixed methods approaches.
3) Interpretation and beliefs - ontological problems
Much of current research is embedded into specific theories of science, and many scientists are not even aware to which theory of science they are counted. Despite a growing criticism of positivism, most of the current knowledge production still falls into this domain, and this creates also problems of mixed methods research. Positivism does not only create a ranking where some forms of science are - at least indirectly - considered of different value. Positivism also claims to generate objective truths, which - not surprisingly - creates a problem if other domains of science also claim to create objective truths about the very same mechanisms, entities and patterns. Modern science needs to integrate an active and reflexive theory of science, and some would argue that this demands an integration of ethics as well. How else would we evaluate the knowledge that science produces if we do not try to evaluate through different ontological lenses if it actually makes sense? There is a pronounced disconnection between the scientific knowledge production and how we make sense of it. Recent claims highlighted that there are some dimensions of knowledge that may be restricted to ethics, which could be seen as an argumentation that we need active knowledge about ethics in order to claim responsibility for our research results. Empirical research is currently mostly far away from claiming ethical responsibility for its research results, let alone the consequences of these. Time will tell if the disconnection between the epistemological and the ontological may be overcome. To this end, mixed methods research may pose a vital and effective cornerstone, as it can facility reflection through active research. By using research as a boundary object to bridge diverse domains of science, we may overcome our differences, and focus instead on our joint goals. We will have to see how quickly this may emerge.
Concrete design problems of mixed models
Beside the theoretical considerations highlighted above, the design criteria of methods allow for a clearer identification of concrete challenges that mixed method research faces. By going through the design criteria step by step, I would like to highlight some known problems, yet have to indicate that right now, the challenges seem endless, as mixed methods research is only starting to emerge.
The great divide between quantitative and qualitative research is probably the largest hurdle science faces, with a long established traditional distrust between these two domains in most branches of science. Data formats were already mentioned as a mere mechanical problem. However, there are more problems from the perspective of quantitive science. The larger challenges lie in a different direction: How can we translate quantitive knowledge into qualitative knowledge, or at least connect the two? The current distrust some people have in scientific results is a complex problem that highlights how quantitative knowledge exists, but how some things are probably best understood through qualitative knowledge. Contextualisation, perceptions and transformational knowledge are challenges that we only start to unravel, and longer conceptual boundary objects such as agency or emergence highlight the difficulties when we try to investigate phenomena and mechanisms that probably demand a mixed method agenda. Even the analogy of pieces of the puzzles we look at seems insufficient, because it implies a materialistic dimension, and much knowledge we lack is in fact vague and tacit.
The divide between inductive and deductive knowledge is equally severe, yet probably less recognised. Today, much research that claims to work deductively is however shifting towards a more abductive agenda, while still being embedded into schools of thought that claim to be deductive. This creates a deep rift in terms of the validity of knowledge, since it is exactly the positivist and their deductive approaches that claim the highest validity to the knowledge they create. One might even suggest that positivist knowledge encompasses a strong system of valuation, if not judgment, regarding individual forms of knowledge, where one might wonder what the purpose of these evaluations actually is? Despite these conflicts, is it increasingly clear that deductive knowledge faces severe problems (e.g. reproducibility crisis), and in addition, it becomes increasingly clear that many of the challenges we face cannot be solely answered through deductive knowledge production. The rift between deductive and inductive knowledge become deeper over time, yet the bridging between the two through abductive knowledge creates a connection that is slowly widening. Time will tell if both domains can get down their high horses, and value each other's strengths while acknowledging one's own weaknesses.
Integration of spatial scales is an obvious problem in mixed methods research, and the examples are almost too numerous to even find any starting point that makes sense. Global supply chains may serve as an example where the needs and wishes of individuals create ripple effects on a global scale, with organisations, countries and cooperations being examples of important mediators between these two scales. These dynamics create a complexity that is only slowly unfolding, and the persisting divide between Microeconomics and Macroeconomics showcases that many forms of knowledge need to be integrated, and the relations between diverse methods are as of yet widely unclear. Another prominent example is the relation between the global biodiversity crisis and local conservation measures. While there is an emerging agenda, and even policy efforts are underway, it is still a long way from an encapsulated system towards an active knowledge exchange across spatial scales.
The last design criterion mentioned here is time. For many reasons, science focuses strongly on knowledge about the present. While a stronger recognition of knowledge about the future emerged over the last decades, and there is also a long tradition to investigate knowledge about the past, all these domains are typically disconnected. This is - again - rooted in the different knowledge domains that we have, but in addition also showcases the isolation of certain disciplines that investigate either past, present or future. It is clear that a historian would not necessarily team up with a policy maker, yet is it seems necessary considering the challenges we face. There is work to do to learn how we can integrate these diverse knowledge domains and agendas.
Mixed method metaphors
Bokeh
The picture on the right allows us very clearly to depict a person. We can see a lot of details about this person, such as the clothes, a not too cheap umbrella, and much more. We can also see that it is obviously in a part of the world where is rains, where you have public transportation (both tram and buses), we see high buildings, but also low building, traffic, street lights - hence a lot of information that can be depicted despite the blurriness.
One of the nicest metaphors of interlinking methods is the Japanese word Bokeh. Bokeh means basically 'depth of field', which is the effect when you have a really beautiful camera lens, and keep the aperture - the opening of the lens - very wide. Photographs made in this setting typically have a laser sharp foreground, and a beautifully blurry background. You have one very crisp level of distance, and the other level is like a washed watercolour matrix. A pro photographer or tech geek will appreciate such a photo with a "Whoa, nice Bokeh". Mixed method designs can be set up in a similar way. While you have one method focusing on something in the foreground, other methods can give you a blurry understanding of the background. Negotiating and designing each method's role in a mixed method setting is central in order to clarify which method demands which depth and focus, to follow with the allegory of photography. Way too often, we demand each method to have the same importance in a mixed method design. More often than not, I am not sure if this is actually possible, or even desirable. Instead, I would suggest to stick to the Bokeh design, and harmonise what is in the foreground, and what is in the background. Statistics may give you some general information about a case study setting and its background, but deep open interviews may allow for the focus and depth needed to really understand the dynamics in a case study.
An example of a paper where a lot of information was analysed that is interconnected is Hanspach et al 2014, which contains scenarios, causal-loop analysis, GIS, and many more methodological approaches that are interlinked (see Additional Information).
Paintings
Another example of a metaphor that may help to understand a mixed method approach is classical landscape painting. You try to include all the details from a landscape in a painting, leaving out uncountable details, and generalising some information into something altogether different. The setting can be quite designed, with a canvas, a certain perspective, some brushes and a pallet of colours. Yet already considering the diversity of colours showcases the different ways how the natural colours can be approximated. Before the chemical industry thrived, painters went through great efforts to get specific colours, often at a high price. Yet analyses of such paintings show us the superiority of these paints, and indeed how many of such paintings stand the test of time. Another thing to consider is the way how many painters paint, maybe sketching out ideas first, creating charcoal sketches and experimenting with perspectives. Paint was often not only mixed but also applied in layers. Brush techniques were often an essential part of the specific character of a painting. Now consider all these details when looking at scientific methods. Just as landscape painting, scientific methods can be a combination of a canvas, some brushes and a few colours. Yet consider the diversity of painting we know today, and compare this to the diversity of knowledge we may gain if we combine a slightly different methods with some other perspective. What if a specific technique to document data may lead to novel insights? And how can we alter our pallet to gain a different representation of reality? And, last but not least, how does our knowledge integrate into previous knowledge? Many painters were strongly influenced by other painters, and schools of thought can be traced in paintings quite well, and are in the focus of diverse forms of research.
Integration, reflection and normativity are components of methodology that are slowly starting to emerge. While deep focus will probably remain the main strategy of most researchers, the wicked problems we currently face demand novel approaches in order to enable a transformation, and to investigate the associated mechanisms and structures. Much is still unknown, and we need to overcome the focus on resources, the claims of scientists to rank and judge knowledge of other scientists, and last but not least the deep entrenchment of scientific disciplines when it comes to their diverse methodologies.
To quote Nietzsche: There has never been such a new dawn and clear horizon, and such an open sea.
Additional Information
- Hanspach et al. 2014. A holistic approach to studying social-ecological systems and its application to southern Transylvania. Ecology and Society 19(4): 32.
A paper that illustrates the combination of various methods (including Scenario Planning, GIS, and Causal-Loop Diagrams) in an empirical study.
- Schreier, M. & Odag, Ö. Mixed Methods. In: G. Mey K. Mruck (Hrsg.). Handbuch Qualitative Forschung in der Psychologie. VS Verlag für Sozialwissenschaften | Springer Fachmedien Wiesbaden GmbH 2010. 263-277.
A good introduction to the Mixed Methods approach.
The author of this entry is Henrik von Wehrden.
