Ripening stages of methods
In a nutshell: Some scientific methods are widely established, other methods have only just been postulated. There are some general patterns on how the range between these two extremes can be structured. Let us try and dissect the different stages.
All methods evolved at some point. Take the example of the Analysis of Variance in statistics, which was postulated by one person - Fisher - and then went on to be developed further by many other statisticians, and was subsequently applied by millions of scientists. Yet, there are some disagreements on how to utilize the methods, which preconditions are to be met, how or if at all to reduce an ANOVA model, and how to interpret it. In other words, after the original proposal by Fisher a normative flexibility evolved how to use and interpret ANOVA models best. Any such norm how to use a method best can be either more positive or negative. On the one end a norm can testify that diverse modifications were tested and stood the test of time. On the other hand, methods may also be stuck and be held onto for sheer dogma or out of latency. Here, we focus on methods in terms of depth and division, asking mainly how long a method has been established and how much it unites or divides a scientific community or group of researchers applying a certain method. An important preamble is that this text will ignore the complex effects of linguistic differences, because sometimes methods which evolve in parallel in different disciplines are perceived as distinct only because a different terminology is used. For example are Principle Component Analysis abundantly used in psychology and ecology, but both disciplines do not use the same wording for components, processes and results of this model. Hence, there is often a large linguistic fuzziness concerning the application of a certain method, yet the intention may be the same. Much of such fuzziness can also be part of a larger paradigmatic difference that goes beyond one single method. For example, some disciplines have a tendency to be more inductive, while other disciplines are more deductive, yet both utilize the same method. There are important ramifications and consequences between such higher order problems that create deep ripples from a philosophy of science stance. We will build on some examples for illustration, yet it is clear that these examples are far from being comprehensive. Time will tell how we solve them. Let's begin.
A level necessary to know but not technically a ripening stage are the preconditions that have to be met for a scientific method. Many methods depend on assumptions that don’t really have anything to do with the method itself, but are still important to consider. For example, the setting for interviews is important to consider, but this is something that is considered before the actual knowledge production process begins. Another example is frequentist statistics, which depend on the data distribution as a precondition. Yet, this says very little about the statistical tests and models depending on the preconditions. However, for beginners, these preconditions are often already a struggle, which is why it is so important to note them as a level 0.
Level 1 is defined as everything that is united by universal agreement. For example, documentation is essential in every form of interview, given the interviewee gave their consent. Equally would anyone modeling a regression check the R2 value to know how much the model explains. About all these factors exists a universal agreement, at least for the time being, and any dispute or discourse has been settled a long time ago. Such basics have not only been established, but also stood the test of time. There are different time scales at work here, though. In genetics an analysis may be outdated within a few years, while in statistics it may be several decades until an established norm changes. P-values are such an example, as these were established more than a decade ago, yet now slowly get replaced by other evaluative criteria. Another extreme example is Bayesian statistics, which was postulated centuries ago and is now slowly unleashing the Bayesian revolution. This example leads us towards level 2, because Bayesian vs Frequentist statistics is an example of a paradigmatic clash.
Level 2 - different paradigms
Within the development of scientific methods, there are often divisions between different paradigms. These can be defined as classes between different schools of thought. More often than not, these are divisions along the lines of progressivists vs. conservatives, or top-down vs bottom-up approaches. A prominent example is the division along the lines of induction vs. deduction. It’s perceived as an important demarcation line within scientific discourses yet rather shows two flipsides of the same coin. Sometimes, such debates about demarcation lines become atomized, and every larger research group suggests their own paradigmatic approach towards it. These debates at first seem like valuable discussions, but they are often linked to reputation, funding, dogmatism, and insistence on being right. Such divisions become hard to heal, and may only quite literally die out with the people keeping them alive. At this point, it is clear that we do not talk about active developments of methods any more, but more of stubbornly clinging to one's opinion. Here, the scientific community usually takes over and solves the problem through peer-review and the rule of majorities. While surely not every majority is always right, it can be a pragmatic step towards progress, especially if the partition between different communities creates more harm than good. After all, such divisions are often more clear in the scientists that postulated them as compared to the other researchers that try to understand them through reading available scientific sources. Peer-review and other scientific publications can thus become a filter for a certain paradigm for better or worse. After all, it may even be textbooks that are creating the ultimate breakthrough moment for a certain method, school of thinking, or paradigm. Hence, it is on level 2 where it is mainly decided whether methods thrive or die. However, the rise of methods starts much earlier, and this brings us to level 3.
Level 3
Methodological innovation usually starts with a single person or a smaller group of people somewhere. These starting points are blurry and often unclear to track down, because the information spreads often fast and cannot be tracked down to its origin. What is however often the case is that there is a certain resistance to novel ideas or innovation, and few people actually try to utilize or tame a new method. Despite science being supposed to constantly innovate, many scientists are quite prone to resist change. Hence, a limbo phase follows any proposal of a new method: the tinkering phase starts during which researchers tinker and toy with a method, often with diverse outcomes. In this phase, knowledge about the method is slowly building up, and recognition from other scientists finally paves the road towards a broader methodological innovation. Like a watery soup, the new method is slowly simmering into something more substantial. Yet, since most researchers - or at least the normal scientists - still avoid the novel method or lack knowledge for implementation, usage is still sparse and spread out. When there is little exchange between different supporters of the method, there is no systematic support that merits the label of a school of thinking. It is, however, encouraging to see how many researchers come to the same conclusion despite being completely unconnected. On the one hand, this is testimony of the rigor and value of science, on the other hand, this is often a phase where accusations of plagiarism and intellectual theft emerge, and tensions run high. To give this a positive spin, good ideas will flourish and multiply. Hence, when several scientists come up with the same idea independently, it confirms that the idea is valuable.. Level 3 ist the level furthest from normal sciences. Researchers operating in this space either do this once and then stick to their one innovative idea and become one-trick-ponies, or are constant innovators. Surprisingly, there are very few researchers that operate in between. Yet methodological innovation can often take decades, hence a designation of scientists from a methodological standpoint into normal or- non-normal scientists is somewhat piecemeal. Citations are an indicator whether a method became established, as are textbooks. Another indicator can be the ratio between conceptual and empirical papers. However,t these factors all depend very much on the respective discipline and are hardly generalisable.
What is however overall true is that all scientific methods were conceived at some -or often several - point(s). The methods subsequently may spread without becoming the accepted gospel, and either become generally established or contribute in parts to became different schools of thinking. We can thus find several mechanisms how scientific methods evolve. The bigger question would be now, why scientific methods evolve or are conceived. All novel methods fill a void in our knowledge that are strata of knowledge that were previously unknown. Time will tell if the method already lingers and vanishes that early, or if it may grow into a more mature state.
The author of this entry is Henrik von Wehrden.
