Is ecology replicable?

There are very few domains of ecology for which we know any general laws. This is particularly true in the “mess” that is community ecology. Even most recent attempts at conceptual unification rarely go beyond the fact that community ecology is driven by selection, drift, speciation, and dispersal. But so does everything else, as far as living organisms are concerned. Not that we should abandon community ecology, but we need to recognize how little we are able to generalize any of the things we know. Can we replicate ecological results?

Equilibrium in nature is the exception.

There is an excellent paper by Benincà and colleagues, which you should read if you are interested in population dynamics. They isolated a part of a food web in a mesocosm, and surveyed the abundance of different species, for seven years. Over this time period, the best predictions they could make covered two to three weeks. Similar results have been observed in microbial food webs or at the scale of ecosystem processes. Even in an isolated, stable, protected system, ecological systems behave in a chaotic way: the slightest change will send ripples propagating throughout the entire system.

The implications are many: if you measure the same system at two different times, you will observe two (possibly widely) different results. Is there a correlation between biomass and trophic rank? It depends! Is the system locally stable? It depends! And since within-system replication is difficult, this leaves very little hope to have satisfying cross-system replication. Is ecology is a series of noteworthy examples tied together by broad theoretical frameworks? Let’s hope not!

But in a way, the inability of our field to identify general laws stems not from the fact that mechanisms are complicated, but from the fact that simple mechanisms tend to generate chaos. If a system is sensitive to initial conditions, and if there are positive feedback loops acting at different timescales, then the potential for replication decreases.

And it will get worse before it gets better.

As the climate and environment and communities and almost everything else around us changes, any observation made of an ecological system is the product of intrinsic mechanisms (those we might want to uncover to understand what the general laws are) as much as it is the product of extrinsic ones. We never observe the same system twice, because its surroundings have almost certainly changed. So in addition to intrinsic chaotic dynamics, we have to deal with the fact that ecological systems are pushed and pressed in all sorts of directions by external forces.

It is therefore unclear how much any observation is the outcome of the expression of general laws, of internal chaos, or of transient dynamics due to long-term, broad-scale change. With this in mind, if ecology faces a replication crisis (which I do not think it does, if only because it is not immediately clear that we even have a strong culture of replication), it may be due to the fickle nature of ecological systems before being due to any statistical corner-cutting by ecologists. That’s right, I am blaming nature.

This has far-reaching implications for the way we do science.

The problem then becomes that, if you observe a pattern (I can’t believe I am using this word, and will come back to it in the next post), what more do you know? The hypothesis that you will know the process through the observation (or the repetition thereof) may be true in a relatively stable world, but I do not think it holds in a variable one. And for this reason, we should be very careful about declaring that an empirical observation contradicts a theory: if there is such a high degree of variation in empirical observations, then it takes a large enough number of them to confidently reject a proposition.

Not that there are no zombie ideas in ecology or evolution. Not that if the facts are not matching with the theory, we should change the facts. Not that empirical ecology is a lost cause. Either of these statements are laughable. Instead, there should be a greater sense of caution when we evaluate the evidence in support of, or against, a particular theory. And since Occam’s razor cuts both ways, we should be extremely cautious about claiming a model “works” because it predicts some empirical data.

I doubt that ecological results are easily replicated. Because ecological systems vary a lot, and on scales and in ways which might be difficult to apprehend, it is unlikely that our ability to make predictions is going to increase any time soon (and for the same reason, replication will remain difficult). In a sense, it reinforces the need for a more global approach: data synthesis, meta-reviews, and larger cross-system comparisons.

There may not be general laws, but we won’t know for sure until we take a step back.