The Challenges of Replicating Research on Endangered Species

– We are currently witnessing a mass extinction event. In this context, behavior and cognition research can play a vital role in our efforts to conserve biodiversity. However, research on threatened species also poses additional challenges for maintaining rigorous reproducibility standards. We identify four main barriers to carrying out replication studies: resource availability, publication bias, regulatory constraints, and social factors. We argue that all four barriers are exacerbated when the focus is on threatened species, and that they are likely to persist in the future. Considering this, we suggest that researchers develop a systematic approach to identify and prioritize studies where replication is possible and likely to significantly improve both our knowledge of a species and its conservation. Where replication is not an option, we provide several recommendations aimed at ensuring the integrity of research on threatened species

High on the slopes of Hawai'i's largest volcano, researchers are closely monitoring the world's rarest corvid, the 'alalā (Corvus hawaiiensis). Tremendous preparation and planning have gone into the reintroduction of these birds to the wild. Previous attempts have failed, but the latest reintroductions have been preceded by carefully designed training to teach captive-reared birds about the environment prior to their release (Masuda et al., 2017). For the 'alalā, an understanding of cognition and behavior, built on scientifically rigorous experimentation and monitoring, could be essential to ensuring the survival of the species in the wild. The case of the 'alalā also represents an extreme example of the difficulties of replicating behavior and cognition research. When you work with every individual of a species on the planet, replication in the classical sense is impossible.
Reproducibility, as demonstrated through replication studies, is central to the scientific method (Popper, 1959). The reproducibility crisis in the social sciences (Aarts et al., 2015) has sparked widespread concern about the reproducibility of findings across multiple disciplines (Baker, 2016a;Forstmeier et al., 2017;Fraser et al., 2018). Replicating studies can be vital to ensuring the integrity of the scientific process, not least because it may help prevent questionable or fraudulent research practices (Forstmeier et al., 2017). Moreover, effective applications of animal behavior or cognition research, such as in conservation or farm animal management, rely on the existence of a wide and accessible evidence base (Berger-Tal et al., 2019;Sutherland et al., 2004). Surprisingly, while applied work requires strong empirical evidence (Sutherland & Wordley, 2017), replicating prior studies is not currently a priority in the field of conservation behavior. This may be because conserving endangered species or mitigating human-wildlife conflict is an immediate problem requiring novel solutions, and thus "learning while doing," i.e., adaptive management, is often employed to reduce uncertainty along the way (Allen et al., 2011). Yet, conservation interventions vary widely in their success , highlighting a need to understand how behavior and cognition contribute to these failures (Greggor et al., 2020). Basing conservation decisions on single studies, or extrapolating results from small sample sizes or effect sizes, is a high-risk gamble when the outcome could have large economic, societal or ecological costs, or the fate of an endangered species is at stake. This is even more concerning when we consider that the generalizability and reproducibility of behavioral and cognitive research findings is often hampered by the fact that animal research frequently uses biased samples from wider populations (i.e., a subset of 'strange' individuals, sensu Webster & Rutz, 2020).
Research on threatened, endangered or evolutionarily distinct species is vital, not only for aiding efforts to conserve biodiversity, but also for deepening our understanding of broader evolutionary theories. For example, evolution on islands can follow distinct patterns (Adler & Levins, 1994), with island species often exhibiting an extreme divergence in body size (i.e., dwarfism or giantism; Lomolino, 1985), or naivety toward predators (Blumstein, 2002). In the case of 'alalā, it is one of only a few known corvid species that naturally forage with stick tools, a behavior which is hypothesized to have evolved in island conditions (Rutz et al., 2016). Thus, studying island species, which are frequently among the most endangered in the world (Kier et al., 2009), allows us to test evolutionary hypotheses about the evolution of behavior and brains (Sayol et al., 2018). However, adopting a broad comparative approach and working with threatened species imposes distinct challenges for replication.
Here we identify what we consider to be the main barriers to replicating behavioral research on threatened species and propose potential solutions. Many of the barriers we highlight can also limit any new behavioral research on threatened species, but all are exacerbated in the context of replication. As many threatened species are on the path to extinction, there is often an increased urgency to use behavioral and cognitive research findings to inform effective solutions, with limited scope for 'correcting/identifying' mistakes in previous research. How do we balance these dire needs with the principles of robust science?

What Are the Barriers to Replicating Studies of Threatened Species?
Replication, in its strictest sense, requires that independent researchers apply the same experimental procedures in different populations. However, even if multiple populations of a species exist, there are many other barriers preventing new researchers from replicating previous work with rare or threatened species. These barriers fall broadly into four main, interconnected categories: resource availability, publication bias, regulatory constraints and social factors (summarized in Figure 1).
Resource availability is perhaps the most important barrier to replicating research on threatened species. Both working in the wild and maintaining populations in laboratories, zoos, other facilities, or for conservation breeding are resource intensive, requiring large investments of time and money. It is well established that there is a distinct lack of funding available for replication studies (Baker, 2016b). This lack of funds is exacerbated in the context of research on species that are under threat, where different agencies may have different funding priorities. For instance, science-focused funding agencies may emphasize novelty in proposed basic science research, while conservation-focused funding agencies may focus on direct application or conservation interventions. As a result, the application of behavior and cognition research to conservation often requires complementary funding from different sources. If some of the tenets of basic science research (e.g., satisfactory sample size) cannot be met in field studies of endangered species, obtaining large-scale funding is prohibitively difficult.
Compounding this issue, a widespread bias toward novelty in scientific publishing reduces the perceived value of replication studies (Nosek et al., 2012). Moreover, studies that are tied to conservation outcomes may suffer from another type of publication bias due to the perceived negativity surrounding "failed" interventions, which can be especially challenging when taking place under greater regulatory and public scrutiny, and can result in overestimates of success (e.g., Miller et al., 2014). As many funders partly base their decisions about who to fund on a proven track record of publishing in highly ranked journals, publication biases for novelty or positive results further diminish funding prospects and reduce the appeal of doing important replication work for individual researchers.

Barriers, Solutions and Benefits for Encouraging Research Replication in Small/Endangered Populations
Note. Barriers (in red boxes) stem from a variety of underlying factors (grey boxes). Solutions (in blue boxes) will depend on the location of the research and the nature of barriers that are present. Where full replication is deemed impossible, solutions are aimed at ensuring that original work is of the highest quality possible. Where replication is possible, improving the research environment and receptibility to such studies will make it more likely to happen. Ultimately, overcoming the barriers will provide a range of benefits (green boxes) to researchers working with threatened species.
A lack of resources for setting up new study populations for replication attempts may also be exacerbated by regulatory factors. Many authorities control the kinds of research that can be done with threatened species and may not grant permission for fundamental behavior or cognition research when it does not have a clear conservation benefit. For example, New Zealand's Wildlife Act (Parliamentary Counsel Office of New Zealand Legislation, 1953) regulates research with threatened species; the application process to carry out research with threatened species often requires a researcher to demonstrate the conservation benefits of their proposed activity. Thus, much of the fundamental research that has been done in a laboratory or zoo context on the cognitive abilities of New Zealand's endemic and endangered alpine parrot, the kea (Nestor notabilis; e.g., Heaney et al., 2017;Huber et al., 2008) will likely never be able to be replicated in wild populations. Similarly, if cognitive or behavioral research on animals that are deemed to be "charismatic" (e.g., elephants or dolphins) is not directly relevant for conservation, it may face greater scrutiny from conservationists and policymakers alike who may prefer resources to be allocated towards conservation interventions. From the perspective of policy makers and regulators, who face greater public scrutiny and must demonstrate progress in species conservation plans (Merkle et al., 2019), the time and resources spent on replication may take away from new projects, further diminishing the support for replication studies.
Finally, scientists working with endangered species must first build relationships with local and indigenous communities. These relationships can enrich research outcomes (Rayne et al., 2020;Ward-Fear et al., 2019) and are individually rewarding, but they are also complex to navigate, requiring time and the recognition that communities will have their own priorities that may not align with a researchers' goals (Berkes, 2009). There is likely to be minimal incentive for any of the parties involved to build these relationships if the only goal is to gain access to a new population for replication work. In human-wildlife conflict areas, for instance, local villagers and farmers may be more willing to help with the collection of behavioral data if the scientists' work benefits not only the wildlife, but the humans involved as well. Thus, the importance of recognizing both human and wildlife perspectives when implementing research is critical to long-term success (Mumby & Plotnik, 2018). It is also important to recognize that while a priority of the scientist may be to collect data (with a clear but longer-term interest in helping the wildlife and the humans), the affected community's main priority is the immediate protection of their resources and families (Barua et al., 2013). Working with local communities on projects that aim to affect longerterm, gradual change can be a sensitive process that requires trust (Hoare, 2015). Thus, careful decisions must be made about what studies should be done and when, meaning replications of previous work cannot often be a priority.

How Can We Work to Overcome These Barriers to Replication?
In the long term, the ideal outcome for promoting replication of behavior and cognition research would be the reduction or elimination of some of the key barriers outlined in Figure 1. However, without a fundamental shift in the way that scientific publishing, regulatory and funding decisions are made, these barriers to replication will remain for the foreseeable future. With the limited opportunities and resources available for replication, as well as the work required to build collaborative research programs with local and indigenous communities that include opportunities for replication, there is a pressing need to develop a system for prioritizing which behavior and cognition research on threatened species should be the focus of replication efforts. Here we focus on actions that researchers can take, although we acknowledge that many of the barriers can also be addressed by refocusing the objectives of academic funding and publishing institutions as well.
Developing a decision-making framework for how and when to replicate will require considering how the information will be used. For example, higher priority may be given to replicate a single study making a novel scientific claim that has yet to be replicated, and/or that could have important conservation applications. By contrast, if a study is reporting on the existence of an already well-studied phenomenon, but in a new species, it may merit a lower ranking. Clarity on what we already consider to be reliable evidence is also key to developing a prioritization system (Salafsky et al., 2019). Does peerreviewed, published literature alone count as evidence, or should we include "grey literature," such as student theses, when deciding how to prioritize replication efforts? Justifying why the existing evidence base is insufficient is also important for convincing funders what is at stake when seeking funds for replication. To support the development of a decision-making framework for prioritizing replication efforts, researchers should ensure that their studies are searchable (e.g., by using informative key terms) to increase the likelihood that their research is picked up as evidence and acknowledged.
Ultimately, replicating studies that are deemed high priority will likely still require the efforts of multiple independent research groups (Farrar et al., 2020). If more than one research group works on the species, forming reciprocal replication partnerships may be a crucial step in terms of achieving replication with the least resource use possible (e.g., Many Primates et al., 2019). Running multi-laboratory studies can also yield more representative study samples, potentially improving the reproducibility of findings (Voelkl et al., 2018). Moreover, if an entire study cannot be replicated outright, researchers could still aim to incorporate key methodological elements of previous work into new protocols, to at least attempt to reproduce select findings of previous research. However, if any attempt at replication is prohibitively challenging, what are the alternatives?
In some cases, inferences may be possible from more common, closely related species. Early research into hand rearing for the critically endangered 'alalā and aga (Corvus kubaryi) used the much better studied raven (Corvus corax) as a proxy species (Valutis & Marzluff, 1999). This example highlights how knowledge generated in behavioral and cognitive research on closely related species may help to inform conservation interventions, or the development of research designs for endangered species. However, the assumption that closely related species will share cognitive and behavioral traits may not always be accurate. For example, Asian (Elephas maximus) and African elephants (Loxodonta africana and L. cyclotis) are often assumed to be similar in behavior and cognition but recent evidence suggests this may not always be the case (for a review, see Jacobson & Plotnik, in press). Thus, if a researcher's aim is to strengthen the evidence base for species-specific behavioral or cognitive traits, then inferences from closely related species cannot, in principle, replace within-species replication.
Where replication is deemed infeasible, there are other ways that researchers working with rare or threatened species can ensure that their research is at least as robust as possible. There are key steps that behavior and cognition researchers can take to maximize the clarity and quality of their research practices. These include greater transparency in the data collection and analysis process and, when possible, an effort to increase the reproducibility of a study's methodology even before it is conducted (e.g., via the use of Registered Reports; Nosek & Lakens, 2014). During the initial stages of research development, computational models may also aid in the development of key predictions. In addition, researchers should archive or allow access to footage of experiments whenever possible, as this allows for independent review by researchers who are blind to experimental conditions or the identities of individual animals. Similarly, following gold standards of open access publication by making all data and code used for statistical analysis freely available (e.g., by archiving in data repositories) allows others to verify that the analytical approach used supports the claims being made (Farrar et al., 2020;Powers & Hampton, 2019). Even when there are no additional populations with which replication work can be conducted, by seeking training in best research practices and adopting these measures, researchers working with small populations can still ensure that their research is robust and transparent.
As scientists working with threatened or endangered species, we recognize the importance of conducting behavior and cognition research both to our overall understanding of evolutionary processes and to the application of basic science to conservation practice. Although we argue that reproducibility is often an unrealistic goal when working with such species, here we have proposed a solution to this problem in two parts. First, in cases where replication can be done, a rigorous, collaborative approach should be taken to prioritize where limited resources should be used and to maximize the benefits of replication efforts, while also considering the impacts on in-situ wildlife populations and local communities of people. Second, where replication is not possible, we suggest that scientists try to use a range of tools and approaches both before they begin, as well as during any novel research, to ensure their work is robust and transparent. Unfortunately, inherent in the study of endangered species is the real likelihood that extinction may occur before existing research can be satisfactorily replicated. Ultimately then, when working with species under threat, we should always strive to maximize the quality of the work we can do to ensure it can stand on its own.