The incredible value of rapid review synthesis contributions #synthesis #openscience #evidencebased

Contemporary, complex issues in society and natural systems are best examined and informed by evidence. A recent contribution was done to explore two direct questions associated with the COVID-19 pandemic and pets. The description of the two driving questions and subsequent link to the paper is provided here. This is a brilliant tool to inform decision making and planning when a decision is needed immediately. One can envision numerous contexts wherein this tool can be applied to crisis management or situations in other disciplines in ecology and the environment wherein a rapid response is needed to mitigate a collapse or challenge. If you click though and read the review, there are several salient elements in this unique reporting formatting.

Key elements of a rapid synthesis review

  1. Apply a standard and replicable synthesis workflow. Here is the one I use, but any similar approach is viable provided you can support a preferred reporting item schema for meta-analysis and systematic reviews (i.e. PRISMA) but there are other guidelines.
  2. Scrape the literature with direct, testable questions as the focus. I propose testable be defined as need-to-know for crisis, but the examples I have seen also ensure that questions have the capacity to be answered in relatively binary terms and with a relatively high capacity to disambiguate related terms. You do not have the luxury of time to explore related concepts and synonyms.
  3. Use these questions to rigorously review the literature rapidly but transparently.
  4. Summarize the landscape of findings clearly by providing the number of publications and key sample sizes including the relative frequencies of key term conjunctions and concepts.
  5. Given the need for a rapid response, answer the questions with number of studies and not effect sizes. Describe the number of samples and methodology of studies that warrant description of outcomes to address a question.
  6. Finally, I would prefer to see a clear statement at the end of a rapid review reminding the reader/decision-maker that a. there are different forms of ‘no effect’ conclusions – i.e. limited evidence, many tests but no significant effects reported, likely heterogeneity, limited subset of methods; and b. a clear re-statement of the scope of the questions – i.e. how far can the answers be generalized based on the evidence summarized. In the example of COVID-19 and pets, describe taxonomic diversity in the literature to be able to safely, i.e. with reasonable confidence, conclude that pets are not vectors. This was evident in the reported results, but it is worthwhile to restate.

@ESA_org #ESA2020 abstract: The even bigger picture to contemporary scientific syntheses

Background/Question/Methods 

Scientific synthesis is a rapidly evolving field of meta-science pivotal to numerous dimensions of the scientific endeavor and to society at large. In science, meta-analyses, systematic reviews, and evidence mapping are powerful explanatory means to aggregate evidence. However, direct compilation of existing primary evidence is also increasingly common to explore the big picture for pattern and process detection and is used to augment more common synthesis tools. Meta-analyses of primary study literature can be combined with open data assets reporting frequency, distribution, and traits of species. Climate, land-use, and other measures of ecosystem-level attributes can also be derived to support literature syntheses. In society, evidence-based decision making is best served through a diversity of synthesis outcomes in addition to meta-analyses and reviews. The hypothesis tested in this meta-science synthesis is that the diversity of tools and evidence to scientific syntheses has changed in contemporary ecology and environmental sciences to more comprehensively reuse and incorporate evidence for knowledge production. 

Results/Conclusions

Case studies and a formal examination of the scope and extent of the literature reporting scientific synthesis as the primary focus in the environmental sciences and ecology were done. Topically, nearly 700 studies use scientific synthesis in some capacity in these two fields.  Specifically, less than a dozen formally incorporate disparate evidence to connect related concepts. Meta-analyses and formal systematic reviews number at over 5000 publications. Syntheses and aggregations of existing published aggregations are relatively uncommon at less than 10 instances. Reviews, discussions, forums, and notes examining synthesis in these two fields are also frequent at 2500 offerings. Analyses of contemporary subsets of all these publications in the literature identified at least three common themes. Reuse and reproducibility, effect sizes and strength of evidence, and a comprehensive need for linkages to inform decision making. Specific novel tools used to explore derived data for evidence-based decision making in the environmental sciences and ecology included evidence maps, summaries of lessons, identification of flagship studies in the environmental studies that transformed decision making, reporting of sample sizes at many levels that supported effect size calculations, and finally, reporting of a path forward not just for additional research but for application. Collectively, this meta-synthesis of research demonstrated an increasing capacity for diverse scientific syntheses to inform decision making for the environmental sciences.   

Session
Meta-Analysis and Beyond: Applying Big Secondary Data to Environmental Decision-Making

#rstats adventures in the land of @rstudio shiny (apps)

Preamble
Colleagues and I had some sweet telemetry data, we did some simple models (& some relatively more complex ones too), we drew maps, and we wrote a paper. However, I thought it would be great to also provide stakeholders with the capacity to engage with the models, data, and maps. I published the data with a DOI, published the code at zenodo (& online at GitHub), and submitted paper to a journal. We elected not to pre-print because this particular field of animal ecology is not an easy place. My goal was to rapidly spin up some interactive capacity via two apps.

Adventures
Map app is simple but was really surprising once rendered. Very different and much more clear finding through interactivity. This was a fascinating adventure!
Model app exploring the distribution of data and the resource selection function application for this species confirmed what we concluded in the paper.

Workflow
Shiny app steps development flow is straightforward, and I like the logic!!
1. Use RStudio
2. Set up a shiny app account (free for up to 25hrs total use per month)
3. Set up a single r script with three elements
(i) ui
(ii) server
(iii) generate app (typically single line)
4. Click run app in RStudio to see it.
5. Test and play.
6. Publish (click publish button).

There is a bit more to it but not much more.

Rationale
A user interface makes it an app (haha), the server serves up the rstats or your work, and the final line generates app using shiny package. I could have an interactive html page published on GitHub and use plotly and leaflet etc, but I wanted to have the sliders and select input features more like a web app – because it is.

Main challenge to adventure was leaflet and reactive data
The primary challenge, adventure time style, was the reactive data calls and leaflet. If you have to produce an interactive map that can be updated with user input, you change your workflow a tiny bit.
a. The select input becomes an input$var that is in essence the name of vector you can use in your rstats code. So, this intuitive in conventional shiny app to me.
b. To take advantage of user input to render an updated map, I struggled a bit. You still use the input but want to filter your data to replot map. Novel elements include introducing a reactive function call to rewrite your dataframe in server chunk and then in leaflet first renderLeaflet map but them use an observe function to update the map with the reactive, i.e. user-defined, subset of the data. Simple in concept now that I get it, but it was still a bit tweaky to call specific elements from reactive data for mapping.

Summary
Apps from your work can illuminate patterns for others and for you.

Apps can provide a mechanism to interact with your models and see the best fits or outcomes in a more parallel, extemporary capacity

Apps are a gratifying mean to make statistics and data more accessible

Updates
Short-cut/parsimony coding: If you wrap your data script or wrangling into the renderPlot call, your data becomes reactive (without the formal reactive function).

The position of scripts is important – check this – numerous options where to read in data and this has consequences.

Also, consider modularizing your code.

Check out conditionalPanel function for customization across tabPanels. Tips in general here for shiny.

Controls, controls everywhere, and change continues nonetheless

Typically, everyone chases experimental treatments – i.e. direct interventions – as the fundamental means to advance science. We do something to one set of subjects whether plants, animals, or people, and we monitor the other set designated as controls.  The paper ‘Ambient changes exceed treatment effects on plant species abundance in global change experiments‘ very nicely shows that in many if not all natural systems, the ‘controls’ change too.  We cannot ignore the fact that natural ecosystems are changing just as rapidly and just as much due to global change and climate as the treatments we might test.
Solution – do the work and check!

Significant ‘developments’ this last few days associated with how we use lands #evidence #synthesis

This last week has been busy with numerous evidence syntheses highlighting that location, location, location and land use patterns are critical issues.

(1) ‘North American diets require more lands than we have’ was published in PLOS ONE and discussed widely. A compelling map of land spared showed little remains.

(2) On the other side of the coin, retiring lands because of water regulations, limitations, and drought are an opportunity for conservation and restoration was published in Ecosphere. Tools mapped for California studying three endangered animal species highlighted that we do know enough to begin to make evidence-based decisions for strategic retirement.

(3) A compelling map of how America uses land was published at Bloomberg.

(4) Hydraulic fracking is now being considered in the region used a case study for the retired land synthesis in #2 listed above including a map of proposed lands open for leasing.

Implications

(a) Scientific synthesis rapidly advances the big picture and both different synthesis tools (maps, systematic reviews, and ideally meta-analyses too) and syntheses with different purposes facilitate a more balanced weighting of issues. Even better, reproducible syntheses provided by different sets of stakeholders would elevate discussion and decision making.

(b) Agriculture, restoration, and energy development (both sustainable and non) must be better balanced through contrasted, transparently aggregated evidence.

(c) The ecological services and functions we get from lands ‘for free’ are precarious and precious.

(d) Whilst we cannot ignore human needs (and their likelihood of continued increases),  it is hard not to imagine that a buffer for other living creatures should also be factored into proposed land use trajectories.

(e) Ecology, socioeconomics, and other fields need to much more rapidly crunch current evidence because the clock is ticking.