informational issues with health research: scale, structure, and trust (version for faculty)

https://hardsci.wordpress.com/2016/08/11/everything-is-fucked-the-syllabus/

who am i?

• Pharmacy Librarian
• Previously worked with post-graduate medicine and public health at UBC
• Also support reproducible research, open science, research commercialization

First: Let's talk about my problems

it's surprisingly hard to get people to care about information

things best learned by doing

https://www.flickr.com/photos/aslakr/14507479/

you need to care about the content to get adequate feedback when learning information systems

information *seems* so easy now

https://www.flickr.com/photos/christoph_straessler/10142109363/

Problems are more interesting than solutions

3 types of problems:

• Problems of scale and structure
• Problems of trust
• Problems of method

problems of scale and structure 

https://www.flickr.com/photos/worldoflard/8264273951/

http://www.sustainable.soltechdesigns.com/peak-science.html

https://www.nlm.nih.gov/bsd/stats/cit_added.html

Bastian, H., Glasziou, P., & Chalmers, I. (2010). Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Medicine, 7(9), e1000326. http://doi.org/10.1371/journal.pmed.1000326

The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta‐analysesThe Milbank Quarterly

Sources and the fragmentation of health information

Article Databases

• Disciplinary Databases
  • Medline/Pubmed (& PMC)
  • EMBASE
  • SciFinder
  • CINAHL
  • Cochrane
  • ERIC (education)
  • PsycInfo
  • Sociological Abstracts
• Dissertations & Thesis
• Conference proceedings
• Institutional Repositories
• National/region databases (LILACS)
• Unindexed Publications

Data & Grants

• Research Data
  • Article Supplementary Files
  • Disciplinary Repositories
  • Institutional Repositories
  • Some server somewhere
  • The authors desk
  • Lost forever
• Grants
  • Funding agencies websites
  • Grants.gov
  • Foundations
  • Pivot

Grey Literature

"That which is produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers."

The Fourth International Conference on Grey Literature

Created September 2016

Updated December 2017

(information privilege and the academic library)

<dc:problems>metadata</dc:problems>

• Language is variable
• Errors are everywhere
• Author's don't write to be found
• Indexing is too complex for most users
• Indexing is bad for interdisciplinary and social determinants of health research

Organizational Complexity 

• Cannabis
• Medical marijuana
• Marijuana smoking
• Marijuana abuse
• Cannabinoids
  • Cannabidiol
  • Cannabinol
  • Dronabinol

Representations of Marijuana in MeSH

Strategies

• Pragmatism & the usefulness equation
• Clinical Searching VS Research Searching
• Think about stakeholders and authors
• Pareto principle (80/20 rule)
• Evidence-Based Practice 
  • To what extent is critical appraisal even possible on an article level?
  • Problems w/ Systematic Reviews

Value of medical information =

relevance x validity / work

problems of trust

(publication bias)

(missing data)

Moving goalposts

Vedula, S. S., Bero, L., Scherer, R. W., & Dickersin, K. (2009). Outcome reporting in industry-sponsored trials of gabapentin for off-label use. The New England Journal of Medicine, 361(20), 1963–71. http://doi.org/10.1056/NEJMsa0906126

• alltrials + opentrials
• pre-registration (clinicaltrials.gov)

Fraud

Retractions

Bakker, C. J., & Riegelman, A. L. (2017). The Complexities of Retractions: Data Integrity, Discovery, and Tools. DOI: 10.17605/OSF.IO/7CXAK

Errors

Prevalence of gene name errors in supplementary Excel files. a Percentage of published papers with supplementary gene lists in Excel files affected by gene name errors. b Increase in gene name errors by year

Ziemann, M., Eren, Y., El-Osta, A., Zeeberg, B., Riss, J., Kane, D., … Tomashevsky, M. (2016). Gene name errors are widespread in the scientific literature. Genome Biology, 17(1), 177. http://doi.org/10.1186/s13059-016-1044-7

Reproducibility

Content (Science)

• Was the study design appropriate
• Was the study adequately powered?

• Was the statistical analysis appropriate?

Package (Reporting)

• Was published?
• Is it findable and accessible?
• Does it have complete the there enough methodology detail?
• Are the data, code, & research materials available?
• Is there a way to verify that the study was conducted according to the original study plan?

steps towards reproducible research

• Improved study power/design, & statistical analysis
• Methodological reporting and reporting guidelines
• Pre-registration
• Data, code, and materials sharing and citation
• Computational methods****
• Incentives
• Promoting replications

OPEN

Flickr user: scottishgovernment

• Open Access
• Open Data
• Open Science

Open Access

Open Data

Open Science

(thank you)