Revolutionizing scholarly writing with Manubot

a foreshadow of publishing's future

slides.com/dhimmel/force2018

#FORCE2018

October 12, 2018 1:30 PM

New Residence Hall Room C14

McGill University, Montreal

Abstract

Manubot is a system for writing scholarly documents on GitHub. It aims to transform publishing to be transparent & reproducible, immediate & permissionless, versioned & automated, collaborative & open, linked & provenanced, decentralized & hackable, interactive & annotated, and free of charge. Manubot accomplishes these goals by applying the workflow from open source software to academic writing. This talk will introduce Manubot and its features. Compared to other methods for writing scholarly documents, Manubot is designed for more technical users as well as infrastructure providers. However, we hope many of Manubot's innovations — such as citation-by-identifier and end-to-end reproducibility — will be adopted more broadly. Hence, this talk should engage all conference participants by foreshadowing a future of publishing towards which we strive!

https://force2018.sched.com/event/F7uX

what is Manubot?

system for writing papers via GitHub

powering the next generation of scholarly manuscript

  • transparent & reproducible
  • immediate & permissionless
  • versioned & automated
  • collaborative
  • open & free of charge

Open collaborative writing with Manubot
Daniel S. Himmelstein, David R. Slochower, Venkat S. Malladi, Casey S. Greene, Anthony Gitter
Manubot Preprint (2018) https://greenelab.github.io/meta-review/

Manubot mechanics

1. modify the manuscript source

2. continuous integration rebuilds the manuscript

Timestamped on the Bitcoin blockchain via OpenTimestamps

3. continuous deployment back to GitHub

https://greenelab.github.io/scihub-manuscript/v/17dd2015ec9396e59cdbc3c2b0abb7d33481bcbf/

Beyond the PDF First Day Notes by De Jongens van de Tekeningen. CC BY 3.0

input

output

manubot process

open source software workflow

This is a sentence with 5 citations [
  @doi:10.1038/nbt.3780;
  @pmid:29424689;
  @pmcid:PMC5938574;
  @arxiv:1407.3561;
  @url:https://greenelab.github.io/meta-review/
].

citation by persistent identifier

References

  1. Reproducibility of computational workflows is automated using continuous analysis
    Brett K Beaulieu-Jones, Casey S Greene
    Nature Biotechnology (2017-03-13) https://doi.org/f9ttx6
    DOI: 10.1038/nbt.3780 · PMID: 28288103 · PMCID: PMC6103790
     
  2. Sci-Hub provides access to nearly all scholarly literature.
    Daniel S Himmelstein, Ariel Rodriguez Romero, Jacob G Levernier, Thomas Anthony Munro, Stephen Reid McLaughlin, Bastian Greshake Tzovaras, Casey S Greene
    eLife (2018-03-01) https://www.ncbi.nlm.nih.gov/pubmed/29424689
    DOI: 10.7554/elife.32822 · PMID: 29424689 · PMCID: PMC5832410
     
  3. Opportunities and obstacles for deep learning in biology and medicine
    Travers Ching, Daniel S. Himmelstein, Brett K. Beaulieu-Jones, Alexandr A. Kalinin, Brian T. Do, Gregory P. Way, Enrico Ferrero, Paul-Michael Agapow, Michael Zietz, Michael M. Hoffman, … Casey S. Greene
    Journal of the Royal Society Interface (2018-04) https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5938574/
    DOI: 10.1098/rsif.2017.0387 · PMID: 29618526 · PMCID: PMC5938574
     
  4. IPFS - Content Addressed, Versioned, P2P File System
    Juan Benet
    arXiv (2014-07-14) https://arxiv.org/abs/1407.3561v1
     
  5. Open collaborative writing with Manubot
    Daniel S. Himmelstein, David R. Slochower, Venkat S. Malladi, Casey S. Greene, Anthony Gitter
    (2018-08-03) https://greenelab.github.io/meta-review/
This is a sentence with 5 citations [1,2,3,4,5].

versus

  1. B. Durmaz, M. Dalgalar, R. Durmaz, Anaerobe 11 (2005) 318-321.
  2. J.I. Keenan, A. Aitchison, R.V. Purcell, R. Greenlees, J.F. Pearson, F.A. Frizelle, Anaerobe 40 (2016) 50-53.
  3. J. Silva, D. Leite, M. Fernandes, C. Mena, P.A. Gibbs, P. Teixeira, Front. Microbiol. 2 (2011) 200.

Example from my uBiome SmartGut report

References

  1. Sci-Hub provides access to nearly all scholarly literature.
    Daniel S Himmelstein, Ariel Rodriguez Romero, Jacob G Levernier, Thomas Anthony Munro, Stephen Reid McLaughlin, Bastian Greshake Tzovaras, Casey S Greene
    eLife (2018-03-01) https://www.ncbi.nlm.nih.gov/pubmed/29424689
    DOI: 10.7554/elife.32822 · PMID: 29424689 · PMCID: PMC5832410

manubot cite

https://github.com/greenelab/manubot

manubot on PyPI

python command line utility

pip install manubot
Prefix Resource Backend API Call
doi DOI Content Negotation curl --location --silent \
  --header "Accept:   application/vnd.citationstyles.csl+json" \
  https://doi.org/10.1098/rsif.2017.0387
pmcid NCBI Citation Exporter https://www.ncbi.nlm.nih.gov/pmc/utils/ctxp?report=citeproc&ids=PMC4304851
pmid NCBI E-Utilities https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=pubmed&id=pmid&id=29424689&rettype=full
arxiv arXiv API https://export.arxiv.org/api/query?id_list=1407.3561
url Greycite http://greycite.knowledgeblog.org/json?uri=url:https://greenelab.github.io/meta-review/

Additional sources under consideration:

ISBNs, Wikidata, Zotero

Technical details of citation metadata retrieval

[
  {
    "publisher": "eLife Sciences Publications, Ltd",
    "abstract": "<jats:p>The website Sci-Hub enables users to download PDF versions of scholarly articles, including many articles that are paywalled at their journal’s site. Sci-Hub has grown rapidly since its creation in 2011, but the extent of its coverage has been unclear. Here we report that, as of March 2017, Sci-Hub’s database contains 68.9% of the 81.6 million scholarly articles registered with Crossref and 85.1% of articles published in toll access journals. We find that coverage varies by discipline and publisher, and that Sci-Hub preferentially covers popular, paywalled content. For toll access articles, we find that Sci-Hub provides greater coverage than the University of Pennsylvania, a major research university in the United States. Green open access to toll access articles via licit services, on the other hand, remains quite limited. Our interactive browser at https://greenelab.github.io/scihub allows users to explore these findings in more detail. For the first time, nearly all scholarly literature is available gratis to anyone with an Internet connection, suggesting the toll access business model may become unsustainable.</jats:p>",
    "DOI": "10.7554/elife.32822",
    "type": "article-journal",
    "source": "Crossref",
    "title": "Sci-Hub provides access to nearly all scholarly literature",
    "volume": "7",
    "author": [
      {
        "given": "Daniel S",
        "family": "Himmelstein"
      },
      {
        "given": "Ariel Rodriguez",
        "family": "Romero"
      },
      {
        "given": "Jacob G",
        "family": "Levernier"
      },
      {
        "given": "Thomas Anthony",
        "family": "Munro"
      },
      {
        "given": "Stephen Reid",
        "family": "McLaughlin"
      },
      {
        "given": "Bastian",
        "family": "Greshake Tzovaras"
      },
      {
        "given": "Casey S",
        "family": "Greene"
      }
    ],
    "container-title": "eLife",
    "language": "en",
    "issued": {
      "date-parts": [
        [
          2018,
          3,
          1
        ]
      ]
    },
    "URL": "https://doi.org/ckcj",
    "PMCID": "PMC5832410",
    "PMID": "29424689",
    "id": "IhliSZDo"
  }
]
manubot cite doi:10.7554/eLife.32822
manubot cite --allow-invalid-csl-data \
  doi:10.7554/eLife.32822
[
  {
    "indexed": {
      "date-parts": [
        [
          2018,
          10,
          11
        ]
      ],
      "date-time": "2018-10-11T04:22:36Z",
      "timestamp": 1539231756463
    },
    "reference-count": 126,
    "publisher": "eLife Sciences Publications, Ltd",
    "license": [
      {
        "URL": "http://creativecommons.org/licenses/by/4.0/",
        "start": {
          "date-parts": [
            [
              2018,
              3,
              1
            ]
          ],
          "date-time": "2018-03-01T00:00:00Z",
          "timestamp": 1519862400000
        },
        "delay-in-days": 0,
        "content-version": "vor"
      },
      {
        "URL": "http://creativecommons.org/licenses/by/4.0/",
        "start": {
          "date-parts": [
            [
              2018,
              3,
              1
            ]
          ],
          "date-time": "2018-03-01T00:00:00Z",
          "timestamp": 1519862400000
        },
        "delay-in-days": 0,
        "content-version": "am"
      },
      {
        "URL": "http://creativecommons.org/licenses/by/4.0/",
        "start": {
          "date-parts": [
            [
              2018,
              3,
              1
            ]
          ],
          "date-time": "2018-03-01T00:00:00Z",
          "timestamp": 1519862400000
        },
        "delay-in-days": 0,
        "content-version": "tdm"
      }
    ],
    "funder": [
      {
        "DOI": "10.13039/100000936",
        "name": "Gordon and Betty Moore Foundation",
        "doi-asserted-by": "publisher",
        "award": [
          "GBMF 4552"
        ]
      }
    ],
    "content-domain": {
      "domain": [],
      "crossmark-restriction": false
    },
    "abstract": "<jats:p>The website Sci-Hub enables users to download PDF versions of scholarly articles, including many articles that are paywalled at their journal’s site. Sci-Hub has grown rapidly since its creation in 2011, but the extent of its coverage has been unclear. Here we report that, as of March 2017, Sci-Hub’s database contains 68.9% of the 81.6 million scholarly articles registered with Crossref and 85.1% of articles published in toll access journals. We find that coverage varies by discipline and publisher, and that Sci-Hub preferentially covers popular, paywalled content. For toll access articles, we find that Sci-Hub provides greater coverage than the University of Pennsylvania, a major research university in the United States. Green open access to toll access articles via licit services, on the other hand, remains quite limited. Our interactive browser at https://greenelab.github.io/scihub allows users to explore these findings in more detail. For the first time, nearly all scholarly literature is available gratis to anyone with an Internet connection, suggesting the toll access business model may become unsustainable.</jats:p>",
    "DOI": "10.7554/elife.32822",
    "type": "article-journal",
    "created": {
      "date-parts": [
        [
          2018,
          2,
          9
        ]
      ],
      "date-time": "2018-02-09T13:00:23Z",
      "timestamp": 1518181223000
    },
    "source": "Crossref",
    "is-referenced-by-count": 3,
    "title": "Sci-Hub provides access to nearly all scholarly literature",
    "prefix": "10.7554",
    "volume": "7",
    "author": [
      {
        "ORCID": "http://orcid.org/0000-0002-3012-7446",
        "authenticated-orcid": true,
        "given": "Daniel S",
        "family": "Himmelstein",
        "sequence": "first",
        "affiliation": [
          {
            "name": "Department of Systems Pharmacology and Translational Therapeutics, University of Pennsylvania, Philadelphia, United States"
          }
        ]
      },
      {
        "ORCID": "http://orcid.org/0000-0003-2290-4927",
        "authenticated-orcid": true,
        "given": "Ariel Rodriguez",
        "family": "Romero",
        "sequence": "additional",
        "affiliation": [
          {
            "name": "Bidwise, Inc, Miami, United States"
          }
        ]
      },
      {
        "ORCID": "http://orcid.org/0000-0003-1563-7314",
        "authenticated-orcid": true,
        "given": "Jacob G",
        "family": "Levernier",
        "sequence": "additional",
        "affiliation": [
          {
            "name": "Library Technology Services and Strategic Initiatives, University of Pennsylvania, Philadelphia, United States"
          }
        ]
      },
      {
        "ORCID": "http://orcid.org/0000-0002-3366-7149",
        "authenticated-orcid": true,
        "given": "Thomas Anthony",
        "family": "Munro",
        "sequence": "additional",
        "affiliation": [
          {
            "name": "School of Life and Environmental Sciences, Deakin University, Melbourne, Australia"
          }
        ]
      },
      {
        "ORCID": "http://orcid.org/0000-0002-9888-3168",
        "authenticated-orcid": true,
        "given": "Stephen Reid",
        "family": "McLaughlin",
        "sequence": "additional",
        "affiliation": [
          {
            "name": "School of Information, University of Texas at Austin, Austin, United States"
          }
        ]
      },
      {
        "ORCID": "http://orcid.org/0000-0002-9925-9623",
        "authenticated-orcid": true,
        "given": "Bastian",
        "family": "Greshake Tzovaras",
        "sequence": "additional",
        "affiliation": [
          {
            "name": "Department of Applied Bioinformatics, Institute of Cell Biology and Neuroscience, Goethe University, Frankfurt, Germany"
          }
        ]
      },
      {
        "ORCID": "http://orcid.org/0000-0001-8713-9213",
        "authenticated-orcid": true,
        "given": "Casey S",
        "family": "Greene",
        "sequence": "additional",
        "affiliation": [
          {
            "name": "Department of Systems Pharmacology and Translational Therapeutics, University of Pennsylvania, Philadelphia, United States"
          }
        ]
      }
    ],
    "member": "4374",
    "published-online": {
      "date-parts": [
        [
          2018,
          3,
          1
        ]
      ]
    },
    "reference": [
      {
        "key": "bib1",
        "author": "Archambault",
        "year": "2014",
        "unstructured": "Archambault É, Amyot D, Deschamps P, Nicol A, Provencher F, Rebout L, Roberge G. 2014. Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels: 1996–2013. University of Nebraska-Lincoln. http://digitalcommons.unl.edu/scholcom/8 [Accessed February 7, 2018].",
        "article-title": "Proportion of Open Access Papers Published in Peer-Reviewed Journals at the European and World Levels: 1996–2013",
        "volume-title": "University of Nebraska-Lincoln"
      },
      {
        "key": "bib2",
        "author": "Association of Research Libraries",
        "year": "2017",
        "unstructured": "Association of Research Libraries. 2017. Graph 4: Expenditure Trends in ARL Libraries, 1986-2015. ARL Statistics 2014–2015. http://www.arl.org/storage/documents/expenditure-trends.pdf [Accessed February 7, 2018].",
        "article-title": "Graph 4: Expenditure Trends in ARL Libraries, 1986-2015",
        "volume-title": "ARL Statistics 2014–2015"
      },
      {
        "key": "bib3",
        "author": "Barnes A",
        "year": "2017",
        "unstructured": "Barnes A L, Weslow D E, Gardner M J. 2017. Complaint – Document #1 of American Chemical Society v. Does – Case 1:17-cv-00726-LMB-JFA. Eastern District Court of Virginia. https://www.courtlistener.com/docket/6146630/1/american-chemical-society-v-does-1-99/ [Accessed February 7, 2018].",
        "article-title": "Complaint – Document #1 of American Chemical Society v. Does – Case 1:17-cv-00726-LMB-JFA",
        "volume-title": "Eastern District Court of Virginia"
      },
      {
        "key": "bib4",
        "doi-asserted-by": "publisher",
        "first-page": "342",
        "DOI": "10.1038/nbt.3780",
        "article-title": "Reproducibility of computational workflows is automated using continuous analysis",
        "volume": "35",
        "author": "Beaulieu-Jones",
        "year": "2017",
        "journal-title": "Nature Biotechnology"
      },
      {
        "key": "bib5",
        "doi-asserted-by": "publisher",
        "first-page": "11",
        "DOI": "10.1038/535011f",
        "article-title": "Publisher under fire for fake article webpages",
        "volume": "535",
        "author": "Becker",
        "year": "2016",
        "journal-title": "Nature"
      },
      {
        "key": "bib6",
        "doi-asserted-by": "publisher",
        "first-page": "e608",
        "DOI": "10.1016/S2214-109X(16)30188-7",
        "article-title": "Sci-Hub and medical practice: an ethical dilemma in Peru",
        "volume": "4",
        "author": "Bendezú-Quispe",
        "year": "2016",
        "journal-title": "The Lancet Global Health"
      },
      {
        "key": "bib7",
        "author": "Benet",
        "year": "2014",
        "unstructured": "Benet J. 2014. IPFS - Content Addressed, Versioned, P2P File System. arXiv. https://arxiv.org/abs/1407.3561v1 [Accessed February 7, 2018].",
        "article-title": "IPFS - Content Addressed, Versioned, P2P File System",
        "volume-title": "arXiv"
      },
      {
        "key": "bib8",
        "doi-asserted-by": "publisher",
        "first-page": "9425",
        "DOI": "10.1073/pnas.1403006111",
        "article-title": "Evaluating big deal journal bundles",
        "volume": "111",
        "author": "Bergstrom",
        "year": "2014",
        "journal-title": "PNAS"
      },
      {
        "key": "bib9",
        "doi-asserted-by": "publisher",
        "first-page": "101",
        "DOI": "10.1007/s12525-017-0249-2",
        "article-title": "Scholarly journal publishing in transition- from restricted to open access",
        "volume": "27",
        "author": "Björk",
        "year": "2017",
        "journal-title": "Electronic Markets"
      },
      {
        "key": "bib10",
        "author": "Bodó",
        "year": "2015",
        "volume-title": "Copyrighting Creativity",
        "DOI": "10.2139/ssrn.2616636",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib11",
        "author": "Bodó",
        "year": "2016",
        "unstructured": "Bodó B. 2016. Pirates in the Library An Inquiry into the Guerilla Open Access Movement. 8th Annual Workshop of the International Society for the History and Theory of Intellectual Property.",
        "article-title": "Pirates in the Library An Inquiry into the Guerilla Open Access Movement",
        "DOI": "10.2139/ssrn.2816925",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib12",
        "doi-asserted-by": "publisher",
        "first-page": "511",
        "DOI": "10.1126/science.352.6285.511",
        "article-title": "The frustrated science student behind Sci-Hub",
        "volume": "352",
        "author": "Bohannon",
        "year": "2016",
        "journal-title": "Science"
      },
      {
        "key": "bib13",
        "doi-asserted-by": "publisher",
        "first-page": "508",
        "DOI": "10.1126/science.352.6285.508",
        "article-title": "Who's downloading pirated papers? Everyone",
        "volume": "352",
        "author": "Bohannon",
        "year": "2016",
        "journal-title": "Science"
      },
      {
        "key": "bib14",
        "author": "Bosch",
        "year": "2017",
        "unstructured": "Bosch S, Henderson K. 2017. New World, Same Model | Periodicals Price Survey 2017. Library Journal. http://lj.libraryjournal.com/2017/04/publishing/new-world-same-model-periodicals-price-survey-2017/ [Accessed February 7, 2018].",
        "article-title": "New World, Same Model | Periodicals Price Survey 2017",
        "volume-title": "Library Journal"
      },
      {
        "key": "bib15",
        "author": "Bosman",
        "year": "2018",
        "article-title": "Open access levels: a quantitative exploration using Web of Science and oaDOI data",
        "volume-title": "PeerJ Preprints",
        "DOI": "10.7287/peerj.preprints.3520v1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib16",
        "doi-asserted-by": "publisher",
        "first-page": "1453",
        "DOI": "10.1007/s11192-016-2225-6",
        "article-title": "Availability of digital object identifiers in publications archived by PubMed",
        "volume": "110",
        "author": "Boudry",
        "year": "2017",
        "journal-title": "Scientometrics"
      },
      {
        "key": "bib17",
        "author": "Brinkema L",
        "year": "2017",
        "unstructured": "Brinkema L M. 2017. Order – Document #37 of American Chemical Society v. Sci-Hub – Case 1:17-cv-00726-LMB-JFA. Eastern District Court of Virginia. https://regmedia.co.uk/2017/11/07/sci_hub_block_order_short.pdf [Accessed February 7, 2018].",
        "article-title": "Order – Document #37 of American Chemical Society v. Sci-Hub – Case 1:17-cv-00726-LMB-JFA",
        "volume-title": "Eastern District Court of Virginia"
      },
      {
        "key": "bib18",
        "author": "British Standards Institute",
        "year": "2012",
        "unstructured": "British Standards Institute. 2012. BS ISO 26324:2012 Information and Documentation. Digital Object Identifier System.",
        "volume-title": "BS ISO 26324:2012 Information and Documentation. Digital Object Identifier System",
        "DOI": "10.3403/30177056",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib19",
        "author": "Brook",
        "volume": "20",
        "year": "2014",
        "article-title": "The social, political and legal aspects of text and data mining (TDM)",
        "journal-title": "D-Lib Magazine",
        "DOI": "10.1045/november14-brook",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib20",
        "author": "Buranyi",
        "year": "2017",
        "unstructured": "Buranyi S. 2017. Is the staggeringly profitable business of scientific publishing bad for science?. The Guardian. https://www.theguardian.com/science/2017/jun/27/profitable-business-scientific-publishing-bad-for-science [Accessed February 7, 2018].",
        "article-title": "Is the staggeringly profitable business of scientific publishing bad for science?",
        "volume-title": "The Guardian"
      },
      {
        "key": "bib21",
        "doi-asserted-by": "publisher",
        "first-page": "874",
        "DOI": "10.1002/asi.23445",
        "article-title": "Bibliogifts in LibGen? A study of a text-sharing platform driven by biblioleaks and crowdsourcing",
        "volume": "67",
        "author": "Cabanac",
        "year": "2016",
        "journal-title": "Journal of the Association for Information Science and Technology"
      },
      {
        "key": "bib22",
        "author": "Cabanac",
        "year": "2017",
        "article-title": "Scimag catalogue of LibGen as of January 1st, 2014",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.4906367.v1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib23",
        "author": "Ching",
        "year": "2017",
        "article-title": "Opportunities and obstacles for deep learning in biology and medicine",
        "volume-title": "bioRxiv",
        "DOI": "10.1101/142760",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib24",
        "author": "ConsenSys",
        "year": "2016",
        "unstructured": "ConsenSys. 2016. Decentralized Storage: The Backbone of the Third Web. ConsenSys Media. https://media.consensys.net/decentralized-storage-the-backbone-of-the-third-web-d4bc54e79700 [Accessed February 7, 2018].",
        "article-title": "Decentralized Storage: The Backbone of the Third Web",
        "volume-title": "ConsenSys Media"
      },
      {
        "key": "bib25",
        "author": "Davis",
        "year": "2016",
        "unstructured": "Davis P. 2016. Two-step authentication: Finally coming to a university near you. The Scholarly Kitchen. https://scholarlykitchen.sspnet.org/2016/06/21/two-step-authentication-finally-coming-to-a-university-near-you/ [Accessed February 7, 2018].",
        "article-title": "Two-step authentication: Finally coming to a university near you",
        "volume-title": "The Scholarly Kitchen"
      },
      {
        "key": "bib26",
        "author": "DeMarco",
        "year": "2015",
        "unstructured": "DeMarco JV, Hirschberg DM, Sen U. 2015a. Complaint – Document #1 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS. Southern District Court of New York. https://www.courtlistener.com/docket/4355308/1/elsevier-inc-v-sci-hub/ [Accessed February 7, 2018].",
        "article-title": "Complaint – Document #1 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS",
        "volume-title": "Southern District Court of New York"
      },
      {
        "key": "bib27",
        "author": "DeMarco",
        "year": "2015",
        "unstructured": "DeMarco JV, Hirschberg DM, Sen U. 2015b. Memorandum of Law in Support of Motion – Document #6 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS. Southern District Court of New York. https://www.courtlistener.com/docket/4355308/6/elsevier-inc-v-sci-hub/ [Accessed February 7, 2018].",
        "article-title": "Memorandum of Law in Support of Motion – Document #6 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS",
        "volume-title": "Southern District Court of New York"
      },
      {
        "key": "bib28",
        "author": "Dingledine",
        "year": "2004",
        "unstructured": "Dingledine R, Mathewson N, Syverson P. 2004. Tor: The Second-Generation Onion Router. http://www.dtic.mil/docs/citations/ADA465464 [Accessed February 7, 2018].",
        "article-title": "Tor: The Second-Generation Onion Router"
      },
      {
        "key": "bib29",
        "doi-asserted-by": "publisher",
        "first-page": "e112",
        "DOI": "10.2196/jmir.3331",
        "article-title": "Is Biblioleaks inevitable?",
        "volume": "16",
        "author": "Dunn",
        "year": "2014",
        "journal-title": "Journal of Medical Internet Research"
      },
      {
        "key": "bib30",
        "author": "EBSCO",
        "year": "2017",
        "unstructured": "EBSCO. 2017. Five Year Journal Price Increase History (2013-2017). EBSCO. https://www.ebscohost.com/promoMaterials/Five_Year_Journal_Price_Increase_History_EBSCO_2013-2017.pdf [Accessed February 7, 2018].",
        "article-title": "Five Year Journal Price Increase History (2013-2017)",
        "volume-title": "EBSCO"
      },
      {
        "key": "bib31",
        "author": "Elbakyan",
        "year": "2016",
        "article-title": "Sci-Hub download data",
        "volume-title": "Dryad Digital Repository",
        "DOI": "10.5061/dryad.q447c/1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib32",
        "author": "Elbakyan",
        "year": "2015",
        "unstructured": "Elbakyan A. 2015. Letter – Document #50 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS. Southern District Court of New York. https://www.courtlistener.com/docket/4355308/50/elsevier-inc-v-sci-hub/ [Accessed February 7, 2018].",
        "article-title": "Letter – Document #50 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS",
        "volume-title": "Southern District Court of New York"
      },
      {
        "key": "bib33",
        "author": "Elbakyan",
        "year": "2016",
        "unstructured": "Elbakyan A. 2016a. Sci-Hub is a goal, changing the system is a method. Engineuring. https://engineuring.wordpress.com/2016/03/11/sci-hub-is-a-goal-changing-the-system-is-a-method/ [Accessed February 7, 2018].",
        "article-title": "Sci-Hub is a goal, changing the system is a method",
        "volume-title": "Engineuring"
      },
      {
        "key": "bib34",
        "author": "Elbakyan",
        "year": "2016",
        "unstructured": "Elbakyan A. 2016b. Why Sci-Hub is the true solution for Open Access: reply to criticism. Engineuring. https://engineuring.wordpress.com/2016/02/24/why-sci-hub-is-the-true-solution-for-open-access-reply-to-criticism/ [Accessed February 7, 2018].",
        "article-title": "Why Sci-Hub is the true solution for Open Access: reply to criticism",
        "volume-title": "Engineuring"
      },
      {
        "key": "bib35",
        "author": "Elbakyan",
        "year": "2017",
        "unstructured": "Elbakyan A. 2017. Some facts on Sci-Hub that Wikipedia gets wrong. Engineuring. https://engineuring.wordpress.com/2017/07/02/some-facts-on-sci-hub-that-wikipedia-gets-wrong/ [Accessed February 7, 2018].",
        "article-title": "Some facts on Sci-Hub that Wikipedia gets wrong",
        "volume-title": "Engineuring"
      },
      {
        "key": "bib36",
        "author": "Else",
        "year": "2017",
        "unstructured": "Else H. 2017. Dutch lose access to OUP journals in subscription standoff. Times Higher Education. https://www.timeshighereducation.com/news/dutch-lose-access-oup-journals-subscription-standoff [Accessed February 7, 2018].",
        "article-title": "Dutch lose access to OUP journals in subscription standoff",
        "volume-title": "Times Higher Education"
      },
      {
        "key": "bib37",
        "author": "Esposito",
        "year": "2016",
        "unstructured": "Esposito JJ. 2016. Sci-Hub and the four horsemen of the internet. The Scholarly Kitchen. https://scholarlykitchen.sspnet.org/2016/03/02/sci-hub-and-the-four-horsemen-of-the-internet/ [Accessed February 7, 2018].",
        "article-title": "Sci-Hub and the four horsemen of the internet",
        "volume-title": "The Scholarly Kitchen"
      },
      {
        "key": "bib38",
        "author": "Esposito",
        "year": "2017",
        "unstructured": "Esposito JJ. 2017. Sci-Hub moves to the center of the ecosystem. The Scholarly Kitchen. https://scholarlykitchen.sspnet.org/2017/09/05/sci-hub-moves-center-ecosystem/ [Accessed February 7, 2018].",
        "article-title": "Sci-Hub moves to the center of the ecosystem",
        "volume-title": "The Scholarly Kitchen"
      },
      {
        "key": "bib39",
        "author": "Fernandez",
        "year": "2014",
        "article-title": "Factors in Science Journal Cancellation Projects: The Roles of Faculty Consultations and Data",
        "journal-title": "Issues in Science and Technology Librarianship",
        "DOI": "10.5062/F4G73BP3",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib40",
        "author": "Fister",
        "year": "2017",
        "unstructured": "Fister B. 2017. Inevitably open. Inside Higher Ed. https://www.insidehighered.com/blogs/library-babel-fish/inevitably-open [Accessed February 7, 2018].",
        "article-title": "Inevitably open",
        "volume-title": "Inside Higher Ed"
      },
      {
        "key": "bib41",
        "author": "Gagnon",
        "year": "2017",
        "unstructured": "Gagnon S. 2017. UdeM Libraries cancel Big Deal subscription to 2,231 periodical titles published by Taylor & Francis Group. Communiqués, Bibliothèques, Université de Montréal. http://www.bib.umontreal.ca/communiques/20170504-DC-annulation-taylor-francis-va.htm [Accessed February 7, 2018].",
        "article-title": "UdeM Libraries cancel Big Deal subscription to 2,231 periodical titles published by Taylor & Francis Group",
        "volume-title": "Communiqués, Bibliothèques, Université de Montréal"
      },
      {
        "key": "bib42",
        "author": "Gardner",
        "year": "2017",
        "unstructured": "Gardner GJ, McLaughlin SR, Asher AD. 2017. Shadow Libraries and You: Sci-Hub Usage and the Future of ILL. ACRL 2017. https://hdl.handle.net/10760/30981 [Accessed February 7, 2018].",
        "article-title": "Shadow Libraries and You: Sci-Hub Usage and the Future of ILL",
        "volume-title": "ACRL 2017"
      },
      {
        "key": "bib43",
        "doi-asserted-by": "publisher",
        "first-page": "98",
        "DOI": "10.1016/j.joi.2015.11.008",
        "article-title": "Availability of digital object identifiers (DOIs) in Web of Science and Scopus",
        "volume": "10",
        "author": "Gorraiz",
        "year": "2016",
        "journal-title": "Journal of Informetrics"
      },
      {
        "key": "bib44",
        "author": "Greshake",
        "year": "2016",
        "article-title": "Correlating the Sci-Hub data with World Bank Indicators and Identifying Academic Use",
        "journal-title": "The Winnower",
        "DOI": "10.15200/winn.146485.57797",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib45",
        "doi-asserted-by": "publisher",
        "first-page": "541",
        "DOI": "10.12688/f1000research.11366.1",
        "article-title": "Looking into Pandora's Box: The Content of Sci-Hub and its Usage",
        "volume": "6",
        "author": "Greshake",
        "year": "2017",
        "journal-title": "F1000Research"
      },
      {
        "key": "bib46",
        "author": "Greshake",
        "year": "2017",
        "article-title": "Data and Scripts for Looking into Pandora's Box: The Content of Sci-Hub and its Usage",
        "volume-title": "Zenodo",
        "DOI": "10.5281/zenodo.472493",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib47",
        "author": "Himmelstein",
        "year": "2017",
        "article-title": "A user-friendly extract of the LibGen scimag metadata SQL dump on 2017-04-07",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.5231245.v1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib48",
        "author": "Himmelstein",
        "year": "2017",
        "article-title": "Metadata for all DOIs in Crossref: JSON MongoDB exports of all works from the Crossref API",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.4816720",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib49",
        "author": "Himmelstein",
        "year": "2016",
        "unstructured": "Himmelstein D. 2016. The licensing of bioRxiv preprints. Satoshi Village. http://blog.dhimmel.com/biorxiv-licenses/ [Accessed February 7, 2018].",
        "article-title": "The licensing of bioRxiv preprints",
        "volume-title": "Satoshi Village"
      },
      {
        "key": "bib50",
        "author": "Himmelstein",
        "year": "2017",
        "article-title": "Sci-Hub provides access to nearly all scholarly literature",
        "volume-title": "PeerJ Preprints",
        "DOI": "10.7287/peerj.preprints.3100v1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib51",
        "author": "International DOI Foundation",
        "year": "2017",
        "volume-title": "DOI Handbook",
        "DOI": "10.1000/182",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib52",
        "author": "Jha",
        "year": "2012",
        "unstructured": "Jha A. 2012. Open access to research is inevitable, says Nature editor-in-chief. The Guardian. https://www.theguardian.com/science/2012/jun/08/open-access-research-inevitable-nature-editor [Accessed February 7, 2018].",
        "article-title": "Open access to research is inevitable, says Nature editor-in-chief",
        "volume-title": "The Guardian"
      },
      {
        "key": "bib53",
        "author": "Kaiser",
        "year": "2017",
        "article-title": "Are preprints the future of biology? A survival guide for scientists",
        "journal-title": "Science",
        "DOI": "10.1126/science.aaq0747",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib54",
        "doi-asserted-by": "publisher",
        "first-page": "e93949",
        "DOI": "10.1371/journal.pone.0093949",
        "article-title": "The number of scholarly documents on the public web",
        "volume": "9",
        "author": "Khabsa",
        "year": "2014",
        "journal-title": "PLoS One"
      },
      {
        "key": "bib55",
        "doi-asserted-by": "publisher",
        "first-page": "17",
        "DOI": "10.3897/jhr.58.12919",
        "article-title": "Ichneumonidae (Hymenoptera) associated with xyelid sawflies (Hymenoptera, Xyelidae) in Mexico",
        "volume": "58",
        "author": "Khalaim",
        "year": "2017",
        "journal-title": "Journal of Hymenoptera Research"
      },
      {
        "key": "bib56",
        "author": "Kikkawa",
        "year": "2016",
        "article-title": "DOI Links on Wikipedia",
        "journal-title": "Digital Libraries: Knowledge, Information, and Data in an Open Access Society",
        "DOI": "10.1007/978-3-319-49304-6_40",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib57",
        "doi-asserted-by": "publisher",
        "first-page": "246",
        "DOI": "10.1080/00987913.2005.10764998",
        "article-title": "Transforming access to research literature for developing countries",
        "volume": "31",
        "author": "Kirsop",
        "year": "2005",
        "journal-title": "Serials Review"
      },
      {
        "key": "bib58",
        "doi-asserted-by": "publisher",
        "first-page": "13",
        "DOI": "10.6087/kcse.2014.1.13",
        "article-title": "CrossRef developments and initiatives: an update on services for the scholarly publishing community from CrossRef",
        "volume": "1",
        "author": "Lammey",
        "year": "2014",
        "journal-title": "Science Editing"
      },
      {
        "key": "bib59",
        "doi-asserted-by": "publisher",
        "first-page": "e0127502",
        "DOI": "10.1371/journal.pone.0127502",
        "article-title": "The oligopoly of academic publishers in the digital era",
        "volume": "10",
        "author": "Larivière",
        "year": "2015",
        "journal-title": "PLoS One"
      },
      {
        "key": "bib60",
        "author": "Laskow",
        "year": "2016",
        "unstructured": "Laskow S. 2016. The Rise of Pirate Libraries. Atlas Obscura. http://www.atlasobscura.com/articles/the-rise-of-illegal-pirate-libraries [Accessed February 7, 2018].",
        "article-title": "The Rise of Pirate Libraries",
        "volume-title": "Atlas Obscura"
      },
      {
        "key": "bib61",
        "author": "Lawson",
        "year": "2015",
        "article-title": "Journal subscription costs - FOIs to UK universities",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.1186832.v23",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib62",
        "author": "Lawson",
        "year": "2017",
        "article-title": "Journal subscription expenditure in the UK 2015-16",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.4542433.v6",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib63",
        "doi-asserted-by": "publisher",
        "first-page": "25",
        "DOI": "10.1629/uksg.333",
        "article-title": "Access, ethics and piracy",
        "volume": "30",
        "author": "Lawson",
        "year": "2017",
        "journal-title": "Insights the UKSG Journal"
      },
      {
        "key": "bib64",
        "doi-asserted-by": "publisher",
        "first-page": "493",
        "DOI": "10.5860/crl-299",
        "article-title": "The Inevitability of Open Access",
        "volume": "73",
        "author": "Lewis",
        "year": "2012",
        "journal-title": "College & Research Libraries"
      },
      {
        "key": "bib65",
        "doi-asserted-by": "publisher",
        "first-page": "518",
        "DOI": "10.12688/f1000research.11415.1",
        "article-title": "We can shift academic culture through publishing choices",
        "volume": "6",
        "author": "Logan",
        "year": "2017",
        "journal-title": "F1000Research"
      },
      {
        "key": "bib66",
        "author": "McKenzie",
        "year": "2017",
        "article-title": "Sci-Hub’s cache of pirated papers is so big, subscription journals are doomed, data analyst suggests",
        "journal-title": "Science",
        "DOI": "10.1126/science.aan7164",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib67",
        "author": "Meadows",
        "volume": "18",
        "year": "2015",
        "article-title": "Beyond Open: Expanding Access to Scholarly Content",
        "journal-title": "The Journal of Electronic Publishing",
        "DOI": "10.3998/3336451.0018.301",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib68",
        "doi-asserted-by": "publisher",
        "first-page": "e0185673",
        "DOI": "10.1371/journal.pone.0185673",
        "article-title": "Use, knowledge, and perception of the scientific contribution of Sci-Hub in medical students: Study in six countries in Latin America",
        "volume": "12",
        "author": "Mejia",
        "year": "2017",
        "journal-title": "PLoS One"
      },
      {
        "key": "bib69",
        "author": "Milova",
        "year": "2017",
        "unstructured": "Milova E. 2017. Alexandra Elbakyan – Science Should be Open to all Not Behind Paywalls. Life Extension Advocacy Foundation. http://www.leafscience.org/alexandra-elbakyan/ [Accessed February 7, 2018].",
        "article-title": "Alexandra Elbakyan – Science Should be Open to all Not Behind Paywalls",
        "volume-title": "Life Extension Advocacy Foundation"
      },
      {
        "key": "bib70",
        "author": "Mongeon",
        "year": "2015",
        "article-title": "The journal coverage of Web of Science and Scopus: a comparative analysis",
        "journal-title": "Scientometrics",
        "DOI": "10.1007/s11192-015-1765-5",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib71",
        "author": "Morrison",
        "year": "2012",
        "unstructured": "Morrison HG. 2012. Freedom for scholarship in the internet age. Simon Fraser University. http://summit.sfu.ca/item/12537 [Accessed February 7, 2018].",
        "article-title": "Freedom for scholarship in the internet age",
        "volume-title": "Simon Fraser University"
      },
      {
        "key": "bib72",
        "author": "Murphy",
        "year": "2016",
        "unstructured": "Murphy K. 2016. Should All Research Papers Be Free?. New York Times. https://www.nytimes.com/2016/03/13/opinion/sunday/should-all-research-papers-be-free.html [Accessed February 7, 2018].",
        "article-title": "Should All Research Papers Be Free?",
        "volume-title": "New York Times"
      },
      {
        "key": "bib73",
        "author": "OpenCitations",
        "year": "2017",
        "article-title": "OCC dataset of all the identifiers, made on 2017-07-25",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.5255368.v1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib74",
        "author": "OpenCitations",
        "year": "2017",
        "article-title": "OCC dataset of all the bibliographic resources, made on 2017-07-25",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.5255365.v1",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib75",
        "author": "Oxenham",
        "year": "2016",
        "unstructured": "Oxenham S. 2016. Meet the Robin Hood of Science. Big Think. http://bigthink.com/neurobonkers/a-pirate-bay-for-science [Accessed February 7, 2018].",
        "article-title": "Meet the Robin Hood of Science",
        "volume-title": "Big Think"
      },
      {
        "key": "bib76",
        "author": "Paul",
        "year": "2017",
        "article-title": "The transition to open access: The state of the market, offsetting deals, and a demonstrated model for fair open access with the open library of humanities",
        "journal-title": "Stand Alone",
        "DOI": "10.3233/978-1-61499-769-6-118",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib77",
        "doi-asserted-by": "publisher",
        "first-page": "253",
        "DOI": "10.1108/JD-12-2013-0166",
        "article-title": "Setting our bibliographic references free: towards open citation data",
        "volume": "71",
        "author": "Peroni",
        "year": "2015",
        "journal-title": "Journal of Documentation"
      },
      {
        "key": "bib78",
        "author": "Peroni",
        "year": "2016",
        "article-title": "Metadata for the OpenCitations Corpus",
        "volume-title": "Figshare",
        "DOI": "10.6084/m9.figshare.3443876.v3",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib79",
        "doi-asserted-by": "publisher",
        "first-page": "e4375",
        "DOI": "10.7717/peerj.4375",
        "article-title": "The State of OA: A large-scale analysis of the prevalence and impact of Open Access articles",
        "volume": "6",
        "author": "Piwowar",
        "year": "2018",
        "journal-title": "PeerJ"
      },
      {
        "key": "bib80",
        "author": "Piwowar",
        "year": "2017",
        "article-title": "Data from: The State of OA: A large-scale analysis of the prevalence and impact of Open Access articles",
        "volume-title": "Zenodo",
        "DOI": "10.5281/zenodo.837902",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib81",
        "author": "Piwowar",
        "year": "2016",
        "unstructured": "Piwowar H. 2016. Introducing oaDOI: resolve a DOI straight to OA. Impactstory Blog. http://blog.impactstory.org/introducting-oadoi/ [Accessed February 7, 2018].",
        "article-title": "Introducing oaDOI: resolve a DOI straight to OA",
        "volume-title": "Impactstory Blog"
      },
      {
        "key": "bib82",
        "doi-asserted-by": "publisher",
        "first-page": "148",
        "DOI": "10.1038/530148a",
        "article-title": "Does it take too long to publish research?",
        "volume": "530",
        "author": "Powell",
        "year": "2016",
        "journal-title": "Nature"
      },
      {
        "key": "bib83",
        "doi-asserted-by": "publisher",
        "first-page": "60",
        "DOI": "10.1629/2460",
        "article-title": "Reassessing the value proposition: first steps towards a fair(er) price for scholarly journals",
        "volume": "24",
        "author": "Prosser",
        "year": "2011",
        "journal-title": "Serials: The Journal for the Serials Community"
      },
      {
        "key": "bib84",
        "author": "Rogers",
        "year": "2012",
        "unstructured": "Rogers J. 2012. Walking away from the American Chemical Society. Attempting Elegance. http://www.attemptingelegance.com/?p=1765 [Accessed February 7, 2018].",
        "article-title": "Walking away from the American Chemical Society",
        "volume-title": "Attempting Elegance"
      },
      {
        "key": "bib85",
        "doi-asserted-by": "publisher",
        "first-page": "123",
        "DOI": "10.1300/J123v18n01_09",
        "article-title": "The Serials Crisis Revisited",
        "volume": "18",
        "author": "Roth",
        "year": "1990",
        "journal-title": "The Serials Librarian"
      },
      {
        "key": "bib86",
        "author": "Rovner",
        "year": "2014",
        "unstructured": "Rovner SL. 2014. Online Access To ACS Publications Is Restored After Some Customers Were Unintentionally Blocked. Chemical & Engineering News. http://cen.acs.org/articles/92/web/2014/04/Online-Access-ACS-Publications-Restored.html [Accessed February 7, 2018].",
        "article-title": "Online Access To ACS Publications Is Restored After Some Customers Were Unintentionally Blocked",
        "volume-title": "Chemical & Engineering News"
      },
      {
        "key": "bib87",
        "author": "Royster",
        "year": "2016",
        "unstructured": "Royster P. 2016. A Brief History of Open Access. Library Conference Presentations and Speeches. http://digitalcommons.unl.edu/library_talks/123 [Accessed February 7, 2018].",
        "article-title": "A Brief History of Open Access",
        "volume-title": "Library Conference Presentations and Speeches"
      },
      {
        "key": "bib88",
        "doi-asserted-by": "publisher",
        "first-page": "131",
        "DOI": "10.2307/271031",
        "article-title": "Logit-Based Interval Estimation for Binomial Data Using the Jeffreys Prior",
        "volume": "17",
        "author": "Rubin",
        "year": "1987",
        "journal-title": "Sociological Methodology"
      },
      {
        "key": "bib89",
        "doi-asserted-by": "publisher",
        "first-page": "341",
        "DOI": "10.1109/TVCG.2016.2599030",
        "article-title": "Vega-Lite: A Grammar of Interactive Graphics",
        "volume": "23",
        "author": "Satyanarayan",
        "year": "2017",
        "journal-title": "IEEE Transactions on Visualization and Computer Graphics"
      },
      {
        "key": "bib90",
        "doi-asserted-by": "publisher",
        "first-page": "13",
        "DOI": "10.1038/nature.2016.21223",
        "article-title": "Scientists in Germany, Peru and Taiwan to lose access to Elsevier journals",
        "volume": "541",
        "author": "Schiermeier",
        "year": "2016",
        "journal-title": "Nature"
      },
      {
        "key": "bib91",
        "author": "Schiermeier",
        "year": "2015",
        "article-title": "Pirate research-paper sites play hide-and-seek with publishers",
        "journal-title": "Nature",
        "DOI": "10.1038/nature.2015.18876",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib92",
        "author": "Schiermeier",
        "year": "2017",
        "article-title": "US court grants Elsevier millions in damages from Sci-Hub",
        "journal-title": "Nature",
        "DOI": "10.1038/nature.2017.22196",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib93",
        "author": "Schiermeier",
        "year": "2017",
        "article-title": "Pirate paper website Sci-Hub dealt another blow by US courts",
        "journal-title": "Nature",
        "DOI": "10.1038/nature.2017.22971",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib94",
        "doi-asserted-by": "publisher",
        "first-page": "137",
        "DOI": "10.1038/d41586-018-00093-7",
        "article-title": "Germany vs Elsevier: universities win temporary journal access after refusing to pay fees",
        "volume": "553",
        "author": "Schiermeier",
        "year": "2018",
        "journal-title": "Nature"
      },
      {
        "key": "bib95",
        "doi-asserted-by": "publisher",
        "first-page": "230",
        "DOI": "10.1186/s12916-015-0469-2",
        "article-title": "'Predatory' open access: a longitudinal study of article volumes and market characteristics",
        "volume": "13",
        "author": "Shen",
        "year": "2015",
        "journal-title": "BMC Medicine"
      },
      {
        "key": "bib96",
        "author": "Silver",
        "year": "2017",
        "unstructured": "Silver A. 2017. Sci-Hub domains inactive following court order. The Register. http://www.theregister.co.uk/2017/11/23/scihubs_become_inactive_following_court_order/ [Accessed February 7, 2018].",
        "article-title": "Sci-Hub domains inactive following court order",
        "volume-title": "The Register"
      },
      {
        "key": "bib97",
        "author": "Singh Chawla",
        "year": "2017",
        "article-title": "Publishers go after networking site for illicit sharing of journal papers",
        "journal-title": "Science",
        "DOI": "10.1126/science.aaq0132",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib98",
        "author": "Singh Chawla",
        "year": "2017",
        "article-title": "Publishers take ResearchGate to court, alleging massive copyright infringement",
        "journal-title": "Science",
        "DOI": "10.1126/science.aaq1560",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib99",
        "doi-asserted-by": "publisher",
        "first-page": "399",
        "DOI": "10.1038/d41586-017-05922-9",
        "article-title": "Need a paper? Get a plug-in",
        "volume": "551",
        "author": "Singh Chawla",
        "year": "2017",
        "journal-title": "Nature"
      },
      {
        "key": "bib100",
        "author": "Singh Chawla",
        "year": "2017",
        "article-title": "Unpaywall finds free versions of paywalled papers",
        "journal-title": "Nature",
        "DOI": "10.1038/nature.2017.21765",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib101",
        "author": "Standish",
        "year": "2017",
        "unstructured": "Standish R. 2017. The World’s Largest Free Scientific Resource Is Now Blocked in Russia. Foreign Policy. http://foreignpolicy.com/2017/09/06/the-worlds-largest-free-scientific-resource-is-now-blocked-in-russia/ [Accessed February 7, 2018].",
        "article-title": "The World’s Largest Free Scientific Resource Is Now Blocked in Russia",
        "volume-title": "Foreign Policy"
      },
      {
        "key": "bib102",
        "author": "Suber",
        "year": "2008",
        "unstructured": "Suber P. 2008. Gratis and libre open access. SPARC Open Access Newsletter. https://dash.harvard.edu/handle/1/4322580 [Accessed February 7, 2018].",
        "article-title": "Gratis and libre open access",
        "volume-title": "SPARC Open Access Newsletter"
      },
      {
        "key": "bib103",
        "author": "Suber",
        "year": "2017",
        "unstructured": "Suber P. 2017. Open Access. MIT Press. https://mitpress.mit.edu/books/open-access.",
        "article-title": "Open Access"
      },
      {
        "key": "bib104",
        "doi-asserted-by": "publisher",
        "first-page": "642",
        "DOI": "10.5860/crln.72.11.8671",
        "article-title": "Is free inevitable in scholarly communication?: The economics of open access",
        "volume": "72",
        "author": "Sutton",
        "year": "2011",
        "journal-title": "College & Research Libraries News"
      },
      {
        "key": "bib105",
        "author": "Sweet",
        "year": "2015",
        "unstructured": "Sweet RW. 2015. Memorandum & Opinion – Document #53 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS. Southern District Court of New York. https://www.courtlistener.com/docket/4355308/53/elsevier-inc-v-sci-hub/ [Accessed February 7, 2018].",
        "article-title": "Memorandum & Opinion – Document #53 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS",
        "volume-title": "Southern District Court of New York"
      },
      {
        "key": "bib106",
        "author": "Sweet",
        "year": "2017",
        "unstructured": "Sweet RW. 2017. Judgement – Document #87 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS. Southern District Court of New York. https://www.documentcloud.org/documents/3878258-2017-06-21-Elsevier-Sci-Hub-Final-Judgement.html [Accessed February 7, 2018].",
        "article-title": "Judgement – Document #87 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS",
        "volume-title": "Southern District Court of New York"
      },
      {
        "key": "bib107",
        "doi-asserted-by": "publisher",
        "first-page": "632",
        "DOI": "10.12688/f1000research.8460.3",
        "article-title": "The academic, economic and societal impacts of Open Access: an evidence-based review",
        "volume": "5",
        "author": "Tennant",
        "year": "2016",
        "journal-title": "F1000Research"
      },
      {
        "key": "bib108",
        "author": "Travis",
        "year": "2016",
        "article-title": "In survey, most give thumbs-up to pirated papers",
        "journal-title": "Science",
        "DOI": "10.1126/science.aaf5704",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib109",
        "author": "Tzovaras",
        "year": "2018",
        "article-title": "Sci-Hub download log of 2017",
        "volume-title": "Zenodo",
        "DOI": "10.5281/zenodo.1158301",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib110",
        "author": "Van der Sar",
        "year": "2015",
        "unstructured": "Van der Sar E. 2015a. Elsevier Cracks Down on Pirated Scientific Articles. TorrentFreak. https://torrentfreak.com/elsevier-cracks-down-on-pirated-scientific-articles-150609/ [Accessed February 7, 2018].",
        "article-title": "Elsevier Cracks Down on Pirated Scientific Articles",
        "volume-title": "TorrentFreak"
      },
      {
        "key": "bib111",
        "author": "Van der Sar",
        "year": "2015",
        "unstructured": "Van der Sar E. 2015b. Court Orders Shutdown of Libgen, Bookfi and Sci-Hub. TorrentFreak. https://torrentfreak.com/court-orders-shutdown-of-libgen-bookfi-and-sci-hub-151102/ [Accessed February 7, 2018].",
        "article-title": "Court Orders Shutdown of Libgen, Bookfi and Sci-Hub",
        "volume-title": "TorrentFreak"
      },
      {
        "key": "bib112",
        "author": "Van der Sar",
        "year": "2015",
        "unstructured": "Van der Sar E. 2015c. Sci-Hub, BookFi and LibGen Resurface After Being Shut Down. TorrentFreak. https://torrentfreak.com/sci-hub-and-libgen-resurface-after-being-shut-down-151121/ [Accessed February 7, 2018].",
        "article-title": "Sci-Hub, BookFi and LibGen Resurface After Being Shut Down",
        "volume-title": "TorrentFreak"
      },
      {
        "key": "bib113",
        "author": "Van der Sar",
        "year": "2017",
        "unstructured": "Van der Sar E. 2017a. Sci-Hub Ordered to Pay $15 Million in Piracy Damages. TorrentFreak. https://torrentfreak.com/sci-hub-ordered-to-pay-15-million-in-piracy-damages-170623/ [Accessed February 7, 2018].",
        "article-title": "Sci-Hub Ordered to Pay $15 Million in Piracy Damages",
        "volume-title": "TorrentFreak"
      },
      {
        "key": "bib114",
        "author": "Van der Sar",
        "year": "2017",
        "unstructured": "Van der Sar E. 2017b. New Lawsuit Demands ISP Blockades Against “Pirate” Site Sci-Hub. TorrentFreak. https://torrentfreak.com/new-lawsuit-demands-isp-blockades-against-pirate-site-sci-hub-170629/ [Accessed February 7, 2018].",
        "article-title": "New Lawsuit Demands ISP Blockades Against “Pirate” Site Sci-Hub",
        "volume-title": "TorrentFreak"
      },
      {
        "key": "bib115",
        "doi-asserted-by": "publisher",
        "first-page": "821",
        "DOI": "10.1098/rstl.1677.0003",
        "article-title": "Observations, communicated to the publisher by Mr. Antony van Leewenhoeck, in a dutch Letter of the 9th Octob. 1676. here English'd: concerning little animals by him observed in rain-well-sea- and snow water; as also in water wherein pepper had lain infused",
        "volume": "12",
        "author": "van Leewenhoeck",
        "year": "1677",
        "journal-title": "Philosophical Transactions of the Royal Society of London"
      },
      {
        "key": "bib116",
        "doi-asserted-by": "publisher",
        "first-page": "134",
        "DOI": "10.1038/483134a",
        "article-title": "Trouble at the text mine",
        "volume": "483",
        "author": "Van Noorden",
        "year": "2012",
        "journal-title": "Nature"
      },
      {
        "key": "bib117",
        "doi-asserted-by": "publisher",
        "first-page": "386",
        "DOI": "10.1038/500386a",
        "article-title": "Half of 2011 papers now free to read",
        "volume": "500",
        "author": "Van Noorden",
        "year": "2013",
        "journal-title": "Nature"
      },
      {
        "key": "bib118",
        "doi-asserted-by": "publisher",
        "first-page": "426",
        "DOI": "10.1038/495426a",
        "article-title": "Open access: The true cost of science publishing",
        "volume": "495",
        "author": "Van Noorden",
        "year": "2013",
        "journal-title": "Nature"
      },
      {
        "key": "bib119",
        "doi-asserted-by": "publisher",
        "first-page": "161",
        "DOI": "10.1038/508161a",
        "article-title": "Funders punish open-access dodgers",
        "volume": "508",
        "author": "Van Noorden",
        "year": "2014",
        "journal-title": "Nature"
      },
      {
        "key": "bib120",
        "author": "Van Noorden",
        "year": "2015",
        "article-title": "Nature owner merges with publishing giant",
        "journal-title": "Nature",
        "DOI": "10.1038/nature.2015.16731",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib121",
        "author": "Van Noorden",
        "year": "2016",
        "article-title": "Alexandra Elbakyan: Paper pirate",
        "journal-title": "Nature",
        "DOI": "10.1038/540507a",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib122",
        "author": "Vogel",
        "year": "2017",
        "article-title": "A bold open-access push in Germany could change the future of academic publishing",
        "journal-title": "Science",
        "DOI": "10.1126/science.aap7562",
        "doi-asserted-by": "publisher"
      },
      {
        "key": "bib123",
        "doi-asserted-by": "publisher",
        "first-page": "161",
        "DOI": "10.1080/00987913.2007.10765116",
        "article-title": "Digital Object Identifiers and Their Use in Libraries",
        "volume": "33",
        "author": "Wang",
        "year": "2007",
        "journal-title": "Serials Review"
      },
      {
        "key": "bib124",
        "doi-asserted-by": "publisher",
        "first-page": "e1005962",
        "DOI": "10.1371/journal.pcbi.1005962",
        "article-title": "A comprehensive and quantitative comparison of text-mining in 15 million full-text articles versus their corresponding abstracts",
        "volume": "14",
        "author": "Westergaard",
        "year": "2017",
        "journal-title": "PLoS Computational Biology"
      },
      {
        "key": "bib125",
        "author": "Woltermann",
        "year": "2015",
        "unstructured": "Woltermann A. 2015. Declaration in Support of Motion – Document #8 Attachment #23 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS. Southern District Court of New York. https://www.courtlistener.com/docket/4355308/8/23/elsevier-inc-v-sci-hub/ [Accessed February 7, 2018].",
        "article-title": "Declaration in Support of Motion – Document #8 Attachment #23 of Elsevier Inc. v. Sci-Hub – Case 1:15-cv-04282-RWS",
        "volume-title": "Southern District Court of New York"
      },
      {
        "key": "bib126",
        "author": "Woolston",
        "year": "2016",
        "article-title": "Paper piracy sparks online debate",
        "journal-title": "Nature",
        "DOI": "10.1038/nature.2016.19841",
        "doi-asserted-by": "publisher"
      }
    ],
    "container-title": "eLife",
    "original-title": [],
    "language": "en",
    "link": [
      {
        "URL": "https://cdn.elifesciences.org/articles/32822/elife-32822-v3.pdf",
        "content-type": "application/pdf",
        "content-version": "vor",
        "intended-application": "text-mining"
      },
      {
        "URL": "https://cdn.elifesciences.org/articles/32822/elife-32822-v3.xml",
        "content-type": "application/xml",
        "content-version": "vor",
        "intended-application": "text-mining"
      }
    ],
    "deposited": {
      "date-parts": [
        [
          2018,
          8,
          24
        ]
      ],
      "date-time": "2018-08-24T15:08:18Z",
      "timestamp": 1535123298000
    },
    "score": 1.0,
    "subtitle": [],
    "short-title": [],
    "issued": {
      "date-parts": [
        [
          2018,
          3,
          1
        ]
      ]
    },
    "references-count": 126,
    "alternative-id": [
      "10.7554/eLife.32822"
    ],
    "URL": "https://doi.org/ckcj",
    "relation": {
      "references": [
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.4906367.v1",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.5061/dryad.q447c/1",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.5281/zenodo.472493",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.5231245.v1",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.4816720",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.1186832.v23",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.4542433.v6",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.5255368.v1",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.5255365.v1",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.6084/m9.figshare.3443876.v3",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.5281/zenodo.837902",
          "asserted-by": "subject"
        },
        {
          "id-type": "doi",
          "id": "10.5281/zenodo.1158301",
          "asserted-by": "subject"
        }
      ],
      "cites": []
    },
    "ISSN": [
      "2050-084X"
    ],
    "subject": [
      "General Biochemistry, Genetics and Molecular Biology",
      "General Immunology and Microbiology",
      "General Neuroscience",
      "General Medicine"
    ],
    "PMCID": "PMC5832410",
    "PMID": "29424689",
    "id": "IhliSZDo"
  }
]

CSL Schema Pruning

CSL Data Validation

accoutrements

end-to-end reproducibility

The provenance of every statement, measure, and figure can be traced to its origin.

![**Downloads per day on Sci-Hub for months with access logs.**](https://raw.githubusercontent.com/greenelab/scihub/e35cc7b0d3b6dd65bf8ce18945007d2b44a6be1e/download/scihub-logs-2017/scihub-log-downloads.svg?sanitize=true)

Embed images using versioned links to source 

{
  "creation_time_utc": "2018-06-15T14:25:26.787993",
  "commit": "63d2468883ea69ad7ad638c39efab0fcbe026298",
  "total_commits": 755,
  "manuscript_commits": 349,
  "deep_review_authors": 36,
  "deep_review_post_submission_authors": 14,
  "github_stars": 616,
  "github_forks": 165,
  "merged_pull_requests": 315,
  "declined_pull_requests": 22,
  "open_pull_requests": 0,
  "open_issues": 322,
  "closed_issues": 215
}
the Deep Review repository accumulated {{total_commits}} git commits, {{merged_pull_requests}} merged pull requests, {{open_issues + closed_issues}} issues, and {{github_stars}} GitHub stars.

Directly insert output using template variables

JSON provides analysis output

the future: living but versioned

“Finally, we estimate that over a six-month period in 2015–2016, Sci-Hub provided access for 99.3% of valid incoming requests.”

— DOI: 10.7287/peerj.preprints.3100v1

“In the first version of this study, we mistakenly treated the log events as requests rather than downloads. Fortunately, Sci-Hub reviewed the preprint in a series of tweets, and pointed out the error…”

— DOI: 10.7287/peerj.preprints.3100v2

tracking prose in git: newlines

no newlines

See blog post at https://rhodesmill.org/brandon/2012/one-sentence-per-line/

Manubot is an ideal authoring protocol for real-time open science, especially for projects that are already using an open source software workflow to manage their research. While Manubot does require technical expertise, the benefits are manyfold. Specifically, Manubot demonstrates a system for publishing that is transparent, reproducible, immediate, permissionless, versioned, automated, collaborative, open, linked, provenanced, decentralized, hackable, interactive, annotated, and free of charge. These attributes empower integrating Manubot with an ecosystem of other community-driven tools to make science as open and collaborative as possible.
Manubot is an ideal authoring protocol for real-time open science, especially for projects that are already using an open source software workflow to manage their research.
While Manubot does require technical expertise, the benefits are manyfold.
Specifically, Manubot demonstrates a system for publishing that is transparent, reproducible, immediate, permissionless, versioned, automated, collaborative, open, linked, provenanced, decentralized, hackable, interactive, annotated, and free of charge.
These attributes empower integrating Manubot with an ecosystem of other community-driven tools to make science as open and collaborative as possible.
Manubot is an ideal
authoring protocol for
real-time open science,
especially for projects
that are already using
an open source software
workflow to manage their
research. While Manubot
does require technical
expertise, the benefits
are manyfold.
Specifically, Manubot
demonstrates a system
for publishing that is
transparent,
reproducible, immediate,
permissionless,
versioned, automated,
collaborative, open,
linked, provenanced,
decentralized, hackable,
interactive, annotated,
and free of charge.
These attributes empower
integrating Manubot with
an ecosystem of other
community-driven tools
to make science as open
and collaborative as
possible.

word wrap at 25 chars

one sentence per line

"semantic newlines" create the best diffs & minimize conflict

current usage

As of October 2018, there are about 40 public Manubot manuscripts. https://git.io/fx4g9

The Deep Review

  • review article on deep learning in precision medicine
  • 27 authors from 20 different institutions
  • readers appreciate the breadth of perspectives

most viewed bioRxiv preprint of 2017

real-time open science

Sci-Hub Coverage Study was most viewed PeerJ Preprint of 2017!

undergraduate progress reports

grant progress reports

climate dataset publications

dataviz OSS manuscripts

personal literature reviews

genomics research papers

historical reproductions

getting started

manubot-rootstock

Acknowledgments

G-2018-11163

GBMF4552

Open collaborative writing with Manubot
Daniel S. Himmelstein, David R. Slochower, Venkat S. Malladi, Casey S. Greene, Anthony Gitter
Manubot Preprint (2018) https://greenelab.github.io/meta-review/

Deep Review Authors:
Paul-Michael Agapow, Amr M. Alexandari, Brett K. Beaulieu-Jones, Anne E. Carpenter, Travers Ching, Evan M. Cofer, Dave DeCaprio, Brian T. Do, Enrico Ferrero, David J. Harris, Michael M. Hoffman, Alexandr A. Kalinin, Anshul Kundaje, Jack Lanchantin, Christopher A. Lavender, Benjamin J. Lengerich, Zhiyong Lu, Yifan Peng, Yanjun Qi, Gail L. Rosen, Avanti Shrikumar, Srinivas C. Turaga, Gregory P. Way, Laura K. Wiley, Stephen Woloszynek, Wei Xie, Jinbo Xu, and Michael Zietz

Manubot contributors & users:
Ogun Adebali, Evan M. Cofer, Robert Gieseke, Alexander Dunkel, Ansel Halliburton, Achintya Rao, and other GitHub users.

Funding

devs needed!

$ frontend $

Questions?

@dhimmel

0000-0002-3012-7446

Getting started
https://tiny.cc/manubot
Slides
https://slides.com/dhimmel/force2018

Made with Slides.com