Making research more equitable and more effective
Simon Hettrick
26 June 2025 - NTO Workshop, Liverpool ORCID: 0000-0002-6809-5195
If we don't recognise everyone that is vital to research,
we limit our ability to conduct research.
See no
NTOs
Hear about no NTOs
Discuss no NTOs
Vicious cycle: NTO version
A bit of history
www.software.ac.uk
Use software
Fundamental to results
69%
92%
n=417
Researcher
Software
Engineer
Research
Software
Engineer
>10,000 RSEs worldwide
Image courtesy Dan Katz
Why did it take until the 2010s to recognise that software is so fundamentally important to research?
WHY ARE WE HERE?
Discovery, reliability, reproducibility, transparency.
Publications are the Bhurj Khalifa
Height = 830 m
Exhibitions are a typical UK bungalow
Height = 3.5 m
Software is a lego person wearing a top hat
Height = 5 cm
2021 REF at Liverpool
What are the dangers of using...
Publications
...as the only metric of research success?
Show me the incentive and I will show you the outcome
- Charlie Munger
People who are
vital to research
People who are named in publications
Academics
RSEs, Technicians,
Librarians, Data
Stewards, PRISMS,
RMAs... and more
The Venn of academic importance and recognition
Recognising not-traditionally submitted outputs (NTOs) and all the people needed to conduct research (Hidden Roles) makes research more equitable and more effective.
What are the barriers to change?
- Fear of affecting funding
- Lack of awareness
- Lack of guidance
- Inherent difficulty of comparison
Recognising all research outputs and every role that makes research possible
www.hidden-ref.org
- Training materials and courses
- Grimpact
- Citizen science
- Enabling access to facilities
- Community building
- Standards
New categories
The Hidden Role
| CC-BY William Murphy |
Data stewards and managers, Librarians, Technicians, Lived Experience Contributors (including patients), Research Software Engineers, PRISMS (Professional Research Investment and Strategy Managers), Professional Services Personnel, Public Engagement Professionals (PEPs), RMAs (Research Managers and Administrators) and Clinical Trials Managers.
Impact
BREAKING
NEWS
2025-2030
Text
Thinktank, Birmingham
7-8 October 2025
If we recognise everyone that is vital to research,
we advance our ability to conduct research.
Exercise 1:
What NTOs have you worked with?
A group discussion about the NTOs that make research possible. These may be from your own research or from the research you work with or facilitate. You can stick to the REF output list or deviate from it.
Once you have identified NTOs, discuss their role and whether they are appropriately recognised.
Notes at: https://tinyurl.com/NTOLiverpoolEx1
Thank you!
@sjh5000
ORCID: 0000-0002-6809-5195, EPSRC Grant no: EP/S021779/1
Licence
© Simon Hettrick. These slides are licensed under a Creative Commons Attribution 4.0 International : https://creativecommons.org/licenses/by/4.0/.
Link to slides
https://slides.com/simonhettrick/talk-1-nto-workshop-liv-june-2025
Image credits
- Earth rise: public domain with thanks to NASA
- Bhurj Khalifa: CC-BY-SA courtesy Donaldytong
- Bungalow: CC-BY-SA courtesy Peter Wyatt
- Lego man: CC-BY courtesy Brickset