Aligning Expert Intelligence
HMIA 2025
Agenda
1m YOU ARE HERE
10m PreClass Work
5m TERMINOLOGY
10m Five Slides
2m Handout intro
5m Heads Down Research, Brainstorm and Suggest Two Wizard Ethics Rules
10m Meet with your task force and reach consensus on a set of ethical guidelines for modern applied wizards
5m (separate/together): commentary on magic risk for each guideline
20m (whole class): review and talk about alternate mechanisms)
Human Intelligence Alignment
Organizational Intelligence Alignment
Expert Intelligence Alignment
(Align agents to sustain cooperation)
(collaborate to solve collective action problems but safety challenged by delegation fails, information problems, goal shift, porous accountability, fragile rules, and power concentration)
(concentrate expertise and deliver through special agents but safety challenged by verification/validation problems, boundary issues, and trust)
YOU ARE HERE
From HIA to OIA to EIA
HMIA 2025
Aligning Expert Intelligence
...public welfare requires that the engineer speak out when there are risks to life or property (i.e. whistleblowing).
...necessitates that engineers adhere to standards, maintain confidentiality as required
...I may have to resist or even report unsafe directions, even if that conflicts with short-term client demands
From Preclass Work
HMIA 2025
Aligning Expert Intelligence
SO, WHISTLE BLOW OR CONFIDENTIALITY?
As an engineer, I have to not only design safe and fair solutions but also avoid conflicts of interest, disclose anything that could influence my judgment, and refuse any outside compensation that could compromise trust in me.
From Preclass Work
HMIA 2025
Aligning Expert Intelligence
WHY THE CONCERN WITH CONFLICTS?
Highest measure of my success is whether my work contributes to the well-being of the community I serve.
Prioritize the common good even when it’s harder or less profitable In reality, though, it’s true alignment with society’s trust in engineers.
Being an engineer well-aligned with society's values means designing systems that match society's requests and intentions.
From Preclass Work
HMIA 2025
Aligning Expert Intelligence
HOW DO YOU KNOW "WHAT SOCIETY WANTS"?
A well-aligned engineer will act to serve the public interest even without constant surveillance, making decisions that prioritize safety over personal profit.
From Preclass Work
HMIA 2025
Aligning Expert Intelligence
HOW DO WE BUILD THIS ENGINEER?
divination, necromancy
self-regulation
epistemic asymmetry
mystification
authority
conflicts of interest
honesty vs integrity
role fidelity
(engineering) standards
Terminology
HMIA 2025
Aligning Expert Intelligence
tell the truth vs act consistent with values
personal interest conflicts with acting in interest of others
legitimated power
presenting knowledge as obscure to wield power
one party knows things other party does not
an agent is trusted to oversee itself or a group its members
foresee the future, consulting with the dead
performing what is expected in a given role
explicit codification of knowledge, rules to take personal out
What is Expert Intelligence?
HMIA 2025
Aligning Expert Intelligence
Publicly recognized and institutionally backed specialization...
...people whose know-how far exceeds everyday problem-solving (think doctors, lawyers, plumbers).
We trust experts to be channeling from body of specialized knowldge.
All the knowledge in the universe
Stuff everybody knows
Rare Unusual Knowldge
Expert
special bit of knowledge
transaction
Alignment Challenges of Expert Intelligence
We rely on domain-specific "superintelligences” to solve problems beyond our training—but EI users face safety risks:
STOP+THINK: Name one signal you’d want before trusting an expert in a high-stakes decision? What does the signal "tell" you? What worries you? Why do you need the signal?
HMIA 2025
Aligning Expert Intelligence
Verification Problem
Users cannot tell whether the expert’s knowledge is sound or advice is correct. Trained Doctor vs. Youtube wellness "guru", how would you verify their claims?
Boundary Problem
Users may not know the limits of their own competence - WHEN and WHICH expert to seek help from. Should you consult a civil engineer for tax advice?
Trust Problem
Willingness to use EI is based on (relatively) blind trust: experts wield authority that can be abused, politicized, or captured.
STOP+THINK: What are some expert intelligences that you rely on?
Profession is modern idea with ancient roots
HMIA 2025
Aligning Expert Intelligence
Started as vocations (callings) trusted with rare skills—priests, scribes, healers; rulers and ordinary people relied on them for problems beyond everyday reason
In medieval Europe three "professions" emerged (c 12th c): Theology, Law, and Medicine. Practitioners emerged from long (often university-based) training with specialized knowledge, while other trades were organized and regulated by guilds. Our notion of "profession" comes from this tradition. But other cultures had analogous forms:
-
In China, the imperial "Keju" (since the Song dynasty, c. 960-1276) examinations produced scholar-officials bound to serve the common good.
-
In India social roles were organized by Varna (order) and Jati (birth group): priests and scholars occupied one varna, traders and farmers others, with artisans and laborers in another 'Varna', while 'Jati' fixed hereditary occupations (weavers, potters, temple musicians) and Dharma prescribed role-specific duties and taboos—ethical norms analogous to professional codes.
-
In African cultures, healers/diviners (Sangomas/Shamans) were apprenticed experts bound by ritual codes; In the Islamic world, expertise was often organized through guilds (Futuwwa, Asnaf), which regulated craft, conduct, and training. Religious scholars (Ulama) formed a professional class of jurists and theologians, legitimized by both knowledge and moral authority and medicine was highly professionalized in Islamic civilizations.
In the contemporary world, the epitome of expert intelligence is the profession.
Key Components of Professions
HMIA 2025
Aligning Expert Intelligence
At its core, a profession is organized expertise. It is not just an occupation but a social institution by which "society" grants special authority and privileges in exchange for commitments to certain rules and regulations.
-
Specialized Knowledge: a distinct body of skills acquired through training, concentrated domain expertise.
-
Standards and credentials: exams or training that certify competence before practice.
-
Ethical codes and norms: explicit commitments to guide practice and bind experts to public welfare
- Autonomy: the right to exercise discretion, with outsiders expected to defer
-
Service orientation: in world of self-interest, commitment public good rather than private gain
-
Self-regulation: professionals commit to peer review of expertise and rule following
Concrete Problems in Expert Intelligence Safetys
HMIA 2025
Aligning Expert Intelligence
If a community deploys expert intelligences, what kinds of "accidents" can occur?
-
Specialized Knowledge: by definition, EI users cannot verify correctness or aptness of advice
-
Standards and credentials: exams may not measure what matters; credentials can be faked.
-
Ethical codes and norms: depends completely on integrity and fiduciary responsibilities
- Autonomy: who regulates the self-regulators?
-
Service orientation: we assume public interest but trust can erode, privilege/power weaponized, exploited
-
Self-regulation: what stops professionals from "protecting their own"? covering up?
“It occurs to me that nobody has said much about ethical screening and training of mages. Consider: If you are going to enable someone to throw fireballs and petrify people and screw around with the weather, wouldn’t you want to be pretty sure they won’t run amok with the ability?”
Dear ________, Johns Hopkins University has recently announced the worlds first university program in Applied Modern Wizardry. The complex problems we face in the contemporary world require more than mere science and creativity and rationality. We need magic and this program will train the kinds of magicians and wizards we need to solve the problems of today - and tomorrow. Wizardy offers untold potential benefits to humankind, but also poses risks. We want to ensure that modern wizardry is aligned with human values and put to uses that are beneficial for humans. And so we have come to ask your help. Would you join our new task force on ethical wizardry? We have asked you and several other experts to collaborate to produce the first ever "Modern Wizards' Code of Ethics."
HMIA 2025
Aligning Expert Intelligence
Resources
Author. YYYY. "Linked Title" (info)
HMIA 2025
Aligning Expert Intelligence
HMIA 2025
Aligning Expert Intelligence
HMIA 2025 Aligning Experts
By Dan Ryan
