Here is a recommended weekly rhythm:
New technology creates a huge effect on people and the planet
Anything that changes the world will have both positives and negatives, called a trade-off
tech has the potential to: make human lives safer, freer, healthier, more interconnected, more full of beauty, and less plagued by poverty
it also has the power to
dramatically increase inequality
enable people with malicious intentions to harm others
create conditions for people to make poor decisions
Burton, Emanuelle; Goldsmith, Judy; Mattei, Nicholas; Siler, Cory; Swiatek, Sara-Jo. Computing and Technology Ethics: Engaging through Science Fiction (p. 4). MIT Press.
Morals | Ethics | Law |
---|
Personal beliefs about right and wrong
Moral acts conform to what an individual believes to be the right thing to do
System of rules, enforced by a set of institutions, that tells us what we can and cannot do
Legal acts are acts that conform to the law
From: Reynolds, G. (2019) Ethics in Information Technology. Cengage Learning.
Personal Ethics | Professional Ethics | Social+Political Ethics |
---|
What does it mean to you to be a good person?
How can you make decisions you can live with?
How to create rules and laws that facilitate
well-being, cooperation, and human rights?
Governance , law, regulation, and policy.
profession definition [1]:
Questions of governance:
Cruess SR, Johnston S, Cruess RL. "Profession": a working definition for medical educators. Teach Learn Med. 2004 Winter;16(1):74-6. doi: 10.1207/s15328015tlm1601_15. PMID: 14987179.
Many professionals "care" about ethics because those professions require a license to practice
professional license:
An official authorization issued by a governmental or regulatory body that allows an individual to legally practice a specific profession. This licensing process ensures that practitioners meet the necessary qualifications and adhere to established standards of competence and ethical conduct. Professional licenses are often required in fields where specialized knowledge, skills, and public trust are critical.
Licensing bodies maintain ethical standards for the profession
B. C. Stahl, J. Timmermans, and B. D. Mittelstadt, “The Ethics of Computing: A Survey of the Computing-Oriented Literature,” ACM Comput. Surv., vol. 48, no. 4, pp. 1–38, May 2016, doi: 10.1145/2871196.
Many professionals "care" about ethics because those professions require a license to practice
Licensing bodies maintain ethical standards for the profession
Software Engineering is not a licensed profession!!
B. C. Stahl, J. Timmermans, and B. D. Mittelstadt, “The Ethics of Computing: A Survey of the Computing-Oriented Literature,” ACM Comput. Surv., vol. 48, no. 4, pp. 1–38, May 2016, doi: 10.1145/2871196.
Many professionals "care" about ethics because those professions require a license to practice
Licensing bodies maintain ethical standards for the profession
Software Engineering is not a licensed profession!!
...
Should it be?
B. C. Stahl, J. Timmermans, and B. D. Mittelstadt, “The Ethics of Computing: A Survey of the Computing-Oriented Literature,” ACM Comput. Surv., vol. 48, no. 4, pp. 1–38, May 2016, doi: 10.1145/2871196.
Many professionals "care" about ethics because those professions require a license to practice
Licensing bodies maintain ethical standards for the profession
Software Engineering is not a licensed profession!!
...
Should it be?
B. C. Stahl, J. Timmermans, and B. D. Mittelstadt, “The Ethics of Computing: A Survey of the Computing-Oriented Literature,” ACM Comput. Surv., vol. 48, no. 4, pp. 1–38, May 2016, doi: 10.1145/2871196.
Why is it difficult to conceive of a Software Engineer professional license?
Excerpt from the Windows 10 EULA:
Except for any repair, replacement, or refund that Microsoft, or the device manufacturer or installer, may provide, you may not under this limited warranty, under any other part of this agreement, or under any theory, recover any damages or other remedy, including lost profits or direct, consequential, special, indirect, or incidental damages.
"Harnessing powerful technologies and hidden algorithms, Match intentionally designs the platforms with addictive, game-like design features, which lock users into a perpetually pay-to-play loop that prioritizes corporate profits over its marketing promises and customers' relationship goals," lawyers for the plaintiffs wrote in the suit."
"The question the lawsuit poses is: Does Match Group have to disclose the potentially addictive quality of such commonplace design features? And have the company's lack of warnings constituted a violation of consumer protection laws?"
"The legal action against Match Group joins a new crop of lawsuits challenging tech companies, including Google, Instagram owner Meta and TikTok, in an attempt to hold platforms accountable for exacerbating the youth mental health crisis."
Ethics are uniquely important in software engineering:
For example, people in many countries are notoriously sensitive to the representation of disputed border territories on maps. In 2010, an error in Google maps led to Nicaragua dispatching forces to its border with Costa Rica. Google then worked with US State Department officials to correct the error. Read more here.
Providing radiation therapy to cancer patients, Therac-25 had malfunctions that resulted in 6 deaths. Who is accountable when technology causes harm?
Barriers to accountability:
https://ethicsunwrapped.utexas.edu/case-study/therac-25
Iran Air Flight 655 was a civilian passenger flight that was shot down by the United States Navy guided-missile cruiser USS Vincennes on July 3, 1988.
At 10:17 AM local time, the USS Vincennes fired two SM-2MR surface-to-air missiles at the aircraft. One missile hit the plane, causing it to disintegrate and crash into the sea. All 290 passengers and crew on board were killed, including 66 children and 16 crew members.
Iran Air Flight 655 was a civilian passenger flight that was shot down by the United States Navy guided-missile cruiser USS Vincennes on July 3, 1988.
At 10:17 AM local time, the USS Vincennes fired two SM-2MR surface-to-air missiles at the aircraft. One missile hit the plane, causing it to disintegrate and crash into the sea. All 290 passengers and crew on board were killed, including 66 children and 16 crew members.
Barriers to accountability:
"The naval officer responsible for authorizing a missile launch, the watch's Anti-Air Warfare Coordinator (AAWC), pushed wrong buttons no fewer than five times in response to a system message to select a weapon."
Iran Air Flight 655 was a civilian passenger flight that was shot down by the United States Navy guided-missile cruiser USS Vincennes on July 3, 1988.
At 10:17 AM local time, the USS Vincennes fired two SM-2MR surface-to-air missiles at the aircraft. One missile hit the plane, causing it to disintegrate and crash into the sea. All 290 passengers and crew on board were killed, including 66 children and 16 crew members.
Findings from the Incident Report:
Radar and Identification Systems:
Misinterpretation of Transponder Signals:
who is responsible for bad design?
I am a good person
Evil is done by evil people
I don't need to worry about ethics
To prevent, we need to imagine the unintended consequences and outcomes, and try to prevent them
https://www.cambridge.org/core/journals/business-ethics-quarterly/article/why-a-right-to-an-explanation-of-algorithmic-decisionmaking-should-exist-a-trustbased-approach
What is fairness to an algorithm?
due process:
a legal principle ensuring that everyone is entitled to a fair and impartial procedure before being deprived of life, liberty, or property.
...
For algorithms in society, this means that people should have the right to:
Burton, Emanuelle; Goldsmith, Judy; Mattei, Nicholas; Siler, Cory; Swiatek, Sara-Jo. Computing and Technology Ethics: Engaging through Science Fiction (pp. 117-118). MIT Press.
hungry judges
a study of parole boards in 2011 found that parole was granted nearly 65% of the time at the start of a session, barely above 0% right before a meal break, and again nearly 65% after a break
...
hungry judges are harsher judges!
What is fairness to an algorithm?
Think about the difference between using automated systems with different levels of:
Human-out-of-the-loop (Quadrant 3)
Where the probability of harm is low & severity of harm is low, the system no longer needs humans to help make a decision. In this scenario, the AI runs on its own without human supervision.
Human-over-the-loop (Quadrants 1 and 4)
Humans mediate when it's determined an automated system has failed. However, if humans are not paying attention, AI will continue without human intervention.
Human-in-the-loop (Quadrant 2)
Probability of harm and the severity of harm is high, requires a human to be part of that decision. AI provides suggestions to humans. If there is no human to make a decision, no action is executed.
HUMAN OVER THE LOOP
Example 1: Traffic prediction systems. Automated system will suggest shortest route to next destination. Humans can overturn that decision.
Example 2: Some cybersecurity solutions. Important company data is protected by a firewall and encrypted. Hackers are unlikely to penetrate your firewall and decrypt encrypted data. However, if they do, severity is high. For insidious new attacks, humans should pay close attention to what is happening.
HUMAN OUT OF THE LOOP
Example 1: Recommendation engines. Many e-commerce sites will help consumers find the products they are most likely to buy, Spotify recommends songs you want to listen to next. The probability of damage is low, and the severity of seeing disliked shoes or hearing disliked songs is also low. Humans are not necessary. What about recommending news articles?
Example 2: Translators. Except in highly delicate situations, AI-based translation systems are improving at such a rapid rate that we will not need humans soon. AI is learning how to conduct basic translation and how to translate the meaning of slang and localized meanings of words.
HUMAN IN THE LOOP
Example 1: Medical diagnosis and treatment. AI can help a doctor to determine what is wrong with you. Furthermore, AI can suggest treatment for you by examining the outcomes from other patients with a similar diagnosis. The AI will be aware of patient outcomes that our doctor is not aware of. AI is making our doctors more educated, and we want our doctors to make the final decision.
Example 3: Lethal Automated Weapon Systems (LAWS). Severity of harm is so high (shooting the wrong target) that it overrides whatever the probability of harm is (for now). In "rights-conscious nations," no LAWS can fire without human confirmation.
...
DODD 3000.09 requires that all systems, including LAWS, be designed to “allow commanders and operators to exercise appropriate levels of human judgment over the use of force.” [...for now]
Minmaxing:
A strategy often used in games, optimization problems, and decision-making processes, where a player or decision-maker seeks to maximize their strengths or advantages while minimizing their weaknesses or disadvantages. The term originates from the concept of minimizing the possible maximum loss, or maximizing the possible minimum gain, depending on the context.
In video games and role-playing games (RPGs), players often use minmaxing to optimize their characters by allocating resources such as skill points, equipment, and abilities in a way that maximizes their effectiveness in desired areas (e.g., combat prowess, magic ability) while minimizing their effectiveness in less important areas.
https://web.stanford.edu/class/cs182/
How do we balance the value of an algorithm in terms of achieving public safety against other values beyond fairness?
Example: Ford Pinto
https://www.classiccarstodayonline.com/classic-car-print-advertisements/ford-1974-ford-pinto-ad-a/
https://www.tortmuseum.org/ford-pinto/#images-1
Since $137 M > $49.5 M, they said "meh"
It did not go well
500-900 people burned to death.
Ford found not guilty of homicide. Did the recall afterwards. Incurred legal fees.
Example: Ford Pinto
Expected payout for potential accidents:
Cost to fix the fuel tank design:
Since $137 M > $49.5 M, they said "meh"
It did not go well
500-900 people burned to death.
Ford found not guilty of homicide. Did the recall afterwards. Incurred legal fees.
Example: Ford Pinto
Expected payout for potential accidents:
Cost to fix the fuel tank design:
Moral intuition: an unreasoned reaction (not unreasonable)
Learning objectives:
A computer hardware designer could create a new phone that was feature rich but also designed to be obsolete within a year so that the user would need to replace it quickly. This would allow for a significant revenue stream for the company. However, the plethora of features could be too complex for an elderly person to manage, and the planned obsolescence could result in excessive waste.
On the flip-side, the designer could implement a version of the phone where the initial setup included a choice of “essentials only” resulting in a screen that was easy to navigate and included a large, clean, display. There could be a prominent “help” button that connected the user with a predetermined relative or caregiver. The phone could come with an offer of a discount if the buyer’s previous phone was turned in for recycling (materials reuse) or donation to a non-profit.