Course Home Page for Ethical Algorithm Design (CIS 4230/5230) University of Pennsylvania, Spring 2023

Ethical Algorithm Design CIS 4230/5230
Spring 2023
Tuesdays and Thursdays 10:15 11:45AM ET
Annenberg 110

Instructor:

Prof. Michael Kearns
mkearns@cis.upenn.edu
Office hours: Tuesday right after class until 1PM, in the lobby area right outside Annenberg 110 or by appointment

Teaching Assistants:

Neha Dohare
neha75@seas.upenn.edu
Office hours: Wednesday 10:30-11:30AM in Levine 5th floor bump space or by appointment

Declan Harrison
declanh@seas.upenn.edu upenn.edu
Office hours: Thursday 9-10AM in 4th floor 3401 Walnut or by appointment

Natalie Ho
natabnho@sas.upenn.edu
Office hours: Wednesday 5-6PM in GRW 5th floor bump space or by appointment

Jordan Hochman
jhawkman@seas.upenn.edu
Office hours: Thursday 5:15-6:15PM in GRW 5th floor bump space or by appointment

Aakash Jajoo
aakashj1@seas.upenn.edu
Office hours: Tuesday 1:45-2:45PM in Levine 5th floor bump space or by appointment

Course Description

This course is about the social and human problems that can arise from algorithms, AI and machine learning, and how we might design these technologies to be "better behaved" in the first place. It is first and foremost a science or engineering course, since we will be developing algorithm design principles. You can get a rough sense of course themes and topics by visiting the websites for the pilot versions of this course offered in 2021, 2020 and 2019. The first formal offering of the course was in Spring 2022.

Here are the lecture videos from the last pilot version. Please note that they will not correspond exactly to this year's lectures, and should not be viewed as a substitute for mandatory lecture attendance.

Prerequisites: Familiarity with some machine learning, basic statistics and probability theory will be helpful. While this is not a theory class, you need to be comfortable with mathematical notation and formalism. There will be some simple coding and data analysis assignments, so some basic programming ability is needed.

Course content will include readings from the scientifc literature, the mainstream media and other articles and books.

Grades will be based on a mixture of quizzes, coding assignments, written homeworks, and a written midterm and final.

CIS 423/523 fulfills the SEAS Engineering Ethics Requirement for these programs: ASCS, BE, CMPE, CSCI, DMD and NETS (but you should confirm with your academic adivsor to be certain).

Lecture Dates

Topic

Slides, Readings, Assignments, Announcements

Thu Jan 12
Course Introduction and Overview

Lecture Notes
A general-audience introduction to some of the themes of the course is given in the (recommended but not required) book The Ethical Algorithm: The Science of Socially Aware Algorithm Design, by M. Kearns and A. Roth.
Also recommended but not required: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, by C. O'Neil.

Tue Jan 17
Tue Jan 19
Foundations of Machine Learning
Lecture Notes

Thu Jan 24
Tue Feb 26
Bias in Machine Learning: COMPAS and ProPublica
Lecture Notes
The following readings are required; you should read the two ProPublica pieces before the Jan 24 lecture so we can discuss them then.
ProPublica article on COMPAS
ProPublica analysis
Practitioner's Guide to COMPAS Core (no need to read, but we'll peruse a bit together in lecture)
COMPAS Risk Assesment Survey (just skim)
Northpointe response to ProPublica
ProPublica github repository, including dataset (we'll look at the dataset a bit in lecture)(technical, just skim)

Tue Jan 31
Thu Feb 2
Tue Feb 7
Thu Feb 9
Tue Feb 14
Thu Feb 16
Tue Feb 21
Science of Fair ML: Models and Algorithms
Lecture Notes
Readings:
Inherent Trade-Offs in the Fair Determination of Risk Scores. J. Kleinberg, S. Mullainathan, M. Raghavan. (First 8 pages required)
The Frontiers of Fairness in Machine Learning. Alexandra Chouldechova, Aaron Roth. (Required)
Please play around with the following Google demo site on fairness and ML. (Required)
Equality of Opportunity in Supervised Learning. M. Hardt, E. Price, N. Srebro. (Intro required, rest optional; this is the post-processing/bolt-on method)
A Reductions Approach to Fair Classification. A. Agarwal, A. Beygelzimer, M. Dudík, J. Langford, H. Wallach. (Intro required, rest optional; this is the in-processing/constrained optimization/game theory method)

An Empirical Study of Rich Subgroup Fairness for Machine Learning. MK, S. Neel, A. Roth, S. Wu. (Intro required, rest optional; this the rich subgroup/preventing fairness gerrymandering method)
An Algorithmic Framework for Bias Bounties. I. Globus-Harris, MK, A. Roth. (Read at least the Intro and Sections 5 and 6, which preview your upcoming group project)

Thu Feb 23
Tue Feb 28
Thu Mar 2
Tue Mar 14
Tue Mar 21
Thu Mar 23
Tue Mar 28
Thu Mar 30
Tue Apr 4
Differential Privacy and Related Topics Lecture Notes
Readings:
Confidence-ranked reconstruction of census microdata from published statistics. T. Dick, C. Dwork, MK, T. Liu, A. Roth, G. Vetri, S. Wu. (Read Abstract/Intro and Sections A,B,C)

Tue Mar 7
Thu Mar 9
Spring Break, no lectures .
Thu Mar 16
Midterm Exam (in person, written, closed book/notes) .
Last Two Weeks of Lecture
Ethical Algorithm Design in the Generative Era Lecture Notes

Lecture Dates	Topic	Slides, Readings, Assignments, Announcements
Thu Jan 12	Course Introduction and Overview	Lecture Notes A general-audience introduction to some of the themes of the course is given in the (recommended but not required) book The Ethical Algorithm: The Science of Socially Aware Algorithm Design, by M. Kearns and A. Roth. Also recommended but not required: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, by C. O'Neil.
Tue Jan 17 Tue Jan 19	Foundations of Machine Learning	Lecture Notes
Thu Jan 24 Tue Feb 26	Bias in Machine Learning: COMPAS and ProPublica	Lecture Notes The following readings are required; you should read the two ProPublica pieces before the Jan 24 lecture so we can discuss them then. ProPublica article on COMPAS ProPublica analysis Practitioner's Guide to COMPAS Core (no need to read, but we'll peruse a bit together in lecture) COMPAS Risk Assesment Survey (just skim) Northpointe response to ProPublica ProPublica github repository, including dataset (we'll look at the dataset a bit in lecture)(technical, just skim)
Tue Jan 31 Thu Feb 2 Tue Feb 7 Thu Feb 9 Tue Feb 14 Thu Feb 16 Tue Feb 21	Science of Fair ML: Models and Algorithms	Lecture Notes Readings: Inherent Trade-Offs in the Fair Determination of Risk Scores. J. Kleinberg, S. Mullainathan, M. Raghavan. (First 8 pages required) The Frontiers of Fairness in Machine Learning. Alexandra Chouldechova, Aaron Roth. (Required) Please play around with the following Google demo site on fairness and ML. (Required) Equality of Opportunity in Supervised Learning. M. Hardt, E. Price, N. Srebro. (Intro required, rest optional; this is the post-processing/bolt-on method) A Reductions Approach to Fair Classification. A. Agarwal, A. Beygelzimer, M. Dudík, J. Langford, H. Wallach. (Intro required, rest optional; this is the in-processing/constrained optimization/game theory method) An Empirical Study of Rich Subgroup Fairness for Machine Learning. MK, S. Neel, A. Roth, S. Wu. (Intro required, rest optional; this the rich subgroup/preventing fairness gerrymandering method) An Algorithmic Framework for Bias Bounties. I. Globus-Harris, MK, A. Roth. (Read at least the Intro and Sections 5 and 6, which preview your upcoming group project)
Thu Feb 23 Tue Feb 28 Thu Mar 2 Tue Mar 14 Tue Mar 21 Thu Mar 23 Tue Mar 28 Thu Mar 30 Tue Apr 4	Differential Privacy and Related Topics	Lecture Notes Readings: Confidence-ranked reconstruction of census microdata from published statistics. T. Dick, C. Dwork, MK, T. Liu, A. Roth, G. Vetri, S. Wu. (Read Abstract/Intro and Sections A,B,C)
Tue Mar 7 Thu Mar 9	Spring Break, no lectures	.
Thu Mar 16	Midterm Exam (in person, written, closed book/notes)	.
Last Two Weeks of Lecture	Ethical Algorithm Design in the Generative Era	Lecture Notes