Teaching Responsible Computing Playbook

Choosing Computing Courses

Courses are the primary mode of interaction between faculty and the vast majority of undergraduate students. Hence, courses are one strong vehicle to educate students about responsible computing. However, not all courses nor all students are the same, and some are better targets for responsible computing material than others.

By way of analogy, we first argue why responsible computing should be included in computing courses themselves (as opposed to say a course in philosophy). The “[X] across the curriculum” models challenge the academic division of labor by which a particular discipline/department is supposed to teach “skills” and a different discipline/department is supposed to teach “content”. For example, a Writing Across the Curriculum model challenges the idea that the Composition, English, or Writing program is where students learn the writing “skills” that they will then apply to the “content” they’re learning in, say, their coursework in the sciences. That idea presumes that “good writing” consists of purely formal or technical elements that are context- or task-independent. Instead, a Writing Across the Curriculum model begins from a more substantive, contextual, and task-oriented understanding of “good writing”. It doesn’t deny that there are formal elements common to all good writing. However, it emphasizes that students should learn their major discipline’s standards of good communication (including good writing) from suitably-trained instructors in that discipline, perhaps in specially designated “writing-in-the-major” courses. This means that students are learning that learning to write/communicate well is a priority, both in their general curriculum coursework and in their major-specific coursework. Applying that model here, to responsible computing concepts and habits, means not subcontracting the teaching of all of those concepts or habits to some other department, but instead thinking about a relatively small number of core things [methods, skills, concepts, habits] that the students will encounter repeatedly throughout their coursework in computing. As in the case of writing-in-the-major courses, the responsible computing-focused courses should include concepts that are authentic to the discipline or area.

This section focuses on three kinds of courses: (i) required courses (i.e. courses that are required to be taken by every student in your program); (ii) existing elective courses (we’ll assume these are junior/senior-level courses that satisfy certain program requirements but are not required courses) and (ii) new courses (i.e. a course focused on responsible computing but does not yet exist in your curriculum).

Introducing responsible computing material in required courses has the advantage of reaching every student in a computing program. However, such courses typically have a standard set of topics to be covered and thus may be perceived by instructors not to have”room” for new topics. As an example, most computing majors are required to take an algorithms course. Instructors of this course might ask, “how do you talk about responsible computing in an algorithms/complexity course amidst all the proofs?”

Existing elective courses, if chosen appropriately, can be a natural fit to introduce responsible computing material. Electives are an attractive target because course material may naturally encourage discussions about responsible computing (e.g. a machine learning course could talk about an impact of machine learning in society) and may moreover have a receptive student audience. However, implementing responsible computer science in electives may be ineffective if the course is not widely popular. And a receptive audience may also be an audience already familiar with concepts in responsible computing, resulting in a situation where students who need responsible computing education the least are the most likely to receive it. Further, the elective courses also tend to be technical and have similar “no room in schedule” issues as with the required courses (though to a lesser extent).

Creating a new course focused on responsible computing has the advantage of giving students a deep dive into the relevant issues but also has the disadvantage of needing the required legwork of creating a completely new course. There is an enormous benefit to students spending time discussing and talking about uncomfortable issues at the heart of responsible computing topics, which just by its nature is difficult to do by limited intervention in a more traditional course.

The other important dimension to consider when introducing responsible computing is the target student audience. For example, how we talk about algorithms/machine learning systems contributing to racism would be different if we are in an institution with a majority of underrepresented students versus when our student population is mostly white. Module(s) in a course that aims to include responsible computing can generally use multiple real-life scenarios to ground the discussion: the challenge is picking one that will resonate with our students.

Finally, none of this is possible without having faculty members who are interested in adding responsible computing material in existing courses or creating new ones. Forcing an unwilling faculty member to include responsible computing in a course that they teach (e.g. they teach the required course for first-year students in your program and you want to reach the students early) can do more harm than good. The decision on whether a department targets a single, dedicated course vs. multiple courses also depends on faculty expertise: a dedicated course will need an instructor who has the relevant experience/background (especially in areas beyond computing: see the related section on structured ways of thinking about computing and society) to create and teach a new course.

Key Questions:

Which kinds of courses will be targeted? In particular, which groups (required, existing electives and/or new course) will be targeted? Note that the decision for this question might not be possible to make independent of the decision on who would teach the targeted courses.
If required and/or existing electives were chosen, which specific courses in the curriculum will be chosen? If a specific course is being left out, why is that the case? If the answer is that the course is too abstract (e.g. theory courses), please consider revisiting the decision.
If a new course was chosen, what would the course be on? There is no need to narrow the scope (at least at the planning stage): impact (especially negative) of computing on society is all around us (COVID-19 and Black Lives Matter to take recent 2020 examples). Consider incorporating increased societal awareness when creating the new course.
Which faculty members would be willing to include responsible computing material in the courses they teach? The first step is to determine their interest in general. The next step is to then see which of the courses that have been picked are/can be taught by these faculty members. As mentioned earlier, this question might be relevant when answering the previous three questions. Another aspect to consider are the logistical issues of coordinating the responsible computing material across multiple courses vs. not having to deal with the coordination logistics if a single dedicated course is being targeted. Finally, faculty expertise/background as well as their teaching load/duties will also need to be taken into account.
For required and existing elective courses, what will be the delivery mode? Lectures while good for an introduction are ineffective if not backed by assignments. Will the intervention target homework or a project?
How will responsible computing be incorporated in the assignments? For required courses, one option is to use responsible computing to tell a story around the assigned problem (but the story should resonate with the students). For electives one option is to create a project that asks students to consider various options when looking at societal impacts of computing. In both cases, the instructor has to meet the students where they are at and not where the instructor would like them to be (this might mean that the story might not be related to what is being covered in the media currently).
For a new dedicated course who is the target audience? Will there be students who are not in a computing major (e.g. from the humanities) in the course?
Do the instructors have the required support (e.g. TAs/developers) to develop/implement the assignments/projects? Depending on the scope of your assignment, the instructors might need to get help to implement the assignments/projects.
Are the instructors going to get feedback from students? Ideally, faculty should involve students from the target audience (e.g. UG TAs for an UG course) to help with the development or at least get critical feedback (multiple times) from the target audience.
If the instructor is using an existing assignment developed by someone else, are they modifying it to suit your students? The “story” for the assignment chosen by another school might not resonate that well with your students, in which case some modification might be required.

Checklist

☐ Identify which groups of courses (required, existing elective, new course) to target

☐ Identify the relevant courses within each group

☐ Identify which faculty would be interested in incorporating responsible computing and which courses they (can) teach

☐ If the intersection of courses from the last two steps is empty, revisit previous steps. Is the intersection empty because the instructors need more resources or does the department need to manage faculty resistance?

☐ For each chosen course, pick the delivery mode

☐ Design the targeted assignment/project with your students’ interests in mind

☐ Identify ways to include diverse voices in the class (in particular, from outside of computing disciplines)

☐ Instructor has adequate resources to develop/implement the assignment

☐ Feedback from target audience has been solicited during development of the assignment

University at Buffalo

All three kinds of courses were targeted:

Required courses
- First year seminar (CSE 199): Module on history, society and diversity and module on societal computing were targeted. Each module is a two week long combination of lectures and in-class activities. CSE 199 is required for BS Computer Science and BS Computer Engineering programs.
- Algorithms course (CSE 331): Homework assignments and a project based on access to high-speed Internet in WNY were introduced. CSE 331 is required for the BS Computer Science program.
- Capstone course (CSE 442): Students review a made-up dossier of government data to identify if an individual should be added to the government’s No-Fly list. Students must then discuss biases in the data provided and, finally, how they would respond if their job assigned them to the team automating this analysis. A second assignment asks students to document the ethical implications of the capstone project they chose. CSE 442 is the capstone course for the BS Computer Science program.
Existing elective
- Machine learning course (CSE 474): Students worked in a group project exploring bias in data driven criminal risk assessment
New course
- Algorithms and Society: Course that had separate sections for CSE and non-CSE majors (but seated in the same room). The goal was to give non-CSE students a non-trivial understanding of various stages of the ML pipeline and for CSE students to get a deep dive into societal implications of machine learning systems. The class was a mix of lectures and discussions.

Check-list walk-through. We now give a quick walk-through of the check-list items to show how they were resolved at UB. (Warning: The checklist above did not exist when the effort was started so the mapping might not be perfect. However, the hope is that this gives an example of how you might go about using the checklist.)

Identify which groups of courses (required, existing elective, new course) to target
- We decided early in the process that we wanted to target the required courses (at least for the BS Computer Science program) so as to reach as many students as possible. We were also open to elective courses if they were among the popular ones (e.g. the Machine Learning course). Creating the new course was not originally planned but an appropriate opportunity arose in fall of 2019.
Identify the relevant courses within each group
- We decided to target roughly one course for each of the four years of a student’s stay in the department. Our BS Computer Science curriculum has all the required courses in the first two years (except the senior capstone course). The first year seminar and the senior capstone course already had modules related to ethics so those were two natural choices. We decided to pick the algorithms course as the sophomore year course because we wanted to pick a course that on the surface might not seem amenable to having a responsible computing assignment to really drive home the point to students that responsible computing considerations are relevant to all the computing courses they take a UB. For the junior level course, we decided to target the machine learning course since (i) It was popular and (ii) machine learning is in news very regularly these days to to (un)ethical uses of ML and hence it seemed to be a nice fit.
  - This step, however, was not completely independent of the next step.
Identify which faculty would be interested in incorporating responsible computing and which courses they (can) teach
- We started with a small sample of faculty members. In particular, we talked with CSE faculty who were involved in an Ethical AI working group that was formed at the same time based on some small seed funding from SUNY (this took care of the Algorithms and ML courses). We also had revised the BS Computer Science curriculum (starting Fall 2018) and during that process some faculty had indicated they were interested in including ethical assignments in their courses, which then took care of the first year seminar and the capstone course (we were already ethics related assignments in the capstone course due to ABET accreditation needs).
If the intersection of courses from the last two steps is empty, revisit previous steps.
- The intersection for us was fortunately not empty.
For each chosen course, pick the delivery mode
- This was left completely to the instructors for the targeted course and following choices were made:
  - The first year seminar is composed of six two-weeks modules (consisting of four lectures and two in-class activities) and we picked two modules in the first year seminar.
  - The algorithms and ML courses picked new programming projects. Both courses had projects. The algorithms course already had some homework questions that were related to societal impact so those still continued.
  - The capstone course already had an ethics in-class activity (again due to ABET accreditation requirements).
Design the targeted assignment/project with the students’ interests in mind
- The first year seminar already had a well-received module on social media and machine learning, which was adapted to include responsible computing material. A new module on history and ethics of computing was also created.
- The ML course went with the COMPAS recidivism dataset and data driven criminal risk assessment. There already had been media coverage of this topic and so we figured we could use the coverage to introduce the topic to the students.
- The decision for the algorithms course took longer. One constraint we put in early was to make sure that the underlying technical concept that was covered early in the course. Given the current course structure the best option seemed to pick something related to the shortest path problem. On the other side, acknowledging that our student population is skewed towards students from WNY (and NYS in general), we wanted to pick a scenario that was local. During the discussion stage someone suggested we look into the NYS Attorney General’s case against Spectrum that alleged serious unethical behavior of Spectrum towards their customer. Given the ubiquity of the Internet in what we do as computing professionals this seemed like a scenario that all of our students would relate to. The choice was finalized when we realized that we could create a programming project related to routing of packets in this context, which would tie directly to the shortest path problem that we had covered in class (and on which students have a programming assignment early in the semester).
- As mentioned earlier, the capstone course already had an existing ethics related assignment but a second activity was introduced to allow students to see how these ethical considerations will grow once they begin their careers.
Identify ways to include diverse voices in the class (in particular, from outside of computing disciplines)
- We could not figure out a way to involve students from outside the department in the four courses discussed above. However, we ran the planned assignments by the UB faculty who were involved in the Mozilla RCS group, which included faculty from Architecture, Industrial Engineering, Law school and Media Study. E.g. for the Spectrum related coding project in the algorithms course we benefited immensely from the lawyer in the group both in navigating the NYS AG’s summons and complaint against Spectrum and making sure that our interpretation of the allegation was correct.
- For the Algorithms and Society course we purposefully designed the course to have a CSE majors section and a non-CSE majors section (seated in the same lecture) so that CSE students could interact with students outside of their major when thinking about societal impacts of computing. We also actively recruited non-CSE students via the Honors college on campus.
Instructor has adequate resources to develop/implement the assignment
- The Mozilla RCS award funded the faculty time but even more importantly funded a dedicated undergraduate TA pool for each of the four targeted courses in the proposal (first year seminar, algorithms, machine learning and the senior capstone course). These wonderful undergraduate TAs developed the relevant assignments over the summer of 2019 (and in the case of the algorithms and machine learning courses essentially developed the entire assignment on their own).
Feedback from target audience has been solicited during development of the assignment
- One of the big advantages of having undergraduate TAs developing the various assignments was that they also gave feedback as students as well (in all cases the UTAs had been students in the respective courses before).

Haverford College

At Haverford College Sorelle Friedler in the Computer Science department has worked together with Jon Wilson in Environmental Studies to teach students about the environmental impact of their code. The students are taught that the CO2 emissions of their code relates to the time it takes to run as well as the computer’s location and the energy mix of the electricity grid in that location. For example, locations that get a high percentage of their energy from coal power plants will have higher CO2 emissions than those locations for which low carbon sources such as wind power are predominant. These CO2 emissions calculations are related to computational complexity since code that takes longer to run will also have a higher energy usage and thus higher CO2 emissions. Coarse emissions calculations can be programmed easily by students or more nuanced calculations can be determined using the CodeCarbon python package.

In integrating these ideas of environmental sustainability into the computer science curriculum we focused on the connection to computational complexity since that allowed integration into the existing curriculum to reinforce existing curricular goals without needing to supplant them. While the underlying ideas and mechanisms for measuring code CO2 emissions were introduced in class, we focused on designing programming assignments and other problem sets that would allow students to explore and solidify their understanding of these connections outside of class. Resources for these assignments can be found here.

Analysis of Algorithms Course (complexity and environmental sustainability)

In Analysis of Algorithms (a senior-level algorithms course using the Kleinberg and Tardos Algorithm Design text) the analysis of the CO2 emissions was introduced as part of the introduction to complexity along with other ways that computational complexity can consider resources beyond time (e.g., along with a basic introduction of space complexity). The introduced problem set extends typical time complexity questions to include experimental measurement of energy usage and analysis and comparisons of the resulting time complexity to the resulting energy usage.

Data Structures Course (sorting, hash tables, complexity and environmental sustainability and voter suppression)

In the Data Structures course (a traditional CS2 course using the Goodrich, Tamassia, and Goldwasser Data Structures and Algorithms in Java text) we introduced a multi-week programming assignment motivated by the issue of de-duplication in voting rolls. As part of voter suppression efforts, voters have been purged from the rolls if their government id does not exactly match their voter registration information. To consider this issue of de-duplication, students examine voter registration data and count the number of duplicate entries. They use three strategies to perform this de-duplication:

The “all pairs” method, where each voter is compared to every other voter.
Sorting, where the voters are sorted and compared to their neighbors in the sorted list.
The hash table method, where voters are entered into a hash table and collisions are considered.

Students implement all three methods and experimentally compare their computational complexity as well as the energy usage of each method. Students are led through some coarse calculations and estimates that will allow them to report their energy usage in terms of, e.g., cups of coffee made, or other tangible energy equivalents.