Artificial Intelligence Fall 2023

Introduction to Artificial Intelligence

CS 4260/5260, Section 2

Fall 2023

This is the syllabus for Section 2 only — see Professor Meiyi Ma for the syllabus for Section 1 of CS 4260/5260 in Fall 2023.

Time and Place: MWF 1:25 pm to 2:15 pm, FGH 211

Instructor: Douglas H. Fisher (Doug, Professor/Dr Fisher, Professor/Dr Doug) douglas.h.fisher@vanderbilt.edu

Instructor Office hours: See my main page for Fall 2023 office hours

TA: Kyle Moore

TA Office Hours:

  • Where: FGH 226
    • Mondays 12:00 pm – 1:00 pm
    • Wednesdays 10:00 am to 11:00 am
    • Fridays 11:00 am – 12:00 pm

Motivation and Overview

With the likes of chatGPT, artificial intelligence (AI) is being discussed in public as never before.  I didn’t think I would live to see such performance by an AI, though I was sure it was coming and though its still very imperfect. While AI has always been a popular topic of science fiction, high-profile technologists now actively contemplate the potential dangers of AI, particularly the prospect of general (or autonomous, or strong) AI and the subsequent displacement of the human species as the dominant entities on our planet. In contrast, optimistic views of general AI focus on the potential of AIs as companions, caregivers, advisers, and collaborators in wide ranging areas. In the realm of general AI, the optimistic view is also that AIs can be positive role models to humans and that collective intelligence composed of both humans and AIs will be more powerful and compassionate than either “species” would be alone.

Historically, most introductory AI courses take a tools-based perspective of AI, where the study of the nuts-and-bolts concepts and algorithms used by task-specific (or narrow, or weak) AIs dominate a course. While AIs that support very specific tasks may seem less exciting than general AIs, it is these task-specific AIs that are driving much public debate: for example, about using machine learning to automate recommendations for prisoner parole and job-candidate interviews and about the use of computer vision and image processing for facial recognition in retail and law enforcement. On the other hand, task-specific AI may be used to strengthen human decision-making in areas such as environmental sustainability, resource management, health care, and education.

Treatment of the bread-and-butter, task-specific material will continue, just as it has for decades, but I’ve adapted the course in the face of advances of deep learning, enabled by massive data capture, computer processing and memory performance, and algorithmic improvements. This latter material is typically not covered at the introductory level, but I view coverage as a necessity, at least at a suitable level of abstraction. My view is that the future of AI will be that most intelligent processing is ultimately implemented at deep (aka fine grained, sub-symbolic, neural, compiled) levels, while concepts and algorithms that are generally presented in AI courses will continue to be the high-level concepts and languages that AI developers use to communicate among themselves in software development, and that human developers will use to communicate to AIs through prompting. If prompting really is to be the next necessary skill in software development, then consider the material in this class as vital for optimal prompting of AI collaborators – at least in the near term, until AIs themselves prompt AIs in what will be an AI instance of Julian Jaynes’  theory of a bicameral mind.

Course Coverage

In this introductory course, students will learn the concepts and methods of core AI areas, including search and exploration, knowledge representation, inference and planning, reasoning with and without various forms of uncertainty, and machine learning. These are the high level programming concepts and languages that AI developers must know. But we will also cover artificial neural networks, so called deep representations and deep learning, that may be the machine level languages and processes of AIs. In one semester we will not touch on all broad areas of AI, but in addition to mastering what we do study, my expectation is that students will be better prepared and informed for advancing in AI, be it from formal coursework or from independent study.

Textbook and other sources

Our textbook is

  • Poole, D. L., & Mackworth, A. K. (2023). Artificial intelligence: Foundations of computational agents (3rd ed.). Cambridge University Press. Poole & Mackworth (PM) is available in hard copy and free online at https://artint.info/.

An optional textbook, recommended particularly for the AI enthusiast is

  • Russell, S. J., & Norvig, P. (2021). Artificial intelligence: A modern approach (4th ed.). Pearson Education, Inc. This book has the advantage of including material that goes well beyond what we can expect to do in one semester, covering areas such as vision and robotics, thus supplying ample material for continued study. Russell & Norvig (RN) is available in various modalities. Information is available at http://aima.cs.berkeley.edu/.

My lectures draw on publicly available auxiliary material (e.g., slides, exercises) from the websites of both PM and RN, as well as other sources.

There are other textbooks that can be good resources. Some are out of print and online:

  • Winston, P. H. (1992). Artificial intelligence (3rd ed.). Pearson Education, Inc. Out of print. This classic text is now available at https://courses.csail.mit.edu/6.034f/ai3/rest.pdf. Winston arguably wrote the first AI textbook in the mid-1970s, from which this third edition grew. His Third edition has 10 chapters on various styles of machine learning (yikes!). I will use this textbook for supplemental material during one or two weeks.

Another text on the history of AI by one of the founders may interest some:

The course schedule (below) lists the required and optional readings from these various texts for each week.

Other Materials: You will be reading other papers, watching videos, and listening to podcasts throughout the semester. These are listed in the course schedule below for each week. Over Summer I emailed preregistered students suggesting that they take the Vanderbilt-Coursera course on prompt engineering. You can still work it in to your schedule (optional) and it will come up in class and discussion, along with two papers along the same lines (“ChatGPT Prompt Patterns for Improving Code Quality,
Refactoring, Requirements Elicitation, and Software Design“, “A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT“).

Course Organization

The course requires attendance in class, homework, exams, programming projects, and encourages you to participate in discussion, even if only to read the posts.

Attendance in class is required, not just by me but by School of Engineering policy. I will take attendance with Top Hat and I will sometimes ask questions on Top Hat, where participation and correctness of your answers will be factored into your grade. So bring a device that allows you to interact with Top Hat. See more about grading of attendance under Grading.

There are three exams during the semester, each of which covers about one month of material, as well as a comprehensive final exam. Our final exam is scheduled for Monday, December 11 at 9:00 am – 12:00 Noon CENTRAL TIME. There are no alternative times.

We will be using Piazza as the class discussion forum. The TA (Kyle) and I will typically use Piazza to make announcements too. If you have not been previously enrolled by me, you can join the class forum by following the Piazza link. The discussion forum is for students to ask questions and give comments on any aspect of the course. Kyle and I will respond as appropriate, but students are encouraged to respond too. We also encourage knowledge sharing of ideas, specifications, prompts, and designs regarding the programming project. In general, students can and should be learning from other students in this course. There may also be prompts from me and Kyle for questions and comments regarding the my slides and instructor notes, source reference texts, practice problems, the programming project, and societal issues of current relevance. And if you find an interesting paper or topic, then feel free to post that, along with your commentary.

Homework and programming are elaborated in their own sections below.

Homework

There are two aspects to each homework. You are given questions that ask for free form answers (e.g., show a travel route from X to Y under specified constraints). You will turn in a pdf of your free form answers to Brightspace under Assignments. But for primary purposes of grading you will take a “quiz” on those questions, or close approximations, as well as other material from lectures and instructor notes for the corresponding week.

  • You must submit the free form pdf BEFORE you take the quiz portion of the homework.
  • You will take the quiz portion of the homework for the first time IN CLASS on Wednesdays. You take the quiz component on Brightspace, so have uploaded the free form before class and bring your laptops.
  • You can take the quiz component on Brightspace (under Quizzes) one additional time outside of class until Wednesday 11:59 pm.
  • Your grade on the quiz component is the average of your quiz attempts.

Kyle and I may consult the free form answers under Assignments to check your work. We may adjust the homework score accordingly but the default is that your score is reflected entirely by the quiz component of a homework. Think of the free form version as “showing your work” for the quiz version that will actually count as your grade. See us if you think that your free form answers show a greater degree of understanding than what was suggested by the quiz component — examination of the free form can raise your grade (if your free form answers appear to be the result of an earnest effort), as well as lower your grade (e.g., if your free form answer is nonsense), but either change is an exception rather than the rule.

Programming Projects

There are four coupled programming assignments distributed across the semester. The programs are done as team of 3 students.

The default language is Python, though if your team has a compelling reason for using a different language please let us know. You are free to use AIs, like chatGPT,  to design, implement, document, and evaluate the AI system consistent with the specification. Think of an AI as an additional team member, smart but flawed, that doesn’t count against your human total.

For each deliverable, you will be submitting (a) a program listing that is well documented; (b) the most significant prompts you used to co-create the code with an AI collaborator, if  applicable; and (c) sample runs on test cases. As an example of “well-documented” see sample code for Othello in C.

For the final deliverable you will also submit a video where you illustrate technical details and experimental studies of your code, which may be above and beyond the requirements of the assignment. You can optionally post these videos to the Discussion Forum for all students to see and comment on.

The programming project is intended to promote individual creativity but also community knowledge sharing. You are encouraged to share ideas, designs, and prompts on the Discussion Forum. You can share on the forum as an individual or on behalf of a team, but do not to share code unless approved by me or Kyle. An important rule of thumb is that if you share with anyone (outside of your team), then you share with everyone. Another important rule is to cite the sources of your ideas and designs, most notably those of students from other teams. See the course Honor Code below. Programming project deliverables are uploaded to Brightspace.

A community score is also computed, in part, from the number of citations you receive from others who have adopted/adapted your intellectual contributions and, to a lesser extent, when you adopt/adapt (and cite) the contributions of others. Other aspects of the community score are your engagement in the discussion forum and the live session. Your community score can contribute up to 2% extra credit on your total grade.

Grading

The following are the breakdown of factors and weights used to compute your grade.

  • Attendance and in-Class (Top-Hat) Questions: 5%
  • Exam 1: 10%
  • Exam 2: 10%
  • Exam 3: 10%
  • Final Exam: 15%
  • Homework: 20%
  • Programming Project: 30%

Again, in-class attendance is required. If you attend all classes (as recorded on Top Hat at the start of class), excepting previously-excused absences, then attendance counts for 5% of your grade. If you miss all classes, then your (lack of) attendance counts for 30% of your grade (i.e., 0%/30%), with the extra 25% taken proportionally out of the weights of other factors. Intermediate between perfect attendance and zero attendance is something intermediate, which will effect the factor weight and percentage attendance proportionally. If you record yourself present on Top Hat from outside the classroom then that is an honor offense.

The letter grade breakdown is

  • [92, 100] A (a very few A+s may be given)
  • [90, 92) A-
  • [87, 90) B+
  • [82, 87) B
  • [80, 82) B-
  • [77, 80) C+
  • [72, 77) C
  • [70, 72) C-
  • [67, 70) D+
  • [62, 67) D
  • [60, 62) D-
  • [0, 60) F

Don’t assume that I will round up.

Honor Code

Fundamentally, the honor code is about giving credit to others where credit is due, which allows for a more accurate assessment of your knowledge and it better ensures community trust and the societal benefits that stem from that trust.

For this class

  • you are encouraged to use AI tools to design, implement, document, and evaluate your programming project;
  • you can work freely with your project team members about the project (this should go without saying, but I am saying it anyways);
  • you may ONLY discuss the programming project with other students (outside your team) over public posts to the discussion forum; you may NOT discuss the project with other non-team students privately (see the rule of thumb under Programming Project);
  • you may NOT consult texts, notes, slides, or communicate with or use others when you take exams (nothing, except your own brain);
  • you MAY consult with texts, notes, slides, AIs, and with other people on the open format components of homework, and you must precisely declare the nature of any help you received or gave to other people (and from/to whom) in the open format submission; your grade may be adjusted based on the nature of help from other people, but its very unlikely; do NOT post answers to the homework on the discussion form;
  • you may NOT consult with others on the quiz portion of any homework. You may NOT consult with AIs, texts, notes, slides, etc while you are actually taking a quiz attempt, but you can consult sources (except other people) between attempts;
  • when recording your attendance on Top Hat you must be in the classroom.

Schedule

Week 1 Class and AI Overview

  • Read
  • Do Week 1 Homework Free Form due Wednesday August 30 before class (see Top Hat)
  • Class meetings
    • Wednesday, August 23: Overview, AI architectures
    • Friday, August 25: AI Architecture and capabilities

Week 2: Exploring Alternatives through Search

  • Read
    • PM Section 2.4
    • PM Chapter 3 through 3.12 (all of it)
    • Optional: chapters 3 of RN.
  • Do Week 2 Homework Free Form due Wednesday Sept 6 before class (see Top Hat)
  • Class meetings
    • Monday, August 28: Graph Search
    • Wednesday, August 30:  Quiz HW1; Informed Search, State-Space Search;
    • Friday, September 1: State-Space Search

Week 3: Other Search Paradigms

  • Read
    • PM Chapter 4 through 4.3
    • PM Chapter 14 through 14.3
    • Optional: chapter 6 through 6.3 of RN
    • Optional: chapter 5 through 5.3 of RN
  • Do Week 3 Homework Free Form due Wednesday September 13 before class (see Top Hat)
  • Class meetings
    • Monday, September 4: Search implicit graphs; Utility-driven search; anytime search;
    • Wednesday, September 6: Quiz HW2;  learning macro operators
    • Friday, September 8: Adversarial search; Search with constraints

Week 4: Reasoning, Planning, and Learning

  • Read
    • PM Chapter 5 through 5.3
    • PM Chapter 6 through 6.1
    • PM Chapter 7 through 7.1
  • Do Week 4 Homework Free Form due Wednesday September 20 before class
  • Class Meetings
    • Monday, September 11: Propositional Theorem Proving;
    • Wednesday, September 13: Quiz HW3; Propositional Planning;
    • Friday, September 15: Machine learning introduction;

Week 5: Machine Learning; Exam 1

  • Read
    • PM Chapter 7 through 7.4
  • Do Week 5 Homework Free Form due Wednesday September 27 before class
  • Class Meetings
    • Monday, September 18:  Decision Tree Learning
    • Wednesday, September 20: Quiz HW4;  Overfitting; learning ensembles
    • Friday, September 22: Exam 1 (August 23 – September 15). See Key on Top Hat.

Week 6: Introduction  to Artificial Neural Networks

  • Read
    • PM Chapter 8 through 8.1
    • Continue studying PM Chapter 7 through 7.4
  • Do Week 6 Homework Free Form due Wednesday October 4 before class
  • Class meetings
    • Monday, September 25: Feedforward Neural Networks
    • Wednesday, September 27:  Quiz HW5; The Delta Learning Rule
    • Friday, September 29: Program 1 finishing

Week 7: Program 2; Uncertain reasoning

  • Read
    • PM Chapter 9 through Section 9.3
  • Week 7 Homework Free Form due Wednesday October 11 before class
  • Class Meetings
    • Monday, October 2: Second programming project ASSIGNED
    • Wednesday, October 4:  Quiz HW6;
    • Friday, October 6: Probability basics; Probabilistic inference;

Week 8: Reasoning with Uncertainty

  • Read
    • PM Chapter 9 through Section 9.5
  • Do Week 8 Homework Free Form due Tuesday October 17 by 11:59 pm
  • Class Meetings
    • Monday, October 9: Bayesian Networks
    • Wednesday, October 11:  Quiz HW7;  Learning Bayesian Networks
    • Friday, October 13:  Full and partial joint probability computatiuons

Week 9: Preferences; Program 3; Exam 2

  • Read
    • PM Chapter 12. through 12.1
  • Do Week 9 Homework Free Form due Wednesday October 25 before class
  • Class Meetings
    • Monday, October 16: Third programming project ASSIGNED
      • Quiz HW8 by 11:59 pm on Tuesday October 17
    • Wednesday, October 18: Quiz HW8; Exam 2 (Comprehensive, but focus on September 18 to October 13)
    • Friday, October 20: Fall break

Week 10: Sequential Reasoning

  • Read
    • PM Section 8.5, 9.6
  • Do Week 10 Homework Free Form due Wednesday November 1 before class
  • Class Meetings
    • Monday, October 23: Sequential Probabilistic Models
    • Wednesday, October 25:  Quiz HW9; Language models
    • Friday, October 27: Back propagation

Week 11: Decision Making

  • Read
    • PM Chapter 12 through 12.3.2
  • Do Week 11 Homework Free Form due Wednesday November 8 before class
  • Class Meetings
    • Monday, October 30: One-off Decision Making;
    • Wednesday, November 1:  Quiz HW10; Sequential Decision Making;
    • Friday, November 3: Learning Policies

Week 12:  First Order Inference and Planning

  • Read
    • PM Chapter 15 through 15.5
  • Do Weeks 12 Homework Free Form due Wednesday November 15 before class
  • Monday, November 6: First Order Representations
  • Wednesday, November 8: Quiz HW11; First Order Representations; Theorem Proving
  • Friday, November 10: First Order Theorem Proving and Planning

Week 13: Knowledge graphs, Program 4, Exam

  • Read
    • PM Chapter 16 through 16.1
  • Do Week 13 Homework Free Form due Wednesday November 29 before class
  • Monday, November 13: Knowledge Graphs
  • Wednesday, November 15:  Quiz HW12; Program 4
  • Friday, November 17: Exam 3 (Comprehensive, but Focus on October 16 – November 13)

Thanksgiving Break: November 18 – November 26

Week 14: Program 4, MDPs revisited

  • Weeks 14 Homework Free Form due Wednesday December 6 before class
  • Monday, November 27:  Welcome back; work in groups on Program 4
  • Wednesday, November 29:  Quiz HW13; work in groups on Program 4
  • Friday, December 1: Reinforcement Learning

Week 15: Transformers; Final exam

  • Monday, December 4: Transformers
  • Wednesday, December 6: Quiz HW14; Review
    • Final Exam review and study guide on Piazza

Final Exam: Monday, December 11 9:00 am – 12:00 pm (Comprehensive) (https://registrar.vanderbilt.edu/documents/Fall_2023_exam_schedule.pdf).