CIS 5150 Handout 1

CIS 5150, fall 2024 Fundamentals of Linear Algebra and Optimization

Course Information August 28, 2024

Coordinates:

Monday-Wednesday, 12pm-1:29pm, Wu and Chen 101

Instructor:

Jean H. Gallier, Levine 476, 8-4405, jean@seas.upenn.edu

TAs:

Jocelyn Quaintance (jocelynq@seas.upenn.edu)
Dominic Olaguera Delogu (delogu@seas.upenn.edu)
Muskaan Beriwal (muskb@seas.upenn.edu)

Office Hours:

Jean: zoom link and time TBA

TA/Graders:

Jocelyn Quaintance (jocelynq@seas.upenn.edu):
Dominic Olaguera Delogu (delogu@seas.upenn.edu):
Muskaan Beriwal (muskb@seas.upenn.edu):

Welcome to CIS 5150 !

Since a lot of material for the fully online version of this course, MCIT 515, is available online, I plan to make use of this material, supplemented by extra slides. I plan to cover roughly the same amount of material this Fall 2024 than I covered in the Spring. In particular, I will cover some elements of optimization theory (the Lagrangian framework, ADMM) and some topics from machine learning, including

Hard Margin Support Vector Machines (SVM)
Soft Margin Support Vector Machines (SVM)
Solving Hard Margin SVM and Soft Margin SVM using ADMM
Linear Regression; Learning a Linear or an Affine Function
Lasso Regression
Solving Lasso Regression using ADMM
Elastic Net
Solving Elastic Net using ADMM
Introduction to Deep Learning (most likely will not be covered due to lack of time)

Deep Neural Nets
Stochastic Gradient Descent
Back Propagation and the Chain Rule
Convolutional Neural Networks

Syllabus (pdf)
Link to Workshop on Equivariance and Data Augmentation, September 4, 2020 (html)

Course Format

In order to increase the level of interation between the students and the instructor(s) I propose to use the following scenario.

Online lessons will be available every week on the CANVAS account for the class.
Every student is expected to listen to recorded lessons and read the corresponding material in the book before every class.
A list of the material to be listened to and read will be available on this web page a week before the actual lecture (see under CANVAS account).
During lecture time, I intend to
Lecture.
Take and answer questions about the material presented online for the lesson.
Occasionally present important proofs.
In general, attempt to motivate, demistify, and put in context the material of the lesson.
Give an idea about applying the material to solve the homework problems.

Classes will be recorded and uploaded to CANVAS.
Consequently, there will be a heavier burden and a greater requirement of self-discipline placed on the student to listen to and read the lessons to keep up with the course.
On the other, you will have greater flexibility in deciding when to listen and read the lessons in preparation for the actual class, which I hope, will be more of an interactive class.
We will try this learning mode but past experience showed that it is difficuilt too implement so I will most likely switch back to a more traditional lecturing mode.
There will be no midtems, no final exam, but instead homework problems (some challenging) and (Matlab) projects (about seven)

CANVAS Account

There is a CANVAS account for the course: BAN_CIS-5150-001 202430 (course number 1814314)
You should have access to it using your Pennkey.
This account contains the video recordings and reading material that
you should consult each week prior to class (a zoom link will be provided).
Look for Class Recordings and Files.
In addition to the recorded lessons in "Class Recordings" of Canvas, there are slides corresponding to these lessons in the "Files" Section of Canvas.
Make sure you look at these slides because some of them do not exist as recorded lessons.
Unless specified otherwise, a Module corresponds to two lectures (one week's worth).
In preparation for this week, please watch the videos in Class Recordings, Module 1,
and read the files in Files specified under Content (In Module 1).
Also read Pages 29-69 of the slides https://www.cis.upenn.edu/~cis5150/cis515-20-sl1-a.pdf.
Details and proofs are given on Page 29-61 of https://www.cis.upenn.edu/~cis5150/linalg-I-f.pdf
You should skim Section 2.3, but ignore the details.

Textbook: The official textbooks are Linear Algebra and Optimization with Applications to Machine Learning, Vol I and Vol II, by Gallier and Quaintance, World Scientific (2020). Relevant Chapters will be available as needed; see Slides and Notes

Latex Help:

html

[ Grade (Homeworks, Projects) | Additional Resources | Slides and Notes ]

A Word of Advice :

Expect to be held to high standards, and conversely! In addition to slides, I will post lecture notes. Please, read the course notes regularly, and start working early on the problems sets. They will be hard! Take pride in your work. Be clear, rigorous, neat, and concise. Preferably, use a good text processor, such as LATEX, to write up your solutions.

Due to the difficulty of the homework problems and in order to give you an opportunity to learn how to collaborate more effectively (I do not mean "copy"), I will allow you to work in small groups. A group consists of AT MOST THREE students.

You are allowed to collaborate with the same person(s) an unrestricted number of times.
Only one homework submission per group. All members of a group will get the SAME grade on a homework or a project (please, list all names in a group).

It is forbidden to use solutions of problems posted on the internet. If you use resources other than the textbook (or the recommended textbooks) or the class notes, you must cite these references.

Plagiarism Policy

I assume that you are all responsible adults.
Copying old solutions verbatim or blatantly isomorphic solutions are easily detectable.
DO NOT copy solutions from old solution sheets, from books, from solutions posted on the internet, or from friend!
Either credit will be split among the perpetrators, or worse!

Back to Gallier Homepage

published by:

CIS 5150, fall 2024 Fundamentals of Linear Algebra and Optimization Course Information August 28, 2024