Advanced Linear Algebra

Math 173, Fall 2018
Harvey Mudd College
Professor: Weiqing Gu
Teaching Assistant: Conner DiPaolo

Meeting Time

M 07:00-09:45PM. SHAN 3485

Course Description

This course is designed to fill the linear algebra background of students to a graduate level. To complement this theory, emphasis on applications in various areas is intended to allow students to bring this knowledge to their own application area. Application areas include: machine learning, convex optimization, compressed sensing, and randomized data compression, among others.

Structure

We will loosely follow our textbook at a rate of around one chapter a week to ensure continuity and make sure every student can fill their mathematical background and speak the language of linear algebra and matrix analysis. In order to achieve fluency using this language in practice, a significant portion of our course will be devoted to applications of the theory of linear algebra and matrix analysis, sometimes from the text, sometimes not. Contrary to the nomenclature, some of these applications will be quite rich mathematically. Homework will follow this split model as well, with roughly half of the problems covering theory and half covering applications.

Textbook

Lax, Peter D. Linear Algebra and Its Applications, 2nd Edition. Wiley Press. 2007

Supplementary Materials

(Background) Axler, Sheldon. Linear Algebra Done Right, 3rd Edition. Springer. 2015
(Infinite Dimensions) Roman, Steven. Advanced Linear Algebra, Third Edition. Springer. 2008
(Some Special Topics) Woodruff, David. Sketching as a Tool For Numerical Linear Algebra. Now Publishers. 2014

Grading

• 40% Homework
• 30% Midterm Project
• 30% Final Project
• [Up to 5% Extra Credit]

Homework

Problem sets will be due (virtually) every week, on Monday outside Professor Gu's office. Usually there will be 4 problems per week. At least half of the problems from the course will come from the text. Occasionally problems will require coding. For this we recommend either Python (using numpy and scipy) or Matlab, but feel free to use whatever works for you.

Midterm Project

Details given in class, but this will reflect the final project in nature. If the final project is to be a continuation of the midterm project (which is expected), significant additional progress must be made.

Final Project

The final project is intended to give students, in groups of 2-3, the opportunity to deep-dive into a specific area of interest in linear algebra or matrix analysis. This could be theoretical or applied, but in both cases should be originated from a single question. For example, such questions might be:

• How close should random matrices be to their mean?
• How can we estimate the spectral norm of a matrix using limited space?
• Which properties of matrices can we approximate only by testing a few inputs?
• Which/can Banach spaces look like Hilbert Space?
• How can large scale linear system solvers from numerical linear algebra be used to speed up kernel methods?

This motivating question will be turned in as the single sentence alone, typed and stapled onto the back of their midterm project. Students are encouraged to discuss their ideas with teaching staff before committing to a question. Staff can also help students less fluent in the field to find topics that might intersect their interests. Questions should be quite focused.

About a month after proposing their initial question, students will submit a literature review of work that attempts to answer their question. This will be at most four pages in article class LaTeX, one inch margins, not including references. The review should include important definitions, discuss the body of work surrounding the question. At the top of the paper, as an abstract, the student should include a refined version of their motivating question.

By the end of the course, students are expected to continue investigating their question. In particular, students should be able to find a concrete open problem in the area or blind spot in the research body. (Hint: look at the end of recent papers). Open problems can be empirical (e.g. investigating the geometry of neural network loss surfaces through the spectral information in the Hessians), applied (e.g. finding efficient algorithms for estimating the L-p norm of an operator with limited space), or theoretical (e.g. characterizing the distortion of embedding L-p into L-2 for different p).

Before the end of the course, using this prior work on the project, the student will create a paper of at least 12 pages that details the background and progress of the research body on their open problem, promising directions, and demonstrations of results (computations or proofs). If the student is able to solve or even make concrete progress towards the open problem they will get an A on the final paper. Otherwise, experimental evidence towards their open problem is expected. The group will also give a presentation of at most 15 minutes detailing this adventure.

Deadlines

• (Oct 19) Motivating question. Hand in stapled onto back of midterm project.
• (Nov 19) Literature Review. Turn into Prof Gu's office before 6:00pm.
• (Dec 17) Presentation; Final Report due in class.

Disabilities

Students who need disability-related accommodations are encouraged to discuss this with the instructor as soon as possible.