Syllabus

Topics. This course will be a survey of the theory and practice of constructing locally p-dimensional parameterizations of subsets in R^d where p is much smaller than d. Topics will include linear regression, singular value decomposition and CUR factorization, principal component analysis in linear subspaces, smooth manifolds and the implicit function theorem, nonlinear manifold estimation and machine learning with multidimensional scaling and diffusion maps. There will be an emphasis on applications and algorithms.

Prerequisites. Prerequisites are linear algebra and basic Fourier analysis. It will be useful to have some knowledge of statistics and some experience in computer programming.

Time. Class will meet 2-4pm beginning Jan.18th, in Room A.201 at PMF. In-person attendance should be possible and is encouraged. Lectures will be livestreamed and I will request that they be videorecorded for later viewing.

Texts. The lectures will follow a series of book chapters and survey papers on this tentative list of topics:

• Manifolds -- Lecture Graphics 01
  • linear manifolds
  • charts, atlases, and parametrizations
  • implicit function theorem
  Details:
  • inverse function theorem
  • tangent vectors, spaces, bundles
  • partitions of unity
  
  References:
  • WOMPtalk-Manifolds.pdf (Jenny Wilson's talk on manifolds).
  • PartOfUnityLocFiniteRefinements.pdf (Thurston's notes on partitions of unity).
  • Brahim-Abdenbi-presentation-Whitney.pdf (Brahim Abdenbi paper on the Whitney Embedding Theorem).
  • WhitEmb-Lec09.pdf (Whitney Embedding Theorem proofs).
  • lecture10-WhitEmbed.pdf (More Whitney Embedding Theorem proofs).
• Regression -- Lecture Graphics 02
  
• Compression -- Lecture Graphics 03
  • hulls and tesselations
  • least squares
  • SVD
  • best orthogonal bases
  • CUR factorization
  Details:
  • Optimality of K-L
  References:
  • Convex Hulls, Voronoi Diagrams and Delaunay Triangulations (JDB-ens-lyon-I.pdf) Graphics from Jean-Daniel Boissonnat's Jan, 2010 talk at ENS-Lyon.
  • new281-282.pdf Excerpt (renumbered) from Jean Gallier's book Aspects of Convex Geometry Polyhedra, Linear Programming, Shellings, Voronoi Diagrams, Delaunay Triangulations (see p.332)
  • KQBrown.pdf (Kevin Q. Brown's paper on Delaunay tesselations via convex hulls.)
  • QuickHull.pdf (Barber, et al. paper on the convex hull algorithm.)
  • DirkGregoriusImplementingQuickHull.pdf (QuickHull implementation issues.)
  • Triangulationlecture09.pdf (Glenn Eguchi notes on triangulations.)
  • DelaunayStructuresSurfaces.pdf (Dyer, Zhang, and Moller, "A survey of Delaunay structures for surface representation.")
  • Hrvoje Lukatela's website (Paper from AUTO-CARTO 8, Baltimore, March 1987)
  • SVDNotes.pdf (Gilbert Strang's chapter on Singular Value Decomposition.)
  • hilbertmat.pdf (Pavel Stovicek's article on the Hilbert matrix.)
  • cur-talk.pdf (Mark Embree's presentation on the CUR factorization.)
  • CS6220-Lecture14-CUR.pdf (Anil Damle lecture notes on the CUR factorization.)
• Dimensionality reduction -- Lecture Graphics 04
  • principal components
  • linear discriminants
  • multidimensional scaling
  • isometric feature maps
  References:
  • lec8-MultiDimScaling.pdf (Sungkyu Jung. "Lecture 8: Multidimensional scaling...")
  • lec9mds.pdf (Guangliang Chen. "Lecture 9: Multidimensional Scaling (MDS)." )
  • chapter3.pdf ("Chapter 3: Multidimensional scaling.")
  • melchior_isomap_demo.pdf (Nik Melchior. "Manifold Learning: ISOMAP and LLE)
  • souvenir-ManifoldClusteringByMDS.pdf (Richard Souvenir and Robert Pless. "Manifold Clustering.")
  • https://octave.sourceforge.io/statistics/function/cmdscale.html (Octave function documentation for cmdscale(), classical multidimensional scaling.)
  • https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/isomap (R function documentation for isomap()multidimensional scaling.)
• Graph Laplacians
  • ch1-EigGraphLapl.pdf ("CHAPTER 1: Eigenvalues and the Laplacian of a graph.")
  • GraphLaplacians-hein07a.pdf (Hein, et al. "Graph Laplacians and their Convergence on Random Neighborhood Graphs.")
  • GraphLaplacian-tutorial.pdf (Radu Horaud. "A Short Tutorial on Graph Laplacians, Laplacian Embedding, and Spectral Clustering.")
  • spielman+teng.pdf (Dan Spielman and Shang-Hua teng. "Spectral partitioning works: Planar graphs and finite element meshes.")
  • laplacian.pdf (J.S.Caughman and J.J.P.Veerman. "Kernels of Directed Graph Laplacians.")
  • ting-huang-jordan-icml10-GraphLapl.pdf (Daniel Ting, Ling Huang, and Michael I. Jordan. "An Analysis of the Convergence of Graph Laplacians.")
• Perron-Frobenius theorem -- Lecture Graphics 05
  • 12-jcf.pdf Stephen Boyd's notes on Jordan canonical form and the Cayley-Hamilton theorem.
  • math108b_w2014_lecture8.pdf Padraic Bartlett's notes on Jordan canonical form.
  • mfmm30-32.pdf my proof that all norms on finite-dimensional vector spaces are equivalent.
• Diffusion maps -- Lecture Graphics 06
  • Coifman and Lafon (Lafon06.pdf) original paper on diffusion maps.
  • An Introduction to Diffusion Maps (delaPorte-Herbst-Hereman-vanderWalt-PRASA-2008.pdf) survey paper.
  • Python implementation of diffusion maps, by Rahul Raj
  • 06difmap.txt Octave commands to build diffusion maps, with an example.
  • Diffusion maps software web page by Wes Barnett.
  • 06swiss.txt Octave commands for Swiss Roll diffusion map.
  • sr2000x3.dat Plain text file of Swiss Roll points in 3D.
  • dmgk.m Octave function to return first 6 diffusion map coordinates. (Save this file in your Octave home folder.)
• Markov chains Lecture Graphics 07
  • MetropHastingsEtc.pdf, Stanley Sawyer's notes on MCMC.
  • Yunis.pdf, David Yunis' notes on ergodic theorems.
  • 07multi.txt, R commands to plot multinomial probabilities on 3 parameters.
  • 07diric.txt, R commands to plot Dirichlet densities on 3 parameters.
  • 07metro.txt, R commands to estimate Beta posteriors with MCMC.

Homework. Homework will be assigned from time to time. Collaboration on homework is permitted and encouraged, but each student must submit individually written solutions. [List of homework problems (hw.pdf), extracted from the lectures.]

Tests. There will be no examinations. Near the end of the course, each student will be required to present a lecture on one of the topics covered during the course. [List of suggested projects (projects.txt), for the end-of-course presentation.]

Grading. One grade will be assigned for all homework and one for the class presentation. These grades will contribute equally to the course grade.

Office Hours. See the instructor after class or by appointment.