What is the curse of dimensionality?

Highly recommend reading the lecture delivered by David Donoho titled “High Dimensional Data Analysis: on the Curses and Blessings of Dimensionality“. The “blessings” of data in high dimensions is that the information of interest effectively lies in a lower dimension which we can then leverage, typically, this involves sparse representations or low dimensional manifolds.

Here are some papers that greatly enhanced mathematical understanding of dimensionality reduction

Linear models:

Non-linear dimensionality reduction

Reference: Advanced Machine Learning course materials by Tony Jebara: http://www.cs.columbia.edu/~jeba…

Also see