DIMENSIONALITY REDUCTION AND MANIFOLD ESTIMATION
Mlden Victor Wickerhauser
PMF/Matematika -- University of Zagreb
Winter, 2022
Example Computer Programs (for Octave/Matlab)


3. Compression:


%% Screeplots
N=100; d=10;
xn=randn(d,N); % N samples of d-variate independent std. normals
x=hilb(d)*xn; % introduce correlation with the Hilbert matrix
xbar=sum(x,2)/N; % mean of the d-variate x's, should be close to 0
x0=x-xbar; % xbar is subtracted from each column of x
sum(x0,2)  % check: this should be all 0s
covx= x0*x0'/(N-1); % dxd covariance matrix
plot(eig(covx));  % screeplot of eigenvalues=singular values
% Scatter plots of various pairs of coordinates
figure; plot(x0(1,:),x0(2,:),"*"); title("X0: 1 versus 2");
figure; plot(x0(1,:),x0(5,:),"*"); title("X0: 1 versus 5");
figure; plot(x0(1,:),x0(10,:),"*"); title("X0: 1 versus 10");
figure; plot(x0(3,:),x0(4,:),"*"); title("X0: 3 versus 4");
figure; plot(x0(6,:),x0(8,:),"*"); title("X0: 6 versus 8");
figure; plot3(x0(1,:),x0(2,:),x0(3,:),"*"); title("X0: 1 vs 2 cs 3"); axis("equal");

%% Karhunen-Loeve coordinates via SVD
[U,S,V]=svd(covx); % decreasing singular values on diag(S)
norm(U*S*V'-covx)  % check that covx=U*S*V' to high precision
plot(diag(S)); % screeplot of the singular values
ed=sum(diag(S)/S(1,1)>0.001)  % effective dimension: #{sing.vals. > 0.1%}
xKL=V'*x0;  % samples x0 transformed into KL coordinates
figure; plot(xKL(1,:),xKL(2,:),"*"); title("KL: 1 versus 2"); % most variance is in 1,2
figure; plot3(xKL(1,:),xKL(2,:),xKL(3,:),"*"); title("KL: 1 vs 2 cs 3"); axis("equal");