UFLDL练习(PCA and Whitening && Softmax Regression)

文章由Byrx.net分享于2019-03-22 11:03:03评论（230）

UFLDL练习(PCA and Whitening && Softmax Regression)

softmax纠结了两天，原因是自己不小心改了主程序

还是照例只是贴贴代码，如果你有需要的话可以去看UFLDL的教程

至于效果和UFLDL都是一样的，我就不重复贴图了啊，ps：代码是matlab的，不是python的

PCA and Whitening：

pca_gen.m

[python]

%%================================================================

x = sampleIMAGESRAW();

figure('name','Raw images');

randsel = randi(size(x,2),200,1); % A random selection of samples for visualization

display_network(x(:,randsel));

%%================================================================

avg = mean(x,2);

x = x - repmat(avg,1,size(x,2));

%%================================================================

xRot = zeros(size(x)); % You need to compute this

[u,s,v]=svd(x);

xRot = u'*x;

%%================================================================

covar = zeros(size(x, 1)); % You need to compute this

covar = diag(diag(cov(x')));

%%================================================================

figure('name','Visualisation of covariance matrix');

imagesc(covar);

%%================================================================

k = 0; % Set k accordingly

egis=eig(covar)

egis=sort(egis,'descend')

for i=1:size(covar,1)

if (sum(egis(1:i))/sum(egis)>0.99)

k=i

break;

end

%%================================================================

xHat = zeros(size(x)); % You need to compute this

xHat = u*[xRot(1:k,:);zeros(size(xHat(k+1:end,:)))];

% Visualise the data, and compare it to the raw data

% You should observe that the raw and processed data are of comparable quality.

% For comparison, you may wish to generate a PCA reduced image which

% retains only 90% of the variance.

figure('name',['PCA processed images ',sprintf('(%d / %d dimensions)', k, size(x, 1)),'']);

display_network(xHat(:,randsel));

figure('name','Raw images');

display_network(x(:,randsel));

%%================================================================

%% Step 4a: Implement PCA with whitening and regularisation

% Implement PCA with whitening and regularisation to produce the matrix

% xPCAWhite.

epsilon = 0.1;

xPCAWhite = zeros(size(x));

avg = mean(x, 1); % Compute the mean pixel intensity value separately for each patch.

x = x - repmat(avg, size(x, 1), 1);

sigma = x * x' / size(x, 2);

[U,S,V] = svd(sigma);

xRot = U' * x; % rotated version of the data.

xTilde = U(:,1:k)' * x; % reduced dimension representation of the data,

% where k is the number of eigenvectors to keep

xPCAWhite = diag(1./sqrt(diag(S) + epsilon)) * U' * x;

%%================================================================

% Visualise the covariance matrix. You should see a red line across the

% diagonal against a blue background.

covar = diag(diag(cov(xPCAWhite')));

figure('name','Visualisation of covariance matrix');

imagesc(covar);

%%================================================================

xZCAWhite = zeros(size(x));

xZCAWhite = U * diag(1./sqrt(diag(S) + epsilon)) * U' * x;

%%================================================================

figure('name','ZCA whitened images');

display_network(xZCAWhite(:,randsel));

figure('name','Raw images');

display_network(x(:,randsel));

Softmax Regression

softmaxCost.m

[python]

function [cost, grad] = softmaxCost(theta, numClasses, inputSize, lambda, data, labels)

% numClasses - the number of classes

% inputSize - the size N of the input vector

% lambda - weight decay parameter

% data - the N x M input matrix, where each column data(:, i) corresponds to

% a single test set

% labels - an M x 1 matrix containing the labels corresponding for the input data

% Unroll the parameters from theta

theta = reshape(theta, numClasses, inputSize);

numCases = size(data, 2);

groundTruth = full(sparse(labels, 1:numCases, 1)); %numClasses*M

cost = 0;

thetagrad = zeros(numClasses, inputSize);

M = theta*data; % (numClasses,N)*(N,M)

M = bsxfun(@minus, M, max(M, [], 1));

h = exp(M);

h = bsxfun(@rdivide, h, sum(h));

cost = -1/numCases*sum(sum(groundTruth.*log(h)))+lambda/2*sum(sum(theta.^2));

thetagrad = -1/numCases*((groundTruth-h)*data')+lambda*theta;%log(h)

下面一段是关键部分没有Vectorization版本的代码

%for i=1:numCases

% s=groundTruth(:,i).*log(h(:,i));

% cost=cost+sum(s);

%end

%cost=cost*(-1)/numCases+lambda/2*sum(sum(theta.^2));

%for i=1:numClasses

% for j=1:numCases

% %groundTruth(:,j)

% %h(:,j)

% k=((groundTruth(:,j)-h(:,j))*data(:,j)');

% thetagrad(i,:)=thetagrad(i,:)+k(i,:);

% end

% thetagrad(i,:)=-thetagrad(i,:)/numCases+lambda*theta(i,:);

%end

grad = [thetagrad(:)];

end

热门文章：

用Python核心编程破解游戏练习题中的难题

UFLDL练习(PCA and Whitening && Softmax Regression)