Summary: We present a penalized matrix decomposition (PMD), a new framework for computing a rank-\(K\) approximation for a matrix. We approximate the matrix \(\mathbf{X}\) as \(\widehat{\mathbf{X}}=\sum^K_{k=1}d_k\mathbf{u}_k\mathbf{v}^T_k\), where \(d_{k}\), \(\mathbf{u}_{k}\), and \(\mathbf{v}_{k}\) minimize the squared Frobenius norm of \(\mathbf{X}-\widehat{\mathbf{X}}\), subject to penalties on \(\mathbf{u}_{k}\) and \(\mathbf{v}_{k}\). This results in a regularized version of the singular value decomposition. Of particular interest is the use of \(L_1\)-penalties on \(\mathbf{u}_{k}\) and \(\mathbf{v}_{k}\), which yields a decomposition of \(\mathbf{X}\) using sparse vectors. We show that when the PMD is applied using an \(L_1\)-penalty on \(\mathbf{v}_{k}\) but not on \(\mathbf{u}_{k}\), a method for sparse principal components results. In fact, this yields an efficient algorithm for the “SCoTLASS” proposal [I. Jolliffe et al., “A modified principal component technique based on the Lasso”, J. Comput. Graph. Stat. 12, No. 3, 531–547 (2003; doi:10.1198/1061860032148)] for obtaining sparse principal components. This method is demonstrated on a publicly available gene expression data set. We also establish connections between the SCoTLASS method for sparse principal component analysis and the method of H. Zou et al. [“Sparse principal component analysis”, J. Comput. Graph. Stat. 15, No. 2, 265–286 (2006; doi:10.1198/106186006X113430)]. In addition, we show that when the PMD is applied to a cross-products matrix, it results in a method for penalized canonical correlation analysis (CCA). We apply this penalized CCA method to simulated data and to a genomic data set consisting of gene expression and DNA copy number measurements on the same set of samples.