C Proofs of Section 4
C.2 Proof of Theorem 4.2
We now verify Assumption 4.2 for the Adaptive Lasso.
Lemma C.7. For adaptive lasso,
(i) mini6=j,(i,j)∈SU|Σu0,ij|γmaxi6=j,(i,j)∈SUwij =Op(1).
(ii) δTγmax(i,j)∈SLwij =Op(1),
(iii) ω−Tγ(min(i,j)∈SLwij)−1 =Op(1) (recall that ωT =N−1/2+T−1/2(logN)).
Proof. By Lemma B.5 maxi≤N,j≤N|Σb∗u,ij −Σu0,ij| = Op(τ). Given this result and the as-sumption that min(i,j)∈SU|Σu0,ij| ≫ωT, we have result (i). For any (i, j)∈SL, the following inequality holds: δT−γ ≤ w−ij1 ≤ (|Σu0,ij|+|Σu0,ij −Σbu,ij|+δT)γ, which then implies results (ii) and (iii), due to the assumptions that δT =o(ωT), and Σu0,ij =O(ωT).
Proof of Assumption 4.2 for Adaptive Lasso
It follows from the previous lemma that αT =Op(ωγT(mini6=j,Σu0,ij∈SU|Σu0,ij|)−γ) =op(1), and βT =Op((ωT/δT)γ). By the assumption that D=O(N),
ζ = min
(s T logN
N D,
T logN
1/4r N
D, N
√DlogN )
≫min
( T logN
1/4
, s N
logN )
. HenceαT =Op(ζ). This together with the lower bound assumption onδT yields Assumption 4.2 (i).
For part (ii), note that αT =op(1) implies that with probability approaching one, min{N,N2
D ,N2
D α−T2}=N, min{N D,
rN D,N
Dα−T1}= rN
D. By Lemma C.7(ii), (recall that KT = P
(i,j)∈SL|Σu0,ij|) and the lower bound δT ≫ ωT(KT/N)1/γ, µT max(i,j)∈SLwijKT =Op(µTδT−γKT) = op(N).
By Lemma C.7(i) and the assumptions that D=O(N) and mini6=j,(i,j)∈SU|Σu0,ij| ≫ωT, we have µT maxi6=j,(i,j)∈SUwij = OpµT(mini6=j,(i,j)∈SU|Σu0,ij|γ)−1 = op(p
N/D), due to the upper bound onµT =o(ωTγ).Finally, by Lemma C.7(iii) and the assumption thatµT ≫ωT1+γ, we have µT min(i,j)∈SLwij ≫ωT.
Proof of Assumption 4.2 for SCAD
Since µT/mini6=j,(i,j)∈SU|Rij| = op(1) and max(i,j)∈SL|Rij| = op(µT), it is easy to verify that with probability approaching one, maxi6=j,(i,j)∈SUwij = 0, min(i,j)∈SLwij = max(i,j)∈SLwij = µT. Hence αT = 0 and βT = 1. This immediately implies the desired result.
References
Alessi, L., Barigozzi, M. and Capassoc, M. (2010). Improved penalization for deter-mining the number of factors in approximate factor models.Statistics and Probability Letters, 80, 1806-1813.
Bai, J. (2003). Inferential theory for factor models of large dimensions. Econometrica. 71 135-171.
Bai, J. and Li, K. (2012). Statistical analysis of factor models of high dimension. Ann.
Statist. 40, 436-465.
Bai, J.andNg, S.(2002). Determining the number of factors in approximate factor models.
Econometrica. 70 191-221.
Bickel, P. and Levina, E. (2008a). Covariance regularization by thresholding. Ann.
Statist. 36 2577-2604.
Bickel, P. and Levina, E. (2008b). Regularized estimation of large covariance matrices.
Ann. Statist. 36 199-227.
Bien, J.andTibshirani, R.(2011). Sparse estimation of a covariance matrix.Biometrika.
98, 807-820.
Breitung, J. and Tenhofen, J. (2011). GLS estimation of dynamic factor models. J.
Amer. Statist. Assoc. 106, 11501166.
Cai, T.andLiu, W.(2011). Adaptive thresholding for sparse covariance matrix estimation.
J. Amer. Statist. Assoc. 106, 672-684.
Cai, T.andYuan, M.(2012). Adaptive covariance matrix estimation through block thresh-olding. Forthcoming in Ann. Statist.
Cai, T. and Zhou, H. (2012). Optimal rates of convergence for sparse covariance matrix estimation. Forthcoming in Ann. Statist.
Caner, M. and Fan, M. (2011). A near minimax risk bound: adaptive lasso with het-eroskedastic data in instrumental variable selection. Manuscript. North Carolina State Uni-versity.
Chamberlain, G. and Rothschild, M. (1983). Arbitrage, factor structure and mean-variance analysis in large asset markets. Econometrica.51 1305-1324.
Choi, I. (2012). Efficient estimation of factor models. Econometric Theory.28 274-308.
Deng, X. and Tsui, K. (2010). Penalized covariance matrix estimation using a matrix-logarithm transformation.Manuscript, University of Wisconsin-Madison
Dias, F., Pinherio, M.andRua, A.(2008). Determining the number of factors in approx-imate factor models with global and group-specific factors.Manuscript. Technical University of Lisbon.
Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model. J. Econometrics. 147, 186-197.
Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360
Fan, J., Liao, Y.andMincheva, M.(2012). Large covariance estimation by thresholding principal orthogonal complements. Manuscript. Princeton University.
Forni, M., Hallin, M., Lippi, M. and Reichlin, L. (2000). The generalized dynamic factor model: identification and estimation.Review of Economics and Statistics.82540-554.
Forni, M. and Lippi, M. (2001). The generalized dynamic factor model: representation theory. Econometric Theory. 17 1113-1141.
van de Geer, S., B¨uhlmann, P.andZhou, S.(2011). The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso). Electronic Journal of Statistics.5, 688-749.
Hallin, M. and Liˇska, R. (2007). Determining the number of factors in the general dy-namic factor model. J. Amer. Statist. Assoc. 102, 603-617.
Han, X. (2012). Determining the number of factors with potentially strong cross-sectional correlation in idiosyncratic shocks.Manuscript. North Carolina State University
Huang, J., Ma, S. and Zhang, C. (2006). Adaptive lasso for sparse high-dimensional regression models. Manuscript. University of Iowa.
Johnstone, I.M.and Lu, A.Y. (2009). On consistency and sparsity for principal compo-nents analysis in high dimensions. Jour. Ameri. Statist. Assoc., 104, 682-693.
Jung, S. and Marron, J.S. (2009). PCA consistency in high dimension, low sample size context. Ann. Statist.,37, 4104-4130.
Kapetanios, G. (2010). A testing procedure for determining the number of factors in approximate factor models with large datasets.Journal of Business and Economic Statistics.
28, 397-409.
El Karoui, N.(2008). Spectrum estimation for large dimensional covariance matrices using random matrix theory. Ann. Statist. 36 2757-2790.
Lam, C. and Fan, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Statist. 37 4254-4278.
Lam, C. and Yao, Q. (2012). Factor modelling for high-dimensional time series: inference for the number of factors. Forthcoming in Ann. Statist.
Lawley, D.and Maxwell, A.(1971).Factor analysis as a statistical method.Second ed.
London, Butterworths.
Ledoit, O. and Wolf, M. (2012). Nonlinear shrinkage estimation of large-dimensional covariance matrices. Ann. Statist. 40, 1024-1060.
Onatski, A. (2010). Determining the number of factors from empirical distribution of eigenvalues. The Review of Economics and Statistics. 92, 1004-1016.
Pati, D., Bhattacharya, A., Pillai, N. and Dunson, D. (2012) Posterior contrac-tion in sparse Bayesian factor models for massive covariance matrices. Manuscript, Duke University
Ravikumar, P., Wainwright, M., Raskutti, G.andYu, B.(2011), High-dimensional covariance estimation by minimizing l1-penalized log-determinant divergence, Electronic Journal of Statistics. 5 935-980.
Rohde, A. and Tsybakov, A. (2011), Estimation of high-dimensional low-rank matrices.
Ann. Statist. 39 887-930.
Rothman, A., Levina, E. and Zhu, J. (2009). Generalized thresholding of large covari-ance matrices. J. Amer. Statist. Assoc. 104 177-186.
Stock, J. and Watson, M. (1998). Diffusion Indexes, NBER Working Paper 6702.
Stock, J. and Watson, M. (2002). Forecasting using principal components from a large number of predictors. J. Amer. Statist. Assoc. 97, 1167-1179.
Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Ser. B, 58 267-288
Wang, P.(2009). Large dimensional factor models with a multi-level factor structure: iden-tification, estimation and inference.Manuscript. Hong Kong University of Science and Tech-nology.
Witten, D.M., Tibshirani, R. and Hastie, T. (2009). A penalized matrix decompo-sition, with applications to sparse principal components and canonical correlation analysis.
Biostatistics, 10, 515-534.
Xue, L. and Zou, H. (2012). Regularized rank-based estimation of high-dimensional non-paranormal graphical models. Forthcoming in Ann. Statist.
Yuan, M. (2010). High dimensional inverse covariance matrix estimation via linear pro-gramming. J. Machine Learning Research. 2010, 2261-2286
Zhang, C.(2010). Nearly unbiased variable selection under minimax concave penalty,Ann.
Statist., 38 894-942
Zhou, S., R¨utimann, P., Xu, M. and B¨uhlmann, P.(2011), High-dimensional covari-ance estimation based on Gaussian graphical models.Journal of Machine Learning Research.
12, 2975-3026.
Zou, H.(2006). The adaptive Lasso and its oracle properties. J. Amer. Statist. Assoc.101 1418-1429