Abstract
In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.
| Language | English (US) |
|---|---|
| Pages | 235-253 |
| Number of pages | 19 |
| Journal | Statistica Sinica |
| Volume | 26 |
| Issue number | 1 |
| DOIs | |
| State | Published - Jan 1 2016 |
Fingerprint
Keywords
- Composite loss function
- Lasso
- Model selection
- Oracle property
- Overlapping
- Penalization
- Sparsity
ASJC Scopus subject areas
- Statistics and Probability
- Statistics, Probability and Uncertainty
Cite this
Adaptive estimation with partially overlapping models. / Shin, Sunyoung; Fine, Jason; Liu, Yufeng.
In: Statistica Sinica, Vol. 26, No. 1, 01.01.2016, p. 235-253.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Adaptive estimation with partially overlapping models
AU - Shin,Sunyoung
AU - Fine,Jason
AU - Liu,Yufeng
PY - 2016/1/1
Y1 - 2016/1/1
N2 - In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.
AB - In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.
KW - Composite loss function
KW - Lasso
KW - Model selection
KW - Oracle property
KW - Overlapping
KW - Penalization
KW - Sparsity
UR - http://www.scopus.com/inward/record.url?scp=85011347231&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85011347231&partnerID=8YFLogxK
U2 - 10.5705/ss.2014.233
DO - 10.5705/ss.2014.233
M3 - Article
VL - 26
SP - 235
EP - 253
JO - Statistica Sinica
T2 - Statistica Sinica
JF - Statistica Sinica
SN - 1017-0405
IS - 1
ER -