Adaptive estimation with partially overlapping models

Sunyoung Shin, Jason Fine, Yufeng Liu

Research output: Research - peer-reviewArticle

  • 1 Citations

Abstract

In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.

LanguageEnglish (US)
Pages235-253
Number of pages19
JournalStatistica Sinica
Volume26
Issue number1
DOIs
StatePublished - Jan 1 2016

Fingerprint

Adaptive Estimation
Overlapping
Model
Adaptive estimation
Overlap
Loss function
Individual model
M-estimation
Efficient Estimation
Penalization
Loss Function
Composite
Simulation study
Parameter estimation
Efficient estimation
Covariates
Composite function
Data-driven
Parameter Estimation
Linear Combination

Keywords

  • Composite loss function
  • Lasso
  • Model selection
  • Oracle property
  • Overlapping
  • Penalization
  • Sparsity

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Adaptive estimation with partially overlapping models. / Shin, Sunyoung; Fine, Jason; Liu, Yufeng.

In: Statistica Sinica, Vol. 26, No. 1, 01.01.2016, p. 235-253.

Research output: Research - peer-reviewArticle

Shin S, Fine J, Liu Y. Adaptive estimation with partially overlapping models. Statistica Sinica. 2016 Jan 1;26(1):235-253. Available from, DOI: 10.5705/ss.2014.233
Shin, Sunyoung ; Fine, Jason ; Liu, Yufeng. / Adaptive estimation with partially overlapping models. In: Statistica Sinica. 2016 ; Vol. 26, No. 1. pp. 235-253
@article{c47444be8635482bafe1e036057432bd,
title = "Adaptive estimation with partially overlapping models",
abstract = "In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.",
keywords = "Composite loss function, Lasso, Model selection, Oracle property, Overlapping, Penalization, Sparsity",
author = "Sunyoung Shin and Jason Fine and Yufeng Liu",
year = "2016",
month = "1",
doi = "10.5705/ss.2014.233",
volume = "26",
pages = "235--253",
journal = "Statistica Sinica",
issn = "1017-0405",
publisher = "Institute of Statistical Science",
number = "1",

}

TY - JOUR

T1 - Adaptive estimation with partially overlapping models

AU - Shin,Sunyoung

AU - Fine,Jason

AU - Liu,Yufeng

PY - 2016/1/1

Y1 - 2016/1/1

N2 - In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.

AB - In many problems, one has several models of interest that capture key parameters describing the distribution of the data. Partially overlapping models are taken as models in which at least one covariate effect is common to the models. A priori knowledge of such structure enables efficient estimation of all model parameters. However, in practice, this structure may be unknown. We propose adaptive composite M-estimation (ACME) for partially overlapping models using a composite loss function, which is a linear combination of loss functions defining the individual models. Penalization is applied to pairwise differences of parameters across models, resulting in data driven identification of the overlap structure. Further penalization is imposed on the individual parameters, enabling sparse estimation in the regression setting. The recovery of the overlap structure enables more efficient parameter estimation. An oracle result is established. Simulation studies illustrate the advantages of ACME over existing methods that fit individual models separately or make strong a priori assumption about the overlap structure.

KW - Composite loss function

KW - Lasso

KW - Model selection

KW - Oracle property

KW - Overlapping

KW - Penalization

KW - Sparsity

UR - http://www.scopus.com/inward/record.url?scp=85011347231&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85011347231&partnerID=8YFLogxK

U2 - 10.5705/ss.2014.233

DO - 10.5705/ss.2014.233

M3 - Article

VL - 26

SP - 235

EP - 253

JO - Statistica Sinica

T2 - Statistica Sinica

JF - Statistica Sinica

SN - 1017-0405

IS - 1

ER -