Collaborative Research: Generalized Fiducial Inference for Massive Data and High Dimensional Problems

Project: Research project


R. A. Fisher, the father of modern statistics, proposed the idea of “Fiducial Inference” in the 1930’s. While his ideas led to some interesting methods for quantifying uncertainty, other prominent statisticians of the time did not accept Fisher’s approach because it went against the ideas of statistical inference of the time. Beginning around the year 2000, the PIs and collaborators started to re-investigate the ideas of fiducial inference and discovered that Fisher’s approach, when properly generalized, would open the doors to solve many important and difficult problems of uncertainty quantification. After many years of preliminary investigations, the team was able to put together a coherent, well thought out plan for a systematic research program in this area. This plan led to the PIs’ initial NSF award in 2007 and a continuation award in 2010. The PIs termed their generalization of Fisher’s ideas as generalized fiducial inference (GFI).
With the help of NSF funding, the PIs have been able to develop an in-depth understanding of GFI, provide solutions to many difficult practical problems, propose many useful applications in different fields of science and industry, and have trained many students in these techniques. Overall, the PIs have demon- strated that GFI is a valid, useful, and promising approach for tackling the problem of uncertainty quan- tification in complex problems. Their work is being used in the framework of FDA bioequivalence trials, metrology at NIST, and many other regulatory situations.
This proposal is a natural and important continuation of the above successful activities. The PIs are now working towards scaling up their GFI methodology to handle big data problems and other difficult problems that have emerged due to our ability to collect massive amounts of data rapidly. In particular the PIs propose to conduct research into the following topics:
1. a thorough theoretical study of some fundamental issues of GFI using higher order asymptotics;
2. an investigation of fiducial methods for the analysis of massive data sets;
3. sparse covariance estimation using GFI in the “large p small n” context;
4. development of the idea of Fiducial Selector so that a sparsity of the fiducial distribution is induced as a natural outcome of a minimization problem;
5. uncertainty quantification for the matrix completion problem using GFI, and
6. applications of GFI to a wide variety of practical problems, such as volatility estimation in finance and international key comparison experiments in measurement science.
Effective start/end date9/1/158/31/18


  • National Science Foundation (NSF)


Uncertainty Quantification
Matrix Completion Problem
Covariance Estimation
Higher-order Asymptotics
Statistical Inference
Minimization Problem