Titles and Abstracts
Titles/abstracts for the Second Workshop on Higher-Order Asymptotics
and Post-Selection
Inference
(WHOA-PSI)^{2}. Click here
to go to the main conference page, where you can find more information.
The schedule will be emailed to registered participants. It will not be
available online until after the workshop has concluded. Contact: Todd Kuffner, email:
kuffner@wustl.edu
Talks --
Slides are available by following the links to the discussion board
threads.
Rina Foygel Barber,
University of Chicago
AllAnswered.com Discussion Thread: click
here
Title: Distribution-free
inference for estimation and classification
Abstract: I
shall Given a "black box" algorithm that estimates Y given a
high-dimensional feature vector X, is it possible to perform inference
on the quality of our estimates, without any distributional
assumptions? Specifically, the black box algorithm fits some regression
function mu(x) that attempts to estimate the mean of Y given X=x (or,
in a binary response setting, P(Y=1|X=x) ). We would like to then
construct a confidence interval for this estimate without any
assumptions on the distribution of the data. This talk will cover some
ongoing work on this open problem. I will discuss preliminary results
for data-driven inference on isotonic regression, where a black box
algorithm computing an estimated regression function mu(x) can be
recalibrated to provide a confidence interval for this estimate. Joint
work with Fan Yang.
Yoav Benjamini, Tel Aviv University
AllAnswered.com Discussion Thread: click
here
Title: Challenges in Large
Selective Inference Problems
Abstract: I
shall classify current approaches to multiple inferences according to
goals, and discuss the basic approaches being used. I shall then
highlight a few challenges that await our attention: some are simple
inequalities, others arise in particular applications.
Yuval Benjamini, Hebrew University of Jerusalem
AllAnswered.com Discussion Thread: click
here
Title: Measuring bumps:
Selection-corrected inference for detected regions in high-throughput
data
Abstract: High-throughput
technologies allow scientist to simultaneously measure signal at
numerous locations simultaneously. Such data allows scientists to
search for spatially-coherent regions which are correlated with a
covariate of interest. A popular approach is to (a) estimate a
statistic at each point, (b) threshold this map and (c) merge
neighboring points passing the threshold into regions. However,
treating these regions as if they were known a-priori leads to biases
in the estimation, especially considering the large spatial process
from which they were selected. Currently, methods for removing these
biases accommodate only significance testing, and require strong
assumptions or very strict procedures.
In this talk I will present a conditional-inference
based approach that also allows estimating and forming confidence
intervals for regional parameters. The proposed method is based on
sampling from a conditional distribution, and therefore can accommodate
the non-stationary covariance in each region. I will discuss common use
cases such as evaluating differentially methylated regions (mDNA)
between tissue types, and well as cortical (fMRI) region detection.
This is based on work with Jonathan Taylor, Rafael Irizarry and Amit
Meir.
Andreas Buja, University of Pennsylvania
AllAnswered.com Discussion Thread: click
here
Title: Higher-Order von
Mises Expansions, Bagging, and Assumption-Lean Inference
Abstract: The
starting point of this work is the fact that models obtained from model
selection procedures cannot be assumed to be correct. They therefore
require assumption-lean inference and, in particular, assumption-lean
standard errors. These can be obtained in two ways: (1)
asymptotic plug-in, resulting in sandwich standard errors, and (2)
bootstrap and, more specifically in regression, ``pairs" or ``x-y"
bootstrap (as opposed to the residual bootstrap). Two questions arise:
Is there a connection between the two types of standard errors?
Is one superior to the other? We give some partial answers by showing
that (a) asymptotic plug-in standard errors are a limiting case of
bootstrap standard errors when the resample size varies, (b) bootstrap
standard errors can be thought of as resulting from a bagging
operation, (c) bagging results in statistical functionals that have
finite von Mises expansions, and finally (d) bootstrap standard errors
are ``smoother" than sandwich standard errors in the sense of von Mises
expansions. We conclude that bootstrap standard errors have some
advantages over sandwich standard errors. However, we do not yet fully
understand the trade-offs in choosing specific bootstrap resample
sizes. Joint with: W. Stuetzle, L. Brown, R. Berk, L. Zhao, E. George,
A. Kuchibhotla, K. Zhang.
Florentina Bunea, Cornell University
AllAnswered.com Discussion Thread: click
here
Title: Overlapping
clustering with LOVE
Abstract: The
area of overlapping variable clustering, with statistical guarantees,
is largely unexplored. We propose a novel Latent model-based
OVErlapping clustering method (LOVE) to recover overlapping sub-groups
of a p dimensional vector X from a sample of size n on X, with p
allowed to be larger than n. In our model-based formulation, a cluster
is given by variables associated with the same latent factor. Clusters
are anchored by a few components of X that are respectively associated
with only one latent factor, while the majority of the X-components may
have multi-factor association. We prove that, under minimal conditions,
these clusters are identifiable, up to label switching. LOVE estimates
first the set of variables that anchor each cluster, and also estimates
the number of clusters. In a second step, clusters are populated with
variables with multiple associations. Under minimal signal strength
conditions, LOVE recovers the population level overlapping clusters
consistently. The practical relevance of LOVE is illustrated through
the analysis of a RNA-seq data set, devoted to determining the
functional annotation of genes with unknown function.
Gerda Claeskens, KU Leuven
AllAnswered.com Discussion Thread: click
here
Title: Asymptotic
post-selection inference after model selection by the Akaike
information criterion
Abstract: The
asymptotic distribution of parameter estimators after model selection
by the Akaike information criterion is studied. We exploit the
overselection property of this criterion in the construction of a
selection region, and obtain the asymptotic distribution of parameter
estimators and linear combinations thereof in the selected model. The
proposed method does not require the true model to be in the model set.
We investigate the method by simulations in linear and generalized
linear models. This is joint work with A. Charkhi (KU Leuven).
Noureddine El Karoui, UC Berkeley
AllAnswered.com Discussion Thread: click
here
Title: Can we trust the
bootstrap? (for moderately difficult statistical problems)
Abstract: The
bootstrap is an important and widely used tool for answering
inferential questions in Statistics. It is particularly helpful in many
analytically difficult situations.
I will discuss the performance of the bootstrap for simple inferential
problems in moderate and high-dimension. Those are typically
analytically still challenging and practically relevant, hence a good
testing ground for resampling methods.
For instance, one can ask whether the bootstrap
provides valid confidence intervals for individual parameters in linear
regression when the number of predictors is not infinitely small
compared to the sample size. Similar questions related to Principal
Component analysis are also natural from a practical standpoint.
We will see that the answer to these questions is
generally negative. Our assessment will be done through a mix of
numerical and theoretical investigations, and include a few other
resampling methods. Joint work with Elizabeth Purdom (UC Berkeley).
Jianqing Fan,
Princeton University
AllAnswered.com Discussion Thread: click
here
Title: Robust inference for
high-dimensional data with application to false discovery rate control
Abstract: Heavy
tailed distributions arise easily from high-dimensional data and they
are at odd with commonly used sub-Gaussian assumptions. This prompts us
to revisit the Huber estimator from a new perspective. The key
observation is that the robustification parameter should adapt to
sample size, dimension and moments for an optimal bias-robustness
tradeoff: a small robustification parameter increases the robustness of
estimation but introduces more biases. Our framework is able to handle
heavy-tailed data with bounded (1 + delta)-th moment for any
delta > 0. We establish a sharp phase transition for robust
estimation of regression parameters in both low and high dimensions:
when delta > 1, the estimator exhibits the optimal sub-Gaussian
deviation bound, while only a slower rate is available in the
regime 0 < delta < 1. The transition is smooth and optimal.
Moreover, a nonasymptotic Bahadur representation for finite-sample
inference is derived with finite variance, which further yields
two important normal approximation results, the Berry-Esseen inequality
and Cramer-type moderate deviation. As an important application,
we apply these robust normal approximation results to analyze a
dependence-adjusted multiple testing procedure. It is shown that this
robust, dependence-adjusted procedure asymptotically controls the
overall false discovery proportion (FDP) at the nominal level under
mild moment conditions. Thorough numerical results on both
simulated and real datasets are also provided to back up our theory.
Will Fithian,
UC Berkeley
AllAnswered.com Discussion Thread: click
here
Title: Adaptive Sequential
Model Selection
Abstract: Many
model selection algorithms produce a path of fits specifying a sequence
of increasingly complex models. Given such a sequence and the data used
to produce them, we consider the problem of choosing the least complex
model that is not falsified by the data. Extending the selected-model
tests of Fithian, Sun and Taylor (2014), we construct p-values for each
step in the path which account for the adaptive selection of the model
path using the data. In the case of linear regression, we propose two
specific tests, the max-t test for forward stepwise regression
(generalizing a proposal of Buja and Brown (2014)), and the next-entry
test for the lasso. These tests improve on the power of the
saturated-model test of Tibshirani et al. (2014), sometimes
dramatically. In addition, our framework extends beyond linear
regression to a much more general class of parametric and non-
parametric model selection problems. To select a model, we can feed our
single-step p-values as inputs into sequential stopping rules such as
those proposed by G’Sell et al. (2016) and Li and Barber (2016),
achieving control of the family- wise error rate or false discovery
rate (FDR) as desired. The FDR- controlling rules require the null
p-values to be independent of each other and of the non-null p-values,
a condition not satisfied by the saturated-model p-values of Tibshirani
et al. (2014). We derive intuitive and general sufficient conditions
for independence, and show that our proposed constructions yield
independent p-values.
Max G'Sell, Carnegie Mellon University
AllAnswered.com Discussion Thread: click
here
Title: Adaptive sequential
model selection for the graphical lasso
Abstract: The
graphical lasso is often used in practice to model the dependence
structure among variables. However, inferential questions about
the resulting solution have traditionally been difficult to
answer. We focus on sequential goodness-of-fit tests for the
graphcal lasso, using the sequential framework of Fithian et. al
(2015+). This serves as an illustration of this approach in a
reasonably complex, non-regression setting, and highlights some
interesting issues that arise in its application. Time
permitting, we will also introduce significance testing for particular
edges, and its interplay with the reliability of graphical lasso
estimates.
Dalia Ghanem,
UC Davis
AllAnswered.com Discussion Thread: click
here
Title: Model selection and
prediction after selection in linear fixed effects models with
high-frequency regressors
Abstract: We
propose a cross-validation model selection procedure for linear fixed
effects modeling of mixed-frequency time series data. Conditions
ensuring consistency of this procedure for model selection are
considered. As such modeling techniques are commonly used for
forecasting, e.g. in climate change impact studies, we construct valid
post-selection prediction intervals. Simulations illustrate the
improvements over existing methods, and the properties of our
post-selection prediction intervals. Joint work with Todd Kuffner.
Lucas Janson,
Harvard University
AllAnswered.com Discussion Thread: click
here
Title: Knockoffs as
Post-Selection Inference
Abstract: The
motivation behind post-selection inference is that data analysts often
look at their data before choosing inferential questions, and that
doing so invalidates inferential results when using procedures that
assume the inferential question is fixed ahead of time. A natural
(although highly non-trivial) solution that has gained quite a bit of
traction has been to develop corrected or altogether new procedures for
performing inference that adjust for whatever selection or data
snooping preceded that inference. Knockoffs, particularly the more
recent model-free knockoffs, was developed without post-selection
inference in mind, but can be interpreted and used to do just that for
the controlled variable selection problem, i.e., selecting a set of
important variables that controls the false discovery rate. The
knockoffs procedure augments the data with artificial null variables
*before* the data analyst views it, and does so in a way that
automatically and exactly protects against selection effects in the
final inference step. Because the main work of correcting for selection
occurs before the analyst sees the data, the knockoffs approach is
procedurally somewhat similar to the use of differential privacy in
adaptive data analysis, wherein noise is added to the data before the
analyst views it to choose inferential questions, although differential
privacy results usually bound how far from valid the inference is,
while with knockoffs the inference is exact. An advantage of the
knockoffs approach is that it places absolutely no restriction on how
the analyst views, models, or performs selection on the augmented data,
and yet this generality does not make its inference conservative at
all. In this talk, I will briefly review the knockoffs procedure and
then go into detail about how it can be used to immunize inference to
the effects of selection.
Arun Kumar Kuchibhotla,
University of Pennsylvania
AllAnswered.com Discussion Thread: click
here
Title: An approach to valid
post selection inference in assumption-lean linear regression analysis
Abstract: In
this talk, I will present three different but closely related
confidence regions that provide valid post-selection inference in an
assumption-lean linear regression analysis. Here the phrase
``assumption-lean linear regression" is used to mean that we do not
assume any of the classical linear regression assumptions on the data
generating process. We allow the observations to be structurally
dependent and non-identically distributed. In trend with the current
high dimensional literature we let the number of covariates that can be
used in model selection be almost exponential in the sample size. The
guarantee provided by these confidence regions is uniform in the sense
of Berk et al. (2013) in that they have valid coverage (asymptotically)
for any (random) model-selection procedure. Most importantly, all these
three confidence regions can be computed in polynomial-time. Some
important extensions and other methods will be discussed time
permitting. This is a joint work with Andreas Buja, Richard Berk,
Lawrence Brown, Ed George and Linda Zhao.
Stephen M.S. Lee,
University of Hong Kong
AllAnswered.com Discussion Thread: click
here
Title: A Mixed Residual
Bootstrap Procedure for Least-Squares Regression Post-Model Selection
Abstract: Common
practice in linear regression often assumes a model predetermined by a
data-driven procedure. Taking into consideration the extra uncertainty
introduced by data-driven model selection, recent studies on
distributions of post-model-selection least squares estimators have
uncovered considerable departures from the standard theory established
under the assumption of a fixed model. Consistent estimation of such
distributions poses a challenge with important implications for valid
regression inferences. We propose a mixed residual bootstrap procedure
for estimating the true distributions of post-model-selection least
squares estimators. The procedure integrates residual bootstrap
estimates constructed from candidate models adaptively into a mixture
distribution, which is shown to be consistent under the conditions that
the approximately correct candidate models are sufficiently separated
from the wrong ones and that the model selector is approximately
conservative or consistent. Empirical performance of our estimator is
illustrated via simulation, with models selected by AIC and the LASSO.
Hannes Leeb,
University of Vienna
AllAnswered.com Discussion Thread: click
here
Title: Prediction when
fitting simple models to high-dimensional data
Abstract: We
study linear subset regression in the context of a high-dimensional
linear model. Consider $y = \vartheta + \theta'z + \epsilon$ with
univariate response $y$ and a $d$-vector of random regressors $z$, and
a submodel where $y$ is regressed on a set of $p$ explanatory variables
that are given by $x = M'z$, for some $d\times p$ matrix $M$.
Here, `high-dimensional' means that the number $d$ of available
explanatory variables in the overall model is much larger than the
number $p$ of variables in the submodel. In this paper, we
present Pinsker-type results for prediction of $y$ given $x$. In
particular, we show that the mean squared prediction error of the best
linear predictor of $y$ given $x$ is close to the mean squared
prediction error of the corresponding Bayes predictor $E[y|x]$,
provided only that $p/\log d$ is small. We also show that the
mean squared prediction error of the (feasible) least-squares predictor
computed from $n$ independent observations of $(y,x)$ is close to that
of the Bayes predictor, provided only that both $p/\log d$ and $p/n$
are small. Our results hold uniformly in the regression
parameters and over large collections of distributions for the design
variables $z$.
Jing Lei,
Carnegie Mellon University
AllAnswered.com Discussion Thread: click
here
Title: Cross-validation
with Confidence
Abstract: Cross-validation
is one of the most popular model selection methods in statistics and
machine learning. Despite its wide applicability, traditional
cross-validation methods tend to overfit, unless the ratio between the
training and testing sample sizes is very small. We argue that such an
overfitting tendency of cross-validation is due to the ignorance of the
uncertainty in the testing sample. We develop a new, statistically
principled inference tool based on cross-validation, that takes into
account the uncertainty in the testing sample. Our method outputs
a small set of highly competitive candidate models that contains the
best one with probabilistic guarantees. In particular, our method leads
to consistent model selection in a classical linear regression setting,
for which existing methods require unconventional split ratios. We
demonstrate the performance of the proposed method in simulated and
real data examples.
Xihong Lin,
Harvard University
AllAnswered.com Discussion Thread: click
here
Title: Detection of Signal
Regions Using Scan Statistics With Applications in Whole Genome
Association Studies
Abstract: We
consider in this paper detection of signal regions associated with
disease outcomes in whole genome association studies. The
existing gene- or region-based methods test for the association of an
outcome and the genetic variants in a pre-specified region, e.g.,
a gene. In view of massive intergenic regions in whole genome
association studies, we propose a quadratic scan statistic based method
to detect the existence and the locations of signal regions by
scanning the genome continuously. The proposed method accounts for the
correlation (linkage disequilibrium) among genetic variants, and allows
for signal regions to have both causal and neutral variants, and
causal variants whose effects can be in different directions. We study
the asymptotic properties of the proposed scan statistics. We derived
an asymptotic threshold that controls for the family-wise error rate,
and show that under regularity conditions the proposed method
consistently selects the true signal regions. We performed simulation
studies to evaluate the finite sample performance of the proposed
method. Our simulation results showed that the proposed procedure
outperforms the existing methods, especially when signal regions have
causal variants whose effects are in different directions, or are
contaminated with neutral variants, or the variants in signal regions
are correlated. We applied the proposed method to analyze a lung cancer
genome-wide association study to identify the genetic regions that are
associated with lung cancer risk.
Han Liu,
Princeton University
AllAnswered.com Discussion Thread: click
here
Title: Combinatorial
inference
Abstract: TBA
Po-Ling Loh,
University of Wisconsin
AllAnswered.com Discussion Thread: click
here
Title: Robust estimation,
efficiency, and Lasso debiasing
Abstract: We
present results concerning high-dimensional robust estimation for
linear regression with non-Gaussian errors. We provide error bounds for
certain local/global optima of penalized M-estimators, valid even when
the loss function employed is nonconvex -- giving rise to more robust
estimation procedures. We also present a new approach for robust
location/scale estimation with rigorous theoretical guarantees. We
conclude by discussing high-dimensional variants of one-step estimation
procedures from classical robust statistics and connections to recent
work on confidence intervals based on Lasso debiasing.
Jelena Markovic,
Stanford University
AllAnswered.com Discussion Thread: click
here
Title: Formalizing data
science through selective inference: Selective inference after multiple
views / queries on the data
Abstract: We
present the recent developments in the conditional approach to
selective inference starting with Lee et al. (2016) through three
examples. In all the examples, we perform a model selection procedure
on our data and choose the coefficients to report inference for after
looking at the outcome of this procedure. In order to have valid
post-selection inference for the selected coefficients, we base our
inference using the conditional distribution of the data, where the
conditioning is on the observed outcome of the model selection
procedure.
In the first example, we use cross-validated LASSO
to select the model. Then we use the inferential tools applicable in a
even more general framework that require the constraints coming from a
model selection procedure to be written in terms of asymptotically
Gaussian random vectors. By adding a small randomization to
cross-validation error vector, we show cross-validation error vector
and the data vector corresponding to LASSO are jointly asymptotically
Gaussian.
As most data analysts will want to try various model
selection algorithms when choosing a model, we present a way to do
inference after multiple views / queries on the data as our second
example. Here, we do marginal screening first and then fit LASSO to the
resulting model consisting of the variables selected by marginal
screening and their interactions. A technique that enables us to
achieve the valid post-selection here relies on adding randomization to
each objective of the selection procedures.
Our third example shows the advantage in power of
data carving compared to data splitting. Data splitting uses a part of
the data for model selection and the leftover data for inference. Data
carving reuses the leftover information in the first part of the data
to construct valid p-values and confidence intervals for the selected
coefficients. As a further application, we also present an example of
doing valid inference after multiple splits of the data.
Ryan Martin,
NC State University
AllAnswered.com Discussion Thread: click
here
Title: On valid
post-selection prediction in regression
Abstract: TBA
Peter McCullagh,
University of Chicago
AllAnswered.com Discussion Thread: click
here
Title: Laplace
approximation of high-dimensional integrals
Abstract: TBA
Ian McKeague,
Columbia University
AllAnswered.com Discussion Thread: click
here
Title: Marginal screening
for high-dimensional predictors of survival outcomes
Abstract: A
new marginal screening test is developed for a right-censored
time-to-event outcome under a high-dimensional accelerated failure time
(AFT) model. Establishing a rigorous screening test in this setting is
challenging, not only because of the right censoring, but also due to
the post-selection inference in the sense that the implicit variable
selection step needs to be taken into account to avoid an inflated Type
I error. McKeague and Qian (2015) constructed an adaptive
resampling test to circumvent this problem under ordinary linear
regression. To accommodate right censoring, we introduce an approach
based on a maximally selected Koul-Susarla-Van Ryzin estimator from a
marginal AFT working model. A regularized bootstrap method is
developed to calibrate the test. The talk is based on joint work
with Tzu-Jung Huang and Min Qian.
Xiao-Li Meng,
Harvard University
AllAnswered.com Discussion Thread: click
here
Title: Struggling with
large p and small n? How about (potentially) infinite p and
(essentially) zero n?
Abstract: The
arrival of ``big data" has expanded the statistical asymptotic land
from fixing-p-growing-n to growing-p-&-n in a variety of
ways. But this expansion is still not rich enough to establish a
rigorous theoretical foundation for individualized prediction and
learning (e.g., for deciding personalized treatment) when the
individuality corresponds to (potentially) infinite many
attributes/variables, such as for human beings. Furthermore, for
such individuals, there will be no direct ``training sample" coming
from their genuine guinea pigs because it is impossible to match on
infinitely many attributes. Consequently, the traditional notions
of estimation consistency, unbiasedness, etc., all become unreachable
even in theory. It is possible, however, to set up an
approximation theory via a multi-resolution (MR) perspective, inspired
by wavelets (and other) literature, where we use the resolution level
to index the degree of approximation to the ultimate
individuality. The key strategy here is to seek an ``indirect
learning population (ILP)" by matching enough attributes to increase
the relevance of the results to the individuals and yet still
accumulate sufficient sample sizes for robust estimation.
Theoretically, MR relies on a standard ANOVA type of decomposition,
albeit with indefinitely many terms to permit the resolution level
approaching infinity. This talk reports recent progress on MR
asymptotic results, in terms of the ILP size, that can render
statistical insights, and on adaptive error estimations aiming for
practical implementation of optimal individualized learning from a
given ILP. (This is a joint work with Xinran Li.)
Yang Ning,
Cornell University
AllAnswered.com Discussion Thread: click
here
Title: A General Framework
for High-Dimensional Inference and Multiple Testing
Abstract: We
consider the problem of how to control the false scientific discovery
rate in high-dimensional models. Towards this goal, we focus on the
uncertainty assessment for low dimensional components in
high-dimensional models. Specifically, we propose a novel decorrelated
likelihood based framework to obtain valid p-values for generic
penalized M-estimators. Unlike most existing inferential methods which
are tailored for individual models, our method provides a general
framework for high-dimensional inference and is applicable to a wide
variety of applications, including generalized linear models, graphical
models, classifications and survival analysis. The proposed method
provides optimal tests and confidence intervals. The extensions to
general estimating equations are discussed. Finally, we show that the
p-values can be combined to control the false discovery rate in
multiple hypothesis testing.
Snigdha Panigrahi,
Stanford University
AllAnswered.com Discussion Thread: click
here
Title: A Cure-All Truncated
Approach to Provide Selection Adjusted Effect Size Estimates
Abstract: My
talk will describe new methods to provide adjusted estimates for
effects in GWAS studies post a genome-wide selection. The starting
point in such studies is an exhaustive collection of genetic variants;
the goal being to identify and quantify effect sizes of a very small
subset of variants that are believed to determine an outcome. In
achieving such a goal, the scientist per- forms a genome wide search of
the strongest possible associations and is subsequently, confronted in
defending the significance of her findings. Motivated to measure these
effect sizes as consistent point estimates and intervals with target
coverage, my methods are modeled along the truncated approach to
selective inference in Lee and Taylor (2014) and Fithian et al. (2014).
At the core of my methods is a convex approximation
to the typically intractable truncated likelihood as a function of the
parameters in the model. This allows frequentist inference free from
any MCMC samplers through a tractable and approximate pivotal quantity.
Appending the approximate truncated likelihood to a prior on the
parameters post selection allows Bayesian inference based on a
truncated posterior. Estimates based on the truncated posterior show
superior frequentist properties like Bayesian FCR and Bayes risk in
comparison to the unadjusted estimates. I will illustrate these
inferential gains for effect size estimates using a truncated model,
both for a frequentist and a Bayesian in the context of a GWAS
experiment. The talk will be based on a combination of Panigrahi et al.
(2016, arXiv:1605.08824); Tian et al. (2016, arXiv:1609.05609);
Panigrahi et al. (2017, arXiv:1703.06154); Panigrahi and Taylor (2017,
arXiv:1703.06176).
Annie Qu,
University of Illinois Urbana-Champaign
AllAnswered.com Discussion Thread: click
here
Title: Variable Selection
for Highly Correlated Predictors
Abstract: (Joint
work with Fei Xue) Penalty-based variable selection methods are
powerful in selecting relevant covariates and estimating coefficients
simultaneously. However, variable selection could fail to be consistent
when covariates are highly correlated. The partial correlation approach
has been adopted to solve the problem with correlated covariates.
Nevertheless, the restrictive range of partial correlation is not
effective for capturing signal strength for relevant covariates. In
this paper, we propose a new Semi-standard PArtial Covariance (SPAC)
which is able to reduce correlation effects from other predictors while
incorporating the magnitude of coefficients. The proposed SPAC variable
selection facilitates choosing covariates which have direct association
with the response variable, via utilizing dependency among covariates.
We show that the proposed method with the Lasso penalty (SPAC-Lasso)
enjoys strong sign consistency in both finite-dimensional and
high-dimensional settings under regularity conditions. Simulation
studies and the `HapMap' gene data application show that the proposed
method outperforms the traditional Lasso, adaptive Lasso, SCAD, and
Peter--Clark-simple (PC-simple) methods for highly correlated
predictors.
Aaditya Ramdas,
UC Berkeley
AllAnswered.com Discussion Thread: click
here
Title: Interactive multiple
testing with STAR: selectively traversed accumulation rules for
structured FDR control
Abstract: We
propose a general framework (STAR) for interactive multiple testing
with a human-in-the-loop that combines ideas from the literatures on
post-selection inference, ordered multiple testing and the knockoff
filter. In our setup, we have N independent p-values, each
corresponding to a different hypothesis. Initially, only ``masked''
p-values g(P) are revealed to the scientist, while, akin to
data-carving, the ``leftover'' information h(P) is used to estimate the
FDP of the masked set. At each step of the interaction, the scientist
may freely choose one or more p-values to ``unmask'', by utilizing
information contained in all p-values that were previously unmasked,
the remaining masked p-values, as well as any other side information in
the form of covariates, prior knowledge or intuition. When the chosen
p-value is unmasked and shown to the scientist, they may freely update
any model or prior being used, and revise the estimated FDP of the
remaining masked set. This process continues until the estimated FDP
falls below a pre-determined level, at which point all remaining masked
p-values are rejected, while all unmasked ones are accepted. We prove
that this interactive procedure guarantees FDR control no matter what
unmasking choices are made by the scientist. We also demonstrate how to
apply this procedure when we have only knockoff statistics instead of
p-values, with the absolute value and sign of the statistic
respectively playing the role of g(P) and h(P). We provide heuristics
to help the scientist with the unmasking process for a variety of
structured applications, and show that STAR performs excellently in a
wide range of problems such as convex region detection, bump-hunting,
and FDR control on trees and DAGs. (joint work with Lihua Lei and Will
Fithian, soon to be posted on Arxiv)
Richard Samworth, University of Cambridge
AllAnswered.com Discussion Thread: click
here
Title: High-dimensional
changepoint estimation via sparse projection
Abstract: Changepoints
are a very common feature of Big Data that arrive in the form of a data
stream. In this work, we study high-dimensional time series in
which, at certain time points, the mean structure changes in a sparse
subset of the coordinates. The challenge is to borrow
strength across the coordinates in order to detect smaller changes than
could be observed in any individual component series. We propose
a two-stage procedure called `inspect' for estimation of the
changepoints: first, we argue that a good projection direction
can be obtained as the leading
left singular vector of the matrix that solves a convex optimisation
problem derived from the CUSUM transformation of the time
series. We then apply an existing univariate changepoint
estimation algorithm to the projected series. Our theory provides
strong guarantees on both the number of estimated changepoints and the
rates of convergence of their locations, and our numerical studies
validate its highly competitive empirical performance for a wide range
of data generating mechanisms. Software implementing the
methodology is available in the R package `InspectChangepoint'.
Weijie Su, University of Pennsylvania
AllAnswered.com Discussion Thread: click
here
Title: When Does the First
Spurious Variable Get Selected by Sequential Regression Procedures?
Abstract: Applied
statisticians use sequential regression procedures to produce a ranking
of explanatory variables and, in settings of low correlations between
variables and strong true effect sizes, expect that variables at the
very top of this ranking are true. In a regime of certain sparsity
levels, however, three examples of sequential procedures -- forward
stepwise, the lasso, and least angle regression -- are shown to include
the first spurious variable unexpectedly early. We derive a sharp
prediction of the rank of the first spurious variable for the three
procedures, demonstrating that the first spurious variable occurs
earlier and earlier as the regression coefficients get denser. This
counterintuitive phenomenon persists for independent Gaussian designs
and an arbitrarily large magnitude of the effects. We further gain a
better understanding of the phenomenon by identifying the underlying
cause and then leverage the insights to introduce a simple
visualization tool termed the ``double-ranking diagram" to improve on
sequential methods.
As a byproduct of these findings, we obtain the
first provable result certifying the exact equivalence between the
lasso and least angle regression in the early stages of solution paths
beyond orthogonal designs. This equivalence can seamlessly carry over
many important model selection results concerning the lasso to least
angle regression.
Stefan Wager,
Stanford University
AllAnswered.com Discussion Thread: click
here
Title: Efficient Policy
Learning
Abstract: There
has been considerable interest across several fields in methods that
reduce the problem of learning good treatment assignment policies to
the problem of accurate policy evaluation. Given a class of candidate
policies, these methods first effectively evaluate each policy
individually, and then learn a policy by optimizing the estimated value
function; such approaches are guaranteed to be risk-consistent whenever
the policy value estimates are uniformly consistent. However, despite
the wealth of proposed methods, the literature remains largely silent
on questions of statistical efficiency: there are only limited results
characterizing which policy evaluation strategies lead to better
learned policies than others, or what the optimal policy evaluation
strategies are. In this paper, we build on classical results in
semiparametric efficiency theory to develop quasi-optimal methods for
policy learning; in particular, we propose a class of policy value
estimators that, when optimized, yield regret bounds for the learned
policy that scale with the semiparametric efficient variance for policy
evaluation. On a practical level, our result suggests new methods for
policy learning motivated by semiparametric efficiency theory. Joint
work with Susan Athey.
Huixia Judy Wang,
George Washington University
AllAnswered.com Discussion Thread: click
here
Title: Inference for High
Dimensional Quantile Regression
Abstract: In
this talk I will present a new marginal testing procedure to detect the
presence of significant predictors associated with the quantiles of a
scalar response. The idea is to fit the marginal quantile regression on
each predictor separately, and then base the test on the t-statistic
associated with the chosen most informative predictor at the quantiles
of interest. A resampling method is devised to calibrate this test
statistic, which has non-regular limiting behavior due to the variable
selection. Asymptotic validity of the procedure is established in a
general quantile regression setting in which the marginal quantile
regression models can be misspecified. Even though a fixed dimension is
assumed to derive the asymptotic results, the proposed test is
applicable and computationally feasible for large-dimensional
predictors. The method is more flexible than existing marginal
screening test methods based on mean regression, and has the added
advantage of being robust against outliers in the response. The
approach is illustrated using an application to an HIV drug resistance
dataset. This is a joint work with Ian McKeague and Min Qian at
Columbia University.
Wei Biao Wu,
University of Chicago
AllAnswered.com Discussion Thread: click
here
Title: Testing for Trends
in High-dimensional Time Series
Abstract: We
consider statistical inference for trends of high-dimensional time
series. Based on a modified L2-distance between parametric and
nonparametric trend estimators, we propose a de-diagonalized quadratic
form test statistic for testing patterns on trends, such as linear,
quadratic or parallel forms. We develop an asymptotic theory for the
test statistic. A Gaussian multiplier testing procedure is proposed and
it has an improved finite sample performance. Our testing procedure is
applied to a spatial temporal temperature data gathered from various
locations across America. A simulation study is also presented to
illustrate the performance of our testing method. The work is joint
with Likai Chen.
Min-ge Xie,
Rutgers University
AllAnswered.com Discussion Thread: click
here
Title: A union of BFF
(Bayesian, frequentist and fiducial) inferences by confidence
distribution and artificial data sampling
Abstract: In
this talk, we will briefly introduce some new developments of
confidence distribution and also our new understanding about fiducial
inference. A comparative study of the BFF inferences from a unique
simulation (artificial sample data) angle will be provided. It will be
illustrated that the connections between Bayesian and frequentist
inferences (including modern Bayesian computing procedures, bootstrap
and many other resampling and simulation methods) are much more
congruous and coherent than what have been perceived in the scientific
community. Specifically, we will discuss the ideas of uncertainty
matching by CD-random variable, by bootstrapping, by fiducial samples
and also by Bayesian posterior samples, in which the uncertainty by a
model population is matched with the uncertainty from the same or a
similar artificial data generation scheme. A general theory of
uncertainty matching for exact and asymptotic inference will be
provided. Some preliminary ideas of higher order matching and model
selection under the same artificial data generation framework will also
be explored.
Daniel Yekutieli,
Tel Aviv University
AllAnswered.com Discussion Thread: click
here
Title: Post selection
inference in replicated and repeated experiments
Abstract: I
will discuss selection and post selection inference for cases where an
experiment may be replicated under different conditions and may also be
repeated several times for the same condition.
Alastair Young,
Imperial College London
AllAnswered.com Discussion Thread: click
here
Title: Measuring nuisance
parameter effects in Bayesian inference
Abstract: A
key concern of statistical theory is interpretation of the effects of
nuisance parameters on an inference about an interest parameter.
Especially important for statistical practice is quantification of the
consequences of including potentially high-dimensional nuisance
parameters to provide realistic modelling of a system under study.
Through decomposition of the Bayesian version of an adjusted likelihood
ratio statistic, we consider easily computed measures of nuisance
parameter effects in Bayesian inference and illustrate their utility in
practical examples, including hierarchical modelling.
Russell Zaretzki,
University of Tennessee Knoxville
AllAnswered.com Discussion Thread: click
here
Title: Feasible and
powerful simultaneous inference after lasso
Abstract: We
propose methodology for simultaneous inference after lasso in the
spirit of the PoSI procedure (Berk et al., 2013), but which has the
advantage of yielding considerably narrower intervals, and which is
computationally less-demanding. Theoretical control of familywise error
rate is considered, and simulations are presented to assess the
validity of our coverage error claims in realistic scenarios, as well
as compare the interval lengths with more conservative methods. Joint
work with Todd Kuffner.
Mayya Zhilova,
Georgia Institute of Technology
AllAnswered.com Discussion Thread: click
here
Title: Higher-order
Berry-Esseen inequalities and accuracy of the bootstrap
Abstract: In
this talk, we study higher-order accuracy of a bootstrap procedure for
approximation in distribution of a smooth function of a sample average
in high-dimensional non-asymptotic framework. Our approach is based on
Berry-Esseen type inequalities which extend the classical normal
approximation bounds. These results justify in non-asymptotic setting
that bootstrapping can outperform Gaussian (or chi-squared)
approximation in accuracy with respect to both dimension and sample
size. In addition, the presented results lead to improvements of
accuracy of a weighted bootstrap procedure for general log-likelihood
ratio statistics (under certain regularity conditions). The theoretical
results are illustrated with numerical experiments on simulated data.
Posters
Kenneth Hung, UC Berkeley
Title: Rank Verification
for Exponential Families
Abstract: Many
statistical experiments involve comparing multiple population groups.
For example, a clinical trial may compare binary patient outcomes under
several treatment conditions to determine the most effective treatment.
Having observed the “winner” (largest observed response) in a noisy
experiment, it is natural to ask whether that treatment is actually the
“best” (stochastically largest response). This article concerns the
problem of rank verification --- post hoc significance tests of whether
the orderings discovered in the data reflect the population ranks. For
exponential family models, we show under mild conditions that an
unadjusted two-tailed pairwise test comparing the top two observations
(i.e., comparing the “winner” to the “runner-up”) is a valid test of
whether the winner is the best. We extend our analysis to provide
equally simple procedures to obtain lower confidence bounds on the gap
between the winning population and the others, and to verify ranks
beyond the first.
John Kolassa, Rutgers
University
Title: Critical Values for Multi-stage Nonparametric Testing Regimes
Abstract:
Amit Meir, University of
Washington (Seattle)
Title: Selective Maximum
Likelihood Inference - Theory and Application
Abstract: Applying
standard statistical methods after model selection may yield
inefficient estimators and hypothesis tests that fail to achieve
nominal type-I error rates. In recent years, much progress has been
made in developing inference methods that are valid for selected model.
However, relatively little attention has been given to the problem of
estimation after model selection.
Here, we propose to compute selective maximum
likelihood estimators and construct confidence intervals based on their
asymptotic properties: a route that is a matter of course in many
statistical applications, yet novel in the context of selective
inference. In order to do so, we develop a stochastic optimization
framework for computing the selective MLE and derive its asymptotic
properties.
We demonstrate the usefulness of our methodologies
in two applied settings. In the context of fMRI studies, we use the
proposed optimization framework to estimate the mean signal in selected
regions of interest and construct confidence-intervals using a
profile-likelihood type approach. In the context of rare-variant GWAS,
we perform inference for genetic variants in genes that were selected
via an aggregate test. Specifically, we show that the problem of
computing the (multivariate) MLE for regression models that were
selected via an aggregate test can often be cast as a line-search
problem. This is based on work with Mathias Drton, Yuval Benjamini and
Ruth Heller.
Honglang Wang, Indiana
University-Purdue University Indianapolis
Title: Empirical Likelihood
Ratio Tests for Coefficients in High Dimensional Heteroscedastic Linear
Models
Abstract: This paper considers hypothesis
testing problems for a low-dimensional coefficient vector in a
high-dimensional linear model with heteroscedastic variance.
Heteroscedasticity is a commonly observed phenomenon in many
applications including finance and genomic studies. Several statistical
inference procedures have been proposed for low-dimensional
coefficients in a high-dimensional linear model with homoscedastic
variance. However, existing procedures designed for homoscedastic
variance are not applicable for models with heteroscedastic variance
and the heterscedasticity issue has been rarely investigated and
studied. We propose a simple inference procedure based on
empirical
likelihood to overcome the heteroscedasticity issue. The proposed
method is able to make valid inference even when the conditional
variance of random error is an unknown function of high-dimensional
predictors. We apply our inference procedure to three recently proposed
estimating equations and establish the asymptotic distributions of the
proposed methods. Simulation studies and real data analyses are
conducted to demonstrate the proposed methods.
Qi Wang, Washington
University in St. Louis
Title: Pseudo-Posterior
Inference for Volatility in Levy Models with Infinite Jumps
Abstract: Jump
diffusion models driven by Levy processes are suitable models to
capture the dynamics of financial asset returns. Efficient estimation
of the volatility parameter is important for accurate measurement and
effective control of risk. A complete Bayesian specification of jump
diffusion models with infinite jump activity would require a prior
distribution on the jump process, expressing relative beliefs about the
size and frequency of jumps, as well as how these characteristics are
time-dependent. The model for the jump process is difficult to
evaluate, because it is discretely-observed and nonparametric.
Furthermore, approximating the likelihood function and sampling from
the full posterior can require advanced computational methods.
Therefore, it is complicated to use fully Bayesian methods for
inference about the volatility parameter, which would be based on its
marginal posterior distribution. Martin, Ouyang & Domagni (2017)
proposed to misspecify the model on purpose, and to assume only finite
jump activity. Then inference can be based on the conditional posterior
of the volatility parameter, for a given realization of the jump
process. The misspecification is that the jump process is not modeled
at all. It can be shown that the misspecified model posterior for the
volatility will be centered at the wrong value, but that this can be
corrected by a simple estimate of the location shift, and re-scaling
the log-likelihood. Moreover, the Bernstein-von Mises theorem can be
used to quantify posterior uncertainty, and to construct approximate
credible intervals. Kleijn & van der Vaart (2012) gave conditions
under which the Bernstein-von Mises theorem holds under model
misspecification. Building on this theory, we extend the results of
Martin, Ouyang & Domagni (2017) to models with infinite jump
activity. We provide sufficient conditions for the Bernstein-von Mises
theorem to hold in this setting, and formally prove the result. Joint
work with Jose Figueroa-Lopez and Todd Kuffner.
Qiyiwen Zhang, Washington
University in St. Louis
Title: Bayesian model
selection uncertainty in linear regression
Abstract:
Xinwei Will Zhang, Rutgers
University
Title: A Further
Investigation on Validity of Naive Confidence Intervals after Model
Selection
Abstract:
