To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca

Abstract: The advent of deep learning has completely revolutionized how we perceive data to obtain super-human performance across all fields of modern science. However, despite the remarkable empirical successes of deep learners, the theoretical guarantees for their statistical accuracy remain rather pessimistic. In particular, the data distributions on which deep learners are generally applied, such as natural images, are often hypothesized to have an intrinsic low-dimensional structure in a typically high-dimensional feature space. However, this is often not reflected in the derived rates in the state-of-the-art analyses. This talk aims to bridge the gap between the theory and practice of deep learning from a statistical perspective. We demonstrate that deep learners exhibit a convergence rate determined solely by the intrinsic dimensionality of the data, rather than its nominal high-dimensional feature representation. Our work not only provides practical guidelines for selecting suitable network architectures but also connects the theoretical analyses of these models to established convergence rates in optimal transport and non-parametric statistics literature. In particular, we derive the sharpest convergence rates for various learning scenarios, including Generative Adversarial Networks (GANs), Wasserstein Autoencoders (WAEs), federated learning, Bi-directional GANs, and general deep supervised learners. Furthermore, we introduce a novel measure, called the entropic dimension, to characterize the intrinsic dimension of probability measures and achieve the sharpest known approximation results for neural networks employing Rectified Linear Unit (ReLU) activation, improving upon classical benchmarks.

Bio: Saptarshi Chakraborty is a fifth-year Ph.D. student in Statistics at the University of California, Berkeley, advised by Prof. Peter Bartlett. Prior to joining Berkeley, he earned his M.Stat and B. Stat (Hons.) degrees in Statistics from the Indian Statistical Institute (ISI), Kolkata, India. Saptarshi is the recipient of the Two-sigma PhD fellowship, 2023-24. He is primarily interested in the theoretical and methodological foundations of machine learning, especially, deep learning theory, unsupervised learning, dimensionality reduction, optimal transport, and optimization.

***

Dear STAT news subscribers:

Please be advised that this seminar has been cancelled. We sincerely apologize for any inconvenience.

Best wishes,

UBC Statistics Department

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca

Abstract: Earth System Models (ESMs) are state-of-the-art mathematical formulations used to describe the Earth’s climate system. Supported by supercomputing resources, ESMs can generate large-ensemble, high-resolution global climate simulations, which supplement real-world observations and enhance our understanding of climate changes and their associated variabilities. However, the substantial computational and storage demands of these simulations often limit their broader utility. We propose an efficient statistical emulator—referred to as a stochastic generator (SG)—to address these challenges. By applying the spherical harmonic transformation (SHT), the SG converts climate simulations into a lower-dimensional spectral domain, significantly reducing computational and storage requirements. As a practical complement to ESMs, the SG can rapidly generate multiple emulations of climate simulations. We demonstrate this approach by developing an SG for surface temperature simulations from the newly published CESM2-LENS2 data. To capture non-stationary spatial dependencies, our model incorporates axial symmetry and applies distinct ranks for land and ocean regions. To handle non-Gaussianity in high-temporal-resolution data, we use a modified Tukey g-and-h transformation. The SG successfully emulates CESM2-LENS2 surface temperature simulations across multiple scales, marking the first attempt at reproducing daily data. With the support of supercomputers, we further validate the SG's scalability by emulating ultra-high-resolution climate simulations—an achievement that contributed to winning the prestigious 2024 Gordon Bell Prize for Climate Modelling. Finally, we extend the SG's capabilities to near real-time regional climate simulations by leveraging Slepian concentration on the sphere and an online updating technique. These developments offer a promising complementary pathway for efficient climate modeling and analysis, overcoming critical computational and storage barriers.

***

Please note that this seminar will now be held virtually only, and will not take place in person.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca

Abstract: Knowledge transfer, a core human ability, has inspired numerous data integration methods in machine learning and statistics. However, data integration faces significant challenges: (1) unknown similarity between data sources; (2) data contamination; (3) high-dimensionality; and (4) privacy constraints. This talk addresses these challenges in three parts across different contexts, presenting both innovative statistical methodologies and theoretical insights.

In Part I, I will introduce a transfer learning framework for high-dimensional generalized linear models that combines a pre-trained Lasso with a fine-tuning step. We provide theoretical guarantees for both estimation and inference, and apply the methods to predict county-level outcomes of the 2020 U.S. presidential election, uncovering valuable insights.

In Part II, I will explore an unsupervised learning setting where task-specific data is generated from a mixture model with heterogeneous mixture proportions. This complements the supervised learning setting discussed in Part I, addressing scenarios where labeled data is unavailable. We propose a federated gradient EM algorithm that is communication-efficient and privacy-preserving, providing estimation error bounds for the mixture model parameters.

In Part III, I will introduce a representation-based multi-task learning framework that generalizes the distance-based similarity notion discussed in Parts I and II. This framework is closely related to modern applications of fine-tuning in image classification and natural language processing. I will discuss how this study enhances our understanding of the effectiveness of fine-tuning and the influence of data contamination on representation multi-task learning.

Finally, I will summarize the talk and briefly introduce my broader research interests. The three main sections of this talk are based on a series of papers [TF23, TWXF22, TWF24, TGF23] and a short course I co-taught at NESS 2024 [STL24]. More about me and my research can be found at https://yet123.com.  

[TF23] Tian, Y., & Feng, Y. (2023). Transfer Learning under High-dimensional Generalized Linear Models. Journal of the American Statistical Association, 118(544), 2684-2697. [Link]

[TWXF22] Tian, Y., Weng, H., Xia, L., & Feng, Y. (2022). Unsupervised Multi-task and Transfer Learning on Gaussian Mixture Models. arXiv preprint arXiv:2209.15224. [Link]

[TWF24] Tian, Y., Weng, H., & Feng, Y. (2024). Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithms. ICML 2024. [Link]

[TGF23] Tian, Y., Gu, Y., & Feng, Y. (2023). Learning from Similar Linear Representations: Adaptivity, Minimaxity, and Robustness. arXiv preprint arXiv:2303.17765. [Link]

[STL24] A (Selective) Introduction to the Statistics Foundations of Transfer Learning. (2024). [Link]

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca

Abstract: Probabilistic models describe complex data generating processes and have been applied to a broad range of fields, such as epidemiology, pharmacology, and astrophysics. Inference for probabilistic models poses significant computational challenges, particularly as models grow in complexity and datasets increase in size. Modern hardware, with its parallelization capabilities, offers new opportunities to accelerate statistical inference. However, many traditional methods are not inherently designed for parallel computation. Markov chain Monte Carlo (MCMC), for instance, typically relies on a few long-running chains. I propose an alternative approach: running hundreds or thousands of shorter chains in parallel. To support this paradigm, I introduce the nested “R-hat,” a novel convergence diagnostic tailored for the many-short-chains regime, paving the way for faster and more automated MCMC.

Next I examine variational inference (VI). VI already leverages the parallelization capacities of modern hardware, however it lacks the theoretical guarantees of MCMC and other statistical methods. I present two key theoretical results: (1) a positive result demonstrating that VI can effectively learn symmetries even under misspecified approximations, and (2) a negative result revealing that factorized (or mean-field) approximations lead to an impossibility theorem, preventing the simultaneous estimation of multiple measures of uncertainty . These findings provide practical guidance for selecting VI’s objective function and approximation family, offering a path toward robust and scalable inference.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca

Abstract: AI/ML applications have quickly gained popularity in many scientific domains, and in some cases, even started replacing conventional approaches for data collection. However, the reliability of scientific findings purely based on ML-derived outcomes remains largely unexplored. In this talk, I will demonstrate that genetic association analysis based on ML-derived phenotypic outcomes can lead to pervasive false-positive findings. To address this, I will introduce a statistical framework named “POP-GWAS” for ML-assisted statistical inference for genetic discovery. It ensures valid and efficient inference given arbitrary "black-box" ML prediction. Moreover, the framework only requires summary statistics as input, enabling computationally efficient application at the biobank scale. Using POP-GWAS, I performed the largest genome-wide association study (GWAS) to date on bone mineral density derived from dual-energy X-ray absorptiometry imaging at 14 skeletal sites, achieving a 9.7%-50.7% gain in effective sample size compared to conventional approaches. This new approach identified 89 novel genetic associations and many complex traits showing significant skeletal-site- specific genetic correlations with bone mineral density. In addition, I will discuss the extension of this framework to general statistical tasks, providing both theoretical insights on statistical optimality and practical implications of summary-statistics-based statistical inference. Finally, I will give a brief overview of my research program, covering topics from quantifying gene-environment interactions to advanced genetic risk prediction in diverse ancestries.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca

Abstract: With the ubiquity of data, linking data sets has become crucial for myriad applications including healthcare, official statistics, ecology, fraud detection, and national security. Record linkage is the task of resolving duplicates in two or more partially overlapping sets of records, or files, from noisy data sources without a unique identifier. In any field where multiple sources of messy data are available to address a scientific problem, record linkage is a critical step in the analysis pipeline. In streaming record linkage, files arrive sequentially in time and estimates of the linkage structure are updated after the arrival of each file. The challenge in streaming record linkage is to efficiently update parameter estimates as new data arrive. In this talk, I present the first multi-file Bayesian record linkage model formulated specifically for the streaming data context. This model is fit using recursive updates, incorporating each new batch of data into the model parameters' posterior distribution. A novel Markov chain Monte Carlo algorithm is presented that performs recursive Bayesian updates while avoiding the issue of degradation, common to many recursive algorithms. This sampler achieves near-equivalent posterior inference to non-streaming algorithms at a small fraction of the compute time.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca.

Abstract: In this talk, I will introduce PrairieLearn, an open-source platform designed for creating assessments with powerful randomization and autograding features. I will highlight why PrairieLearn is an ideal choice for courses in computer science, data science, and statistics. Additionally, I will demonstrate our tools for migrating questions from Canvas, building question banks, and generating test files for coding questions—offering practical advice for transitioning to PrairieLearn. Finally, I will provide a brief tutorial on creating assessments and questions using R or Python.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca.

Abstract: The Killer Whale is the world’s largest predator and a cultural icon in the Pacific Northwest. In the past two decades, the Southern Resident Killer Whale population has declined by more than 25 percent, putting the population at risk of extinction. Much of the whale’s habitat in our local waters overlaps with the shipping lanes that connect the Pacific Ocean with ports in southern British Columbia and northern Washington State. The continued decline in SRKW has been linked to disturbance from commercial vessels servicing our regional ports. In recent years, citizen science networks connected through social media are providing real-time sighting information while substantial infrastructure investment has resulted in real-time underwater acoustic monitoring stations. This has opened the possibility of forecasting systems to fuse these real-time data streams with movement models to predict future trajectories of whales. This seminar will present recent progress in building the A.I. detection/classification algorithms, real-time movement models, and computing infrastructure for real-time mitigation of commercial vessel impacts within SRKW critical habitat.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca.

Abstract: Model selection in Gaussian processes scales prohibitively with the size of the training dataset, both in time and memory. While many approximations exist, all incur inevitable approximation error. Recent work accounts for this error in the form of computational uncertainty, which enables -- at the cost of quadratic complexity -- an explicit tradeoff between computational efficiency and precision. Here we extend this development to model selection, which requires significant enhancements to the existing approach, including linear-time scaling in the size of the dataset. We propose a novel training loss for hyperparameter optimization and demonstrate empirically that the resulting method can outperform SGPR, CGGP and SVGP, state-of-the-art methods for GP model selection, on medium to large-scale datasets. Our experiments show that model selection for computation-aware GPs trained on 1.8 million data points can be done within a few hours on a single GPU. As a result of this work, Gaussian processes can be trained on large-scale datasets without significantly compromising their ability to quantify uncertainty -- a fundamental prerequisite for optimal decision-making.

Bio: Jonathan Wenger is a postdoctoral research scientist at Columbia University's Department of Statistics and Zuckerman Institute working with Prof. John Cunningham. He earned a PhD in Computer Science from the University of Tübingen under the supervision of Prof. Philipp Hennig. Jonathan’s research focuses on resource-efficient methods for large-scale probabilistic machine learning. Much of his work contributes to the field of probabilistic numerics, which views numerical algorithms through the lens of probabilistic inference. This perspective enables the acceleration of learning algorithms via an explicit trade-off between computational efficiency and predictive precision.

To join this seminar virtually: Please request Zoom connection details from ea@stat.ubc.ca.

Abstract: Empirical likelihood is a popular non-parametric method for inference. The resulting profile empirical likelihood function has many similar properties to its parametric counterpart. The empirical likelihood confidence intervals achieve higher coverage precision compared to its parametric counterpart but tend to have below nominal-level coverage probability. To address this issue, researchers have proposed adjusted empirical likelihood and Bartlett corrected empirical likelihood methods to achieve high-order coverage precision. Still, when the sample size is small, the coverage remains unsatisfactory. In my thesis, we develop a computer experiment data-driven approach to improve the coverage precisions of empirical likelihood confidence regions.

The maximum empirical likelihood estimator, just like its parametric counterpart, shares many nice properties. However, the optimal properties cannot be utilized unless we know the local maximum at hand is close to the unknown true parameter value. To overcome this obstacle, we first propose a set of conditions under which the global maximum is consistent. We then develop a global maximum test to ascertain if the local maximum at hand is, in fact, the global maximum. Furthermore, we invent a global maximum remedy to ensure global consistency by expanding the set of estimating functions under empirical likelihood.

For non-regular models such as finite normal mixture models, the MLE is not well-defined because of the unboundedness of likelihood. To address this issue, researchers have proposed penalized likelihood and constrained MLE to consistently estimate the mixing distribution. However, the consistency of these method is established under the assumption that the component covariance matrices of the true mixing distribution are non-singular. We relax this restriction to show that the penalized MLE is still consistent when component covariance matrices of the true mixing distribution are singular. We also invent a test for degeneracy of finite normal mixture model.