Theses

Permanent URI for this collectionhttps://dspace.isical.ac.in/handle/10263/2744

Browse

Subject: search.filters.subject.Density Power Divergence

Search Results

Now showing 1 - 3 of 3
  • No Thumbnail Available
    Item
    Robust Matrix Factorization using the Density Power Divergence and its Applications
    (Indian Statistical Institute, Kolkata, 2025-06) Roy, Subhrajyoty
    n the modern era of big data, high-dimensional datasets are becoming increasingly com- mon across a range of disciplines, including machine learning, natural language process- ing, finance, and genomics. Extracting meaningful information from these datasets often requires uncovering low-dimensional structures hidden within the data. Singular Value Decomposition (SVD) and Principal Component Analysis (PCA) are widely used matrix factorization techniques for this purpose. However, the traditional methods to compute these are extremely sensitive to outliers, with even a single aberrant observation poten- tially leading to highly imprecise results. This issue is exacerbated in high-dimensional datasets, where outliers are difficult to detect. Classical robust inference techniques, such as M-estimators, struggle due to their diminishing breakdown points as the data dimension becomes extremely large. This thesis addresses these challenges by proposing a novel class of robust matrix factorization techniques based on the minimum density power divergence estimator (MD- PDE). The MDPDE, a member of the broader class of minimum divergence estimators, is well-known for its robustness and efficiency across diverse applications. Crucially, it offers a dimension-free asymptotic breakdown point, making it particularly well-suited for high- dimensional settings. In this work, we leverage this estimator to develop robust versions of SVD and PCA, referred to as rSVDdpd and rPCAdpd, respectively. The thesis is structured as follows: In Chapter 1, we provide the necessary background on classical matrix factorization techniques, introduce key concepts related to minimum divergence estimators, particularly the MDPDE, and the notations to be used through- out the thesis. Chapter 2 presents the novel rSVDdpd algorithm, detailing its theoretical properties, including different equivariance properties, algorithmic convergence and con- sistency. Through simulation studies, we demonstrate the algorithm’s superior robustness compared to existing methods, particularly in high-dimensional settings. We also ap- ply the rSVDdpd algorithm to the problem of video surveillance background modelling,showcasing its real-world applicability. Chapter 3 extends this methodology to robust PCA, resulting in the rPCAdpd al- gorithm. We establish its theoretical properties such as orthogonal equivariance, con- sistency and asymptotic normality. We also demonstrate that its influence function re- mains bounded, ensuring its robustness to outliers. Comparative studies with benchmark datasets reveal that rPCAdpd outperforms existing robust PCA algorithms, particularly in scenarios with high-dimensional data with a low signal-to-noise ratio. The robust SVD and the PCA algorithms introduced in Chapters 2 and 3 require a robust estimate of the rank of the low-dimensional component of the data matrix. To this end, we propose a new penalized criterion, DICMR, in Chapter 4. Theoretical results on selection consistency and B-robustness are established, and extensive simulation studies show that DICMR is the best-performing among penalized methods, and also provides competitive performance relative to cross-validation methods while being computationally efficient. A key contribution of this thesis, explored in Chapter 5, is the demonstration that the MDPDE has a dimension-free lower bound to its asymptotic breakdown point. This property makes it uniquely robust in high-dimensional settings, a significant improve- ment over classical M-estimators. We further generalize this result in Chapter 6, showing that the dimension-free breakdown point holds for a broader class of estimators known as minimum generalized Alpha-Beta divergence estimators. We derive the necessary and suf- ficient conditions under which the corresponding divergence measures are well-defined and nonnegative, contributing to the theoretical understanding of generating novel statistical divergence measures that may lead to robust estimation in high-dimensional data. Chapter 7 concludes the thesis, summarizing the key findings and outlining directions for future research. This includes potential extensions of the proposed algorithms to other matrix factorization problems and the exploration of more practical applications beyond those demonstrated in the thesis. Overall, this thesis aims to contribute to the field of robust statistics by developing scalable, robust matrix factorization techniques with strong theoretical guarantees and practical relevance in high-dimensional data analysis.
  • No Thumbnail Available
    Item
    Some Applications of Divergences to Robust Inference with Mixed Data
    (Indian Statistical Institute, Kolkata, 2025-06) Pyne, Arijit
    This thesis focuses on the application of the density power divergence to studies involving mixed-data problems. It also develops a unified the- ory of two-sample nonparametric tests for a general class of divergence measures. The main content of the thesis is divided into three parts. The first part explores parameter estimation in ordinal response mod- els, which are prevalent in many scientific studies. A typical data set generated through an ordinal response model includes continuous, non- stochastic regressors and a response variable with ordinal outcomes. The theory of non-homogeneous density power divergence is applicable here, provided appropriate conditions on the regressors and link functions are satisfied. The roles of different link functions in estimation are thor- oughly analyzed, and the robustness of the estimators is evaluated using the influence function, the (explosive) breakdown point, and the implosive breakdown point. The latter two measures are found to be very high, en- suring the robustness of the minimum density power divergence method against various types of outliers. The second part focuses on the estimation and development of Wald- type tests for polychoric correlation. Initially, the standard density power divergence is applied. Subsequently, a two-step approach is introduced, which, while theoretically more complex, substantially reduces the com- putational burden. The results from the two-step approach are highly consistent with those from the initial method. Additionally, a new divergence measure involving two tuning parame- ters, derived from the density power divergence (DPD), is proposed. These estimates perform at least as good as the DPD under pure data condi- tions, up to a threshold defined by the tuning parameters. Moreover, the proposed estimates exhibit enhanced robustness compared to the DPD. Given the effectiveness of polychoric correlation in quantifying associa- tions between categorical variables, this research provides valuable tools for applied scientists. The third part introduces a class of two-sample nonparametric tests based on the class of extended Bregman divergences to assess the equality of two completely unstructured absolutely continuous distributions. The asymptotic distributions of the test statistics are derived under both the null hypothesis and contiguous alternatives. The robustness of the pro- posed method is studied through the influence function and the asymp- totic breakdown point. Numerical studies are conducted for two spe- cific divergence families: the generalized S-Bregman divergence and the Exponential-Polynomial divergence measures. Notably, divergences out- side the power divergence family often perform better within this frame- work. Finally, a generic tuning parameter selection strategy is proposed, en- abling the application of the method to real-world data. The theoretical developments presented in this part hold the potential for extension to various other research areas in the future.
  • No Thumbnail Available
    Item
    Robust Inference using the Extended Bregman Divergence and Optimal Tuning Parameter Selection
    (Indian Statistical Institute, Kolkata, 2022-07) Basak, Sancharee
    The inference procedure based on the minimization of statistical distances has already proved to be a very useful tool in the field of robust inference. One of such commonly used divergences is the Bregman Divergence. Several important divergence families, e.g., the Likelihood Dispar- ity (LD), the Density Power Divergence (DPD) family, the B-Exponential Divergence (BED) family etc. can be represented as subfamilies of the class of Bregman divergences. Yet, there are several other important divergences, e.g., the Power Divergence family, the S-divergence family, etc., which cannot be represented in the Bregman form. We will try to expand the structure of the Bregman divergence so that the above mentioned divergences can be accommodated within the Bregman form with this expanded definition. This we will do by utilizing powers of densities as arguments, rather than the arguments themselves; this leads to the generalized class of the extended Bregman divergences which is one step ahead through the modification of existing popular tools for minimum distance approach used extensively in this literature. Later, using this extension, we have explored the advantage of its use in the field of estimation by construct- ing a new divergence family, namely, the Generalized S-Bregman (GSB) family. Similarly, its contribution in the field of hypotheses testing has also been explored. But, in spite of such modification, sometimes we are not able to get the ‘best’ results due to another burning issue – choice of optimal tuning parameter(s). Inappropriate selection of it can sometimes lead us to dangerous consequences. The emphasis in present times is to find an ‘optimal’ data-based tuning parameter in the estimation process which generates an estimator which represents the best compromise between robustness and efficiency for the data at hand. Selecting this tuning parameter “optimally” is a problem of substantial practical interest, which we have also tried to address through the present work. The DPD has been used as a basic illustrative tool for this purpose. Here, we have tried to refine the attempts to select the optimal tuning parameter taken by Warwick and Jones (2005) as well as Hong and Kim (2001). We have proposed a modified algorithm, namely the Iterated Warwick and Jones (IWJ) algorithm, which helps us to find highly robust estimates along with reasonable efficiency, after removing the pilot dependency to a great extent. Several real life data examples have been used to demonstrate the success of our proposed algorithm. This method can potentially be applied in case of any robust estimation method which depends on the choice of tuning parameter(s).