Determining Weights By Entropy Measures In Case Of Heteroscedasticity

Hatice Ciodem Cin, Atif EVREN

In simple  regression analysis, the dependent variable is assumed to have constant variance at different levels of independent variable. Whenever the assumption of constant variance fails, some remedies like the weighted least squares, or Box-Cox transformation may be helpful. While Box-Cox approach is based on a nonlinear transformation on the dependent variable, in the weighted least squares methodology, independent variable is rescaled by weights to maintain constant variance. Although there is already a large literature on this issue, determining the weights seems a major problem. In this study, the weights are alternatively calculated by entropy approach, since statistical entropy, and variance conveys similar information about a probability distribution. In this study, by exploiting the normality assumption of linear models, the weights are determined by the reciprocals of Shannon, Tsallis and Renyi entropies of normal distribution. The weighting procedure has been applied on some simulated data having nonconstant variance. In some applications we have shown that weighting by Tsallis and/or Rényi entropies produced better goodness of fit results in terms of coefficient of determination, and the mean square error.

https://dx.doi.org/10.31873/IJEAS.6.5.2019.07