📖 Tier 1: Prepare & Study Guide ✓ 100% Solved with Rationales

Linear & Multiple Regression, Correlation & ANOVA (Statistics) Solved Questions & Notes (2026) - Apex Rankers

Statistics & Data Science > Statistics > Linear & Multiple Regression, Correlation & ANOVA

90 Total Solved Questions
~135 mins Estimated Reading Time
1 Subject Areas / Chapters
Select Topic Area / Chapter: Click any section below to switch questions

Linear & Multiple Regression, Correlation & ANOVA

100%
Showing 25 of 90 (28%)
🎯 Practice
Jump:
Q. 1 Statistics
Difficulty: easy (1 Mark)
In the classical simple linear regression model $Y_i = \beta_0 + \beta_1 X_i + u_i$, what does the Gauss-Markov Theorem state regarding the Ordinary Least Squares (OLS) estimator $\hat{\beta}_1$?
A
It is the maximum likelihood estimator for all non-linear equations
B
Under the classical linear regression assumptions, the OLS estimator is the Best Linear Unbiased Estimator (BLUE), having minimum variance among all linear unbiased estimators
✓ Correct
C
It guarantees that $R^2 = 1.0$ in large samples
D
It eliminates all measurement error in the dataset
💡 Step-by-Step Explanation & Concept Rationale
The Gauss-Markov theorem proves that under homoscedasticity, exogeneity, and no autocorrelation, OLS estimators $\hat{\boldsymbol{\beta}}$ achieve the lowest sampling variance among all possible linear unbiased estimators (BLUE).
Q. 2 Statistics
Difficulty: easy (1 Mark)
In the matrix formulation of Multiple Linear Regression $\mathbf{y} = \mathbf{X}\boldsymbol{\beta} + \mathbf{u}$, what is the OLS estimator $\hat{\boldsymbol{\beta}}$?
A
$\hat{\boldsymbol{\beta}} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{y}$
✓ Correct
B
$\hat{\boldsymbol{\beta}} = \mathbf{X}^T \mathbf{y} (\mathbf{X}^T \mathbf{X})$
C
$\hat{\boldsymbol{\beta}} = (\mathbf{X} \mathbf{X}^T)^{-1} \mathbf{y}$
D
$\hat{\boldsymbol{\beta}} = \mathbf{X}^{-1} \mathbf{y}$
💡 Step-by-Step Explanation & Concept Rationale
Minimizing the sum of squared residuals $\mathbf{u}^T \mathbf{u} = (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})^T (\mathbf{y} - \mathbf{X}\boldsymbol{\beta})$ with respect to $\boldsymbol{\beta}$ yields the normal equations $(\mathbf{X}^T \mathbf{X})\hat{\boldsymbol{\beta}} = \mathbf{X}^T \mathbf{y}$, giving $\hat{\boldsymbol{\beta}} = (\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{y}$.
Q. 3 Statistics
Difficulty: medium (1 Mark)
What is the Variance-Covariance matrix of the OLS estimator $\hat{\boldsymbol{\beta}}$ under standard homoscedastic assumptions?
A
$\text{Var}(\hat{\boldsymbol{\beta}}) = \sigma^2 (\mathbf{X}^T \mathbf{X})^{-1}$
✓ Correct
B
$\text{Var}(\hat{\boldsymbol{\beta}}) = \sigma^2 (\mathbf{X}^T \mathbf{X})$
C
$\text{Var}(\hat{\boldsymbol{\beta}}) = \frac{\sigma^2}{n} \mathbf{I}$
D
$\text{Var}(\hat{\boldsymbol{\beta}}) = (\mathbf{X}^T \mathbf{X})^{-1}$
💡 Step-by-Step Explanation & Concept Rationale
Since $\hat{\boldsymbol{\beta}} = \boldsymbol{\beta} + (\mathbf{X}^T \mathbf{X})^{-1}\mathbf{X}^T\mathbf{u}$, $\text{Var}(\hat{\boldsymbol{\beta}}) = (\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T [\text{Var}(\mathbf{u})] \mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1} = \sigma^2 (\mathbf{X}^T \mathbf{X})^{-1}$.
Q. 4 Statistics
Difficulty: easy (1 Mark)
What is the Coefficient of Determination ($R^2$) in a linear regression model?
A
The ratio of the slope to the intercept
B
The proportion of the total variation in the dependent variable $Y$ that is explained by the regression model: $R^2 = 1 - \frac{SSE}{SST} = \frac{SSR}{SST}$
✓ Correct
C
The probability that the regression coefficients are non-zero
D
The correlation coefficient divided by degrees of freedom
💡 Step-by-Step Explanation & Concept Rationale
$R^2 = \frac{SSR}{SST} = 1 - \frac{SSE}{SST}$ quantifies the percentage of variance in the response variable accounted for by the explanatory predictors in the model, bounded in $[0, 1]$.
Q. 5 Statistics
Difficulty: medium (1 Mark)
Why is the 'Adjusted $R^2$' ($\bar{R}^2$) used instead of standard $R^2$ when comparing multiple regression models with different numbers of predictors?
A
Standard $R^2$ never decreases (and almost always increases) whenever a new predictor is added, whereas $\bar{R}^2 = 1 - \frac{SSE/(n-k-1)}{SST/(n-1)}$ penalizes the addition of irrelevant variables
✓ Correct
B
Standard $R^2$ can exceed 1.0 in multiple regression
C
Adjusted $R^2$ can only be computed using non-parametric statistics
D
Adjusted $R^2$ eliminates multicollinearity completely
💡 Step-by-Step Explanation & Concept Rationale
Standard $R^2$ mechanically inflates as more regressors are added, even if they are noise. Adjusted $R^2$ adjusts for degrees of freedom ($df$), increasing only if the new predictor improves the model more than expected by chance.
Q. 6 Statistics
Difficulty: easy (1 Mark)
What is the relationship between the F-statistic for overall model significance in simple linear regression and the individual t-statistic for the slope coefficient?
A
$F = t^2$
✓ Correct
B
$F = \sqrt{t}$
C
$F = t / 2$
D
$F = 2t + 1$
💡 Step-by-Step Explanation & Concept Rationale
In simple linear regression ($k = 1$ predictor), testing $H_0: \beta_1 = 0$ via the two-sided t-test ($df = n-2$) is mathematically identical to the overall ANOVA F-test ($df_1 = 1, df_2 = n-2$), with $F = t^2$.
Q. 7 Statistics
Difficulty: medium (1 Mark)
What is the Hat Matrix $\mathbf{H}$ in linear regression, defined as $\mathbf{H} = \mathbf{X}(\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T$?
A
A matrix that projects the observed response vector $\mathbf{y}$ onto the column space of $\mathbf{X}$ to yield predicted values: $\hat{\mathbf{y}} = \mathbf{H}\mathbf{y}$
✓ Correct
B
A matrix containing the names of the variables
C
A diagonal matrix of residual variances
D
An identity matrix of dimensions $n \times n$
💡 Step-by-Step Explanation & Concept Rationale
The Hat matrix (projection matrix) maps observed values to fitted values: $\hat{\mathbf{y}} = \mathbf{H}\mathbf{y}$. Its diagonal elements $h_{ii}$ measure the 'leverage' of each observation in predictor space.
Q. 8 Statistics
Difficulty: hard (1 Mark)
What is the sum of the diagonal elements (trace) of the Hat Matrix $\mathbf{H} = \mathbf{X}(\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T$ with $p = k + 1$ parameters?
A
$\text{tr}(\mathbf{H}) = n$
B
$\text{tr}(\mathbf{H}) = p$ (the number of estimated regression parameters including intercept)
✓ Correct
C
$\text{tr}(\mathbf{H}) = n - p$
D
$\text{tr}(\mathbf{H}) = 1$
💡 Step-by-Step Explanation & Concept Rationale
Using cyclic property of trace: $\text{tr}(\mathbf{H}) = \text{tr}(\mathbf{X}(\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T) = \text{tr}((\mathbf{X}^T \mathbf{X})^{-1} \mathbf{X}^T \mathbf{X}) = \text{tr}(\mathbf{I}_p) = p$. The average leverage is $\bar{h} = p / n$.
Q. 9 Statistics
Difficulty: medium (1 Mark)
An observation is commonly flagged as having 'High Leverage' in regression diagnostics if its diagonal Hat value $h_{ii}$ exceeds what threshold?
A
$h_{ii} > \frac{2p}{n}$ (or $h_{ii} > \frac{3p}{n}$)
✓ Correct
B
$h_{ii} > 1.0$
C
$h_{ii} < 0$
D
$h_{ii} > 0.5$
💡 Step-by-Step Explanation & Concept Rationale
Since the average leverage is $\bar{h} = p/n$, observations with $h_{ii} > 2p/n$ (or $3p/n$) are considered high leverage points that exert disproportionate potential influence on the fitted plane.
Q. 10 Statistics
Difficulty: medium (1 Mark)
What does Cook's Distance ($D_i$) measure in regression diagnostics?
A
The physical distance between the interviewer and the respondent
B
The overall influence of the $i$-th observation on ALL fitted values when that observation is deleted: $D_i = \frac{\sum_{j=1}^n (\hat{Y}_j - \hat{Y}_{j(i)})^2}{p \cdot MSE}$
✓ Correct
C
The correlation between error terms and explanatory variables
D
The multicollinearity between two predictor variables
💡 Step-by-Step Explanation & Concept Rationale
Cook's Distance combines both the standardized residual (outlier in $Y$) and the leverage $h_{ii}$ (outlier in $X$) to measure how much all regression predictions shift when point $i$ is omitted. $D_i > 1$ or $D_i > 4/n$ indicates high influence.
Q. 11 Statistics
Difficulty: easy (1 Mark)
What is 'Multicollinearity' in multiple regression?
A
The presence of high linear correlation among two or more independent explanatory variables ($X_j$)
✓ Correct
B
A non-linear relationship between the dependent variable and the independent variable
C
Unequal variance in the error terms across observations
D
Correlation between successive error terms over time
💡 Step-by-Step Explanation & Concept Rationale
Multicollinearity occurs when predictors are highly intercorrelated, making the $(\mathbf{X}^T\mathbf{X})$ matrix near-singular, which inflates standard errors of regression coefficients without biasing point estimates.
Q. 12 Statistics
Difficulty: easy (1 Mark)
What is the Variance Inflation Factor (VIF) for predictor $X_j$ defined as?
A
$\text{VIF}_j = \frac{1}{1 - R_j^2}$, where $R_j^2$ is the coefficient of determination from regressing $X_j$ on all other predictors
✓ Correct
B
$\text{VIF}_j = 1 - R_j^2$
C
$\text{VIF}_j = \sqrt{R_j^2}$
D
$\text{VIF}_j = \frac{R_j^2}{n - k}$
💡 Step-by-Step Explanation & Concept Rationale
The VIF measures how much the variance of coefficient $\hat{\beta}_j$ is inflated due to multicollinearity: $\text{VIF}_j = \frac{1}{1 - R_j^2}$. A common rule of thumb considers $\text{VIF} > 5$ or $10$ as evidence of severe multicollinearity.
Q. 13 Statistics
Difficulty: easy (1 Mark)
What is 'Tolerance' in regression diagnostics?
A
The patience of the statistician running the model
B
The reciprocal of the Variance Inflation Factor: $\text{Tolerance}_j = 1 - R_j^2 = \frac{1}{\text{VIF}_j}$
✓ Correct
C
The p-value of the slope coefficient
D
The margin of error in hypothesis testing
💡 Step-by-Step Explanation & Concept Rationale
Tolerance is $\text{Tol}_j = 1 - R_j^2$. A tolerance below $0.10$ or $0.20$ indicates severe collinearity with other explanatory variables.
Q. 14 Statistics
Difficulty: easy (1 Mark)
What is 'Heteroscedasticity' in regression modeling?
A
A violation of the classical assumption where the variance of the error term is NOT constant across observations: $\text{Var}(u_i \mid X_i) = \sigma_i^2 \ne \sigma^2$
✓ Correct
B
Correlation between error terms across different time periods
C
A linear relationship among independent variables
D
A non-normal distribution of the dependent variable
💡 Step-by-Step Explanation & Concept Rationale
Heteroscedasticity means the error terms have non-constant variance (e.g., variance of income errors grows as income increases), rendering standard OLS standard errors invalid and test statistics biased.
Q. 15 Statistics
Difficulty: medium (1 Mark)
What are the consequences of ignoring Heteroscedasticity when using Ordinary Least Squares (OLS)?
A
OLS estimates of $\boldsymbol{\beta}$ become biased and inconsistent
B
OLS coefficient estimates remain unbiased and consistent, but standard errors are biased (usually underestimated), rendering t-tests, F-tests, and confidence intervals invalid, and OLS is no longer BLUE
✓ Correct
C
The $R^2$ becomes negative
D
The regression line passes through all points exactly
💡 Step-by-Step Explanation & Concept Rationale
Under heteroscedasticity, OLS point estimates remain unbiased ($E[\hat{\boldsymbol{\beta}}] = \boldsymbol{\beta}$), but standard formula for $\text{Var}(\hat{\boldsymbol{\beta}})$ is wrong, distorting p-values. Weighted Least Squares (WLS) or Huber-White robust SEs are required.
Q. 16 Statistics
Difficulty: medium (1 Mark)
Which of the following statistical tests is specifically designed to detect Heteroscedasticity in regression models?
A
Durbin-Watson test
B
Breusch-Pagan Test / White's General Heteroscedasticity Test
✓ Correct
C
Dickey-Fuller test
D
Kruskal-Wallis test
💡 Step-by-Step Explanation & Concept Rationale
The Breusch-Pagan test regresses squared OLS residuals on the independent variables (or fitted values), while White's test includes squares and cross-products of regressors to test for general forms of heteroscedasticity.
Q. 17 Statistics
Difficulty: hard (1 Mark)
What is White's Heteroscedasticity-Consistent Covariance Matrix Estimator (HCCME / 'Robust Standard Errors')?
A
$\widehat{\text{Var}}(\hat{\boldsymbol{\beta}}) = (\mathbf{X}^T \mathbf{X})^{-1} \left( \sum_{i=1}^n \hat{u}_i^2 \mathbf{x}_i \mathbf{x}_i^T \right) (\mathbf{X}^T \mathbf{X})^{-1}$ (the 'Sandwich' estimator)
✓ Correct
B
$\widehat{\text{Var}}(\hat{\boldsymbol{\beta}}) = \sigma^2 \mathbf{I}$
C
$\widehat{\text{Var}}(\hat{\boldsymbol{\beta}}) = \frac{1}{n} (\mathbf{X}^T \mathbf{X})$
D
$\widehat{\text{Var}}(\hat{\boldsymbol{\beta}}) = (\mathbf{X}^T \mathbf{X})^{-1}$
💡 Step-by-Step Explanation & Concept Rationale
Halbert White (1980) formulated the sandwich estimator $(X^TX)^{-1}(X^T \hat{\boldsymbol{\Omega}} X)(X^TX)^{-1}$, providing asymptotically valid standard errors and hypothesis tests even when the exact functional form of heteroscedasticity is unknown.
Q. 18 Statistics
Difficulty: easy (1 Mark)
What is 'Autocorrelation' (Serial Correlation) in time series regression?
A
Correlation between error terms at different time points: $\text{Cov}(u_t, u_s) \ne 0$ for $t \ne s$
✓ Correct
B
Correlation between two independent variables
C
Correlation between the intercept and slope
D
The presence of multiple peaks in a histogram
💡 Step-by-Step Explanation & Concept Rationale
Autocorrelation occurs when regression disturbances in period $t$ are correlated with disturbances in earlier periods (e.g., $u_t = \rho u_{t-1} + \epsilon_t$), common in macro-economic and environmental time-series data.
Q. 19 Statistics
Difficulty: medium (1 Mark)
What is the range and ideal null-value of the Durbin-Watson statistic ($d$) for detecting first-order autocorrelation in regression residuals?
A
Bounded between 0 and 4; $d \approx 2$ indicates no first-order autocorrelation ($\rho = 0$)
✓ Correct
B
Bounded between -1 and +1; $d = 0$ indicates no correlation
C
Bounded between 0 and $\infty$; $d = 100$ is ideal
D
Bounded between 0 and 1; $d = 0.5$ is ideal
💡 Step-by-Step Explanation & Concept Rationale
The Durbin-Watson statistic is $d = \frac{\sum_{t=2}^T (e_t - e_{t-1})^2}{\sum_{t=1}^T e_t^2} \approx 2(1 - \hat{\rho})$. $d \approx 2$ means no autocorrelation; $d \approx 0$ indicates strong positive autocorrelation; $d \approx 4$ indicates strong negative autocorrelation.
Q. 20 Statistics
Difficulty: hard (1 Mark)
When is the Durbin-Watson test INVALID for testing serial correlation?
A
When the sample size $n > 100$
B
When the regression model contains a Lagged Dependent Variable among the regressors ($Y_{t-1}$), where the Durbin-h test or Breusch-Godfrey LM test must be used instead
✓ Correct
C
When the error terms are homoscedastic
D
When all variables are positive
💡 Step-by-Step Explanation & Concept Rationale
If a lagged dependent variable $Y_{t-1}$ is included as a predictor, the Durbin-Watson statistic is severely biased toward 2 (masking true serial correlation). The Breusch-Godfrey LM test is the general standard.
Q. 21 Statistics
Difficulty: hard (1 Mark)
What are Newey-West Standard Errors (HAC - Heteroscedasticity and Autocorrelation Consistent)?
A
Standard errors that are robust to both unknown forms of heteroscedasticity AND arbitrary serial correlation up to a specified lag truncation length $L$
✓ Correct
B
Standard errors calculated using only the newest observations
C
Standard errors that are always equal to 1.0
D
Standard errors used exclusively in logistic regression
💡 Step-by-Step Explanation & Concept Rationale
Newey-West (1987) HAC standard errors use a Bartlett kernel weighting scheme to produce positive semi-definite covariance estimates robust to simultaneous heteroscedasticity and autocorrelation in time-series OLS.
Q. 22 Statistics
Difficulty: hard (1 Mark)
What does the Ramsey RESET (Regression Equation Specification Error Test) test for?
A
General model misspecification, including omitted non-linear terms (e.g., squared or cubic regressors) and incorrect functional form, by testing the significance of powers of fitted values $\hat{Y}^2, \hat{Y}^3$
✓ Correct
B
Whether the computer reset properly during estimation
C
Equality of variances across subgroups
D
Multicollinearity among explanatory variables
💡 Step-by-Step Explanation & Concept Rationale
Ramsey's RESET tests for omitted non-linearities and functional form misspecification by adding powers of the fitted values $(\hat{y}_i^2, \hat{y}_i^3)$ to the auxiliary regression and testing their joint significance via F-test.
Q. 23 Statistics
Difficulty: easy (1 Mark)
In Binary Logistic Regression, what is the 'Logit' Link Function defined as for probability $p = P(Y = 1 \mid X)$?
A
$\text{logit}(p) = \ln\left(\frac{p}{1 - p}\right) = \beta_0 + \beta_1 X$
✓ Correct
B
$\text{logit}(p) = \frac{p}{1 - p}$
C
$\text{logit}(p) = e^p$
D
$\text{logit}(p) = \Phi^{-1}(p)$
💡 Step-by-Step Explanation & Concept Rationale
The logit function is the natural logarithm of the odds: $\text{logit}(p) = \ln\left(\frac{p}{1-p}\right)$. It maps probability $p \in (0, 1)$ to $(-\infty, \infty)$, enabling linear parameterization.
Q. 24 Statistics
Difficulty: medium (1 Mark)
In a Logistic Regression model $\ln\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 X$, how is the slope coefficient $\beta_1$ interpreted?
A
For a one-unit increase in $X$, the probability $p$ increases by $\beta_1$
B
For a one-unit increase in $X$, the Odds Ratio changes multiplicatively by a factor of $e^{\beta_1}$ (i.e., log-odds increases by $\beta_1$)
✓ Correct
C
For a one-unit increase in $X$, the variance increases by $\beta_1^2$
D
$\beta_1$ represents the correlation between $X$ and $Y$
💡 Step-by-Step Explanation & Concept Rationale
In logistic regression, exponentiating the slope gives the Odds Ratio: $\text{OR} = e^{\beta_1}$. A one-unit increase in $X$ multiplies the odds of the outcome by $e^{\beta_1}$.
Q. 25 Statistics
Difficulty: medium (1 Mark)
What estimation method is standard for fitting Logistic Regression models?
A
Ordinary Least Squares (OLS)
B
Maximum Likelihood Estimation (MLE) using iterative numerical algorithms like Newton-Raphson or Iteratively Reweighted Least Squares (IRLS)
✓ Correct
C
Method of Moments
D
Sign Test
💡 Step-by-Step Explanation & Concept Rationale
Because the binary response $Y_i \sim \text{Bernoulli}(p_i)$ violates OLS homoscedasticity and normality, Logistic Regression parameters are estimated by maximizing the Bernoulli log-likelihood via Newton-Raphson / IRLS.
Study Stream Progress: Showing 25 of 90 Questions (28%)
Jump to:

Ready to Test Your Retention & Speed?

Now that you have reviewed the study questions and rationales, test yourself in our interactive 1-by-1 practice engine or take the full official timed mock exam.