Multiple Regression Analysis

Purpose of Multiple Regression Analysis

Multiple regression analysis is applied to:

• Establish a functional relationship between one dependent variable $Y$ and two or more independent variables $X_1, X_2, \ldots$
• Predict/estimate the value of the dependent variable from known independent variables
• Measure the separate effect of each independent variable while holding the others constant
• Identify which independent variables are significant contributors to the variation in $Y$
• Provide a more realistic model since real phenomena depend on several factors simultaneously

---

Given Data

$n = 10$

---

Part (i): Multiple Regression Equation

Model: $\hat{Y} = b_0 + b_1 X_1 + b_2 X_2$

Step 1: Summations

$$\sum Y = 681,\ \sum X_1 = 67,\ \sum X_2 = 30,\ \sum X_1^2 = 463,$$ $$\sum X_2^2 = 182,\ \sum X_1 X_2 = 185,\ \sum X_1 Y = 4673,\ \sum X_2 Y = 1810$$

Step 2: Means

$$\bar{Y} = 68.1,\quad \bar{X}_1 = 6.7,\quad \bar{X}_2 = 3.0$$

Step 3: Normal Equations

$$681 = 10b_0 + 67b_1 + 30b_2 \quad (1)$$ $$4673 = 67b_0 + 463b_1 + 185b_2 \quad (2)$$ $$1810 = 30b_0 + 185b_1 + 182b_2 \quad (3)$$

Step 4: Solve

Eliminate $b_0$ using (1) and (2): multiply (1) by $6.7$: $$4562.7 = 67b_0 + 448.9b_1 + 201b_2 \quad (4)$$ $(2) - (4)$: $$110.3 = 14.1b_1 - 16b_2 \quad (5)$$

Eliminate $b_0$ using (1) and (3): multiply (1) by $3$: $$2043 = 30b_0 + 201b_1 + 90b_2 \quad (6)$$ $(3) - (6)$: $$-233 = -16b_1 + 92b_2 \quad (7)$$

From (5): $14.1b_1 - 16b_2 = 110.3$ From (7): $-16b_1 + 92b_2 = -233$

Solve. From (5): $b_1 = \dfrac{110.3 + 16b_2}{14.1}$.

Substitute in (7): $$-16\left(\frac{110.3 + 16b_2}{14.1}\right) + 92b_2 = -233$$

$$-16(110.3 + 16b_2) + 92(14.1)b_2 = -233(14.1)$$

$$-1764.8 - 256b_2 + 1297.2b_2 = -3285.3$$

$$1041.2b_2 = -1520.5$$

$$b_2 = -1.4603$$

Then: $$b_1 = \frac{110.3 + 16(-1.4603)}{14.1} = \frac{110.3 - 23.365}{14.1} = \frac{86.935}{14.1} = 6.1656$$

From (1): $$b_0 = \frac{681 - 67(6.1656) - 30(-1.4603)}{10}$$ $$= \frac{681 - 413.10 + 43.81}{10} = \frac{311.71}{10} = 31.171$$

Regression Equation

$$\boxed{\hat{Y} = 31.171 + 6.166,X_1 - 1.460,X_2}$$

---

Part (ii): Interpretation of $b_1$ and $b_2$

• $b_1 = 6.166$: Holding the number of absent days ($X_2$) constant, a one-unit increase in the aptitude test score ($X_1$) raises the job-satisfaction score ($Y$) by about 6.17 units. The positive sign shows a direct relationship.

• $b_2 = -1.460$: Holding the aptitude test score ($X_1$) constant, each additional day of absence ($X_2$) decreases the job-satisfaction score ($Y$) by about 1.46 units. The negative sign shows an inverse relationship.

---

Part (iii): Estimate for $X_1 = 9,\ X_2 = 6$

$$\hat{Y} = 31.171 + 6.166(9) - 1.460(6)$$ $$= 31.171 + 55.494 - 8.760$$ $$= \boxed{77.91}$$

The estimated job-satisfaction score is approximately 78.

Design of Experiments and Latin Square Design

Design of Experiments

Design of experiments refers to the systematic planning and arrangement of experimental conditions so that valid, unbiased, and reliable conclusions can be drawn from the experimental data with minimum experimental error.

According to R.A. Fisher, the design of experiments is based on three fundamental principles:

Three Principles of Design of Experiments

---

Latin Square Design (LSD)

LSD is a design used for non-homogeneous experimental material. It applies local control simultaneously along both rows and columns, making it more efficient than RBD. The experimental layout is always square (equal number of rows and columns).

Mathematical Model

$$Y_{ijk} = \mu + \alpha_i + \beta_j + T_k + e_{ijk}$$

Where:

• $Y_{ijk}$ = observation in $i^{th}$ row and $j^{th}$ column receiving $k^{th}$ treatment
• $\mu$ = general mean effect
• $\alpha_i$ = effect due to $i^{th}$ row
• $\beta_j$ = effect due to $j^{th}$ column
• $T_k$ = effect due to $k^{th}$ treatment
• $e_{ijk}$ = random error due to chance

---

Layout of a Latin Square Design (4x4 Example)

In an LSD with 4 treatments (A, B, C, D), each treatment appears exactly once in each row and exactly once in each column:

$$ \begin{array}{|c|c|c|c|} \hline \textbf{Col 1} & \textbf{Col 2} & \textbf{Col 3} & \textbf{Col 4} \ \hline A & B & C & D \ \hline B & C & D & A \ \hline C & D & A & B \ \hline D & A & B & C \ \hline \end{array} $$

Here, 4 rows, 4 columns, and 4 treatments -- each appearing exactly once per row and per column.

---

Why the Number of Treatments Should NOT Be Less Than 3

In an LSD with $m$ treatments, the degrees of freedom for error is given by:

$$df_{error} = (m-1)(m-2)$$

If $m = 2$ (only 2 treatments):

$$df_{error} = (2-1)(2-2) = 1 \times 0 = 0$$

• With zero degrees of freedom for error, it is impossible to estimate the experimental error.
• Without an error estimate, no F-test can be performed, and no valid statistical conclusions can be drawn.

If $m = 3$ (minimum acceptable):

$$df_{error} = (3-1)(3-2) = 2 \times 1 = 2$$

• This gives at least 2 degrees of freedom for error, which allows estimation of experimental error and valid hypothesis testing.

Summary Table

Conclusion: The number of treatments in an LSD must be at least 3 so that the degrees of freedom for error is greater than zero, enabling estimation of experimental error and valid F-tests in the ANOVA table.

Chi-Square Test for Association

Step 1: Given Data

Observed contingency table:

Significance level $\alpha = 0.05$.

Step 2: Hypotheses

• $H_0$: No association between duration of marriage and acquaintanceship period (independent).
• $H_1$: There is an association.

Step 3: Expected Frequencies

$$E_{ij} = \frac{R_i \times C_j}{N}$$

Step 4: Chi-Square Statistic

$$\chi^2 = \sum \frac{(O-E)^2}{E}$$

$$\chi^2_{cal} = 0.24545 + 0.36818 + 0.27222 + 0.40833 + 0.05556 + 0.08333 = 1.4331$$

Step 5: Degrees of Freedom and Critical Value

$$df = (r-1)(c-1) = (3-1)(2-1) = 2$$

$$\chi^2_{0.05,,2} = 5.991$$

Step 6: Decision

$$\chi^2_{cal} = 1.433 < \chi^2_{tab} = 5.991$$

Fail to reject $H_0$.

Step 7: Conclusion

At the 5% level of significance, there is no significant association between the duration of marriage and the period of acquaintanceship before marriage. The data do not substantiate an association.

Confidence Level in Estimation and Confidence Interval for a Proportion

Definition: Confidence Level

The confidence level $(1-\alpha)$ is the probability that the interval estimate constructed from sample data will contain the true population parameter. A 96% confidence level means that if repeated random samples were taken and an interval built from each, about 96% of those intervals would enclose the true population proportion.

---

Given Data

---

Step 1: Sample Proportion

$$p = \frac{X}{n} = \frac{20}{400} = 0.05, \qquad q = 1 - p = 0.95$$

Step 2: Critical Value

$$\frac{\alpha}{2} = 0.02 \Rightarrow Z_{\alpha/2} = 2.054 \approx 2.05$$

Step 3: Standard Error

$$S.E.(p) = \sqrt{\frac{pq}{n}} = \sqrt{\frac{0.05 \times 0.95}{400}} = \sqrt{\frac{0.0475}{400}} = \sqrt{0.00011875} = 0.010897$$

Step 4: Confidence Interval

$$p \pm Z_{\alpha/2}\cdot S.E.(p)$$

Margin of error: $$E = 2.054 \times 0.010897 = 0.02238$$

Lower limit: $$0.05 - 0.02238 = 0.02762$$

Upper limit: $$0.05 + 0.02238 = 0.07238$$

Step 5: Express as Percentage

$$\boxed{2.76% \leq P \leq 7.24%}$$

(Using $Z = 2.05$: $2.77%$ to $7.24%$; essentially the same.)

---

Interpretation

We are 96% confident that the true percentage of all toothpaste tubes with leakage from the tail end lies between approximately 2.76% and 7.24%. If the sampling procedure were repeated many times, about 96% of the resulting intervals would contain the true population proportion of leaking tubes.

Hypothesis Test: Mean Drying Time of Spray Paint

Given Data

• Population (claimed) mean: $\mu_0 = 122$ seconds
• Sample size: $n = 50$
• Sample mean: $\bar{X} = 116$ seconds
• Sample standard deviation: $s = 16.8$ seconds
• Significance level: $\alpha = 0.05$

Step 1: Hypotheses

• $H_0: \mu = 122$ (no reduction)
• $H_1: \mu < 122$ (drying time reduced) - left-tailed test

Step 2: Test Statistic

Since $n = 50 > 30$, use the Z-test: $$Z = \frac{\bar{X} - \mu_0}{s/\sqrt{n}}$$

Step 3: Calculation

$$s/\sqrt{n} = \frac{16.8}{\sqrt{50}} = \frac{16.8}{7.0711} = 2.3759$$

$$Z = \frac{116 - 122}{2.3759} = \frac{-6}{2.3759} = -2.525$$

Step 4: Critical Value

Left-tailed at $\alpha = 0.05$: $Z_{tab} = -1.645$

Step 5: Decision

Since $Z_{cal} = -2.525 < -1.645$, we reject $H_0$.

Step 6: P-value

$$p = P(Z < -2.525) = 1 - P(Z < 2.525) = 1 - 0.9942 = 0.0058$$

Since $p = 0.0058 < \alpha = 0.05$, reject $H_0$.

Conclusion

The data provide strong evidence at the 5% level of significance that the new chemical additive reduces the mean drying time below 122 seconds. The small p-value ($\approx 0.0058$) confirms high significance.

Level of Significance and Hypothesis Testing

Concept of Level of Significance

The level of significance (denoted α) is the probability of rejecting the null hypothesis $H_0$ when it is actually true. It is the maximum acceptable probability of committing a Type I error.

• Fixed in advance of the test (typically $1%$, $5%$, or $10%$).
• At $5%$ significance, we accept a $5%$ chance of wrongly rejecting a true $H_0$.
• It determines the critical (tabulated) value used to bound the acceptance/rejection regions.
• A smaller $\alpha$ makes the test more conservative (harder to reject $H_0$).

---

Numerical Solution

Given Data

Sample proportion: $$\hat{p} = \frac{x}{n} = \frac{80}{100} = 0.80$$

---

Step 1: Hypotheses

• $H_0: P = \dfrac{2}{3}$ (two-thirds prefer the first color)
• $H_1: P > \dfrac{2}{3}$ (more than two-thirds prefer the first color)

This is a right-tailed test.

---

Step 2: Test Statistic

$$Z = \frac{\hat{p} - P}{\sqrt{\dfrac{P(1-P)}{n}}}$$

Numerator: $$\hat{p} - P = 0.80 - 0.6667 = 0.1333$$

Denominator: $$\sqrt{\frac{0.6667 \times 0.3333}{100}} = \sqrt{\frac{0.2222}{100}} = \sqrt{0.002222} = 0.04714$$

Therefore: $$Z = \frac{0.1333}{0.04714} = 2.83$$

---

Step 3: Critical Value

For a right-tailed test at $\alpha = 0.05$: $$Z_{tab} = 1.645$$

---

Step 4: Decision

$$Z_{cal} = 2.83 > Z_{tab} = 1.645$$

Since the calculated value falls in the rejection region, reject $H_0$.

---

Step 5: Conclusion

At the $5%$ level of significance, there is sufficient evidence to conclude that more than two-thirds of all customers prefer the first color of the laptop.

Queuing Theory and Barber Shop Problem

Part 1: Brief Explanation of Queuing Theory

Queuing theory is the mathematical study of waiting lines. It models systems in which customers (or jobs) arrive, possibly wait in a queue, receive service from one or more servers, and then leave. Its aim is to predict system performance measures such as average waiting time, queue length, and server utilization so resources can be optimally designed.

Characteristics of a queuing system (Kendall notation A/B/c):

Utilization (traffic intensity): $\rho = \dfrac{\lambda}{c\mu}$; stability requires $\rho < 1$.

---

Part 2: Barber Shop Numerical Solution

Given data

• Mean inter-arrival time $= 12$ minutes
• Mean service time (in barber's chair) $= 10$ minutes
• One server (one-man shop) → M/M/1 model

Step 1: Rates

$$\lambda = \frac{1}{12}\text{ customers/min}, \qquad \mu = \frac{1}{10}\text{ customers/min}$$

Step 2: Traffic intensity

$$\rho = \frac{\lambda}{\mu} = \frac{1/12}{1/10} = \frac{10}{12} = \frac{5}{6} \approx 0.833 \quad (<1,\text{ stable})$$

Step 3: Expected number in queue $L_q$

$$L_q = \frac{\rho^2}{1-\rho} = \frac{(5/6)^2}{1 - 5/6} = \frac{25/36}{1/6} = \frac{25}{6} \approx 4.17 \text{ customers}$$

(Supporting) Expected number in system $L_s$

$$L_s = \frac{\rho}{1-\rho} = \frac{5/6}{1/6} = 5 \text{ customers}$$

Final Answer

$$\boxed{L_q = \frac{25}{6} \approx 4.17 \text{ customers waiting in the queue}}$$

The question specifically asks for customers in the queue, so $L_q \approx 4.17$ is the required answer ($L_s = 5$ is the number in the whole shop).