Paraphrase-this-paragraph-Put-it-in-your-own-words-

https://stats.stackexchange.com/questions/46185/qu…

Multiple regression can be obtained by sequential matching

Returning to the setting of the question, we have one target

y

$y$ and two matchers

x1

$x_1$ and

x2

$x_2$. We seek numbers

b1

$b_1$ and

b2

$b_2$ for which

y

$y$ is approximated as closely as possible by

b1x1+b2x2

$b_1{x}_1+b_2{x}_2$, again in the least-distance sense. Arbitrarily beginning with

x1

$x_1$, Mosteller & Tukey match the remaining variables

x2

$x_2$ and

y

$y$ to

x1

$x_1$. Write the residuals for these matches as

x2â‹…1

$x_{2â‹\dots 1}$ and

yâ‹…1

$y_{â‹\dots 1}$, respectively: the

â‹…1

${}_{â‹\dots 1}$indicates that

x1

$x_1$ has been “taken out of” the variable.

We can write

y=λ1x1+y⋅1 and x2=λ2x1+x2⋅1.

$y={\mathrm{λ}}_1{x}_1+y_{â‹\dots 1}\text{and}{x}_2={\mathrm{λ}}_2{x}_1+x_{2â‹\dots 1}.$

Having taken

x1

$x_1$ out of

x2

$x_2$ and

y

$y$, we proceed to match the target residuals

yâ‹…1

$y_{â‹\dots 1}$ to the matcher residuals

x2â‹…1

$x_{2â‹\dots 1}$. The final residuals are

yâ‹…12

$y_{â‹\dots 12}$. Algebraically, we have written

y⋅1y=λ3x2⋅1+y⋅12; whence=λ1x1+y⋅1=λ1x1+λ3x2⋅1+y⋅12=λ1x1+λ3(x2−λ2x1)+y⋅12=(λ1−λ3λ2)x1+λ3x2+y⋅12.

$\begin{array}{cc}{y}_{â‹\dots 1}& ={\mathrm{λ}}_3{x}_{2â‹\dots 1}+y_{â‹\dots 12};\text{whence}\\ y& ={\mathrm{λ}}_1{x}_1+y_{â‹\dots 1}={\mathrm{λ}}_1{x}_1+{\mathrm{λ}}_3{x}_{2â‹\dots 1}+y_{â‹\dots 12}={\mathrm{λ}}_1{x}_1+{\mathrm{λ}}_3\left(x_2−{\mathrm{λ}}_2{x}_1\right)+y_{â‹\dots 12}\\ & =\left({\mathrm{λ}}_1−{\mathrm{λ}}_3{\mathrm{λ}}_2\right)x_1+{\mathrm{λ}}_3{x}_2+y_{â‹\dots 12}.\end{array}$

This shows that the

λ3

${\mathrm{λ}}_3$ in the last step is the coefficient of

x2

$x_2$ in a matching of

x1

$x_1$ and

x2

$x_2$ to

y

$y$.

We could just as well have proceeded by first taking

x2

$x_2$ out of

x1

$x_1$ and

y

$y$, producing

x1â‹…2

$x_{1â‹\dots 2}$ and

yâ‹…2

$y_{â‹\dots 2}$, and then taking

x1â‹…2

$x_{1â‹\dots 2}$ out of

yâ‹…2

$y_{â‹\dots 2}$, yielding a different set of residuals

yâ‹…21

$y_{â‹\dots 21}$. This time, the coefficient of

x1

$x_1$ found in the last step–let’s call it

μ3

${\mathrm{μ}}_3$–is the coefficient of

x1

$x_1$ in a matching of

x1

$x_1$ and

x2

$x_2$ to

y

$y$.

Finally, for comparison, we might run a multiple (ordinary least squares regression) of

y

$y$ against

x1

$x_1$and

x2

$x_2$. Let those residuals be

yâ‹…lm

$y_{â‹\dots lm}$. It turns out that the coefficients in this multiple regression are precisely the coefficients

μ3

${\mathrm{μ}}_3$ and

λ3

${\mathrm{λ}}_3$ found previously and that all three sets of residuals,

yâ‹…12

$y_{â‹\dots 12}$,

yâ‹…21

$y_{â‹\dots 21}$, and

yâ‹…lm

$y_{â‹\dots lm}$, are identical.