Assume $\boldsymbol{y}=\boldsymbol{\iota}\beta_1+\boldsymbol{x}\beta_2+\boldsymbol{u}$ where $\boldsymbol\iota$ is the n-vector of ones and $\{u_i\}$ are i.i.d. with $E(u_i)=0$ and $E(u_i^2)=\sigma^2$. Now, assume that $\boldsymbol{x’x}/n\to c>0$ and $\boldsymbol{\iota’x}/n\to 0$ as $n\to\infty.$ Suppose there is an estimator $\hat\gamma$ (independent from $\boldsymbol{u}$) for the ratio of the coefficients $\gamma=\beta_1/\beta_2$, and it follows that $$ \sqrt{n}(\hat\gamma-\gamma)\overset{A}{\sim}\mathcal{N}(0,\lambda^2). $$ Define $$ \hat\beta_2=\frac{(\boldsymbol{x}+\boldsymbol{\iota}\hat\gamma)’\boldsymbol{y}}{(\boldsymbol{x}+\boldsymbol{\iota}\hat\gamma)'(\boldsymbol{x}+\boldsymbol{\iota}\hat\gamma)}. $$ What […]

I put this question here as it has a pure maths element to it, even though it has a statistical twist. Basically, I have been given the table of data: $$\begin{matrix} i & \mathrm{Response} \, y_i & \mathrm{Covariate} \, x_i \\ 1 & -1 & -0.5 \\ 2 & -0.25 &-0.25 \\ 3 & 1 […]

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial derivatives (although I might want to do it wituout partials if it’s easier). I’ve posted my attempt below. I don’t know what […]

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the whole equation linear. Is it possible to use least square to solve it with a constant too ( […]

Why use the kernel trick in a support vector machine as opposed to just transforming the data and then using a linear classifier? Certainly, we’ll approximately double the amount of memory required to hold the data, original plus transformed, but beyond that it seems like the amount of computation remains about the same. What are […]

I understand the derivation for $\hat{x}=A^Tb(A^TA)^{-1}$, but I’m having trouble explicitly connecting it to least squares regression. So suppose we have a system of equations: $A=\begin{bmatrix}1 & 1\\1 & 2\\1 &3\end{bmatrix}, x=\begin{bmatrix}C\\D\end{bmatrix}, b=\begin{bmatrix}1\\2\\2\end{bmatrix}$ Using $\hat{x}=A^Tb(A^TA)^{-1}$, we know that $D=\frac{1}{2}, C=\frac{2}{3}$. But this is also equivalent to minimizing the sum of squares: $e^2_1+e^2_2+e^2_3 = (C+D-1)^2+(C+2D-2)^2+(C+3D-2)^2$. I […]

I’m currently in a course learning about neural networks and machine learning, and I came across these two formulas in this textbook page on linear regression: 1) $y(x) = a + bx$ and 2) $y(x) = w^{T}\phi(x)$ What is the difference between these two formulas? How are they related? They seem to perform the same […]

I have a noisy data set (the grey line in the graph below) that corresponds roughly to $y=m(1-2^{-x/k})$ where m and k are unknown constants. How can I determine the best-fit value of m and k? I can get an approximate value for k by guessing m and then doing linear regression on $-\log_2(1-y/m)$… by […]

I am learning machine learning from Andrew Ng’s open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost function for linear regression and then get to my question with logistic regression. (Btw a similar question was asked here, which answers […]

This question How to work out orthogonal polynomials for regression model and the answer https://math.stackexchange.com/a/354807/51020 explain how to build orthogonal polynomials for regression. However they only consider one dimensional functions. How can we use (discrete) orthogonal polynomials for regression with two dimensional functions (i.e., $z = f(x, y)$)?

Intereting Posts

generic regularity of affine varieties
Prove that $N(\gamma) = 1$ if, and only if, $\gamma$ is a unit in the ring $\mathbb{Z}$
Sequence problem involving inequalities
Evaluating $\int_0^\infty \sin x^2\, dx$ with real methods?
Memorizing the unit circle?
Nilradical and Jacobson's radical.
symmetric positive definite matrix question
What numbers are integrally represented by $4 x^2 + 2 x y + 7 y^2 – z^3$
Locally bounded Family
Finding expected number of distinct values selected from a set of integers
How do I understand $e^i$ which is so common?
Counterexample to “ a closed ball in M is a closed subset.”
Maximisation problem
An integral for $2\pi+e-9$
Determine the Winding Numbers of the Chinese Unicom Symbol