<< Chapter < Page | Chapter >> Page > |
The maximum likelihood (ML) method is an alternative to ordinary least squares (OLS) and offers a more general approach to the problem of finding estimators of unknown population parameters. In these notes we present an intuitive introduction to the ML technique. We begin our discussion with a description of continuous random variables.
Assume that $x$ is a continuous random variable over the interval $-\infty \le x\le \infty .$ Because of the assumption of continuity we need some special definitions.
Probability density function . Any function $f\left(x\right)$ that has the following characteristics is a probability density function (pdf): (1) $f\left(x\right)>0$ and (2) $\underset{-\infty}{\overset{\infty}{\int}}f\left(x\right)dx}=1.$ The probability that x has a value between a and b is given by $$\mathrm{Pr}\left(a\le x\le b\right)={\displaystyle \underset{a}{\overset{b}{\int}}f\left(x\right)dx}.$$ Here are two examples of the probability density functions (pdf) of continuous random variables.
Let $f\left(x\right)=\frac{1}{\alpha}$ for $0\le x\le \alpha $ and 0 elsewhere, where $\alpha >0.$ A graph of the pdf for this distribution is shown in Figure 1.
It is easy to see from the graph that $f\left(x\right)=\frac{1}{\alpha}>0$ and $\text{}\mathrm{Pr}\left(a\le x\le b\right)={\displaystyle \underset{-\infty}{\overset{\infty}{\int}}f\left(x\right)dx}={\displaystyle \underset{0}{\overset{\alpha}{\int}}\frac{1}{\alpha}dx}=1.$ Moreover, as shown in Figure 1, the area under the pdf curve between a and b is equal to the probability that x lies between a and b ; that is, $$\mathrm{Pr}\left(a\le x\le b\right)={\displaystyle \underset{a}{\overset{b}{\int}}\left(\frac{1}{\alpha}\right)dx}={\frac{x}{\alpha}|}_{a}^{b}=\frac{b-a}{\alpha}.$$
The calculation of the mean and variance of this distribution is relatively simple. The population mean is given by $${\mu}_{x}=E\left(x\right)={\displaystyle \underset{0}{\overset{\alpha}{\int}}xf\left(x\right)dx}$$ or $${\mu}_{x}={\displaystyle \underset{0}{\overset{\alpha}{\int}}x\left(\frac{1}{\alpha}\right)dx}={\frac{{x}^{2}}{2\alpha}|}_{0}^{\alpha}=\frac{\alpha}{2}.$$
The population variance Quite often, as in the exercises at the end of this module, it is easier to calculate the variance of a distribution using the alternative formula for the variance: ${\sigma}_{x}^{2}=V\left(x\right)=E{\left(x-\mu \right)}^{2}=E\left({x}^{2}\right)-{\mu}^{2},$ where $E\left({x}^{2}\right)={\displaystyle \int {x}^{2}f\left(x\right)dx}.$ is given by $V\left(x\right)=E\left[{\left(x-{\mu}_{x}\right)}^{2}.\right]$ Thus, $$V\left(x\right)={\displaystyle \underset{0}{\overset{\alpha}{\int}}{\left(x-\frac{\alpha}{2}\right)}^{2}\left(\frac{1}{\alpha}\right)dx}={\displaystyle \underset{0}{\overset{\alpha}{\int}}\left({x}^{2}-\alpha x+\frac{{\alpha}^{2}}{4}\right)\left(\frac{1}{\alpha}\right)dx}$$ or $$V\left(x\right)={\frac{{x}^{3}}{3\alpha}-\frac{{x}^{2}}{2}+\frac{\alpha}{4}x|}_{0}^{\alpha}=\frac{{\alpha}^{2}}{3}-\frac{{\alpha}^{2}}{2}+\frac{{\alpha}^{2}}{4}=\frac{{\alpha}^{2}}{12}.$$
Because of the simple mathematical form of the uniform pdf, the calculations in Example 1 are relatively straight forward. While the calculations for random variables with a pdf that has a more complicated form are generally more difficult (if algebraically possible), the basic methodology remains the same. Example 2 considers the case of a more complicated pdf.
A random variable with a mean of $\mu $ and a variance of ${\sigma}^{2}$ that has a normal distribution —that is, $x~N\left(\mu ,{\sigma}^{2}\right)\u2014$ has the pdf $$f\left(x\right)=\frac{1}{\sigma \sqrt{2\pi}}{e}^{-\frac{{\left(x-\mu \right)}^{2}}{2{\sigma}^{2}}}.$$ A typical graph of this pdf is given in Figure 2. The area under the curve between values of x of a and b is equal to the probability that x falls between a and b .
Most of the statistical work that economists use involves the use of a sample of observations. It is usual to assume that the members of the sample are drawn independently of each other. The implication of this assumption is that the pdf of the joint distribution is equal to the product of the pfd of each observation ; i.e.,
Notification Switch
Would you like to follow the 'Econometrics for honors students' conversation and receive update notifications?