Expected Values

The expected value of a random variable \(X\), denoted \(\mathrm{E}[X]\), is its mean. It is computed as the weighted sum over the possible values of \(x\), where the weight for each value is its probability (or density). For discrete \(X\) with probability measure \(P\), we have:

\[\mathrm{E}[X] = \sum_{x \in X} x \mathrm{P}[x]\]

If \(X\) is continuous and has probability density \(p\), we have:

\[\mathrm{E}[X] = \int x p(x) dx\]

If we use the technical definition of a random variable, then we denote:

\[\mathrm{E}_P[f_X] = \int f_X(e) \mathrm{P}[e] de\]

We can also talk about the conditional expectation \(\mathrm{E}[X | A]\), the expected value of \(X\) given that we know event \(A\) happened. It is defined as \(\mathrm{E}[X|A] = \int x p(x|A) dx\).

Variance and Covariance

The variance of a random variable \(X\) is the expected value of its squared deviation from its mean:

\[\mathrm{Var}(X) = \mathrm{E}[(X - \mathrm{E}[X])^2]\]

The standard deviation is the square root of variance (\(\sigma_X = \sqrt{\mathrm{Var}(X)}\)).

The covariance of two random variables is the expected value of the product of their deviations from mean:

\[\mathrm{Cov}(X, Y) = \mathrm{E}[(X - \mathrm{E}[X]) (Y - \mathrm{E}[Y])]\]

The correlation \(r_{XY} = \frac{\mathrm{Cov}(X, Y)}{\sigma_X \sigma_Y}\).

We can also show that \(\mathrm{Var}(X) = \mathrm{Cov}(X, X)\).

Random variables can also be described as independent in the same way as events: knowing one tells you nothing about the other. If two random variables are independent then their covariance \(\mathrm{Cov}(X, Y) = 0\) (this implication is one-directional — there exist non-independent random variables whose covariance is 0).

Properties of Expected Values

Expected value obeys a number of useful properties (\(X\) and \(Y\) are random variables, and \(\alpha\), \(\beta\), etc. are real numbers):

• Linearity of expectation:
  • \(\mathrm{E}[X + Y] = \mathrm{E}[X] + \mathrm{E}[Y]\)
  • \(\mathrm{E}[\alpha X] = \alpha \mathrm{E}[X]\)
• If \(X\) and \(Y\) are independent, then \(\mathrm{E}[XY] = \mathrm{E}[X] \mathrm{E}[Y]\)
• If \(\mathrm{E}[X] = 0\), then \(\mathrm{Var}(X) = \mathrm{E}[X^2]\)
• If \(\mathrm{E}[X] = \mathrm{E}[Y] = 0\), then \(\mathrm{Cov}(X, Y) = \mathrm{E}[X Y]\)

Expectation of Indicator Functions

Sets can be described as an indicator function (or characteristic function) \(\mathbb{I}_A: E \to \{0,1\}\). This function is defined as:

\[\mathbb{I}_A(x) = \begin{cases} 1 & x \in A \\ 0 & x \not\in A \end{cases}\]

Then the expected value of this function is the same as the probability of \(A\):

\[\mathrm{E}[\mathbb{I}_A(X)] = \mathrm{P}[A]\]