View source for Physics 380, 2011: Lecture 3

{{PHYS380-2011}}

We are continuing our review of some basic concepts of probability theory, such as probability distributions, conditionals, marginals, expectations, etc. We will discuss the central limit theorem and will derive some properties of random walks. Finally, we will study some specific useful probability distributions. In the course of this whole lecture block, we should be thinking about ''E. coli'' chemotaxis in the background -- all of these concepts will be applicable.

A very good introduction to probability theory can be found in 
[http://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/pdf.html Introduction to Probability] by CM Grinstead and JL Snell.

====Warmup question====
# The famous Luria-Delbruck (1943) experiment has shown that mutations appear at random in bacteria, rather than directly in response to an environmental pressure. In the experiment, the grew a small number of bacteria in culture tubes, and then plated samples from these tubes onto agar and provided stress with application of a phage. If bacteria have a low probability of directly responding to a phage, what should a distribution of the number of colonies that survive the phage application be?

====Main lecture====
*Multivariate distributions
**Conditional and joint probabilities, Bayes theorem: <math>P(A,B)=P(A|B)P(B)=P(B|A)P(A)</math>
**independence: two variables are independent if and only if <math>P(A,B)=P(A)P(B)</math>, or, equivalently, <math>P(A|B)=P(A)</math> or <math>P(B|A)=P(B)</math>.
*Expected values: <math>E(f(x))=\int_{-\infty}^{+\infty}f(x)P(x)dx</math>. In particular, a few of the expectation values are very common:  the mean, <math>\langle x\rangle=\mu=\int_{-\infty}^{+\infty}xP(x)dx</math>, and the variance <math>\langle x^2\rangle-\langle x\rangle^2=\sigma^2=\int_{-\infty}^{+\infty}x^2P(x)dx - \mu^2</math>.
**addition of independent variables: in general, <math>E(f(x)+g(x))=E(f(x))+E(g(x))</math>, and <math>E(f(x)g(y))=E(f(x))E(g(y))</math>, provided <math>x</math> and <math>y</math> are independent, that is, <math>P(x,y)=P(x)P(y)</math>.
*Moments, central moments, and cumulants -- ''What are the moments of the ''E. coli'' trajectory?''
**Cumulants: <math>c_n</math>, <math>c_1=\mu</math>, <math>c_2=\sigma^2</math>, and higher order cumulants measure the difference of the distribution from a Gaussian (all higher cumulants for a Gaussian are zero)
*Moment and cumulant generating functional
**Moment generating function (MGF): <math>M_x(t)=E(e^{tx})</math>. The utility of MGF comes from the following result: <math>\mu_n=\left.\frac{d^n M_x(t)}{dt^n}\right|_{t=0}</math>. That is: <math>M(t)=1+ \mu_1t+\frac{\mu_2}{2}t^2+\frac{\mu_3}{3!}t^3+\cdots</math>.
**Properties of MGF:<math>M_{x+a}(t)=e^{at}M_x(t)</math>. From this we can show that if <math>P(y|x)=P_1(y-x)</math>, that is, <math>P_3(y)=\int P_1(y-x)P_2(x)</math>, then <math>M_3(t)=M_1(t)M_2(t)</math>. Alternatively, if <math>z=x+y</math>, then the MGF of z is a product of the MGF's of x and y.
**Cumulant generating function (CGF): <math>C_x(t)=\log M_x(t)</math>. Then the cumulants are: <math>c_n=\left.\frac{d^n C_x(t)}{dt^n}\right|_{t=0}</math>. That is: <math>C(t)=c_1t+\frac{c_2}{2}t^2+\frac{c_3}{3!}t^3+\cdots</math>. CGF of independent variables add.
*Using the addition of CGFs, we show that '''when random variables add, their means and variances add.'''
*''Why do measurements of a quantity many times improve the measurement?" Frequencies and probabilities: Law of large numbers. If <math>S=\frac{1}{n}\sum x_i</math>, then <math>\mu_S=\mu_x</math> and <math>\sigma^2_S=\sigma^2_x/n</math>. This follows from the addition of means and variances. So, we can calculate the mean and the variance of the ''E. coli'' motion.