Physics 380, 2010: Fourier Analysis
Lectures 11, 12
Homework (due Oct 8)
- In class, we discussed the rate-distortion theorems, and in particular its Information Bottleneck form (Tishby et al, 2000), which, for a given distribution of a signal and the relevance variable , , tries to compress into , while preserving as much information in as possible about . That is, the method maximizes over , and the parameter controls the tradeoff between the strength of the compression and the amount of information retained about . Let's explore this approach numerically.
- Download the following program bottle2.m, which takes on its input the joint distribution of and , the suggested dimensionality of , and the value of . The routine outputs and .
- You will also need auxiliary functions llog2.m and unzero.m. Download them and place them into the same folder/directory as bottle2.m
- Recall the homework problem from the last week and, as , take a distribution that allows 1, 5, 10, 20 molecules with an equal probability of 0.25. Use the code you wrote the last time to generate , where is obtained by Poisson smearing of .
- Run the Information bottleneck algorithm and plot vs. for different values of and for sizes of of 2, 3, 4, 5. How close do we come to recovering all of information about in ? That is, how close is to 1? How different are the plots for different sizes of ? Discuss.
- Calculate Fourier coefficients for the following three functions defined for . (a) ; (b) ; (c). Plot the first ten partial sums of the Fourier series for all of these functions to observe how the sums converge. Use one figure per function (that is, plot the ten sums on the same figure). Note that the Fourier coefficients decrease the slowest for the function that is discontinuous, (a), and the fastest for the function that is smooth, (c).
Homework (due Oct 15)
Let's play a bit today with Fourier transforms
- Consider the following Langevin differential equation that we discussed in class: , where is a Wiener process, that is, it is a Gaussian variable with , .
- Write simple program that would solve this equation as a difference equation. That is, following our discussion in one of the previous homeworks, for a given , and the temporal step size , we can define , and then , where is a Gaussian random variable with zero mean and unit variance. This then gives , which can be turned into a simple for-loop code for simulating a sequence of x's.
- Simulate steps of this dynamics of . Plot . Describe what you see.
- Take a Fourier transform of this using the Matlab build-in function.
- Plot (in log-log scale) the power spectrum (that is .
- Using the expressions we derived in class, show that the power spectrum for this should look like . Compare this to your plot. Do they look similar? Think about why or why not. (Hint -- they will be similar for about half of your plot).
- For grad students only. Can you also calculate the power spectrum for that satisfies the following equations: , where all 's are positive, and are uncorrelated noises in the two variables. Are there any special features in the power spectrum. How do you think might look like given this spectrum?
For these problems, we wrote the following functions:
- JCondJ.m -- takes a conditional distribution and the joint distribution as an input and produces the joint on the output.
- FSerVal.m -- takes the vector of time, the zeros cosine coefficient, and the rest of the cosine and the sine coefficients and produces the values of the corresponding partial sums.