# Physics 434, 2016: Homework 9

Back to the main Teaching page.

Back to Physics 434, 2016: Physical Biology.

Please turn on the assignment either as a PDF file to me by email, or as a printout/writeup to my mailbox in physics. Detailed derivations (with explanations) and calculations must be present for the problems for full credit. Submit your code if a problem requires you to program.

### Undergraduates

- Problem 11.1
- What is the differential entropy of an exponential distribution?
- How much information can a spiking neuron transmit? This is limited from above by its entropy rate. Let's represent a neuron as releasing action potentials with a Poisson process with a certain rate , and let's calculate the entropy rate of the Poisson process. First represent this process by discretizing time in intervals . Explain why the entropy of the Poisson generated sequence of duration (or, alternatively, symbols) is
*exactly*proportional to time, that is , where is some constant. Thus we only need to calculate the entropy of a single symbol, this , in order to find the entropy rate as . Does this rate have a finite value as ? Why or why not? Estimate the maximum bitrate of a neuron that can control placement of its spikes to the accuracy of 1 ms.

### Graduate Students

- Problem 11.3
- What is the differential entropy of an exponential distribution?
- How much information can a spiking neuron transmit? This is limited from above by its entropy rate. Let's represent a neuron as releasing action potentials with a Poisson process with a certain rate , and let's calculate the entropy rate of the Poisson process. First represent this process by discretizing time in intervals . Explain why the entropy of the Poisson generated sequence of duration (or, alternatively, symbols) is
*exactly*proportional to time, that is , where is some constant. Thus we only need to calculate the entropy of a single symbol, this , in order to find the entropy rate as . Does this rate have a finite value as ? Why or why not? Estimate the maximum bitrate of a neuron that can control placement of its spikes to the accuracy of 1 ms. - Suppose now the neuron has what's called a
*refractory period*. That is, after a spike, a neuron cannot fire for the time . What is the entropy rate of such neuron?