# Physics 434, 2016: Homework 9

Back to the main Teaching page.

Please turn on the assignment either as a PDF file to me by email, or as a printout/writeup to my mailbox in physics. Detailed derivations (with explanations) and calculations must be present for the problems for full credit. Submit your code if a problem requires you to program.

3. How much information can a spiking neuron transmit? This is limited from above by its entropy rate. Let's represent a neuron as releasing action potentials with a Poisson process with a certain rate $r$ , and let's calculate the entropy rate of the Poisson process. First represent this process by discretizing time in intervals $\Delta t$ . Explain why the entropy of the Poisson generated sequence of duration $T$ (or, alternatively, $n=T/\Delta t$ symbols) is exactly proportional to time, that is $S=sn$ , where $s$ is some constant. Thus we only need to calculate the entropy of a single symbol, this $s$ , in order to find the entropy rate as $R={\frac {sT}{\Delta tT}}$ . Does this rate have a finite value as $\Delta t\to 0$ ? Why or why not? Estimate the maximum bitrate of a neuron that can control placement of its spikes to the accuracy of 1 ms.
3. How much information can a spiking neuron transmit? This is limited from above by its entropy rate. Let's represent a neuron as releasing action potentials with a Poisson process with a certain rate $r$ , and let's calculate the entropy rate of the Poisson process. First represent this process by discretizing time in intervals $\Delta t$ . Explain why the entropy of the Poisson generated sequence of duration $T$ (or, alternatively, $n=T/\Delta t$ symbols) is exactly proportional to time, that is $S=sn$ , where $s$ is some constant. Thus we only need to calculate the entropy of a single symbol, this $s$ , in order to find the entropy rate as $R={\frac {sT}{\Delta tT}}$ . Does this rate have a finite value as $\Delta t\to 0$ ? Why or why not? Estimate the maximum bitrate of a neuron that can control placement of its spikes to the accuracy of 1 ms.
4. Suppose now the neuron has what's called a refractory period. That is, after a spike, a neuron cannot fire for the time $\tau _{r}$ . What is the entropy rate of such neuron?