# Physics 434, 2016: Homework 9

Back to the main Teaching page.

Please turn on the assignment either as a PDF file to me by email, or as a printout/writeup to my mailbox in physics. Detailed derivations (with explanations) and calculations must be present for the problems for full credit. Submit your code if a problem requires you to program.

3. How much information can a spiking neuron transmit? This is limited from above by its entropy rate. Let's represent a neuron as releasing action potentials with a Poisson process with a certain rate ${\displaystyle r}$, and let's calculate the entropy rate of the Poisson process. First represent this process by discretizing time in intervals ${\displaystyle \Delta t}$. Explain why the entropy of the Poisson generated sequence of duration ${\displaystyle T}$ (or, alternatively, ${\displaystyle n=T/\Delta t}$ symbols) is exactly proportional to time, that is ${\displaystyle S=sn}$, where ${\displaystyle s}$ is some constant. Thus we only need to calculate the entropy of a single symbol, this ${\displaystyle s}$, in order to find the entropy rate as ${\displaystyle R={\frac {sT}{\Delta tT}}}$. Does this rate have a finite value as ${\displaystyle \Delta t\to 0}$? Why or why not? Estimate the maximum bitrate of a neuron that can control placement of its spikes to the accuracy of 1 ms.
3. How much information can a spiking neuron transmit? This is limited from above by its entropy rate. Let's represent a neuron as releasing action potentials with a Poisson process with a certain rate ${\displaystyle r}$, and let's calculate the entropy rate of the Poisson process. First represent this process by discretizing time in intervals ${\displaystyle \Delta t}$. Explain why the entropy of the Poisson generated sequence of duration ${\displaystyle T}$ (or, alternatively, ${\displaystyle n=T/\Delta t}$ symbols) is exactly proportional to time, that is ${\displaystyle S=sn}$, where ${\displaystyle s}$ is some constant. Thus we only need to calculate the entropy of a single symbol, this ${\displaystyle s}$, in order to find the entropy rate as ${\displaystyle R={\frac {sT}{\Delta tT}}}$. Does this rate have a finite value as ${\displaystyle \Delta t\to 0}$? Why or why not? Estimate the maximum bitrate of a neuron that can control placement of its spikes to the accuracy of 1 ms.
4. Suppose now the neuron has what's called a refractory period. That is, after a spike, a neuron cannot fire for the time ${\displaystyle \tau _{r}}$. What is the entropy rate of such neuron?