As mentioned in lectures, entropy is measure of how much information content (“surprise”) is present in a system.  

Given a set of N symbols, and the probability of each symbol occurring, we can compute the entropy (in bits) as:  

{{/Labs/01/Images/entropy.svg}} 

where p<sub>i</sub> is the probability of encountering a given symbol. A worked example of computing entropy was given in lectures. For another example, consider a system with five symbols: A; B; C; D; and E, each occurring with probabilities: 0.0625; 0.25; 0.5; 0.0625; and 0.125. The entropy of this system is 1.875 bits, the computation of which is outlined in the following table:  

<table>
  <tr>
    <th>Symbol</th>
    <th>i</th>
	 <th>p<sub>i</sub></th>
	 <th>log<sub>2</sub>p<sub>i</sub></th>
	 <th>-pi x log<sub>2</sub>p<sub>i</sub></th>
  </tr>
  <tr>
    <td>A</td><td>1</td><td>0.2</td><td>-2.322</td><td>0.464</td>
  </tr>
    <tr>
    <td>B</td><td>2</td><td>0.1</td><td>-3.322</td><td>0.332</td>
  </tr>
  <tr>
    <td>C</td><td>3</td><td>0.3</td><td>-1.737</td><td>0.521</td>
  </tr>
    <tr>
    <td>D</td><td>4</td><td>0.3</td><td>-1.737</td><td>0.521</td>
  </tr>
  <tr>
    <td>E</td><td>5</td><td>0.1</td><td>-3.322</td><td>0.332</td>
  </tr>
  <tr>
    <td></td><td></td><td></td><td></td><td>s=2.171</td>
  </tr>
</table>
### Exercise  
Use the table below to help you compute the entropy of a system with five symbols (A, B, C, D, E) with the probabilities 0.0625, 0.25, 0.5, 0.0625 and 0.125 (respectively):  
Note:There is an editable worksheet doc on Blackboard you can use for your workings in this lab.
<table>
  <tr>
    <th>Symbol</th>    <th>i</th>	 <th>p<sub>i</sub></th>	 <th>log<sub>2</sub>p<sub>i</sub></th>	 <th>-p<sub>i</sub> x log<sub>2</sub>p<sub>i</sub></th>
  </tr>
  <tr>
    <td>A</td><td>1</td><td>0.0625</td><td></td><td></td>
  </tr>
    <tr>
    <td>B</td><td>2</td><td>0.25</td><td></td><td></td>
  </tr>
  <tr>
    <td>C</td><td>3</td><td>0.5</td><td></td><td></td>
  </tr>
    <tr>
    <td>D</td><td>4</td><td>0.0625</td><td></td><td></td>
  </tr>
  <tr>
    <td>E</td><td>5</td><td>0.125</td><td></td><td></td>
  </tr>
  <tr>
    <td></td><td></td><td></td><td></td><td>s= ?</td>
  </tr>
</table>