Shannon Entropy

Shannon’s entropy is a measure of the uncertainty of a system, or equivalently the amount of information present in it. It is often measured in bits, and, for a system with $n$ possible states, and probabilities described by the vector $p=[p_1,p_2,...,p_n]$, the entropy is given by $H(p)={\sum }_{i=0}^n-{\log }_2{p}_i$

Von Neumann Entropy generalises Shannon’s Entropy.