Self-Information Consider once again the expression of mutual information over a joint ensemble: ; 2

Self-Information Consider once again the expression of mutual information over a joint ensemble: ; 2 ( ) ( ; ) log ( ) X Y k j XY k j X k P ab I ab P a ? ? = bits, 1.3.9 Now, if ( ) P ab X Y? k j ? =1 i.e., if the occurrence of the event y=bj certainly specifies the outcome of x = ak we see that, ; 1 ( ; ) ( ) log ( ) XY k j X k X k I ab I a P a = = 2 bit 1.3.10 This is defined as the self-information of the event x = ak. It is always non-negative. So, we say that self-information is a special case of mutual information over a joint ensemble.