# 2.5 Practical entropy coding techniques

This module introduces practical entropy coding techniques, such as Huffman Coding, Run-length Coding (RLC) and Arithmetic Coding.

In the module of Use of Laplacian PDFs in Image Compression we have assumed that ideal entropy coding has been used in order to calculate the bitrates for the coded data. In practise we must use real codes and we shall now see how this affects the compression performance.

There are three main techniques for achieving entropy coding:

• Huffman Coding - one of the simplest variable length coding schemes.
• Run-length Coding (RLC) - very useful for binary data containing long runs of ones of zeros.
• Arithmetic Coding - a relatively new variable length coding scheme that can combine the best features ofHuffman and run-length coding, and also adapt to data with non-stationary statistics.
We shall concentrate on the Huffman and RLC methods for simplicity. Interested readers may find out more aboutArithmetic Coding in chapters 12 and 13 of the JPEG Book.

First we consider the change in compression performance if simple Huffman Coding is used to code the subimages of the4-level Haar transform.

The calculation of entropy in this equation from our discussion of entropy assumed that each message with probability ${p}_{i}$ could be represented by a word of length ${}_{i}=-\log_{2}{p}_{i}$ bits. Huffman codes require the ${}_{i}$ to be integers and assume that the ${p}_{i}$ are adjusted to become:

$\stackrel{^}{{p}_{i}}=2^{-{}_{i}}$
where the ${}_{i}$ are integers, chosen subject to the constraint that $\sum \stackrel{^}{{p}_{i}}\le 1$ (to guarantee that sufficient uniquely decodable code words are available) and such that the mean Huffman word length(Huffman entropy), $\stackrel{^}{H}=\sum {p}_{i}{}_{i}$ , is minimised.

We can use the probability histograms which generated the entropy plots in figures of level 1 energies , level 2 energies , level 3 energies and level 4 energies to calculate the Huffman entropies $\stackrel{^}{H}$ for each subimage and compare these with the true entropies to see the loss in performance caused by using realHuffman codes.

An algorithm for finding the optimum codesizes ${}_{i}$ is recommended in the JPEG specification [ the JPEG Book , Appendix A, Annex K.2, fig K.1]; and a Mathlab M-file toimplement it is given in M-file code .

Numerical results used in the figure - entropies and bit rates of subimages for qstep=15
Column: 1 2 3 4 5 6 -
0.0264 0.0265 0.0264 0.0266
0.0220 0.0222 0.0221 0.0221 Level 4
0.0186 0.0187 0.0185 0.0186
0.0171 0.0172 0.0171 0.0173 -
0.0706 0.0713 0.0701 0.0705
0.0556 0.0561 0.0557 0.0560 Level 3
3.7106 3.7676 0.0476 0.0482 0.0466 0.0471 -
0.1872 0.1897 0.1785 0.1796
0.1389 0.1413 0.1340 0.1353 Level 2
0.1096 0.1170 0.1038 0.1048 -
0.4269 0.4566 0.3739 0.3762
0.2886 0.3634 0.2691 0.2702 Level 1
0.2012 0.3143 0.1819 0.1828 -
Totals: 3.7106 3.7676 1.6103 1.8425 1.4977 1.5071

shows the results of applying this algorithm to the probability histograms and lists the same results numerically for ease of analysis. Columns 1 and 2 compare theideal entropy with the mean word length or bit rate from using a Huffman code (the Huffman entropy) for the case of theuntransformed image where the original pels are quantized with ${Q}_{\mathrm{step}}=15$ . We see that the increase in bit rate from using the real code is: $\frac{3.7676}{3.7106}-1=1.5%$ But when we do the same for the 4-level transformed subimages, we get columns 3 and 4. Here we see thatreal Huffman codes require an increase in bit rate of: $\frac{1.8425}{1.6103}-1=14.4%$ Comparing the results for each subimage in columns 3 and 4, wesee that most of the increase in bit rate arises in the three level-1 subimages at the bottom of the columns. This is becauseeach of the probability histograms for these subimages (see figure ) contain one probability that is greater than 0.5. Huffman codes cannot allocate a word length ofless than 1 bit to a given event, and so they start to lose efficiency rapidly when $-\log_{2}{p}_{i}$ becomes less than 1, ie when ${p}_{i}> 0.5$ .

where we get a research paper on Nano chemistry....?
what are the products of Nano chemistry?
There are lots of products of nano chemistry... Like nano coatings.....carbon fiber.. And lots of others..
learn
Even nanotechnology is pretty much all about chemistry... Its the chemistry on quantum or atomic level
learn
da
no nanotechnology is also a part of physics and maths it requires angle formulas and some pressure regarding concepts
Bhagvanji
hey
Giriraj
Preparation and Applications of Nanomaterial for Drug Delivery
revolt
da
Application of nanotechnology in medicine
what is variations in raman spectra for nanomaterials
I only see partial conversation and what's the question here!
what about nanotechnology for water purification
please someone correct me if I'm wrong but I think one can use nanoparticles, specially silver nanoparticles for water treatment.
Damian
yes that's correct
Professor
I think
Professor
Nasa has use it in the 60's, copper as water purification in the moon travel.
Alexandre
nanocopper obvius
Alexandre
what is the stm
is there industrial application of fullrenes. What is the method to prepare fullrene on large scale.?
Rafiq
industrial application...? mmm I think on the medical side as drug carrier, but you should go deeper on your research, I may be wrong
Damian
How we are making nano material?
what is a peer
What is meant by 'nano scale'?
What is STMs full form?
LITNING
scanning tunneling microscope
Sahil
how nano science is used for hydrophobicity
Santosh
Do u think that Graphene and Fullrene fiber can be used to make Air Plane body structure the lightest and strongest. Rafiq
Rafiq
what is differents between GO and RGO?
Mahi
what is simplest way to understand the applications of nano robots used to detect the cancer affected cell of human body.? How this robot is carried to required site of body cell.? what will be the carrier material and how can be detected that correct delivery of drug is done Rafiq
Rafiq
if virus is killing to make ARTIFICIAL DNA OF GRAPHENE FOR KILLED THE VIRUS .THIS IS OUR ASSUMPTION
Anam
analytical skills graphene is prepared to kill any type viruses .
Anam
Any one who tell me about Preparation and application of Nanomaterial for drug Delivery
Hafiz
what is Nano technology ?
write examples of Nano molecule?
Bob
The nanotechnology is as new science, to scale nanometric
brayan
nanotechnology is the study, desing, synthesis, manipulation and application of materials and functional systems through control of matter at nanoscale
Damian
Is there any normative that regulates the use of silver nanoparticles?
what king of growth are you checking .?
Renato
What fields keep nano created devices from performing or assimulating ? Magnetic fields ? Are do they assimilate ?
why we need to study biomolecules, molecular biology in nanotechnology?
?
Kyle
yes I'm doing my masters in nanotechnology, we are being studying all these domains as well..
why?
what school?
Kyle
biomolecules are e building blocks of every organics and inorganic materials.
Joe
Got questions? Join the online conversation and get instant answers!