-
Home
- Fundamentals of electrical
- Information communication
- Entropy
Shannon showed the power of probabilistic models for symbolic-valued signals. The dey quantity that characterizes such a signal is the entropy of its alphabet.
Communication theory has been formulated best for
symbolic-valued signals. ClaudeShannon published in 1948
The Mathematical Theory
of Communication , which became the cornerstone of digital
communication. He showed the power of
probabilistic
models for symbolic-valued signals, which allowed him to
quantify the information present in a signal. In the simplestsignal model, each symbol can occur at index
with a probability
,
. What this model says is that for each signal value a
-sided coin is flipped (note that
the coin need not be fair). For this model to make sense, theprobabilities must be numbers between zero and one and must sum
to one.
This coin-flipping model assumes that symbols occur without
regard to what preceding or succeeding symbols were, a falseassumption for typed text. Despite this probabilistic
model's over-simplicity, the ideas we develop here alsowork when more accurate, but still probabilistic, models are
used. The key quantity that characterizes a symbolic-valuedsignal is the
entropy of its alphabet.
Because we use the base-2 logarithm, entropy has units of
bits. For this definition to make sense, we must take specialnote of symbols having probability zero of occurring. A
zero-probability symbol never occurs; thus, we define
so that such symbols do not affect the entropy. The
maximum value attainable by an alphabet's entropy occurswhen the symbols are equally likely
(
). In this case, the entropy equals
. The minimum value occurs when only one symbol
occurs; it has probability one of occurring and the rest haveprobability zero.
Derive the maximum-entropy results, both the
numeric aspect (entropy equals
) and the theoretical one (equally likely symbols
maximize entropy). Derive the value of the minimum entropyalphabet.
Equally likely symbols each have a probability of
. Thus,
. To prove that this is the maximum-entropy
probability assignment, we must explicitly take into accountthat probabilities sum to one. Focus on a particular
symbol, say the first.
appears
twice in the entropy
formula: the terms
and
. The derivative with respect to this probability
(and all the others) must be zero. The derivative equals
, and all other derivatives have the same form
(just substitute your letter's index). Thus, eachprobability must equal the others, and we are done. For the
minimum entropy answer, one term is
, and the others are
, which we define to be zero also. The minimum
value of entropy is zero.
Got questions? Get instant answers now!
A four-symbol alphabet has the following probabilities.
Note that these probabilities sum to one as they should. As
,
. The
entropy of this alphabet equals
Got questions? Get instant answers now!
Questions & Answers
how does Neisseria cause meningitis
is the branch of biology that deals with the study of microorganisms.
studies of microbes
Louisiaste
when we takee the specimen which lumbar,spin,
How bacteria create energy to survive?
Bacteria doesn't produce energy they are dependent upon their substrate in case of lack of nutrients they are able to make spores which helps them to sustain in harsh environments
_Adnan
But not all bacteria make spores, l mean Eukaryotic cells have Mitochondria which acts as powerhouse for them, since bacteria don't have it, what is the substitution for it?
Muhamad
they make spores
Louisiaste
what is sporadic nd endemic, epidemic
the significance of food webs for disease transmission
Abreham
food webs brings about an infection as an individual depends on number of diseased foods or carriers dully.
Mark
explain assimilatory nitrate reduction
Assimilatory nitrate reduction is a process that occurs in some microorganisms, such as bacteria and archaea, in which nitrate (NO3-) is reduced to nitrite (NO2-), and then further reduced to ammonia (NH3).
Elkana
This process is called assimilatory nitrate reduction because the nitrogen that is produced is incorporated in the cells of microorganisms where it can be used in the synthesis of amino acids and other nitrogen products
Elkana
Examples of thermophilic organisms
Give Examples of thermophilic organisms
Shu
advantages of normal Flora to the host
Prevent foreign microbes to the host
Abubakar
they provide healthier benefits to their hosts
ayesha
They are friends to host only when Host immune system is strong and become enemies when the host immune system is weakened . very bad relationship!
Mark
cell is the smallest unit of life
Fauziya
cell is the smallest unit of life
Akanni
cell is the structural and functional unit of life
Hasan
is the fundamental units of Life
Musa
what are emergency diseases
There are nothing like emergency disease but there are some common medical emergency which can occur simultaneously like Bleeding,heart attack,Breathing difficulties,severe pain heart stock.Hope you will get my point .Have a nice day ❣️
_Adnan
define infection ,prevention and control
Innocent
I think infection prevention and control is the avoidance of all things we do that gives out break of infections and promotion of health practices that promote life
Lubega
Heyy Lubega hussein where are u from?
_Adnan
which site have a normal flora
Many sites of the body have it
Skin
Nasal cavity
Oral cavity
Gastro intestinal tract
Safaa
skin,Oral,Nasal,GIt
Sadik
How can Commensal can Bacteria change into pathogen?
Sadik
How can Commensal Bacteria change into pathogen?
Sadik
what are the advantages of normal Flora to the host
Micheal
what are the ways of control and prevention of nosocomial infection in the hospital
Micheal
part of a tissue or an organ being wounded or bruised.
Wilfred
what term is used to name and classify microorganisms?
Binomial nomenclature
adeolu
Got questions? Join the online conversation and get instant answers!
Source:
OpenStax, Fundamentals of electrical engineering i. OpenStax CNX. Aug 06, 2008 Download for free at http://legacy.cnx.org/content/col10040/1.9
Google Play and the Google Play logo are trademarks of Google Inc.