



In this module we consider differential entropy.
Consider the entropy of
continuous random variables. Whereas the (normal)
entropy is the entropy of a
discrete random variable, the differential entropy is the entropy of a continuous random variable.
Differential entropy
Differential entropy
 The differential entropy
$h(X)$ of a continuous random variable
$X$ with a pdf
$f(x)$ is defined as
$h(X)=\int_{()} \,d x$∞
∞
f
x
f
x
Usually the logarithm is taken to be base 2, so that the unit of the differential entropy is bits/symbol. Note that is the discrete case,
$h(X)$ depends only on the pdf of
$X$ . Finally, we note that
the differential entropy is the expected value of
$\lg f(x)$ , i.e.,
$h(X)=E(\lg f(x))$
Now, consider a calculating the differential entropy of some random variables.
Consider a normal distributed random variable
$X$ , with mean
$m$ and
variance
$\sigma ^{2}$ .
Then its density is
$\sqrt{\frac{1}{2\pi \sigma ^{2}}}e^{\left(\frac{(xm)^{2}}{2\sigma ^{2}}\right)}$ .
We can then find its differential entropy as follows, first calculate
$\lg f(x)$ :
$\lg f(x)=\frac{1}{2}\lg (2\pi \sigma ^{2})+\lg e()\frac{(xm)^{2}}{2\sigma ^{2}}$
Then since
$E((Xm)^{2})=\sigma ^{2}$ ,
we have
$h(X)=\frac{1}{2}\lg (2\pi \sigma ^{2})+\frac{1}{2}\lg e=\frac{1}{2}\lg (2\pi \times e\sigma ^{2})$
Got questions? Get instant answers now!
Properties of the differential entropy
In the section we list some properties of the differential entropy.
 The differential entropy can be negative

$h(X+c)=h(X)$ , that is translation
does not change the differential entropy.

$h(aX)=h(X)+\lg \lefta\right$ , that is scaling
does change the differential entropy.
The first property is seen from both
and
. The two latter can be shown by using
.
Questions & Answers
List and explain four factors of production
capital
labour
entrepreneur
natural resources
Thembi
when the supply decreases demand also decreases
Thembi
types of demand and the explanation
other things remaining same if demend is increases supply is also decrease and if demend is decrease supply is also increases is called the demand
Mian
if the demand increase supply also increases
Thembi
you are wrong this is the law of demand and not the definition
Tarasum
Demand is the willingness of buy and ability to buy in a specific time period in specific place.
Mian you are saying law of demand but not in proper way. you have to keep studying more. because its very basic things in Economics.
Hamza
what is commercialization?
How to talk loan for bank?
what is the meaning of gpa?
Answer: GPA stands for Grade Point Average. It is a standard way of measuring academic achievement in the U.S. Basically, it goes as follows: Each course is given a certain number of "units" or "credits", depending on the content of the course.
Yusuf
what is small and Microbuisenes
What is fiscal policy
Dansofo
Who is the funder of Economic
Dansofo
founder , that is Adam Smith
Daniel
the wealth of nations, is it the first?
Umar
Yes very sure it was released in 1759
Yusuf
then when did he died?
Umar
17 July 1790
Born: 16 June 1723, Kirkcaldy, United Kingdom
Place of death: Panmure House, Edinburgh, United Kingdom
Yusuf
that's my today questions, thank you Yusuf it's bed time see u after.
Umar
mode is the highest occurring frequency in a distribution
Bola
mode is the most commonly occurring item in a set of data.
Umar
Please, what is the difference between monopoly and monopsony?
is there monopsony word?
Umar
I have no idea though
Umar
please, in which year Adam smith was born?
Umar
monopsony is when there's only one buyer while monopoly is when there's only one producer.
Bola
who have idea on Banter
Ibrahim
like trade by barter?
Bola
Monopoly is when there's excessively one seller and there is no entry in the market while monopsony is when there is one buyer
kemigisha
Adam smith was born in 1723
Bola
(uncountable)
Good humoured, playful, typically spontaneous conversation.
verb
(intransitive) To engage in banter or playful conversation.
(intransitive) To play or do something amusing.
(transitive) To tease mildly.
Umar
which book Adam smith published first? the first book of Adam smith pls.
Umar
wealth on nation, 1776
Daniel
what is market power and how can it affect an economy?
market power: where a firm is said to be a price setter.market power benefits the powerful at the expense of others.
Umar
Market power refers to the ability of a firm (or group of firms) to raise and maintain price above the level that would prevail under competition is referred to as market or monopoly power. The exercise of market power leads to reduced output and loss of economic welfare
Kartheek
find information about the national budget
Molahlegi
three branches of economics in which tourism is likely to figure
What are those three branches?
IlRegno
Got questions? Join the online conversation and get instant answers!
Source:
OpenStax, Information and signal theory. OpenStax CNX. Aug 03, 2006 Download for free at http://legacy.cnx.org/content/col10211/1.19
Google Play and the Google Play logo are trademarks of Google Inc.