# 4.17 Non-parametric model evaluation

 Page 1 / 1

If very uncertain about model accuracy, assuming a form for the nominal density may be questionable orquantifying the degree of uncertainty may be unreasonable. In these cases, any formula for the underlying probability densities may be unjustified, but the model evaluationproblem remains. For example, we may want to determine a signal presence or absence in an array output (non-zero mean vs. zeromean) without much knowledge of the contaminating noise. If minimal assumptions can be made about the probability densities, non-parametric model evaluation can be used ( Gibson and Melsa ). In this theoretical framework, no formula for the conditional densities isrequired; instead, we use worst-case densities which conform to the weak problem specification. Because few assumptions about theprobability models are used, non-parametric decision rules are robust: they are insensitive to modeling assumptions because sofew are used. The "robust" test of the previous section are so-named because they explicitly encapsulate model imprecision. Ineither case, one should expect greater performance (smaller error probabilities) in non-parametric decision rules than possible froma "robust" one.

Two hypothesized models are to be tested; ${}_{0}$ is intended to describe the situation "the observed data have zero mean" and the other "a non-zero mean is present."We make the usual assumption that the $L$ observed data values are statistically independent. The only assumption we will make about the probabilistic descriptions underlying these models is that the median of the observationsis zero in the first instance and non-zero in the second. The median of a random variable is the "half-way" value: the probability that the random variable is less than themedian is one-half as is the probability that it is greater. The median and mean of a random variable are not necessarily equal;for the special case of a symmetric probability density they are. In any case, the non-parametric models will be stated interms of the probability that an observation is greater than zero. ${}_{0}:({r}_{l}\ge 0)=\frac{1}{2}$ ${}_{1}:({r}_{l}\ge 0)> \frac{1}{2}$ The first model is equivalent to a zero-median model for the data; the second implies that the median is greater thanzero. Note that the form of the two underlying probability densities need not be the same to correspond to the two models;they can differ in more general ways than in their means.

To solve this model evaluation problem, we seek (as do all robust techniques) the worst-case density, the densitysatisfying the conditions for one model that is maximally difficult to distinguish from a given density under theother. Several interesting problems arise in this approach. First of all, we seek a non-parametric answer: thesolution must not depend on unstated parameters (we should not have to specify how large the non-zero mean might be). Secondly,the model evaluation rule must not depend on the form for the given density. These seemingly impossible properties are easilysatisfied. To find the worst-case density, first define ${p}^{+}({r}_{l}, {}_{1}, {r}_{l})$ to be the probability density of the ${l}^{\mathrm{th}}$ observation assuming that ${}_{1}$ is true and that the observation was non-negative. A similar definition for negative values isneeded. ${p}^{+}({r}_{l}, {}_{1}, {r}_{l})=p({r}_{l}, {}_{1}, {r}_{l}\ge 0, {r}_{l})$ ${p}^{-}({r}_{l}, {}_{1}, {r}_{l})=p({r}_{l}, {}_{1}, {r}_{l}< 0, {r}_{l})$ In terms of these quantities, the conditional density of an observation under ${}_{1}$ is given by $p({r}_{l}, {}_{1}, {r}_{l})=({}_{1}, {r}_{l}\ge 0){p}^{+}({r}_{l}, {}_{1}, {r}_{l})-1{p}^{-}({r}_{l}, {}_{1}, {r}_{l})$ The worst-case density under ${}_{0}$ would have exactly the same functional form as this one for positive and negative valueswhile having a zero median.

Don't forget that the worst-case density in model evaluation agrees with thegiven one over as large a range as possible.
As depicted in , a density meeting these requirements is $p({r}_{l}, {}_{1}, {r}_{l})=\frac{{p}^{+}({r}_{l}, {}_{1}, {r}_{l})+{p}^{-}({r}_{l}, {}_{1}, {r}_{l})}{2}$

The likelihood ratio for a single observation would be $2({}_{1}, {r}_{l}\ge 0)$ for non-negative values and $2(1-({}_{1}, {r}_{l}\ge 0))$ for negative values. While the likelihood ratio depends on $({}_{1}, {r}_{l}\ge 0)$ , which is not specified in out non-parametric model, the sufficient statistic will not depend on it! To see this, note that the likelihood ratio varies only withthe sign of the observation. Hence, the optimal decision rule amounts to counting how many of the observations are positive;this count can be succinctly expressed with the unit-step function $u()$ as $\sum_{l=0}^{L-1} u({r}_{l})$ .

We define the unit-step function as $u(x)=\begin{cases}1 & \text{if x> 0}\\ 0 & \text{if x< 0}\end{cases}$ , with the value at the origin undefined. We presume that the densities have no mass at the origin under eithermodel. Although appearing unusual, $\sum u({r}_{l})$ does indeed yield the number of positively values observations.
Thus, the likelihood ratio for the $L$ statistically independent observations is written $2^{L}({}_{1}, {r}_{l}\ge 0)^{\sum u({r}_{l})}(1-({}_{1}, {r}_{l}\ge 0))^{(L-\sum u({r}_{l}))}\underset{{}_{0}}{\overset{{}_{1}}{}}$ Making the usual simplifications, the unknown probability $({}_{1}, {r}_{l}\ge 0)$ can be maneuvered to the right side and merged with the threshold. The optimal non-parametric decision rule thuscompares the sufficient statistic - the count of positive-valued observations - with a threshold determined by the Neyman-Pearsoncriterion.
$\sum_{l=0}^{L-1} u({r}_{l})\underset{{}_{0}}{\overset{{}_{1}}{}}$
This decision rule is called the sign test as it depends only on the signs of the observed data. The sign test isuniformly most powerful and robust.

To find the threshold  , we can use the Central Limit Theorem to approximate theprobability distribution of the sum by a Gaussian. Under ${}_{0}$ , the expected value of $u({r}_{l})$ is $\frac{1}{2}$ and the variance is $\frac{1}{4}$ . To the degree that the Central Limit Theorem reflects the false-alarm probability (see this problem ), ${P}_{F}$ is approximately given by ${P}_{F}=Q(\frac{-\frac{L}{2}}{\sqrt{\frac{L}{4}}})$ and the threshold is found to be $=\frac{\sqrt{L}}{2}Q({P}_{F})^{(-1)}+\frac{L}{2}$ As it makes no sense for the threshold to be greater than $L$ (how many positively values observations can there be?), the specified false-alarmprobability must satisfy ${P}_{F}\ge Q(\sqrt{L})$ . This restriction means that increasing stringent requirements on the false-alarm probability can only be met ifwe have sufficient data.

#### Questions & Answers

what us maxima and minima
MiXUP Reply
Maxima s below equilibrium. Whilst minima s above. Equilibrium
Afran
Wht is demand
Afran
is the willingness and the ability of a consumer to purchase goods at a given price and at a particular point in time.
Assan
Ohhk different question? Ask
Afran
why is the demand curve downwards sloppy?
Assan
3 Reasons.. 1... diminishing marginal utility 2... substitution effect 3...income effect
Harshita
thanks
Assan
Because of the negative or inverse relationship between price and quantity demanded
Afran
what is the law of diminishing returns states?
Assan
ohk
Assan
The law states that all other things being equall as much of variable factor(labour) is employed on fixed factor(land) the marginal product rises..attain a maximum and begins to fall.
Afran
What is income elasticity of demand
Afran
what is monetary policy
Edward
Monetary policy is an attempt to influence the economy by opera ting in such monetary variables
Afran
thanks
Edward
Wlcm
Afran
Wht is disutility?
Afran
is disutility? is rightly writing?
Yhlas
is it i wanna say
Yhlas
Yes please Disutility
Afran
what is macro economics?
Oyas
the branch of economics concerned with large-scale or general economic factors, such as interest rates and national productivity.
idk
in other words it is the study of the economic as a whole
idk
What is an Economic growth
Ayumo Reply
Economic growth is the process whereby the real per capita income of an economy increases over a long period of time.
Nureni
what is the generally accepted defination of economics and by who
IDY Reply
Economics is defined by Lionel Robbins as a social science which studies human behaviour as a relationship between ends and scarce means which have alternative uses
Tba
Importance of economic
Achike Reply
Helps in decision making
MP
I need like 5 importance
Achike
hi
Physcal
Hey
Muhammad
hellow dear.
juwel
hello
Al-ameen
Hello
MP
it helps an individual in rational decision making process
Assan
Fine and u
Buzabaryaho
how does it make individual in rational dicision making decisions
Annor
if an individual is faced with unlimited wants.
Assan
it also helps an individual in arranging their wants in order of their importance.
Assan
ohk
Annor
ok
Al-ameen
Hello guys
Radah
My name is Radah
Radah
Please what is a scale of preference used for?
Radah
it's use for arranging wants in order of their importance.
Assan
in other words when an individual is faced with unlimited wants,scale of preference would help the individual to select the most important wants.
Assan
Thanks
Radah
welcome
Assan
what is tourism
forgive Reply
Tourism is travel for pleasure or business
Yusuf
It is the commercial organization and operation of holidays and visits to places of interest.
Nureni
who is a price taker?
sam Reply
A price taker is a person or a company who have no control to dictate a prices of a goods or services
Unique
Someone who sets price
Nureni
In the trading world, a price taker is a trader who does not affect the price of the stock if he or she buys or sells shares.
Nureni
A price taker refers to a firm or an individual who sets the price of his good and services based on an external factor. In other words he cannot choose and set a price by himself. An example is a firm operating in perfect competition where prices are set through the price mechanism.
Tba
in a common and suitable sense state the law of diminishing returns
Unique Reply
The higher the satisfaction derived from a particular commodity,the lower the demand for it but that law doesn't match in some instances.
Nureni
state the features of an imperfect competitive market
Naomi
@NURENI instance like wat
Unique
imperfect competitive market involves large number of sellers and buyers price makers selling cost product differentiation free entry and exit of a firms
Unique
is economics a science
Alex Reply
yes. a social science.
Carlos
Yes of cause It uses scientific principles in its research. That is to say, analyzing data, making experiment as well as making deductions and drawing conclusions
Aziz
U can understand the scientific nature of economics by learning about the methods used by Abhijit Banerjee(indian) ,the nobel prize laureate 2019.
Harshita
it is considered as a social science
idk
Hence, economics is a science, a social science many can call it, or more appropriately, a young science
Taha
it can be called social science because of behaviour ,which is unpredictable.There r many theroies in economics which make economics a social science But some economic theories makes it science
Harshita
human behaviour*
Harshita
remember science derives from the root words "to know". With that being said most fields of study can be considered as a science or soft science, for they possess key knowledge to attaining understanding of our world.
Alexander
economics is a science cos it deals with human wants, desire or neads in order to satisfy them
Unique
according comparision of political science economic is science.
Hassan
what's the question?
Nureni Reply
Discuss economics system
Henry Reply
discuss institutional system
Henry
Give 3 at most advantages and disadvantages of economics system and institutional syatem
Henry
Give the features characteristics of market or free enterprise
Henry
The structure of an economy is largely determined by the economic system which is a function of the economic ideology of the nation
Nureni
The economic system is grouped into 5 groups: 1: Pure market 2: Developed market 3: Centrally planned or Socialist 4: Mixed market and 5: Market Socialist Economic systems
Nureni
what is inflation
Prince Reply
Inflation is a sustained and general rise in the price of all goods and services of an economy
Tba
hello everyone , I'm New here, third degree price discrimination?
Saeed Reply
2nd degree price discrimination?
Saeed
hi
Kini
hi
Mitchel
Hi
MP
price paid by consumers after the sales tax is called?
Pinias Reply
why government impose price floor on certain products?
Pinias
how can black market be occurred when price ceiling is introduced?
Pinias
How can inflation affect goods and services?
Ph
When prices rise for energy, food, commodities, and other goods and services, the entire economy is affected
Joan
If inflation becomes too high the economy can suffer conversely, if inflation is controlled and at reasonable levels, the economy may prosper. With controlled, lower inflation, employment increases.
Joan
Is it necessary to make decision when it fails you
Evelin Reply
Pls when what fails u
MP
I think so
Kini
well i might naught know what you on about but i gotta tell you, it is necessary
Troy
yep
Ibe
Kk
MP
yep
Ibe
how can the demand side approach solve unemployment
Tshepiso Reply
demand solves unemployment when it is addressed with supply you can't just expect demand to work alone without supply the two are interconnected
Nureni
You have to apply the concept of aggregate demand
Tba
That is apply demand side policies to boost aggregate demand hence increasing need for labour and decreasing unemployment(more people get jobs)
Tba
Difference between extinct and extici spicies
Amanpreet Reply
Researchers demonstrated that the hippocampus functions in memory processing by creating lesions in the hippocampi of rats, which resulted in ________.
Mapo Reply
The formulation of new memories is sometimes called ________, and the process of bringing up old memories is called ________.
Mapo Reply
Got questions? Join the online conversation and get instant answers!
Jobilize.com Reply

### Read also:

#### Get the best Algebra and trigonometry course in your pocket!

Source:  OpenStax, Statistical signal processing. OpenStax CNX. Dec 05, 2011 Download for free at http://cnx.org/content/col11382/1.1
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Statistical signal processing' conversation and receive update notifications?

 By By By By