# 2.3 Machine learning lecture 4 course notes  (Page 5/5)

 Page 5 / 5

Let's put all this together into a theorem.

Theorem. Let $|\mathcal{H}|=k$ , and let any $m,\delta$ be fixed. Then with probability at least $1-\delta$ , we have that

$\epsilon \left(\stackrel{^}{h}\right)\le \left(\underset{h\in \mathcal{H}}{min},\epsilon ,\left(h\right)\right)+2\sqrt{\frac{1}{2m}log\frac{2k}{\delta }}.$

This is proved by letting $\gamma$ equal the $\sqrt{·}$ term, using our previous argument that uniform convergence occurs with probability at least $1-\delta$ , and then noting that uniform convergence implies $\epsilon \left(h\right)$ is at most $2\gamma$ higher than $\epsilon \left({h}^{*}\right)={min}_{h\in \mathcal{H}}\epsilon \left(h\right)$ (as we showed previously).

This also quantifies what we were saying previously saying about the bias/variance tradeoff in model selection. Specifically, suppose we have some hypothesis class $\mathcal{H}$ , and are considering switching to some much larger hypothesis class ${\mathcal{H}}^{\text{'}}\supseteq \mathcal{H}$ . If we switch to ${\mathcal{H}}^{\text{'}}$ , then the first term ${min}_{h}\epsilon \left(h\right)$ can only decrease (since we'd then be taking a min over a larger set of functions). Hence,by learning using a larger hypothesis class, our “bias” can only decrease. However, if k increases, then the second $2\sqrt{·}$ term would also increase. This increase corresponds to our “variance” increasingwhen we use a larger hypothesis class.

By holding $\gamma$ and $\delta$ fixed and solving for $m$ like we did before, we can also obtain the following sample complexity bound:

Corollary. Let $|\mathcal{H}|=k$ , and let any $\delta ,\gamma$ be fixed. Then for $\epsilon \left(\stackrel{^}{h}\right)\le {min}_{h\in \mathcal{H}}\epsilon \left(h\right)+2\gamma$ to hold with probability at least $1-\delta$ , it suffices that

$\begin{array}{ccc}\hfill m& \ge & \frac{1}{2{\gamma }^{2}}log\frac{2k}{\delta }\hfill \\ & =& O\left(\frac{1}{{\gamma }^{2}},log,\frac{k}{\delta }\right),\hfill \end{array}$

## The case of infinite $\mathcal{H}$

We have proved some useful theorems for the case of finite hypothesis classes. But many hypothesis classes, including any parameterized by real numbers(as in linear classification) actually contain an infinite number of functions. Can we prove similar results for this setting?

Let's start by going through something that is not the “right” argument. Better and more general arguments exist , but this will be useful for honing our intuitions about the domain.

Suppose we have an $\mathcal{H}$ that is parameterized by $d$ real numbers. Since we are using a computer to represent real numbers, and IEEE double-precision floating point ( double 's in C) uses 64 bitsto represent a floating point number, this means that our learning algorithm, assuming we're using double-precision floating point, is parameterized by $64d$ bits. Thus, our hypothesis class really consists of at most $k={2}^{64d}$ different hypotheses. From the Corollary at the end of the previous section, we therefore find that, to guarantee $\epsilon \left(\stackrel{^}{h}\right)\le \epsilon \left({h}^{*}\right)+2\gamma$ , with to hold with probability at least $1-\delta$ , it suffices that $m\ge O\left(\frac{1}{{\gamma }^{2}},log,\frac{{2}^{64d}}{\delta }\right)=O\left(\frac{d}{{\gamma }^{2}},log,\frac{1}{\delta }\right)={O}_{\gamma ,\delta }\left(d\right)$ . (The $\gamma ,\delta$ subscripts are to indicate that the last big- $O$ is hiding constants that may depend on $\gamma$ and $\delta$ .) Thus, the number of training examples needed is at most linear in the parameters of the model.

The fact that we relied on 64-bit floating point makes this argument not entirely satisfying, but the conclusion is nonetheless roughly correct: If what we're going to do is try to minimize training error,then in order to learn “well” using a hypothesis class that has $d$ parameters, generally we're going to need on the order of a linear number of training examples in $d$ .

(At this point, it's worth noting that these results were proved for an algorithm that uses empirical risk minimization. Thus, while the linear dependence of samplecomplexity on $d$ does generally hold for most discriminative learning algorithms that try to minimize trainingerror or some approximation to training error, these conclusions do not always apply as readily to discriminative learning algorithms. Giving good theoreticalguarantees on many non-ERM learning algorithms is still an area of active research.)

The other part of our previous argument that's slightly unsatisfying is that it relies on the parameterization of $\mathcal{H}$ . Intuitively, this doesn't seem like it should matter: We had written the classof linear classifiers as ${h}_{\theta }\left(x\right)=1\left\{{\theta }_{0}+{\theta }_{1}{x}_{1}+\cdots {\theta }_{n}{x}_{n}\ge 0\right\}$ , with $n+1$ parameters ${\theta }_{0},...,{\theta }_{n}$ . But it could also be written ${h}_{u,v}\left(x\right)=1\left\{\left({u}_{0}^{2}-{v}_{0}^{2}\right)+\left({u}_{1}^{2}-{v}_{1}^{2}\right){x}_{1}+\cdots \left({u}_{n}^{2}-{v}_{n}^{2}\right){x}_{n}\ge 0\right\}$ with $2n+2$ parameters ${u}_{i},{v}_{i}$ . Yet, both of these are just defining the same $\mathcal{H}$ : The set of linear classifiers in $n$ dimensions.

To derive a more satisfying argument, let's define a few more things.

Given a set $S=\left\{{x}^{\left(i\right)},...,{x}^{\left(d\right)}\right\}$ (no relation to the training set) of points ${x}^{\left(i\right)}\in \mathcal{X}$ , we say that $\mathcal{H}$ shatters $S$ if $\mathcal{H}$ can realize any labeling on $S$ . I.e., if for any set of labels $\left\{{y}^{\left(1\right)},...,{y}^{\left(d\right)}\right\}$ , there existssome $h\in \mathcal{H}$ so that $h\left({x}^{\left(i\right)}\right)={y}^{\left(i\right)}$ for all $i=1,...d$ .

Given a hypothesis class $\mathcal{H}$ , we then define its Vapnik-Chervonenkis dimension , written $\mathrm{VC}\left(\mathcal{H}\right)$ , to be the size of the largest set that is shattered by $\mathcal{H}$ . (If $\mathcal{H}$ can shatter arbitrarily large sets, then $\mathrm{VC}\left(\mathcal{H}\right)=\infty$ .)

For instance, consider the following set of three points:

Can the set $\mathcal{H}$ of linear classifiers in two dimensions ( $h\left(x\right)=1\left\{{\theta }_{0}+{\theta }_{1}{x}_{1}+{\theta }_{2}{x}_{2}\ge 0\right\}$ ) can shatter the set above? The answer is yes. Specifically, we see that, for any of the eight possiblelabelings of these points, we can find a linear classifier that obtains “zero training error” on them:

Moreover, it is possible to show that there is no set of 4 points that this hypothesis class can shatter. Thus, the largest set that $\mathcal{H}$ can shatter is of size 3, and hence $\mathrm{VC}\left(\mathcal{H}\right)=3$ .

Note that the VC dimension of $\mathcal{H}$ here is 3 even though there may be sets of size 3 that it cannot shatter. For instance, if we had a set of three pointslying in a straight line (left figure), then there is no way to find a linear separator for the labeling of the three points shown below (right figure):

In order words, under the definition of the VC dimension, in order to prove that $\mathrm{VC}\left(\mathcal{H}\right)$ is at least $d$ , we need to show only that there's at least one set of size $d$ that $\mathcal{H}$ can shatter.

The following theorem, due to Vapnik, can then be shown. (This is, many would argue, the most important theorem in all of learning theory.)

Theorem. Let $\mathcal{H}$ be given, and let $d=\mathrm{VC}\left(\mathcal{H}\right)$ . Then with probability at least $1-\delta$ , we have that for all $h\in \mathcal{H}$ ,

$|\epsilon \left(h\right)-\stackrel{^}{\epsilon }\left(h\right)|\le O\left(\sqrt{\frac{d}{m}log\frac{m}{d}+\frac{1}{m}log\frac{1}{\delta }}\right).$

Thus, with probability at least $1-\delta$ , we also have that:

$\epsilon \left(\stackrel{^}{h}\right)\le \epsilon \left({h}^{*}\right)+O\left(\sqrt{\frac{d}{m}log\frac{m}{d}+\frac{1}{m}log\frac{1}{\delta }}\right).$

In other words, if a hypothesis class has finite VC dimension, then uniform convergence occurs as $m$ becomes large. As before,this allows us to give a bound on $\epsilon \left(h\right)$ in terms of $\epsilon \left({h}^{*}\right)$ . We also have the following corollary:

Corollary. For $|\epsilon \left(h\right)-\stackrel{^}{\epsilon }\left(h\right)|\le \gamma$ to hold for all $h\in \mathcal{H}$ (and hence $\epsilon \left(\stackrel{^}{h}\right)\le \epsilon \left({h}^{*}\right)+2\gamma$ ) with probability at least $1-\delta$ , it suffices that $m={O}_{\gamma ,\delta }\left(d\right)$ .

In other words, the number of training examples needed to learn “well” using $\mathcal{H}$ is linear in the VC dimension of $\mathcal{H}$ . It turns out that, for “most” hypothesis classes, the VC dimension (assuming a “reasonable” parameterization) is also roughly linear in the number of parameters.Putting these together, we conclude that (for an algorithm that tries to minimize training error) the number of trainingexamples needed is usually roughly linear in the number of parameters of $\mathcal{H}$ .

Hi, I have a question. What factors influence social facilitation and social inhibition in groups?
it really depends on that person on how the can task in life either they can work better with other people or you do worse working with people.
Lametra
Lametra
Lametra
Do you know of any studies done to show social facilitation and social inhibition
Amritpal
social facilitation is often influenced by the person's perception of the situation and their appraisal of the task. for example, a track athlete who views their competition as a challenge rather than a threat, will tend to run the fastest they have ever ran under a large crowd.
Another condition is if the person has mastered the ability to perform that task. A track athlete who trains everyday, will perform above average under a large crowd, however, someone who does not run, will tend to do worse than their actual ability under a large crowd
social inhibition can be explained through evolutionary biology or attachment personality theory. some people develop an avoidance personality trait, which when they feel under pressure, they tend to leave the group or avoid a gathering due to high lev of stress
Does anyone know how I can transfer my higher national diploma credits in usa?
Oh wow. I never realised there was a chat on this app!
Then welcome dearest
Bridget
Thank you! ❤️
Amritpal
thank you
Ana
Hello Ana, good afternoon from here
hello im new student here.
Rai
hello
Rai
I m also student.
Rai
hi
Kavya
what is easier option to psychology?
Dhanaraj
I'm new too
Deniz
As in the the specialisation or ?
Bridget
good morning
Ana
understanding human nature is what we are going to talk about.
Ana
Philip
“ Every child is left to evaluate his experiences for himself, and to take care of his own personal development outside the classroom. There is no tradition for the acquisition of a true knowledge of the human psyche. The science of human nature thus finds itself today in the position that chemis
Ana
well, I think every individual is different and to a great extent we can't say confidently that we understand the human nature in its entirety
Philip
lack determines what we will become in life?.
Ana
Not 100% but it plays a huge role
Philip
Hi! I wasn't aware there was a chat feature on this app 😊
Ophelia
👍Philip Anyaegbu
Ana
cool
Ana
👍
Ana
yes Ana I'm here
Philip
alternatively we are different, we have basic similar qualities
Bridget
Yes true Bridget.
Philip
hello there
Ciel
Hi
Philip
hi
Ej
hi
KUNDAN
hi
Ana
who we are how we think what we do insight
Ana
@ bridget. yes that's true
Ana
Ana ioanidis? do you speak Greek?
Qwanta
Qwanta no I dont speak Greek my husband does
Ana
are you a psychologist? or your husband?
Qwanta
I need some reliable informations about my studies...
Qwanta
not a psychologist
Ana
Ana
so what is the cause why a person experiences a psychological disorder if it not of their own?
it's partly genetics,culture, environment and substance abuse. generally these are the common factors which can lead to most of the disorders
war
waoo
Rai
r u proficcer
Rai
u r psychology subject
Rai
not a professor but a clinical psychologist
war
can i ask you, i keep getting headache, and had a history of hypothyroid ans diagnosed as having mild anxiety disorder. i want to finish. my undergraduate psychology thesis.. but always unable to have good ambition and energy to focus on writing the case and the theories... whats the best remedy
Astaroxche
Ana
297 according to the DSM 5
how many disorders are there
interesting question
Mahmoud
it is😅
Nelly
I'm guessing it's in the tens of thousands, maybe hundreds if you include counter interactions and stuff like it. although I'm sure it's almost impossible to know for sure, unless you're very rich and connected to the right people. but as I said: guessing.
Beenie
There are more than 200 classified forms of mental illness. Some of the more common disorders are: clinical depression, bipolar disorder, dementia, schizophrenia and anxiety disorders. Symptoms may include changes in mood, personality, personal habits and/or social withdrawal. that is what I think
Mahmoud
too difficult to number. diagnosing a disorder is just checking off boxes on a compilation of symptoms that might match any particular condition on the DSM
what is psychology of the guest
I don't understand the question can you elaborate?
Edgar
the study of philosophy gives to the sociologist
why women are viewed as far more emotional than men?
because they actually are ,I guess.
Collins
May be they are biologically milder than man. It does not mean they are not equal with man. Man can also be emotional and can are oppressed with traditional norms. For example, man are not to be cry, in actual man are also emotional being and they cannot have the right to show their sentiment.
shine
sollungal bro sollungal...
Balaji
because of she has hormonal fluctuations than men..
bavi
usually women are more emotional than men and they are multi talented people
lavanya
Both men and women are capable of expressing emotions. But women has the highest percentage of doing that. Because our society is conditioned by nature in such a way that men are expected to suppress their emotions and motivate them through their acts or thoughts, which has it's side effects of....
Santos
denial of any emotion which they feel that useless at that point. Women in the other hand were encouraged/not controlled to suppress their emotions and let them out what they feel about it. I feel that's the wonderful superpower of the women.
Santos
y
Mahmoud
Men's emotion comes mostly with memories triggered by senses. That means, they thoughts have the power to decide whether to let go of emotions or not.
Santos
Q
Mark
because women tend to be more agreeable than men.
Edgar
women at a young age are conditioned to be more in tune with emotion than man should be less
David
The question is "Why women Viewed are as far more emotional than men ?" it's not a question whether women are more emotional than men. This is more an issue about the point of view from the observer, his/her assumption what emotional behavior is or what emotional behavior is.
Mehmet
The anwers are answers more to the question " Are women more emotional than men?"
Mehmet
hi are you on what'sap
no not really a fan of social laziness
Robert
huh
Parmizan
is there any way for a Btech E.C.E. graduate to take on MSC. PSYCHOLOGY
devesh
ya you can just by cracking entrance , if in India
Mansi
Any stream in UG can go for M.Sc (Psychology)
classification of traits and how they are measured
what was Freud's first name?
Sigmund
Dentriodite
sigmund
Margie
Sigismund Schlomo Freud
Sigmund Freud
Talal
sigmund
Ankita
Leon Vygotsky and Sergei Rubenstein please tell me contributions of these personalities in in 4 lines
Uzma
sigmund Freud
Harpreet
Lev Vygotsky was the founder of socio-cultural theory
Jo
So psychology was based off of the Greek gods I that right or no?
psychology is the study. anything -ology is the study of a certain field. Psyche is a mortal woman who becomes divine in Greek mythology. The etymology of the word "psyche" in Greek means "spirit" or "soul"
seriously there is no scope of psychology all over the world.i hear those how have psychology degrees they have no careers
psychology degree is the waste of time qnd waste of money.is this true?...
at least you know people's minds
Mahmoud
hummmmm
Blaq
well, i like this subject for helping myself and others as well, not for the scope or anything. and i think we shouldn't liberate everything on the basis of outcomes etc
Natasha
Ye
Isaiah
very useful skill
Isaiah
It is not at all true that people who have psychology degree have no career scope
Aaprajita
If u complete your M.Phil then you can work in hospitals as a clinical psychologist.You can also work in school, colleges and IT companies as a counsellor or clinical psychologist.
Aaprajita
yes
aravinth
people are so stressful and tense, and they need a psychologist
aravinth
yo fr tbh lol
Isaiah
if some one have only master degree in psychology what will he/she do...where he/she utilizes degree
Not if you have set goals and a plan. Look into what careers are available with your degree where you live. Also it helps to have a specialty along with your psychology degree. For example a Bachelor's degree in Psychology of Science in Addictions allows for you to become an addictions counselor.
Jo
Psychology is really a valuable degree. I would like to become an "EDUCATIONAL PSYCHOLOGIST "
there is no proper society of Psychology. so it doesn't get a good exposure in our country. we along with whole student need to discuss and approach to government
Shekhar
Psychology is not only a diverse field but also one that is expected to grow tremendously in the years to come. As a subject dealing with the study of the human psyche or mind, psychology finds applications across all avenues of life, be it family, work, relationships, sports, corporate spaces
Clinical Psychology As is suggestive, this branch of psychology is associated to the understanding, diagnosis and treatment of psychological disorders in humans. Clinical psychologists help people facing difficulties in their life to get through it using different treatment methods and therapies.
These professionals work with hospitals, NGOs and even drug rehabilitation centers. As people become increasingly aware of different psychological disorders with time, the need for experts in clinical psychology is only expected to grow.
Counseling Psychology Counseling psychology is a branch of psychology that helps solve people’s personal and interpersonal issues. These issues can be problematic but are different from serious mental health issues. Counseling psychologists or counselors help people deal with such issues when they
fail to do it on their own.Career counseling, guidance counseling, marital counseling and rehabilitation counseling are among the many applications of counseling psychology. These professionals either manage their own counseling set-ups or work with therapycenters, career centers, NGOs as well as
schools and universities. Industrial or Organizational Psychology Organizational psychologists are professionals who apply the principles of psychology in organizations or workplaces. They analyze the issues of the workplace at individual as well as organizational level, and work towards resolving
them to enhance the efficiency of the employees. Those specializing in organizational psychology can choose to work as – • Human resource development specialists • HR managers • Organizational consultants With increasing pressure and stress levels at work places, the need for emotional understandi
understanding and gauging social intelligence has become stronger. Child Psychology (Development psychology) Development psychology is a branch of psychology that is dedicated to studying the psychological development of human beings over the course of their lifetime. Child psychology is one of
the more popular variants of development psychology, and deals with emotional, cognitive, social and psychomotor development in infants and children. Child psychologists work with schools, child therapy centers and also NGOs. They also play big roles in special education centers for children.
Sports Psychology This branch of psychology deals with the study of elements that influence an athlete’s performance. These elements or factors could be emotional, cognitive, psychological or motivational. Sports psychologists can go on to work with sports coaching centers, leagues, academies or
sports teams. Sports psychologists aid sportspersons to stay in their best mental forms, thus improving their performances on the ground. With the frenzy regarding sports in India, sports psychology is a promising career to pursue.
Forensic Psychology Forensic psychology offers a unique opportunity to apply psychology to the benefit of legal organizations, especially pertaining to specific contents or witness testimony. Professionals in this field utilize their skills to look deep into the psyche of criminals and figure out
the intent behind the crime. This helps define the quantum and nature of sentence to be rendered. With the onset of new legal era, forensic psychology is expected to influence policy making.
SO,THESE ARE THE CAREER SCOPE IN THE FIELD OF PSYCHOLOGY👥
psychology and the jobs in psychology are the ever growing fields. there are number of jobs available. but, yes, one needs the right qualification, such as, MPhil, PhD, NET, etc.
Paul
No. Psychology has a great scope all over world
Talal
Because of 90% people has stress or anxiety or any other mental problem. So they have a need of Psychologist.
Talal
Got questions? Join the online conversation and get instant answers!