# 2.1 Machine learning lecture 2 course notes  (Page 3/6)

 Page 3 / 6
$\begin{array}{ccc}\hfill \Phi & =& \frac{1}{m}\sum _{i=1}^{m}1\left\{{y}^{\left(i\right)}=1\right\}\hfill \\ \hfill {\mu }_{0}& =& \frac{{\sum }_{i=1}^{m}1\left\{{y}^{\left(i\right)}=0\right\}{x}^{\left(i\right)}}{{\sum }_{i=1}^{m}1\left\{{y}^{\left(i\right)}=0\right\}}\hfill \\ \hfill {\mu }_{1}& =& \frac{{\sum }_{i=1}^{m}1\left\{{y}^{\left(i\right)}=1\right\}{x}^{\left(i\right)}}{{\sum }_{i=1}^{m}1\left\{{y}^{\left(i\right)}=1\right\}}\hfill \\ \hfill \Sigma & =& \frac{1}{m}\sum _{i=1}^{m}\left({x}^{\left(i\right)}-{\mu }_{{y}^{\left(i\right)}}\right){\left({x}^{\left(i\right)}-{\mu }_{{y}^{\left(i\right)}}\right)}^{T}.\hfill \end{array}$

Pictorially, what the algorithm is doing can be seen in as follows:

Shown in the figure are the training set, as well as the contours of the two Gaussian distributions that have been fit to the data in each of thetwo classes. Note that the two Gaussians have contours that are the same shape and orientation, since they share a covariance matrix $\Sigma$ , but they have different means ${\mu }_{0}$ and ${\mu }_{1}$ . Also shown in the figure is the straight line giving the decision boundary at which $p\left(y=1|x\right)=0.5$ . On one side of the boundary, we'll predict $y=1$ to be the most likely outcome, and on the other side, we'll predict $y=0$ .

## Discussion: gda and logistic regression

The GDA model has an interesting relationship to logistic regression. If we view the quantity $p\left(y=1|x;\Phi ,{\mu }_{0},{\mu }_{1},\Sigma \right)$ as a function of $x$ , we'll find that it can be expressed in the form

$p\left(y=1|x;\Phi ,\Sigma ,{\mu }_{0},{\mu }_{1}\right)=\frac{1}{1+exp\left(-{\theta }^{T}x\right)},$

where $\theta$ is some appropriate function of $\Phi ,\Sigma ,{\mu }_{0},{\mu }_{1}$ . This uses the convention of redefining the ${x}^{\left(i\right)}$ 's on the right-hand-side to be $n+1$ -dimensional vectors by adding the extra coordinate ${x}_{0}^{\left(i\right)}=1$ ; see problem set 1. This is exactly the form that logistic regression—a discriminative algorithm—used to model $p\left(y=1|x\right)$ .

When would we prefer one model over another? GDA and logistic regression will, in general, give different decision boundaries when trained on the same dataset. Which is better?

We just argued that if $p\left(x|y\right)$ is multivariate gaussian (with shared $\Sigma$ ), then $p\left(y|x\right)$ necessarily follows a logistic function. The converse, however, is not true; i.e., $p\left(y|x\right)$ being a logistic function does not imply $p\left(x|y\right)$ is multivariate gaussian. This shows that GDA makes stronger modeling assumptions about the data than does logistic regression. It turns out that when these modelingassumptions are correct, then GDA will find better fits to the data, and is a better model. Specifically, when $p\left(x|y\right)$ is indeed gaussian (with shared $\Sigma$ ), then GDA is asymptotically efficient . Informally, this means that in the limit of very large training sets (large $m$ ), there is no algorithm that is strictly better than GDA (in terms of, say, how accurately they estimate $p\left(y|x\right)$ ). In particular, it can be shown that in this setting, GDA will be a better algorithm than logistic regression; and more generally,even for small training set sizes, we would generally expect GDA to better.

In contrast, by making significantly weaker assumptions, logistic regression is also more robust and less sensitive to incorrect modeling assumptions. There are many different sets of assumptions that would lead to $p\left(y|x\right)$ taking the form of a logistic function. For example, if $x|y=0\sim \mathrm{Poisson}\left({\lambda }_{0}\right)$ , and $x|y=1\sim \mathrm{Poisson}\left({\lambda }_{1}\right)$ , then $p\left(y|x\right)$ will be logistic. Logistic regression will also work well on Poisson data like this. But if we were to use GDA on such data—and fit Gaussian distributions tosuch non-Gaussian data—then the results will be less predictable, and GDA may (or may not) do well.

To summarize: GDA makes stronger modeling assumptions, and is more data efficient (i.e., requires less training data to learn “well”)when the modeling assumptions are correct or at least approximately correct. Logistic regression makes weaker assumptions, and is significantly more robust to deviationsfrom modeling assumptions. Specifically, when the data is indeed non-Gaussian, then in the limit of large datasets, logistic regression will almost always do better thanGDA. For this reason, in practice logistic regression is used more often than GDA. (Some related considerations about discriminative vs. generative models also apply forthe Naive Bayes algorithm that we discuss next, but the Naive Bayes algorithm is still considered a very good, and is certainly also a very popular, classification algorithm.)

what is psychology
Study of behavior and mental processes
Koketso
hi koketso
Chukwu
psychology is the study of mind and behaviour
psychology is a scientific study of human & animal behaviour & their cognitive processes
Arnika
what types of sleep disorders
Hello everybody! I am Ann from Russia. I would like to discuss this and text topic to improve my knowledge in psychology and my English.
hello, I'm Olamide from Nigeria.
la
Hello:)
Anna
Are you psychologist or student or somebody else?)
Anna
something else
Yato
здравствуйте Анна! меня зовут Joshua, what would you like to know?As someone who's lived in South Africa,UK, and Australia I have a хорошо understanding of English :)
Universally
Oh
Abba
hi
Precious
Hello again
Anna
hello :) I'm Yasmine and I'm from Tunisia. i have a bachelor's degree in psychology
Yasmine
This app unfortunately work slow(
Anna
wow, I'm from tunisia too but I'm still a student
Faten
what is the phsicology behind female players? i know that male players don't have a close relationship with their feather, so they observe closer their mother, understanding better what females want, having ease with them, but i cant think of the circumstances a woman has to have to become a player
Yato
Faten that's really cool! where do you study? FSHST?
Yasmine
no, in ISSHT
Faten
it's near FSHST you know right?
Faten
Tunisia, I was there
Anna
Which field of psychology is popular in Tunisia?
Anna
@psychologist_bia
Anna
you can write me
Anna
if you want :)
Anna
For me more interesting -psychology of development
Anna
Anna
@faten yes! i studied in FSHST :) but i still haven't finished my master's. it's in my long term plans
Yasmine
amazing Yasmine so you have a license?
Faten
Yasmine
it's my first year so I'm still a beginner hhhhhh
Faten
congratulations btw
Faten
it's my first year so I'm still a beginner hhhhhhh
Faten
You all are psychology students?
Hamna
@yasmine yes I am actually enjoying it. it's super interesting!
Faten
oooo
Precious
i'm currently finishing my 2nd year of my degree :')
Precious
I'm just in high school :3
Hamna
cool! you're planning to take psyc as ur major?
Precious
I don't know yet. I'm just interested in it for now.
Hamna
nice! it certainly is very interesting :')
Precious
:) do u plan on being a psychologist?
Hamna
Hello guys, this is student from highschool. If anyone does live in America, have you guys ever took AP Psychology? I want to find some students or teachers who took this class.
Oh, sorry for inconvenience. I putted in the wrong chat. Sorry!
Hello everyone, I'm nyakuiy from the youngest African country which is south Sudan I'm just from hight school but as I will be entering university later this year my first choice is psychology
Nyakuei
Anna, what exact exploration expecting.. raise doubts and get the answers in psychology.
dcs
mr.king, if want to join or get the pdf, you must complete phd.
dcs
oh, I see but first I'm taking Rpm.
King
hi, i'm Adams from nigeria i likt to be part of this improvement
hello Adams, I'm also from Nigeria studying psychology
la
Mr. Nyakuei, all the best for entering in psychology.
dcs
good choice that is...
dcs
@Mr.King, do check ur mail inbox
Smitha
Hello everybody, I'm sabah from Algeria. I'm psychologist and student in the same time.
Sabah
hello
Jwala
what is mood
mood is an affective state. In contrast to emotions or feelings, moods are less specific, less intense and less likely to be provoked or instantiated by a particular stimulus or event
la
Why some woman are looking up and down why you approach her?
I don't know
GOPAL
It could be a sign of either lack of interest or maybe they are nervous.
Rachna
if a girl don't look directly in your eyes while talking then it is 80 percent sure that she has crush on you
Sanket
or she is scared. fear creates an avoidant-type relationship. as does embarrassment. when a woman is so bold to look you in the eye, or other visual signs of her attention, she is performing a feminine courtship
but all this is also confirmed or denied by cultural background. in some countries women would be frowned on for approaching the man in any form. some societies hold women in highest regard, and thus their judgment is considered an honor.
not looking in the eye, is a sign if a female going through the development of a crush on the person OR that her past experiences and trauma compels her to avoid eye-contact. usually females do act nervous around the men they like but once there is an establishment of comfortness and trust
Tiasha
using myself as an example: if im interested, you will know it by my focus, my curiosity, my desire to know more. much like an interrogation i suppose. every question would ripple through you demanding an answer. then progressing onward. if im not interested, i would find some way out
looking into the eyes is just a matter of time
Tiasha
It is some sort of inhibition and/or trait of famininity.
dcs
if the woman is not interested in you in any way, that is when best she'll look up straight to your eyeballs, but if a woman is actually looking up and down while you ask her out or on a date, believe me, she never believed you in person can ask her out, maybe she feels you are too big to ask her ou
Olaleye
and if you are dressed in a way that is different to the onlooker, of course they are going to examine you. First to discover the apparent trend you are wearing, second to determine if you are foreign or mentally compromised. wink at them and blow a kiss next time. you may get a smile or rejection
if a woman does this to the man she is already dating, it relates to socioeconomical status concerns. "does my man look the part?" "will he make me look foolish among my peers?"
or simply "is he wearing the shoes i recommended?"
its not only happened to women, mens also perceive. mostly shy people feels this often. when shy people makes conversation with opposite genders, the main thinking of their is mostly there deal with opposite gender, its leads to shy and discomfort and makes them avoided straight look
gokul
the eyes are considered soul-bearing. to stay disconnected from sharing that part of yourself says many things, usually fear or discomfort. perhaps inferiority complex. or simply shyness as mentioned by gokul
May be she is Practical Minded. Gold Digger ? Because those type people Frequently look Top to Bottom to Analyse a Persons worth. The Best way to Judge is 1) She Holds an iPhone (just to look Posh) 2) She has a Cringy Behavior While Talking. (doesn't have a cultured attitude)
Mushtaque
if those two parameters match in any Girl. Then Money Speaks to her. Money in sense any form (Gift's / Phone's / Jewellery etc) Check Wisely and Approach Anyone Buddy. Take Care.
Mushtaque
very good Mushtaque. Her arrogant behavior or desire for flashy and valuable items would be signs of this. How would you handle this type of person if you were caught on conversation?
Being a Psychologist, I would look at her as a Subject and Have Conversation and know better about her and her intentions and why and what made her become like that. Hopefully not a One Day Task. In General as a Human, I would just Ignore.
Mushtaque
Mushtaque don't you think a psychologist is not supposed to judge a person? As far as I can see from your above messages you have already come to a conclusion that these are going to be her intentions. As a psychologist, should not you know that humans are complex and every individual differs?
Alisha
In this way you're Judging Psychologist's that they Cant Judge. hmmm. well, ofcourse they are Complex , but Traits are identifiable.
Mushtaque
a part of psychology is on human behavior which as well all know is variable. so, being a psychologist do you think an individual can be confined to first impressions?
Alisha
being psychologist is about being open minded and looking not judging as no individual is the same plus it's about digging deeper into the reasons behind the actions of certain individuals
Rhian
could not be put into better words, rhian.
Alisha
Agree with you Thomas.
Mushtaque
hi 👋
kim
Application Error, Shown Rhian name as Rhian Thomas. previous comment was for Rhian.
Mushtaque
kim
Hi Kim Liza. May I know What Kind of Help you are seeking ?
Mushtaque
I have a talk in humanistic psychology I don't know what to do
kim
i seems not to understand your question please, can you enlighten us on it so as to contribute or proffer solution to it
Olaleye
you have to make a presentation that talks about humanistic psychology
kim
kim
can u help me
kim
Humanistic Psychology: It is a psychological perspective that emphasizes the study of the whole person. Humanistic psychologists look at human behavior not only through the eyes of the observer, but through the eyes of the person doing the behaving.
Olaleye
Proponents of Humanistic Psychology Abraham Maslow Co-Founder of Humanistic Psychology with Carl Rogers Believed we are all biologically driven to achieve self- actualization Came up with Hierarchy of Needs theory that is most important humanistic theory.
Olaleye
thank you so much
kim
Proponents of Humanistic Psychology Abraham Maslow  Co-Founder of Humanistic Psychology with Carl Rogers  Believed we are all biologically driven to achieve self- actualization  Came up with Hierarchy of Needs theory that is most important humanistic theory.
Olaleye
Maslow is known for his Hierarchy of Needs • Maslow proposed 5 levels of need: from basic to more advanced, they are: – Physiological (food, water, sex) – Safety (security, order, stability – Belongingness and love – Esteem (from self and others) – Self-actualization • The more bas
Olaleye
According to Carl Rogers • Important issues must be defined by the individual: importance of an autonomous self: self-insight is what predicts later behavior. • Actualizing tendency: people tend to develop in a positive direction, toward self-actualization
Olaleye
• The way we perceive our environment determines how it affects us. (Roger's theory emphasizes phenomenology) • Incongruence between our self-concept and our experience leads to pathology
Olaleye
In Rogerian therapy, the client • Moves from talking about externals to talking about personal issues • Moves from talking about the past to dealing with the present • Becomes closer to and more comfortable with his/her feelings
Olaleye
• Becomes more open to new experiences, experiences more freedom
Olaleye
The Fully Functioning Person, for Rogers • Is aware of all experiences • Lives in the moment • Trusts in his/her self • Experiences freedom in choices • Is creative and adapts well • Is still growing and experiencing difficulties
Olaleye
i hope you will find this helpful
Olaleye
yeaaa thank you so much
kim
hey Guy's
Joseph
what's the difference between hallucinations and being crazy?
Joseph
they are both abnormalities
Olaleye
false perception in the absence of stimulus is hallucination.
dcs
crazy is a regular word using in daily life, which is passoinated towards objects or creatures.
dcs
Is/did anyone study/ing forensic psychology
AUDREIGH
oshin
thanks everyone
Gimba
Acquisition Skill is the stage during which a new response is learned and gradually strengthened. For instance, Hitting moving objects, Horse riding, car driving etc,.
dcs
start a new thread for acquisition of skills
sumitra
what is the last stage of experimental method
the thing which are used to verify hypotheses in experimental method it is called
sawera
I'm sure if you went to settings you would be able to change the language.
Demi
안 데여. 한번도 못 찾았어요.
Lex
그런데 여기서 미국 사람들이 우리가 영화 쓰는 거는 그렇게 시킨 거야.
Lex
내가 뭐 물어 볼까 봐 그냥 이거 뭐라고
Lex
it's not speaking nonsense cuz you know it is basically nonsense the machine doesn't really hear what I'm saying cuz I can speak English as well. easier to take direction when you say Jolly Good, Tally Ho. God save the Queen for me.Pro Deo et Patriae And other snazzy cliches.Irrumatus Manducare US
Lex
anglo wasp POS
Lex
cave dwelling trogledite.
Lex
forgive my spelling.
Lex
what r u doing
ANGELICA
what's actually going on here?
Demi
ok
ANGELICA
take it easy
King
are you guys studying or working
King
I'm studying, what are you?
Demi
im studying
ANGELICA
are you enjoying it angelica?
Demi
this year, I'm going to take Rpm
King
hoping to pass the exam
King
studying what?
King
I'm studying clinical. good luck with your exams!
Demi
Thanks Demi.
King
hi
Sharavana
hello., how are you
King
I wonder why this app don't have information about statistics
King
that's a very good point. I've found the app very helpful for my introduction into my new career path so I'm more than great full
Demi
I hope to find app like yours
King
this subject I need to review: Psychological Assessment, Theories of Personality, Abnormal Psychology and Industrial Psychology.., I need to refresh my mind even though I'm a fresh Graduate of Bs Psychology.
King
food in classical condition is called
dream anylisis is method of
dude
dream anylisis is method of
dude
psychoanalysis
Kaiser
food is unconditioned stimulus
gokul
it's a necessity
Anthony
its unconditional stimulus
Puja
Unconditional Stimulus
dcs
conditioned stimulus
Laura
classical conditioning
can I get the information about the advantages of ECG in biopsychology
Sheeba
what do you call that moment right before the "emotion (s)" take place? that blink of an eye flash because that is how I I experience being a person with schizophrenia...?
at childhood, focus is upon what?
It shld be on their developmental stages!! Oliver. From cognitive to kinesthetic, intrapersonal to interpersonal, fine motor to gross motor, n strong reasoning skills etc.
Smitha
Check the works of Jean Piaget n Erick Erickson's stages of developments, will be helpful.
Smitha
Child develops in all the biopsychosocial ways, and all the developments at that time are crucial. From biological perspective, neural development and sensory and motor abilities psychological perspective: various cognitive processes develop Social aspect focus on development of various relations.
Pranav
define knowledge and compare it with learning
Is learning a cognitive behaviour?
Its the ability to act instinctly
lidsay
is there motivation in cognitive thinking and behaviour?
Got questions? Join the online conversation and get instant answers!