<< Chapter < Page Chapter >> Page >

Most of the wide variety of classifiers performed within an order of magnitude of each other, except for one called “Hyperpipes" which performed two orders of magnitude better than any of the others. This struck Dr. Padley as very strange as he had never heard of the Hyperpipes algorithm before.

Current work - elec 631 fall 2011

One interesting aspect of the behavior of the Hyperpipes algorithm on the dataset was the very low true positive rate: less than 10% of the t-tbar events were correctly classified as such. However, the false positive rate was much, much lower – this is what accounted for its good performance. At this point the author became suspicious that the default parameters for the WEKA machine learning algorithms were poorly suited to this application and that some of the other algorithms could also demonstrate greatly improved performance with a little tuning. Since logistic regression is relatively simple, and since the author now understands a little about how the algorithm works from the on-line Stanford course, the decision was made to try and tune it to this application.

Cost matrix

The first stage of tuning involved adjusting the cost matrix. This followed from the observation that when training a classifier, we really want to penalize false positives much more harshly than false negatives. Sacrificing up to 50% of our true positives is acceptable if it leads to a very large decrease in background false positive events and thus an improved signal-to-noise ratio. Keeping the true positive rate relatively high is still important as t-tbar events are rare enough that keeping enough for statistical significance is still a problem, as we shall see a bit later.

A variety of cost matrices were created and the logistic regression classifier in WEKA was trained on a training data set with each one of them. The resulting classification models were then tested on a cross-validation data set. Both the training data set and the cross-validation set had 10,000 of each type of event as before. The results are summarized in Table TODO. Note that as expected, increasing the cost of false positives relative to other kinds of errors greatly improved the signal to noise ratio with an acceptable decrease in the true positive rate. Note that the cost matrices are properly scaled by WEKA: only the relative costs of the different types of errors represented by the cost matrix matter. Interestingly, eliminating the costs associated with misclassifying W background events as QCD background events and vice-versa seems to have no effect on the signal-to-noise ratio.

Choosing a cost matrix which penalizes each type of false positive 300 times more than other types of errors seemed to give good results without leading to diminishing returns in the form of decreased true positives. Penalizing false positives caused by QCD events more than W events didn't improve the results on the large test set.

Ratio of signal to background in training set

Dr. Devika Subramanian of the Rice Computer Science department suggested that classifiers need to be trained on data that has each type of event to be classified in a ratio that is roughly the same as that in the real test data. This suggestion indicated that the previous strategy of training classifiers with an equal ratio of each type of event was unsound. Unfortunately, generating training and test sets with the correct ratio of event types is quite difficult because of the huge backgrounds that must be generated, in this case several million QCD events for every t-tbar event. Generating enough background for 100 t-tbar events would take a modern cpu core over 1000 days to compute enough background.

Questions & Answers

what is the difference between demand and supply
Peter Reply
what is the national income
Kamara Reply
oils and resources
What's current account?
Che Reply
Demand refers to goods and services that the buyer is willing and able to buy at a price over a period of time
Can it be possible to have two level of comparative advantage in a country ?
Louise Reply
.no.its not possible
Why ?
I think no possible
why do oligopoly increase on the elastic segment of the demand curve
Tintswalo Reply
what is all about production possibility curve
Nice Reply
help me about the assumption of possibility curve
-The quantity and quality of economic resources are fixed. -only two types of goods can be produce out of this resources (that is,producer and consumer goods). -Resources are fully utilised. -The resources are mobile. -The state of technology is constant.
What is utility
chisom Reply
what is the meaning of money and inflation
Tinuke Reply
Money can be define as anything acceptable as a medium of exchange and mean of payment
inflation is when everything seemed to cost so much less
what is a bar chart
Godwin Reply
what's economic
John Reply
Economics can be define as a study of how human beings make decisions in the face of scarcity it can also be define as using one wealth to make more wealth
Or in Nigerian way Economics is a science (social science) which studies human behavior as a relationship between Ends and Scarce which have alternative uses
Economics is the study of how human make decision in the face of scarcity
what is the meaning of imperfect market structure
Hoyindamolah Reply
the theroy of demand
what is market economy
Jusu Reply
Market Economy:Is a system where the laws of supply and those demand direct the production of goods and services.
what is the meaning of money and inflation
what's demand in economics?
Abi Reply
Demand in economic is the good a consumer is willing or able to purchase at a particular time
explanation of graph,bar chart,pie chart with examples
4. Assume, after completing your economics class, you explain your friend that about 65% of GDP is spending on consumption. Your friend tells you that people are greedy and it is better for GDP if they spend on services or experiences. What would be your answer to your friend?
Javohir Reply
What is power
Ahmad Reply
Got questions? Join the online conversation and get instant answers!
Jobilize.com Reply

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now

Source:  OpenStax, Introductory survey and applications of machine learning methods. OpenStax CNX. Dec 22, 2011 Download for free at http://legacy.cnx.org/content/col11400/1.1
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Introductory survey and applications of machine learning methods' conversation and receive update notifications?