<< Chapter < Page Chapter >> Page >

Reliability and validity

Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.

Unfortunately, being consistent in measurement does not necessarily mean that you have measured something correctly. To illustrate this concept, consider a kitchen scale that would be used to measure the weight of cereal that you eat in the morning. If the scale is not properly calibrated, it may consistently under- or overestimate the amount of cereal that’s being measured. While the scale is highly reliable in producing consistent results (e.g., the same amount of cereal poured onto the scale produces the same reading each time), those results are incorrect. This is where validity comes into play. Validity refers to the extent to which a given instrument or tool accurately measures what it’s supposed to measure. While any valid measure is by necessity reliable, the reverse is not necessarily true. Researchers strive to use instruments that are both highly reliable and valid.

How valid is the sat?

Standardized tests like the SAT are supposed to measure an individual’s aptitude for a college education, but how reliable and valid are such tests? Research conducted by the College Board suggests that scores on the SAT have high predictive validity for first-year college students’ GPA (Kobrin, Patterson, Shaw, Mattern,&Barbuti, 2008). In this context, predictive validity refers to the test’s ability to effectively predict the GPA of college freshmen. Given that many institutions of higher education require the SAT for admission, this high degree of predictive validity might be comforting.

However, the emphasis placed on SAT scores in college admissions has generated some controversy on a number of fronts. For one, some researchers assert that the SAT is a biased test that places minority students at a disadvantage and unfairly reduces the likelihood of being admitted into a college (Santelices&Wilson, 2010). Additionally, some research has suggested that the predictive validity of the SAT is grossly exaggerated in how well it is able to predict the GPA of first-year college students. In fact, it has been suggested that the SAT’s predictive validity may be overestimated by as much as 150% (Rothstein, 2004). Many institutions of higher education are beginning to consider de-emphasizing the significance of SAT scores in making admission decisions (Rimer, 2008).

In 2014, College Board president David Coleman expressed his awareness of these problems, recognizing that college success is more accurately predicted by high school grades than by SAT scores. To address these concerns, he has called for significant changes to the SAT exam (Lewin, 2014).

Summary

A correlation is described with a correlation coefficient, r , which ranges from -1 to 1. The correlation coefficient tells us about the nature (positive or negative) and the strength of the relationship between two or more variables. Correlations do not tell us anything about causation—regardless of how strong the relationship is between variables. In fact, the only way to demonstrate causation is by conducting an experiment. People often make the mistake of claiming that correlations exist when they really do not.

Researchers can test cause-and-effect hypotheses by conducting experiments. Ideally, experimental participants are randomly selected from the population of interest. Then, the participants are randomly assigned to their respective groups. Sometimes, the researcher and the participants are blind to group membership to prevent their expectations from influencing the results.

In ideal experimental design, the only difference between the experimental and control groups is whether participants are exposed to the experimental manipulation. Each group goes through all phases of the experiment, but each group will experience a different level of the independent variable: the experimental group is exposed to the experimental manipulation, and the control group is not exposed to the experimental manipulation. The researcher then measures the changes that are produced in the dependent variable in each group. Once data is collected from both groups, it is analyzed statistically to determine if there are meaningful differences between the groups.

Psychologists report their research findings in peer-reviewed journal articles. Research published in this format is checked by several other psychologists who serve as a filter separating ideas that are supported by evidence from ideas that are not. Replication has an important role in ensuring the legitimacy of published research. In the long run, only those findings that are capable of being replicated consistently will achieve consensus in the scientific community.

Personal application questions

We all have a tendency to make illusory correlations from time to time. Try to think of an illusory correlation that is held by you, a family member, or a close friend. How do you think this illusory correlation came about and what can be done in the future to combat them?

Got questions? Get instant answers now!

Are there any questions about human or animal behavior that you would really like to answer? Generate a hypothesis and briefly describe how you would conduct an experiment to answer your question.

Got questions? Get instant answers now!

Questions & Answers

how do you get the 2/50
Abba Reply
number of sport play by 50 student construct discrete data
Aminu Reply
width of the frangebany leaves on how to write a introduction
Theresa Reply
Solve the mean of variance
Veronica Reply
Step 1: Find the mean. To find the mean, add up all the scores, then divide them by the number of scores. ... Step 2: Find each score's deviation from the mean. ... Step 3: Square each deviation from the mean. ... Step 4: Find the sum of squares. ... Step 5: Divide the sum of squares by n – 1 or N.
kenneth
what is error
Yakuba Reply
Is mistake done to something
Vutshila
Hy
anas
hy
What is the life teble
anas
hy
Jibrin
statistics is the analyzing of data
Tajudeen Reply
what is statics?
Zelalem Reply
how do you calculate mean
Gloria Reply
diveving the sum if all values
Shaynaynay
let A1,A2 and A3 events be independent,show that (A1)^c, (A2)^c and (A3)^c are independent?
Fisaye Reply
what is statistics
Akhisani Reply
data collected all over the world
Shaynaynay
construct a less than and more than table
Imad Reply
The sample of 16 students is taken. The average age in the sample was 22 years with astandard deviation of 6 years. Construct a 95% confidence interval for the age of the population.
Aschalew Reply
Bhartdarshan' is an internet-based travel agency wherein customer can see videos of the cities they plant to visit. The number of hits daily is a normally distributed random variable with a mean of 10,000 and a standard deviation of 2,400 a. what is the probability of getting more than 12,000 hits? b. what is the probability of getting fewer than 9,000 hits?
Akshay Reply
Bhartdarshan'is an internet-based travel agency wherein customer can see videos of the cities they plan to visit. The number of hits daily is a normally distributed random variable with a mean of 10,000 and a standard deviation of 2,400. a. What is the probability of getting more than 12,000 hits
Akshay
1
Bright
Sorry i want to learn more about this question
Bright
Someone help
Bright
a= 0.20233 b=0.3384
Sufiyan
a
Shaynaynay
How do I interpret level of significance?
Mohd Reply
It depends on your business problem or in Machine Learning you could use ROC- AUC cruve to decide the threshold value
Shivam
how skewness and kurtosis are used in statistics
Owen Reply
yes what is it
Taneeya
Got questions? Join the online conversation and get instant answers!
Jobilize.com Reply

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Psychology. OpenStax CNX. Feb 03, 2015 Download for free at https://legacy.cnx.org/content/col11629/1.5
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Psychology' conversation and receive update notifications?

Ask