Measure each person twice:

Two DVs

Scatterplots

How did we get those graphs (for example for Temperature and Cold Drink Sales) in the previous section? We'll talk about how to do it now. Those kinds of graphs are called scatterplots.

Let's use an example. A researcher is going to make two measurements on each person in a group of N = 6 people. Another way to say that is that there will be two DV's. For the first DV, the researcher will get a self-report rating of how much exercise the person does (from 1, little, to 10, lots). The rating is meant to cover the range from "complete couch potato" to "avid exercise fanatic." That's variable X.

The second DV measurement is a 1 to 10 rating of each person's health. That's Y.

The researcher has two dependent (or criterion) variables. S/he has a sample of N = 6 subjects.

Notice on the lecture graphic that Person #1 has an exercise rating (X) of 9 and a health rating (Y) of 10. Look at the graphic until you can clearly identify person # 1's data. The data for person #1 are two numbers, 9 and 10.

Each subject has two data points. Person #2 was rated as a 1 on exercise and a 3 on health. Person #3 was rated as 9 on X (exercise) and 6 on Y (health). And so forth. To keep the example simple, only six subjects are in the study. Each person is measured on two variables.

(NOTE: In the kind of experimental research we've been doing up to now we generally just measure one aspect of the person, their mental health or whatever our examples happen to be about. But this time we measure two aspects of a person, because we want to know if there's a relationship between these two aspects of a person.)

Here's the data from the exercise and health example. Six volunteers are measured twice each. One measure is their exercise level; the other measure is health rating. So the six people generate 12 data points. The formula for r uses N, which is the number of people. So N = 6. (Occasionally students think that N is twelve because there are twelve scores; but that is wrong). N = 6.

What I suggest doing as you learn and on exams is to make several columns. The data are given in an X column (exercise) and a Y column (health). But let's make three more columns.

Let's call the first new column X times Y (often called the cross product). The second new column will be X squared. The third new column will be Y squared. These three columns allow you to get the most important information you will need to calculate r.

CROSS PRODUCTS: (Besides the two data columns, X and Y) the first new column is X times Y (the cross product). So let's concentrate on that column. You'll notice what X times Y is just what it says, we take the first person and multiply his or her X-score times his or her Y score. For the first person we get 9 times 10 and that's 90. This is called the cross product for that person's data.

The second person in the cross-products column has an X score of 1 which multiplied by the Y score of 3. 1 times 3 is 3.

And so forth all the way down, so jot that information down in your notes. It's better learning if you do all the work in your notes first, and then check your answers against the graphic.

X SQUARED: The next column of course, is just X squared, so you just take the X value for the first subject, 9, and square it. That gives you 81. Go all the way down the column squaring the X score for each participant.

Y SQUARED: The last column is the Y scores squared. Again, it is best if you do the work first in your notes and then check your work against the graphic.

You end up with these three new columns of numbers next to your two data columns.

THE SUMS:Finally, you must sum up all five columns (the two data columns, and three new columns). You need sum of X, sum of Y, sum of XY, sum of X squared and sum of Y squared. Those 5 sums will be central to calculating the Pearson r.

Substituting into the formula

The next step is to substitute the various sums we calculated into the formula on the left.

In the previous graphic, we calculated that the sum of the cross-products (the sum of XY) is 225, the sum of X squared is 246, and the sum of Y squared is 219. We also found that the sum of X is 34 and the sum of Y is 33.

I recommend that you substitute those sums into the formula for r right now and then check your work on the next lecture graphic.

Calculating and Interpreting r

Check your substitution against the graphic to be sure that you have done it correctly.

CARRY 5 DECIMALS: Because of the severe problems rounding error can cause in this formula please carry your working calculations to 5 decimals. That way we can check your answer for correctness to 2 or 3 decimals.

DO the arithmetic. I suggest that you do the arithmetic on your own and then check your result against the detailed computations shown on the current lecture graphic.

THE CORRELATION COEFFICIENT: The final result is that r = +0.8497. From our work with visualizing correlations on scatterplots you should have a good idea about how to interpret this coefficient.

INTERPRETATION: r can only vary from -1 to +1. A plus sign indicates a positive (direct) relationship. Because +0.8497 is well toward +1 you have a sense that exercise and health ratings are directly related to a substantial degree.

Look at the scatterplot for exercise and health, think of the range of r, and form an opinion about what an r in the neighborhood of +.85 means.

Imagine an example in which Y is college GPA and X is college entrance exam (ACT or SAT) score. Almost everyone has to take an ACT or SAT test to get into college. We want to know if there's a relationship between college GPA and entrance exam score. If college entrance exams (such as ACT or SAT) work, a reasonable scientific hypothesis would be that there should be a positive relationship between entrance exam score and college GPA.

If you look at the full pattern of data points on the entire scatterplot in the graphic on the left, there seems to be a decent positive relationship between X and Y. As X (entrance exam score) goes up Y (GPA) tends to go up. The relationship is not perfect but it's clearly positive. For the whole graph, the actual r = +0.90.

However, suppose we select the only those dots beyond the green line on the scatterplot. Those are the dots highlighted in the circle. Another way of saying this is that we are going to truncate or restrict the range of X (entrance exam scores). We will only look at the correlation between X and Y for those X scores that are higher than the green line.

Consider the dots within the circle as a small scatterplot. Notice that within the circle it looks like there is no relationship between X and Y. In other words, in the restricted range above the green line there is no relationship between X and Y. The calculated r = +0.05 for the dots in the circle. The correlation coefficient for the dots in the restricted range (above the green line) is spuriously low compared the r computed on the whole scatterplot.

Now you may be asking yourself, why would anyone select just a small part of their data to perform a correlation on? Unfortunately, in this example, you can't help restricting the range.

Let's say that the college where the data is gathered has admission standards. One standard is that the college admits no one below a certain entrance exam score. Suppose the green line represents the college's ACT or SAT admission standards. This means that only those people who score above the green line on X (entrance exam) will be admitted to college. Therefore they will be the only ones ever to get a college GPA at that college.

The potential students to the left of the green line are just that--potential. They were not admitted to that college because their entrance exam score was too low. Therefore, they really don't have GPA's at that college. And so the data points to the left of the green line don't actually exist. How can you measure someone's GPA at a college s/he never attended?

This can be a subtle problem. It might seem sensible to go to a college and measure the relationship between GPA and entrance exam score. But it may not be obvious that admission standards have almost certainly truncated the range of X. In effect you are only measuring the relationship between X and Y in a very restricted range of X. In that restricted range, the r is spuriously low.

This is not really a statistical problem. It is a problem with how the data were collected in the realm of science.

The second scatterplot shows how you could get a spuriously high value of r by restricting the range of X.

Looking at the graphic on the left, you can see that if you consider the entire relationship between X and Y on the scatterplot there appears to be no (or little) relationship between X and Y. The overall correlation between X and Y is +0.05

In contrast, if you only measured in the restricted range above the green line (shown here by the shaded area), you would believe you were seeing a strong positive relationship. In this truncated range r = +0.90, which is spuriously high.

An outlier is a score that so different from all the other scores that it does not seem to be part of the same data set.

On the lower scatterplot on the lecture graphic, the majority of the data is clustered in one spot and then there is one score far from the rest of the data. You can see from that scatterplot that that score can greatly increase the visual impression of a positive correlation. It also affects the correlation coefficient. On the lower scatterplot the correlation coefficient would be +0.70 while on the upper scatterplot, where there is no outlier, r would be +0.06.

One strange piece of data can change r and along with it your scientific conclusion.

Outliers might be theoretically important or they might just be measurement errors. For example you might trace the outlier back to a data sheet where someone misplaced a decimal point, turning a 2.17 into a 21.7. If you're very sure you have found the error then the data can be corrected.

But there is a strong and important scientific ethic that says that data should not be altered or thrown out. It is possible that an outlier, while not understandable right now, may be a hint for future theoretical breakthroughs. If you have an outlier you might report that you have one and then report the statistics, in this case r, both with and without the outlier. That way other people who read your work have the choice of deciding whether the outlier is trivial or important.

Curvilinear Relationships

The statistical theory behind the Pearson r assumes that the relationship between X and Y is linear. (We will define linear relationship in some detail in the next lecture on regression.) Pearson r does not accurately describe relationship unless the relationship is linear.

Look at the scatterplot on the graphic on the left. The shape formed by the dots is an inverted U. It starts out (on the right of the graph) as a positive relationship but then turns downward and becomes a negative relationship (on the left part of the graph). Therefore it indicates there is a curvilinear relationship between X and Y.

One curvilinear relationship which is well known in psychology is between performance and arousal. Arousal and performance are related by an inverted U shaped curve.

Suppose we are researching performance on shooting free throws. Performance for low levels of arousal tends to be poor. If a person is not motivated at all, doesn't care, and is too relaxed and lethargic, then performance is probably going to be poor.

As motivation and arousal go up, say the person is practicing with teammates and watched by coaches, performance will go up.

And as arousal increases beyond some optimum level, performance can decline. If the motivation is extremely high, like shooting free throws in the last seconds of a game in the NBA finals, an excellent free throw shooter can inexplicably miss shots. A certain amount of tension is necessary for good performance, but being too tense can be a detriment to performance. That's why many people work on psychological control of internal state so that they can maintain a peak arousal level even though the external circumstances are changing. A world class athlete like a woman on the balance beam in the Olympics wants to be able maintain her internal state at the perfect level of arousal.

EFFECTS OF CURVILINEARITY ON r: As we said, the Pearson correlation coefficient assumes a linear relationship between two variables. If the relationship is not linear the value of r will be attenuated (moved closer to 0).

In the inverted U shaped relationship we have been discussing, the value of r would be near 0. Look at the curve. At first there is a strong positive relationship. Then there is a strong negative relationship. Little r will average these two trends and give you a value near 0. An r value near 0 should indicate a little or no relationship between variables. But there is a strong and clear relationship between arousal and performance. In effect, r = 0 in this case would be inaccurate and misleading.

When the relationship between variables is not linear, r is not a good measure of relationship.

How do you know if the relationship is linear or not? Draw the scatterplot and look at the shape formed by the dots.

In advanced statistics there are other correlational measures that measure relationship for nonlinear cases.

We are now going to summarize some issues of research design that impact (1) on proper choice of which statistical procedures to use and (2) upon the proper interpretation of statistical results. These issues will be covered in more depth in research design; but they are worth mentioning in a statistics class. Besides the worthwhileness of learning about these ideas, you will have to understand the distinction between a True Experiment and a Correlational Study to do research in Virtual Lab properly.

IV's and DV's. In the Interface to Science lecture we talked about Independent Variables (IV's) and Dependent Variables (DV's). Review IV's and DV's if you need to. Remember that terms, Dependent Variable and Criterion Variable, are often used interchangeably.

In this course we will only need to use the distinction between a True Experiment and a simple Correlational Study. So we have crossed out some of the distinctions that you will study later in research methods. Just focus on the parts of the graphic that are not crossed out.

We won't do any more than mention that there are Quasi-experiments nor will talk about more complicated forms of correlational studies

True Experiment

Correlational Study

A group of scientist hypothesize that high levels of stress will disrupt sleep. They take lab rats and randomly divide them into two groups. The High Stress group is given occasional, harmless but annoying electrical shocks through their metal cages. The occurrence of the shock is unpredictable.

The No Stress control group just live in their cages as they normally do.

No shock is given during the sleep cycle in either group.

The rats are observed during their sleep cycle, and the number of sleep disruptions is counted.

When the study is over, the scientists have one measure (sleep disruption) on each rat. They also know which group each rat was in.

A group of scientists hypothesize that there is a relationship between stress level and sleep disruption. They ask a group of 20 human volunteers to rate the stress level of their lives during the last month. They also ask the volunteers to count the number of nights in the last month that their sleep is seriously disrupted.

When the study is over, the scientists have two measurements on each person (self-rated stress level, and number of disrupted nights of sleep).

Hypothetical Results

The DV (# of Sleep Disruptions) is on the vertical Axis. The IV (Stress) is along the horizontal axis. The grey bar represents the number of Sleep Disruptions in the High Stress group. The black bar represents the number of sleep disruptions in the No Stress group.

Hypothetical Results

Scatterplot for 20 volunteers who rated their stress level and their sleep disruptions for the last month. Sleep Disruption rating is on the vertical axis. Stress rating is on the horizontal axis.

1. Control Groups. There are two or more groups (or conditions). Each group is given a different level of the IV.

In the Stress and Sleep experiment, the 20 rats were randomly divided into two groups which were given two levels of stress (no stress and high stress). The No Stress group acts as a control group for the High Stress group.

1. There is only one group.

In the Stress and Sleep example, there was a single group of 20 human volunteers.

The researchers measured two things (stress and sleep disruption) on each person in that single group.

2. Active Manipulation of IV. The researcher actively manipulates the levels of the IV.

In the Stress and Sleep example, the researchers actively administered the stress to one group and withheld it in the other group.

2. Naturally occurring variables: There is no active manipulation of variables by researchers.

In the Stress and Sleep example, researchers did not cause stress in peoples lives. They simply measured naturally occurring stress levels and correlated them with naturally occurring sleep disruption.

3. Random assignment of participants to groups.

In the Stress and Sleep example, the rats were randomly assigned to the High Stress and No Stress groups.

As you can see from the graphic, common forms of research present us with more complex cases. We have crossed out those because we will not need them in this course.

Strong Implications for causality. Why make these research distinctions? Because the type of research design has strong implications for whether or not we can conclude whether one variable causes changes in another variable.

Virtual Lab. We have not started using Virtual Lab yet. But when we do and when you read a scientific puzzle in Virtual Lab, you will have to decide if the puzzle is asking you to do experimental or correlational research. Then you will have to go and design a study (either experimental or correlational). So Virtual Lab will give you practice using this distinction.

Conclusions of causality require a true experiment. Scientists are often fascinated with causality. What causes what? If you want to discover if an IV actually causes changes in a DV, then you need to do a well-run, true experiment. What makes an experiment well run is a source of constant debate among scientists. The main issues in how to design a well-run experiment are extensive and will be covered in research methods.

Suffice it to say that before you worry about what is well-run, you need to be sure that as a minimum you are running a true experiment if you want to impute causality between variables.

Causal Hypotheses. Suppose the scientific hypothesis is causal. The scientific hypothesis is that changes in the IV cause changes in the DV.

Example. Using the stress and sleep example, the causal scientific hypothesis would be that high levels of stress cause sleep disruption. The scientists want to make the point that stress causes sleep disruption.

Correlational Study. Suppose the scientist do a correlational study like the one we just discussed above. They ask a group of 20 human volunteers to rate the stress level of their lives during the last month. They also ask the volunteers to count the number of nights in the last month that their sleep is seriously disrupted.

Hypothetical Results. Suppose they get hypothetical results we saw above in the scatterplot. Suppose also that the correlation coefficient = +0.72.

A poor conclusion. Suppose they conclude from their results that stress causes sleep disruption. Based on the results of the correlational study alone, this conclusion is open to severe criticism.

Six competing hypotheses. The graphic outlines six possible hypotheses that all compete with each other to explain the correlation between stress and sleep disruption.

The graphic uses the symbol IV for the supposed causal variable (stress) even though the correlational study did not have a true IV; it had only two DV's.

Plausible Competing Hypotheses. The graphic lists 6 hypotheses. As we shall see, they are all plausible and they all compete with each other to explain the correlation found in the study.

4. Third variables. Perhaps sleep have nothing to do with each other. Rather, it can be hypothesized that changes in both are caused by a third variable. Maybe adrenal gland output is the third variable. (The adrenal gland puts out adrenaline, the hormone that gives a person a zap of "fight and flight" energy.) Maybe people whose adrenal glands are producing higher amounts of adrenal experience both stress and sleep disruption. The self reports of stress and the self reports of sleep disruption are correlated because they are both caused by a third variable, adrenal output. In our example, a third variable hypothesis is plausible. You can easily make up more. Maybe the third variable is environmental noise levels, which cause both stress and disrupted sleep.

6. Chance alone. It can be hypothesized that there is no link between stress and sleep disruption. The correlation we found was a quirky, chance occurrence in the data. If we ran the study again, we would not find the same results.

Chance sounds at first hearing like a strange competing hypotheses. But in fact it is a very serious challenge to any empirical support for any scientific hypothesis. A great deal of the last half of this course will focus on how scientists use statistics to argue that their results did NOT occur by chance alone. As odd as it seems when you first come across this idea, chance is considered to be a plausible explanation of results in any research.

Certain criteria are generally held to be minimal requirements for the inference that one variable (let's call it the IV) causes changes in the other (let's call it the DV).

We will just quickly mention a few positive guides toward making causal inferences.

Cause precedes the effect. In logic, the cause must precede the effect. If someone wanted to infer that stress causes sleep disruption, they could not do so if the sleep disruption occurred first and the stress occurred the next day. As a minimum the stressor would have to be first, and the sleep disruption second. Cause first, effect second.

A difficult criterion. To infer causality, a researcher must eliminate other plausible competing hypotheses. As we pointed out in the previous section other scientists will generate many plausible hypotheses which compete to explain to explain the results of research. The power of the experimental method is that it gives researchers tools for eliminating plausible competing hypotheses. For example, in the true experiment with rats, the scientists could eliminate a third variable hypothesis that noise levels are causing sleep disruption because they are housing both groups of rats in the same quiet area. They could eliminate reverse causality because they actively manipulate stress versus no stress and know that it is the stress which comes first, not the sleep disruption. And so on.

The great intellectual adventure of science includes this interplay between plausible competing hypotheses and experimental design.