1
Basic Concepts in Research

1.1 The Scientific Method

This is a textbook about statistics. Simply defined, statistics are the mathematical tools used to analyze and interpret data gathered for scientific study. It is paramount to remember that statistical analyses and interpretations do not exist in a vacuum. They occur within the larger scientific research process. Both how to analyze and how to interpret the data are quite dependent upon the surrounding research context. While the subject of statistics can be singled out and studied in isolation (as this textbook demonstrates), it is inextricably linked to the larger scientific enterprise. As such, it is appropriate to first review the basic features of the practice of science.

The scientific method can be conceptualized as a three‐step recursive process. Each step can be summarized as follows:

Theory. Theories are attempts to explain and organize collections of data observed about a topic (or “phenomenon”) under scrutiny by appealing to general principles and relationships that are independent of the topic itself. Take, for example, a line of research on the endurance of friendships. In theorizing why some acquaintances lead to enduring friendships while others do not, one could propose that personalities are a bit like magnets; similar ones repel one another, while dissimilar ones are drawn together (i.e. opposites attract). This theory, then, appeals to the concepts of “magnets,” “personality,” “similarity,” and “dissimilarity.”

Not all theories can be considered “scientific.” For a theory to qualify as “scientific,” it must be testable. By testable we mean: Is it potentially falsifiable? Can it be placed into jeopardy and potentially observed to be untrue? If it cannot, it remains a theory, but it is not considered to be properly “scientific.” Using testability as a criterion, for example, the theory that each of our choices (past and future) are fully predetermined by some combination of our current environment, our DNA, and behavioral conditioning from our previous experiences, could hardly be considered “scientific.” While many people believe it to be true, how exactly would we go about putting it in jeopardy and testing it?

Hypothesis. In the light of any scientific theory, it should be possible to generate predictions about the data one expects to observe – this is a hypothesis. Sticking with our magnetic theory of friendships, one hypothesis might be as follows: If we measure the personalities of incoming university students who are randomly assigned living arrangements in a dorm, we might expect to find that those with quite discrepant personality profiles are more likely to be friends at the end of the semester than those with similar profiles.

Because it is possible to find evidence that would not support this hypothesis, we can say that this theory is “testable.” However, we cannot stop at hypothesizing. To say anything meaningful, we must complete the research process by going out and doing the work, setting up the study, gathering the participants, and carefully collecting and analyzing the data. This leads us to the third step.

Observation. The gathering of scientific observations is done by careful and systematic measurements of events occurring in the world by using our five senses, often with the aid of various scientific tools and instruments. In our example, we would want to meticulously measure our incoming freshman’s personalities as well as the nature of their friendships at the end of the semester. These observations, then, would be organized and interpreted. Ultimately, what is concluded will be brought back in contact with the stated theory. Observations will either support the theory, fail to support it, or, perhaps, partially support it. The circle is made complete as we compare our findings with our original theoretical proposition.

In our example, we should not be too confident that supporting data will be found – previous research suggests we will probably be disappointed (e.g. Back et al. 2008). And that is an important point – if supporting data are not found, so much the worse for the theory. We may need to think differently about why some friendships begin and endure while others do not. As would be expected, accurate theories will be supported by our observations. Supporting observations can both affirm a theory and lead to clearer and more refined articulations of that theory. More precise theories, in turn, lead to new hypotheses, and the cycle starts over again. The process is circular and recursive, with each cycle spiraling toward a more accurate understanding of the topic at hand.

The specific role of statistical analysis is found in the interpretation of our numerically represented observations. What do the numbers mean and not mean? How certain are we that our conclusions are accurate? On what do we base our sense of certainty? These are often not easy determinations to make. The central purpose of this text is to dissect and explain how this part of the research process works. The remainder of this introductory chapter will lay out an overview of the research enterprise.

1.2 The Goals of the Researcher

Scientific researchers set out with earnest intention to study carefully, logically, and objectively a particular topic of interest. Depending upon what is already known about the topic, what one wants to learn about the topic, and what one realistically can learn about the topic, researchers adopt different “goals” for their projects. Often, the initial goal for a researcher is that of description. Scientific description is the process of defining, identifying, classifying, categorizing, and organizing the topic of interest. Explicit delineation of topic boundaries is crucial. What exactly constitutes the topic? How many forms can it take? How frequently are these various forms found?

For example, if we were interested in studying the various ways in which people take vacations, we would first have to define what a vacation is. Is an afternoon day trip to a community park a vacation? What about an extra day tacked onto a work‐related business trip? It is not a requirement for all researchers to agree on the same definition for a topic to be studied. However, it is imperative that the readers know explicitly what we, the researchers conducting the present study, mean by “vacation” when we use that term. In other words, concepts must be operationally defined. An operational definition is a precise description of the concrete measurement of that concept, as it will be used in a given research project.

Another related issue would be to decide how many ways “vacation” can take place. For example, someone might suggest that there are fundamentally two different kinds of vacations: one kind is designed around relaxation and focuses on bodily rest, while the other kind features action and excitement. Another researcher may come along and suggest that there is a third kind of vacationing – one that combines the two, incorporating dedicated time to both bodily rest and being active. For this reason, it is crucial for investigators to clearly indicate their particular interpretation of the concept in question. For example, one researcher might find that only 20% of vacations are of the relaxation variety, while another, using a different operational definition, might find a different percentage. A proper interpretation of any statistical statement first requires an understanding of variable operationalization. Finally, it should be noted that the statistical needs associated with meeting the goal of “description” are usually not too sophisticated.

Another aim of the researcher would be one of correlation (or prediction or association – these are all analogous terms). Correlation involves a description of the degree of relationship between the topic of interest and other variables. For example, in our study of vacations, we might be interested to see if there was a relationship between the age of the vacationer and the type of vacation chosen. Here, we would be measuring two variables (“vacationer age” and “type of vacation chosen”). As we will learn later in the text, mathematical procedures applied to these measurements can determine if a relationship exists, and if so, the strength of that relationship.

It is critical to realize that research designed to show correlations does not allow us to draw causal conclusions. For example, if we find that older individuals, more so than younger ones, prefer to take vacations centered on rest, we could not justifiably conclude that age causes people to want to take restful vacations. It could very well be, for example, that older people grew up in a time when vacations were generally understood to be restful in nature, and they formed their vacationing expectations and habits accordingly. Another possibility might be that there are not as many exciting vacationing experiences geared toward an older audience compared with those available to a younger crowd. If the set of vacation options were different, then perhaps the numbers of older vacationers choosing active vacations would increase. Understand this clearly: one of the most frequently observed critical thinking errors is the tendency to impose a causal interpretation on data that were gathered...