1
The Correlation‐Based Law of Effect

Foreword

This article was my first attempt to lay out the molar view of behavior. Reading it in the year 2021, I see both strengths and shortcomings. The paper explains the molar view clearly, but with one error and omission of two essential concepts that came later: the link to evolution and the centrality of induction.

If I were writing this now, my vocabulary would be different. In particular, the word reinforcer should be reserved for an event, and the word reinforcement should not be used to denote an event but a process of behavior change.

The error I made was in failing to recognize that the molecular view of behavior constitutes a different paradigm from the molar view and, consequently, differs ontologically. This error led me to use “molecular” as synonymous with “small scale” and “molar” as if it meant “large scale.” I distinguished discrete responses from activities extended in time, but lacked the concepts of scale and individual. Thanks to meeting with a philosophy of biology group at the California Academy of Science from 1999 onwards, I learned about the ontological category of individual and the necessity of part–whole relations. I did not begin applying these concepts until 2001.

Having reviewed the book in which Eve Segal's paper on induction appeared in 1972, I had the concept but did not realize its full significance until later, when I realized that not only non‐operant activities but also operant activities were induced.

As I thought critically about the concept of reinforcement that I had inherited via my undergraduate education, I came to realize that it contained a fundamental deficiency. Whether one adopts the concept of reinforcement or utility or maximization or matching, any of these concepts requires a mechanism. Herrnstein (1970) put the matter well when he pointed out that “adaptation” is a question, not an answer.

The implicit requirement of a mechanism to explain how behavior increases or decreases led behavior analysts to propose “strength” as an unseen driver of response rate. Response strength, however, has all the defects of any hypothetical explanation, particularly circularity and untestability. Worse than strength, however, was the unacknowledged appeal to agency. Economic models were particularly prone to leaving a void on mechanism that agency would fill implicitly. If one claims that an organism behaves so as to maximize utility, the burden of mechanism falls on the organism; the organism maximizes. In the absence of any other mechanism, the agency of the organism fills the void, and we are left with some sort of inner self choosing and perceiving and deciding. We have failed to escape pre‐scientific folk psychology.

I finally realized that induction filled the mechanism gap. The present paper proposed a feedback system including environment and organism—E‐Rules and O‐Rules. Much could be said about E‐Rules—feedback functions and so on—but the paper contains almost nothing about O‐Rules. Nowadays I put induction in as the primary O‐Rule.

Induction deals with two long‐standing problems: how reinforcement and punishment work and what accounts for their efficacy. Adopting the idea of the behavior–environment feedback system, induction, as an O‐Rule, completes the loop, because a reinforcer induces operant activity, the operant activity produces the reinforcer, and the reinforcer induces the operant activity—on and on.

Realizing that the events called reinforcers, punishers, and unconditional stimuli induce non‐operant activities implies a role for evolution to account for their efficacy. Activities that mitigate fitness‐threatening events—encountering a predator, sustaining an injury, and so on—play a role in natural selection, because organisms that fail in that regard leave fewer offspring. Similarly, activities that enhance fitness‐promoting events—encountering a mate, gaining food, and so on—allow organisms that deal with them effectively to leave more offspring. Thus, I refer to these fitness‐affecting events as Phylogenetically Important Events. Instead of value—a term that fails to escape agency—I now call V competitive weight, recognizing that activities compete for the time available, which is always limited. Competitive weight depends on induction and is measurable. These ideas appeared in later publications.

Abstract

It is commonly understood that the interactions between an organism and its environment constitute a feedback system. This implies that instrumental behavior should be viewed as a continuous exchange between the organism and the environment. It follows that orderly relations between behavior and environment should emerge at the level of aggregate flow in time, rather than momentary events. These notions require a simple, but fundamental, change in the law of effect: from a law based on contiguity of events to a law based on correlation between events. Much recent research and argument favors such a change. If the correlation‐based law of effect is accepted, it favors measures and units of analysis that transcend momentary events, extending through time. One can measure all consequences on a common scale, called value. One can define a unit of analysis called the behavioral situation, which circumscribes a set of values. These concepts allow redefinition of reinforcement and punishment, and clarification of their relation to discriminative stimuli.

Keywords: molar view; molecular view; correlation; reinforcement; punishment; behavioral situation;

Source: Originally published in Journal of the Experimental Analysis of Behavior, 20 (1973), pp. 137–153. Reproduced with permission of John Wiley & Sons. Preparation of this manuscript was supported by grants from the National Science Foundation, National Institutes of Health, and National Institute of Mental Health to Harvard University. The author thanks R.J. Herrnstein and H. Rachlin for their helpful advice and criticism.

‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐‐

Introduction

The traditional view of the law of effect makes contiguity between a response and a reinforcer central. It holds that reinforcement strengthens whatever response is contiguous with it, and that a response must be contiguous with reinforcement to be strengthened. Accordingly, a contingency would operate by ensuring contiguity of certain responses with reinforcement.

Recently, a number of authors have criticized this reliance on sheer response‐reinforcer contiguity, and have tried to restate the law of effect in more global terms (Herrnstein, 1969, 1970; Seligman, Maier, & Solomon, 1971; Staddon & Simmelhag, 1971; Bloomfield, 1972). The present paper attempts to show that such a reinterpretation of the law of effect follows directly from the understanding that the organism and its environment constitute a feedback system. It attempts also to elucidate a notion of correlation that can replace mere contiguity. Finally, it attempts to show some of the implications of the new view.

Instrumental Behavior and Feedback

The opening sentence of Schedules of Reinforcement (Ferster and Skinner, 1957) reads: “When an organism acts upon the environment in which it lives, it changes that environment in ways which often affect the organism itself.” This statement implies that an organism's relations with its environment can be treated as a closed chain of events: the environment affects the organism's behavior, the organism's behavior changes the environment, the environmental changes again change the organism's behavior, and so on. Although it has long been recognized that behavior, through its consequences, feeds back to the organism—even before Thorndike (Morgan, 1894)—an obvious, but fundamental, implication of the relation has been overlooked until recently.

The Organism‐Environment System

Consider how the organism and its environment can be likened to a feedback system. Figure 1.1 diagrams the interactions in the organism‐environment system. The experimenter manipulates the E‐rules by which the organism's behavior (output) affects the environment, and attempts to discover the O‐rules (functional relations) by which the environmental consequences (feedback) affect the organism's behavior. Some of the organism's output is also fed back directly by somesthesis. This loop within the organism is a logical necessity in characterizing the system, because variables such as effort expenditure produce important consequences internal to the organism. These variables are measurable, however, because they can produce effects (e.g., force or work) in the environment as well as within the organism. When a procedure differentially reinforces effort expenditure, the external consequences (reinforcement) will tend to increase effort expenditure, while the internal consequences will tend to keep it from increasing. An internal loop is no different in...