In Search of the Preference Reversal Zone

A win of £1,000 today has more value than a win of £1,000 to be delivered 1 year from now because of a process termed delay (or temporal) discounting (Mazur, 1987). As a result of delay-discounting, people and nonhuman animals sometimes forego a larger-later (LL) reward to receive a smaller-sooner (SS) reward (see supplementary Table 3 in Glautier et al., 2022, for a summary listing of symbols and abbreviations used in this paper). The relationship between delay and value is not linear, and one approach is to describe value, V, as a hyperbolic function of delay, δ, and reward magnitude, α, with the rate of decline given by the delay-discounting parameter κ, as shown in Equation 1 (Mazur, 1987).

As can be seen in Equation 1, larger values of κ indicate that reward value is lost more quickly as delay increases and larger values of κ identify a tendency to choose more immediate impulsive options. This hyperbolic model (Equation 1) has been used to summarize delay-discounting in many laboratory studies, including studies of socially important behaviors such as addiction (Amlung et al., 2017), failure to save for the future (Janssens et al., 2017), and failure to conserve energy (Macaskill et al., 2019).

The value of a reward at a particular delay is typically estimated by locating the indifference point – the amount of reward received immediately that has the same value as the delayed amount. Finding indifference points for a range of delays allows least-squares estimates of κ to be obtained using nonlinear regression. Equation 1 generally describes delay-discounting well, with regressions producing R² values over 0.9 (e.g., Kirby, 1997; Madden et al., 1999), but note that Equation 1 is a highly simplified model and does not take into account a wide range of variables that are relevant to delay-discounted decision making (e.g., see Cavagnaro et al., 2016). Estimates of κ are associated with important real-world behaviors such as smoking, drug use, and problem gambling (see MacKillop et al., 2011). The widespread use of the hyperbolic model as a tool for understanding, and possibly intervening in, applied problems makes a thorough test of its predictive accuracy essential.

A plot of V (δ) using Equation 1 is given in Figure 1 alongside a plot using Equation 2.

Equation 2 is an exponential model of delay-discounting (Samuelson, 1937). These hyperbolic and exponential delay-discounting curves are visually similar in form, both produce curves with slopes approaching zero at a negatively accelerated rate. However, the exponential model typically produces lower R² values in model fitting than does the hyperbolic model for humans (e.g., Kirby, 1997; Madden et al., 1999; McKerchar et al., 2009; Myerson & Green, 1995) and nonhumans (e.g., Mazur & Biondi, 2009). More importantly, for the current paper, although not apparent in Figure 1, the exponential model of Equation 2 does not predict preference reversals. A preference reversal occurs when people initially prefer an LL reward but then switch to preferring an SS reward as both come closer in time. This behavior can be modeled with Equation 1, as will be explained in the Method section, but according to Equation 2, an individual will prefer either the SS reward or the LL reward and that choice will be consistent no matter when in time they make the choice (Kirby & Herrnstein, 1995).

This capacity to model preference reversals (e.g., Ainslie & Herrnstein, 1981; Green et al., 1994) is a key difference between Equations 1 and 2. Therefore, well-documented preference reversal effects would be of critical theoretical importance, allowing a choice to be made between two models of delay-discounting. Furthermore, existing demonstrations of preference reversals with humans (Green et al., 1994; Kirby & Herrnstein, 1995; Pope et al., 2019) and pigeons (Ainslie & Herrnstein, 1981; Rachlin & Green, 1972) suggest cross-species generality, so empirical tests of these models are of general interest. However, although there seems to be clear evidence of preference reversal effects, it is less clear that these demonstrations are as theoretically decisive as may be apparent on first glance. This is because there has been no direct link between individual delay-discounting rates and the temporal location of preference reversals.

For example, Pope et al. (2019) found that individual delay-discounting rates derived from the hyperbolic model broadly predicted preference reversals when considered at the group level. Smokers, who had larger κ values, required longer delays to SS (longer “front-end” delays), holding constant the difference between SS and LL before their preferences changed from SS to LL (see also Yi et al., 2016). This suggests some level of correspondence between κ and the delays at which preference reversals occur, but Pope et al. did not examine whether preference reversals occurred at delays specifically predicted using each individual’s estimated κ value.

The few other studies that have tested specific predictions based on estimated κ values have not consistently found that preference reversals occur at the delays predicted by Equation 1. Kable and Glimcher (2010) compared discounting with and without a 60-day front-end delay. Choices were not customized to the individual, so for many choices the participants saw, their κ value did not actually predict that they would shift their preference when the front-end delay was added. When Kable and Glimcher examined only the subset of choices in which the hyperbolic model predicted preference reversal, most participants did not show any change in preference and shifts that did occur were just as likely to be in the SS to LL direction as vice versa (see also Janssens et al., 2017; Luhmann, 2013).

One caveat is that tests of the hyperbolic model’s predictions regarding preference reversal have generally neglected to consider the well-demonstrated effect of reward magnitude on discounting rate. Larger reinforcers have been discounted less steeply than smaller reinforcers in a number of human studies (e.g., Green et al., 1997) and in some nonhuman studies (e.g., Grace et al., 2012), but not others (e.g., Green et al., 2004). We reviewed human studies of magnitude effects on delay-discounting (see supplementary Figure 3 in Glautier et al., 2022, and section Choice of x and d: The “ab” optimization criterion) and found a systematic negative relationship between reward size and delay-discounting, so this is clearly important to take into account. The models of Equation 1 and Equation 2 can easily be developed to allow κ to vary for different reward magnitudes (e.g., Kirby, 1997), and therefore, fair tests of preference reversal predictions should not only be based on individualized κ values but should also allow different κ values for the large and small rewards under consideration. An interesting consequence of introducing two free parameters into the modeling is the fact that the exponential model is also able to predict preference reversals if a larger discounting rate is assumed for the smaller than for the larger reward (Kirby, 1997; Madden et al., 1999).

In summary, the facts (1) that the hyperbolic model predicts preference reversals and (2) that preference reversals occur have been taken as evidence in favor of that model, but researchers have rarely directly tested specific hyperbolic model predictions for preference reversals based upon individual participant estimates of κ. Researchers who have tested specific predictions of the hyperbolic model have not found support for them, but have also overlooked the magnitude effect. Therefore, the current study directly tested specific, quantitative, individual-level predictions of the hyperbolic discounting model for preference reversals, taking into account the possibility of different discount rates for small and large rewards. Modeling with two discount rate parameters means that both the hyperbolic and exponential discounting models could predict preference reversals, so we undertook a comparison of these two models.

In the first of two main analyses, our model comparison used the generalization criterion method (GCM) originally proposed by Mosier (1951) (see also Ahn et al., 2008; Busemeyer & Wang, 2000; Shiffrin et al., 2008). Key features of this method are that it provides a strong test of the predictive power of a model and that it avoids problems of comparing models of different complexities which can arise due to functional form and due to the number of parameters (Myung & Pitt, 1997). The GCM involves calibration of model parameters using a calibration data set and then, without further parameter adjustment, testing model predictions using a test data set. The GCM differs from cross-validation approaches, which also employ partition of data into calibration and test sets, in that the GCM uses different experimental design parameters for the calibration and test data. Thus, GCM tests a model’s capacity to extrapolate beyond the calibration design. One of the features of our investigation, described in detail below, is that when we tested the two discount rate parameter hyperbolic and exponential model predictions, we used experimental designs optimized for each model. It is not generally feasible to use the same design to compare models with respect to preference reversal predictions because a design parameterized by one set of delays and values may be predicted to produce a preference reversal by one model, but not by another. To overcome this, we compared model performance using the best design for each model. Our experimental design optimization found delay-discounting test questions tailored to maximize preference reversals for each model, so the number of correctly predicted preference reversals for each model was used for model evaluation alongside maximum likelihood statistics as indices of model fit to the test data.

In our second main analysis, we assessed the descriptive rather than predictive adequacy of the two discount rate parameter hyperbolic and exponential models alongside single discount rate parameter versions of each model and alongside a baseline guessing model. In this analysis, we used Akaike weight analysis of maximum likelihood statistics (Wagenmakers & Farrell, 2004), which takes into account the number of parameters in a model as a proxy for model complexity.

Method

A two-stage experiment was conducted. In Stage 1, we estimated hyperbolic (Equation 1) and exponential (Equation 2) discounting rates for a small (S) and for a large (L) hypothetical monetary reward for each participant. These estimates were used to design questions for Stage 2. The Stage 2 questions formed a preference reversal zone test in which we asked each participant question pairs of the form “Would you like S in d or L in D?” and “Would you like S now or L in D₁?”. In these question pairs, d is the delay to S, the front-end delay referred to in the introduction, and D is the delay to L. D₁, the delay to L in the second question, is computed as D − d. The selection of values for S, L, d, and D is described below. The question pairs were one of three types based on the values of d and D. Based on the discounting rates established in Stage 1, we generated the following model predictions. For the LLC pairs, we predicted that the participant would choose L for both questions (an LL choice pattern). For the LSC pairs, we predicted that the participant would choose L when S was delayed and would show a preference reversal choosing S when it was immediately available (an LS choice pattern). Finally, for the SSC pairs, we predicted that the participant would choose S for both questions (an SS choice pattern). This process was repeated in parallel for the hyperbolic and exponential models, and our model evaluation involved a comparison of the choice patterns found in the data and the choice patterns predicted by each model.

Stage 1: Estimation of Delay-Discounting Rates

In Stage 1, we obtained maximum likelihood discount rates for two monetary amounts S = £820 and L = £1,000. The value of S was obtained from xL where the specific value of x was selected as part of the Stage 2 design optimization procedure described below. Discount rates were obtained for each model (using the same questions) and used to select the delays used in Stage 2, as described below. For each of these two amounts, participants had a series of questions of the form “Would you like a now or A in D?” where A was fixed and corresponded to either S or L.

The delays at which the values of S and L were estimated were 1 week, 1 month, 6 months, 1 year, and 2 years. Eight values of a were used for the estimation of L at each delay with a ∈ {£5, £100, £250, £400, £550, £700, £850, £1,000}. The corresponding eight values of a for the estimation of the value of S at each delay were obtained by multiplying each element of a by x, just as S was obtained from xL, to give {£4, £82, £205, £328, £451, £574, £679, £820}. Thus, there were 80 different questions – eight values of a to assess discounting at each of five delays for each amount S and L. Each question was repeated twice (160 total), and the order of questions was randomized independently for each participant within each repeat.

Maximum likelihood estimates of κ were obtained for each participant for the hyperbolic and exponential models by modeling the choices made in each of the i trials of Stage 1 (excluding fillers, see below) to find model parameters which minimized Equation 3.

(3)

In Equation 3, P(R) is the probability of the observed response computed using Equation 4 as used by Cavagnaro et al. (2016).

(4)

In Equation 4, the function V is used to compute the value of each choice alternative using Equations 1 and 2, and g > 0 is an additional parameter which determines the sensitivity of the choice to the difference between V (A) and V (a).

Stage 2: Preference Reversal Zone Test

Stage 1 yielded, for each participant, four maximum likelihood estimates of κ, one for S (k) and one for L (K) for each model, Equations 1 and 2. These κ values were used to compute the values of D required in the Stage 2 questions. Calculation of κ and D values was done “on-the-fly” as soon as Stage 1 was complete as described in section Online Experimental Methods. In Stage 2, we identified three pairs of choices for each of the two models, giving six pairs in total. Each pair corresponded to one of three types – an LLC pair, an LSC pair, and an SSC pair, and each type was represented for each model. The types were based on values of D computed from the models. Each pair of questions was repeated twice giving 24 questions in total, and the order of questions was randomized for each participant. The question types corresponded to different patterns of choice expected across the members of the pair, as described below. For example, for the LSC type, we expected, based on model predictions, that participants would choose the large reward L for the first question and the small reward S for the second question.

To explain the derivation of the LLC, LSC, and SSC question types, consider Equation 5.

(5)

Equation 5 is an alternative way to show delay-discounting curves by plotting value against position in time (as opposed to delay, as was done in Figure 1 for Equation 1). Equation 5 gives value against time for rewards which become available with δ set to some offset from t = 0. Plotting two such curves on the same graph, one for a small reward, α = a = 500 and δ = d = 50, and one for a larger reward, α = A = 2a and δ = D = 90, can reveal a theoretically interesting characteristic of Mazur’s equation, namely the presence of a preference reversal zone. This is shown in Figure 2 panel LSC. The implication is that someone might prefer an LL reward (e.g., better health) over an SS reward (e.g., smoking a cigarette) at t = 0, but this preference switches as t → 50 when the cigarette is imminently available.

Figure 2 panel LLC shows two hyperbolic delay-discounting curves constructed using Equation 5. It can be seen here that a prediction can be made that at t = 0 a participant would choose L and that the same choice would be made at t = d – an LLC choice pattern. In panel LSC, the prediction is that the participant would choose L at t = 0, but at t = d they would choose S – an LSC choice pattern which is a prediction for a preference reversal. Panel LSC also shows two vertical lines, marked LD and UD.

These give boundaries on the value of D within which there is a preference reversal zone, and details of calculation of LD and UD are given below. In panel SSC, the prediction is for an SSC choice pattern, and the SS reward should be chosen at t = 0 and at t = d.

Of course Equation 2 can also be written as a function of t as in Equation 6.

(6)

Using Equation 6, the choice patterns shown in Figure 2 (panels LLC, LSC, and SSC) can be qualitatively reproduced if κ for the SS reward (k) and the LL reward (K) are allowed to differ and if k > K. As described in the introduction (e.g., Green et al., 1997), there is evidence that smaller rewards may be discounted more heavily than larger rewards. This puts the hyperbolic and exponential models on an even footing – with two discount parameters, both models can predict preference reversals. We now outline how the value of D, for the LL reward, can be calculated to generate model predictions for both models, for each of the choice patterns, using k and K computed from Stage 1.

LLC Question Pairs

From Figure 2, it can be seen that as the delay D to the LL reward increases at some point a preference reversal is predicted. To design the LLC question pairs, it is necessary to define an upper bound on D below which the LLC choice pattern is predicted. This upper bound is the delay D at which A has the delay-discounted value a. Given K from Stage 1 for an individual, we set d, A (for an LL reward), x, and a (for an SS reward), where a = xA, and using Equation 1, we solve Equation 7 to find the required D.

(7)

(8)

Therefore, referring to Equation 7, xA is the “now” value of the SS reward and following rearrangement Equation 8 sets the upper bound on D below which the now value of the SS reward is less than the value of the LL reward delivered with a delay of D − d. Thus, we have the limit on an LLC zone in which all choices should be for the LL reward. This limit is designated LD_h because it marks the lower bound on D for the preference reversal zone for the hyperbolic model.

Following the logic set out above, we now derive LD_e, the lower bound on D for the preference reversal zone for the exponential model, as given in Equation 9.

(9)

We can now construct LLC question pairs for the hyperbolic and exponential models for which we expect all choices to be for the LL reward:

• –
  “Would you like S in d or L in D?” D ∈ [d, LD_h)
• –
  “Would you like S now or L in D?” D ∈ [0, LD_h − d (hyperbolic questions)
• –
  “Would you like S in d or L in D?” D ∈ [d, LD_e)
• –
  “Would you like S now or L in D?” D ∈ [0, LD_e − d (exponential questions)

SSC Question Pairs

From Figure 2, it can be seen that when D is large all choices are predicted to be for the SS reward. To select appropriate values for D for the SSC questions, we need to identify a lower bound on D above which the SSC choice pattern is predicted. This lower bound is the delay D at which A has the same delay-discounted value as a discounted for delay d, and we designate this limit UD_h because it marks the upper bound on D for the preference reversal zone. Given k and K from Stage 1 for an individual and using Equation 1, we solve Equation 10 for D as in Equation 11.

(10)

(11)

The corresponding UD_e for the exponential model is given by Equation 12.

(12)

We can now construct SSC question pairs for the hyperbolic and exponential models for which we expect all choices to be for the SS reward:

• –
  “Would you like S in d or L in D?” D > UD_h
• –
  “Would you like S now or L in D?” D > UD_h − d (hyperbolic questions)
• –
  “Would you like S in d or L in D?” D > UD_e
• –
  “Would you like S now or L in D?” D > UD_e − d (exponential questions)

LSC Question Pairs

We now have LD_h, UD_h, LD_e, and UD_e which form the lower and upper boundaries of preference reversal zones for the hyperbolic and exponential discounting models, so now we can construct LSC question pairs as follows (in each pair, we expect an L choice for the first question and an S choice for the second question):

• –
  “Would you like S in d or L in D?” D ∈ [LD_h, UD_h)
• –
  “Would you like S now or L in D?” D ∈ (LD_h − d, UD_h − d] (hyperbolic questions)
• –
  “Would you like S in d or L in D?” D ∈ [LD_e, UD_e)
• –
  “Would you like S now or L in D?” D ∈ (LD_e − d, UD_e − d] (exponential questions)

Results

Of the 282 participants completing Stage 1, 89 (31.6%) failed to meet the inclusion criteria set out by Johnson and Bickel (2008). Of the 193 participants whose data were processed further, 84 (43.5%) were excluded because k_e ≤ K_e. Two of the 109 participants who met the inclusion criteria and completed both stages had an optimization error during the calculation of d, so they were excluded. This left 107 participants for analysis – 31 males and 71 females, four reported other gender and 1 declined to answer. The mean age was 33.2 (range 18–63).

Stage 1 Summary

Figure 3 gives the mean Stage 1 indifference points. There was a clear decline in the value of rewards as a function of delay, and the smaller delayed reward was valued less than the larger delayed reward. The average maximum likelihood κ values calculated for the Stage 1 trials for models H3 and E3 (n = 160, excluding fillers) were larger for S than for L, and this was true for the hyperbolic and for the exponential models, consistent with previous research. The mean κ values were 0.26 “v” 0.168 (t(106) = 4.08, p < .001) and 0.166 “v” 0.115 (t(106) = 4.08, p < .001) S “v” L for hyperbolic and exponential models, respectively. The average maximum likelihood g values were 0.017 and 0.016 for the hyperbolic and exponential models, respectively. The average minimized values were 17.3 and 17.4 for the hyperbolic and exponential models, respectively.

Stage 2 Model Predictive Performance

The main results of interest from Stage 2 are shown in Figure 4. For the LLC question pairs, the vast majority of choices were for L, and for the SSC question pairs, the vast majority of choices were for S, and this was true for the questions with the delays derived from both the hyperbolic and exponential models, as described in the Method subsection Stage 2: Preference Reversal Zone Test. These results were expected but, contrary to expectations, the vast majority of choices were also for S in the LSC condition. In the LSC condition, we expected to see preference reversals with L chosen for the first question and S chosen for the second question – both models predicted preference reversals, as shown in Figure 4, but very few were observed.

For all 12 Stage 2 questions, the mean log-likelihoods for models H3 and E3 were _h = 6.31 and _e = 7.023 and the average log-likelihood difference between the models, _D, was −0.713 (t(106) = 1.6, p = .11). For the four LSC trials, the mean likelihoods were _h = 2.971 and _e = 3.178 and the mean log-likelihood difference between the models, _D, was −0.207 (t(106) = 0.692, p = .49). However, the outcome of this test was impacted by some outliers, and there were considerably fewer participants for whom the log-likelihoods favored the exponential model.

For example, in the analysis of all 12 Stage 2 questions, there were 8 cases where the difference between models H3 and E3 was greater than ±2 SDs from the mean. Following this up, a Wilcoxon test produced a highly significant result in favor of the hyperbolic model (V = 1708, p < .001). Furthermore, for this 12-question trial analysis, there were only 32 participants out of 107 for whom _e < _h. In a binomial test with p(success) = .5, the probability of 32 or fewer successes in 107 trials is less than 0.00002.

There was no difference between the number of preference reversals (LL switch to SS, the “Ls” choice pattern in Figure 4) in the condition with the LSC delays chosen using the hyperbolic model and the number of Ls choice patterns with delays chosen using the exponential model (Wilcoxon signed-rank test W = 5,455, p = .42)

Stage 1 and Stage 2 Model Fitting Performance

The results of the Akaike weight analysis are summarized in Table 1. The Akaike weights in column wAIC provide conclusive evidence in favor of model E2, the exponential model with one discount rate parameter. The evidence ratios (Burnham & Anderson, 2002) provide information on the weight of evidence in favor of model E2, and in all cases, the ratios exceed one million. Although the results of this overall analysis are clear-cut, the picture is less clear at the level of individual cases where there are more individuals for whom the hyperbolic model fares better. Table 2 shows the number of participants for whom each model holds a particular rank based on the Akaike weights wAIC. For example, there were 42 cases where model H2 was best and only 36 cases where model E2 was best (the guessing model was ranked 5 in all cases, so it was not considered further). To follow up this observation, the frequencies in Table 2 were collapsed into four cells of a 2 × 2 contingency table. The cells counted the number of cases in which the two exponential models were best (ranked 1 or 2, n = 36 + 14 + 25 + 26 = 101), the number of cases in which the two hyperbolic models were best (n = 113), the number of cases in which the hyperbolic models were worst (ranked 3 or 4, n = 101), and the number of cases in which the exponential models were worst (n = 113). Although the pattern suggested an advantage for the hyperbolic models, this was not borne out in a χ² test which produced χ² = 1.346 (1 df, p = .246). Thus, the hyperbolic and exponential models are on an approximately even footing on this ranking analysis, but this analysis ignores the magnitude of model differences and this is evident in the overall analysis summarized in Table 1 where the single discount parameter exponential model is the clear winner. Table 2 in the supplementary material gives the mean maximum likelihood parameters for the five models fitted in this analysis, and supplementary Figures 5 and 6 provide detailed distributional data in the form of box and whisker plots (Glautier et al., 2022).

Table 1 Overall Akaike weight analyses for data combined over Stage 1 and Stage 2

Model	Parameters		AIC	∆ AIC	wAIC
The column Parameters gives the number of parameters estimated for each participant for each model. There were 107 participants; therefore, for example, the number of parameters estimated for H3 was 3 × 107 = 321. computed over 184 trials (Stage 1, excluding fillers, plus Stage 2).
H3	3	5,060.9	10,763.8	141.3	<0.000001
E3	3	5,011.9	10,665.7	43.3	<0.000001
H2	2	5,127.9	10,683.8	61.4	<0.000001
E2	2	5,097.2	10,622.4	0	→ 1
G	1	12,530.3	25,274.7	14,652.3	<0.000001

Table 1 Overall Akaike weight analyses for data combined over Stage 1 and Stage 2

View as image HTML

Table 2 Cross-tabulation showing the number of participants and ranks, based on wAIC (Rank 1 is best model), of fitted models

	E2	E3	H2	H3
The four quadrants, with totals of 101, 113, 101, and 113 reading clockwise from the top left, were subject to a χ2 analysis (see the section “Stage 1 and Stage 2 Model Fitting Performance).
Rank 1	36	14	42	15
Rank 2	25	26	24	32
Rank 3	35	29	25	18
Rank 4	11	38	16	42

Table 2 Cross-tabulation showing the number of participants and ranks, based on wAIC (Rank 1 is best model), of fitted models

View as image HTML

Discussion

Despite the fact that numerous studies suggest that preference reversals are well-described by hyperbolic models of delay-discounting (e.g., Green et al., 1994; Kirby & Herrnstein, 1995), we found little evidence of preference reversals using delays designed a priori using a two discount parameter version of Equation 1 to produce preference reversals. Our results (cf. Figure 4) showed that the LLC condition produced predominantly LL choice patterns, but these switched over to SS choice patterns in the LSC condition indicating that preference reversals may have been seen if we had used shorter LSC delays, in between those we actually used in the LLC and LSC conditions. We saw the same pattern using a two discount parameter version of Equation 2 for an exponential model. Thus, participants were significantly more impulsive than anticipated by the models. Why should the model parameters obtained in Stage 1 lead to such overestimation of delays for preference reversals?

One possibility is that the model parameters were estimated in Stage 1 at shorter delays (average 4.3 months) than those in Stage 2 (average 54.4 months). Assuming that increasing delay increases discounting more than implied by Equations 1 and 2, we asked whether the observed data might be better understood in these terms. In the supplementary materials (Glautier et al., 2022), we examined models considered by Rodriguez and Logue (1988) and Takahashi et al. (2008) (see also McKerchar et al., 2009; Peters et al., 2012) in which delay is exponentiated according to a sensitivity parameter s, as shown in Equation 13 for a hyperbolic model and in Equation 14 for an exponential model.

For the hyperbolic model, when our Stage 1 data were fitted with Equation 13, we found similar κ values to those obtained using Equation 1. In contrast, for the exponential models, when our Stage 1 data were fitted with Equation 14, we found larger κ values than those obtained using Equation 2 (cf. Glautier et al., 2022, supplementary Table 2). Thus, had we used Equations 13 and 14 to obtain the Stage 2 delays, the preference reversal zone questions for the exponential model would have used shorter delays than those we actually used and may have captured the preference reversals which we assume would be occurring at some point where the pattern of choices switches from LL to SS. Thus, it is of interest to use Equation 14 for a follow-up study with another attempt to identify a priori the temporal location of individual preference reversal zones.

Another perspective on this apparent failure of these models to adequately capture sensitivity to delay is suggested by our finding that our estimates for parameter g from Equation 4 were larger for Stage 2 than for Stage 1. Parameter g reflects sensitivity to the difference between the values of the small and larger rewards, amplifying differences as g increases. Thus, changes in g between stages could have worked in concert with underestimation of the effect of delay, as discussed above, to produce the observed patterns. However, some caution is needed here because there were only 12 Stage 2 trials upon which to base parameter estimates for each of the models.

In addition, looking at model predictions for preference reversals, we evaluated the overall predictive and fitting performance of hyperbolic and exponential models. Stage 2 of the experiment examined the predictive performance of our two discount parameter models, H3 and E3, and the hyperbolic model performed better than the exponential model. Using parameters estimated in Stage 1, the average log-likelihoods of the Stage 2 data were lower for the hyperbolic than for the exponential model, and there were significantly more participants than would be expected by chance for whom the hyperbolic model performed better than the exponential model. Poorer predictive model performance is linked to model complexity and indicates that the exponential model may overfit as compared to the hyperbolic model (e.g., Myung, 2000; Zucchini, 2000).

Following on from this, the fitting performance of the exponential models was better than hyperbolic models in the overall analysis of Stage 1 and Stage 2 with the Akaike weight analysis showing conclusive evidence in favor of the single discount parameter exponential model, E2, against the other four models in our set. However, this result did not extend to the individual level using model ranks (cf. Table 2). Recall that in the Stage 2 predictive performance analysis, we found significantly more participants for whom the hyperbolic model was the winner. In contrast, in the Stage 1 and Stage 2 fitting analysis, the number of participants for whom the exponential models were the best was in fact slightly lower than for the hyperbolic models despite the fact that the exponential models came out on top overall.

In summary, we found (1) little evidence that hyperbolic and exponential models could predict the temporal location of preference reversals, (2) that the predictive performance of the hyperbolic models was better than the exponential models, and (3) that a single discount parameter exponential model provided the best overall fits to the data. In relation to (1), although there was some evidence that participant behavior may have differed between the Stage 1 parameter calibration and the generalization criterion test in Stage 2, it is not clear whether this could explain the results; only the sensitivity parameter g changed between stages, the discount parameters themselves remained constant. An exploratory analysis showed that a modified exponential model (Equation 14) would be worth testing in a future investigation. In relation to (2), although there was no overall difference between the exponential and hyperbolic models in terms of their predictive performance in Stage 2, exclusion of some outlying cases showed that the hyperbolic model was better than the exponential model, and at the level of individual participants, there was a significant majority for whom the hyperbolic model was better. In relation to (3), fitting models to the complete data set, Stage 1 and Stage 2 combined, resulted in a clear overall win for the exponential model, despite the fact that the hyperbolic and exponential models were evenly matched at the level of individual participant fits, counting case numbers where one model or the other won on the basis of Akaike weight ranking.

In conclusion, we note three points. First, our finding of better overall fitting performance for the exponential model runs somewhat against the literature that has generally suggested better fits for hyperbolic models than for exponential models. However, as pointed out by Cavagnaro et al. (2016), the majority of existing studies have used least-squares regression to model value functions estimated with indifference points. We are not aware of any clear reason why this methodology should favor hyperbolic models and why using maximum likelihood methods should favor exponential models, but there is a growing number of studies that have used maximum likelihood methods, and among these studies, there is no clear advantage for hyperbolic models (e.g., Cavagnaro et al., 2016; Hofmeyr et al., 2017).

Second, one motivation for this work was to provide a strong test for models of preference reversals. This is especially important since these models may be used to guide applied work. However, in this study, as with the majority of delay-discounting studies, we used hypothetical rewards. Use of hypothetical rewards is sufficient to provide an internally valid experimental test of our theoretical models, but despite the fact that there has been little reliable evidence of differences between real and hypothetical reward methodologies (e.g., Green & Lawyer, 2014), a note of caution is still required if inferences about real rewards and applications are to be drawn.

Finally, our sample was highly selected. To provide common ground for a comparison of hyperbolic and exponential models, we included participants for whom k_e > K_e because without this constraint the two-discount parameter exponential model does not produce preference reversals. This selection criterion excluded 43.5% of participants and, assuming that participants could show preference reversals without meeting this constraint, this is a serious limitation on the generality of this model.

However, our preliminary exploration of the modified exponential model given in Equation 14 showed that this model can generate preference reversals without this constraint. Given this model might also explain the increased impulsivity that we saw at the longer Stage 2 delays, we are encouraged to use this model in continuation of our search for the preference reversal zone.