 Methodology
 Open access
 Published:
Optimized segmented regression models for the transition period of intervention effects
Global Health Research and Policy volume 8, Article number: 29 (2023)
Abstract
Background
The interrupted time series (ITS) design is a widely used approach to examine the effects of interventions. However, the classic segmented regression (CSR) method, the most popular statistical technique for analyzing ITS data, may not be adequate when there is a transitional period between the pre and postintervention phases.
Methods
To address this issue and better capture the distribution patterns of intervention effects during the transition period, we propose using different cumulative distribution functions in the CSR model and developing corresponding optimized segmented regression (OSR) models. This study illustrates the application of OSR models to estimate the longterm impact of a national free delivery service policy intervention in Ethiopia.
Results
Regardless of the choice of transition length (\(L\)) and distribution patterns of intervention effects, the OSR models outperformed the CSR model in terms of mean square error (MSE), indicating the existence of a transition period and the validity of our model’s assumptions. However, the estimates of longterm impacts using OSR models are sensitive to the selection of L, highlighting the importance of reasonable parameter specification. We propose a datadriven approach to select the transition period length to address this issue.
Conclusions
Overall, our OSR models provide a powerful tool for modeling intervention effects during the transition period, with a superior model fit and more accurate estimates of longterm impacts. Our study highlights the importance of appropriate statistical methods for analyzing ITS data and provides a useful framework for future research.
Background
The interrupted time series (ITS) is most commonly used to evaluate the effects of interventions such as quality improvement programs or health policies [1, 2] and is a powerful quasiexperimental design [2] especially when randomized controlled trials are impossible, unethical, or not feasible [3,4,5].
The most popular statistical methodology for ITS timeseries data of interventions is the classic segmented regression (CSR) [6, 7], which is a potent method for accounting for underlying trends and has a high ability to infer causation [8]. To distinguish between the pre and postintervention phases, the CSR model restricts the interruption to a predetermined time point in the outcome time series [9]. The impact of interventions is frequently portrayed in CSR as instant, immediate, and leapfrogging at a fixed point [7].
However, the immediate effect of the intervention may not always hold, which is inconsistent with CSR assumptions. Many studies have shown that there may be a transition period between the pre and postintervention phases [2, 10, 11]. Because interventions may be effective over a prolonged period, or there may be a brief period of adjustment before interventions’ lasting impacts on the outcome time series become apparent [2]. First, interventions may have been introduced over time. For example, in England's 2012 Health and Social Care Act, the Clinical Commissioning Groups first took on the task in April 2012 and eventually took over full budget responsibility in March 2013; therefore, there was a oneyear transition period before the intervention was fully implemented [12]. Second, interventions may require a brief adjustment period (training required for intervention implementation). For instance, the Clinical Nurse Leader intervention program launched by the American Association of Colleges of Nursing, aimed to enhance healthcare quality, and the nurses involved required training for several months before the program was formally implemented [13]. The nurses put what they learned into practice in their clinical work during the training period, which allows the intervention to generate an adjustment period before its fully functional [9]. Similar intervention training was scheduled for the implementation of the free maternal health services policy in Kenya [14]. Such training is necessary for interventions to improve healthcare quality and equity.
In the aforementioned examples, the effects of interventions may be released during the transition period. That is, the intervention gradually shows the effect after the predetermined interruption (within the usually defined “postintervention phase”). However, the CSR model fails to model this period precisely. For the time points of the transition period, the CSR model either ignores them to model the entire outcome time series directly or removes them and then models the remaining time points [6, 15]. For example, in evaluating the effect of pay for performance on hypertension in the United Kingdom, the period corresponding to the stepwise implementation of the intervention was excluded from the interrupted timeseries analysis [16]. As researchers chose to exclude the transition period, this censoring (removing the transition period) not only leaves out data but could also distort parameter estimations on the effect of interventions [17]. If an intervention is found to be effective based on an inaccurate or biased estimation of its effects, resources may be allocated to scale up the intervention, which could potentially waste resources, and divert attention from other effective interventions [18, 19].
To solve this problem, we propose an optimized segmented regression (OSR) model to capture different distribution patterns of the intervention effects during the transition period using probability density functions (PDFs) types. We then utilize the corresponding cumulative distribution functions (CDFs) of the above PDFs to model the effects of the interventions during the transition period and introduce them into the CSR model. The transition period commenced when the intervention was initially introduced, and the length of the transition period reflects the time horizon over which interventions are effective or the length of the required training time for intervention implementation. Furthermore, CDFs can manifest in various forms, reflecting different distributions of intervention effects. In this study, we discuss four common distributions, namely uniform distribution, normal distribution, lognormal distribution (rightskewed distribution), and lognormal flip distribution (leftskewed distribution), to characterize the possible distributions of intervention effects during the transition period.
In this study, we first describe the steps of the optimized model. Then taking the evaluation of the free delivery service policy in five Ethiopian health centers as an empirical study example [20], we estimated the longterm impact of the free delivery service policy using the CSR model and OSR models with different CDFs. In this process, we suggest a possible datadriven approach for selecting the length of the transition period using the mean squared error (MSE) as a measure of the goodness of fit of the OSR model. By comparing the estimated longterm impacts of the models, we illustrated the advantages and disadvantages of the optimized models and their applicability.
Models
Classic segmented regression (CSR)
\({Y}_{t}\) is the value of the outcome series at time point \(t\). \(time\) is an indicator variable of the time point (\(time = 1,2,3, \ldots ,T_{e}\)) and spans the first and last observation points. \({T}_{0}\) is the time point at which the intervention is implemented (nominal intervention time), and \({T}_{e}\) is the length of the entire time series. A dummy variable, \(intervention\), was used to represent the implementation of the intervention. The dummy variables 0 and 1 values represent pre and postintervention, respectively. The time elapsed after the nominal implementation of the intervention is monitored using the \({post}\text{}{time}\) indicator variable. The value of \({post}\text{}{time}\) is first set to 1 during the postimplementation phase and then increases over time (\({post}\text{}{time} = 1, 2, 3, \ldots , \;T_{e}  T_{0} < T_{e}\)). The random error term for time point \(t\) is\({\varepsilon }_{t}\). Before the implementation, the outcome series' baseline trend is depicted by \({\beta }_{1}\). \({\beta }_{2}\) reflects the instant effect of the intervention on \({Y}_{t}\). The longterm impact of the intervention consists in the change in the trend of the outcome time series (slopes), represented by\({\beta }_{3}\). The matrix expression of Eq. (1) is:
where
Optimized segmented regression (OSR)
In the optimized model (Eq. 3), we model the transition period using different forms of CDFs as follows:
The piecewise function \(F\left(t\right)\) is:
where \({T}_{0}\) is the nominal intervention time and has the same definition as the CSR model. \({T}_{2}\) is the end time of the transition period, \({T}_{2}={T}_{0}+L\), where \(L\) stands for “transition length”. The effect of the intervention is assumed to last from \({T}_{0}\) (first implementation) to \({T}_{2}\) (fully valid): the transition period \(\left[{T}_{0},{T}_{2}\right]\). \(CDF\left(t\right)\) represent the CDFs of the different distribution patterns of the intervention effect during the transition period.
The variable assignments (\({time}\), \(intervention\), and \({post}\text{}{time}\)) of the optimized model and the meanings of the corresponding coefficients were the same as the CSR model. The matrix expression of Eq. (3) is:
where
Distribution patterns of intervention effects—CDFs
\(CDF\left(t\right)\) are the CDFs of the corresponding PDFs for the different distribution patterns of the intervention effect during the transition period. The PDFs represent how the effect of the intervention is distributed during the transition period \([{T}_{0},{T}_{2}]\) and the values of the corresponding CDFs taken at specific points are used for modeling, that is, \(CDF\left(1\right),\dots ,CDF\left(L\right)\). In this study, we mainly discuss the common distributions: (1) uniform distribution, (2) normal distribution, (3) lognormal distribution (rightskewed distribution), and (4) lognormal flip distribution (leftskewed distribution). The CDFs and the corresponding PDFs are shown in Fig. 1.
For the normal and lognormal distributions, their PDFs are respectively defined in the domain \([ \infty , + \infty ]\) and \([0, + \infty ]\). We truncated the PDFs so that we can describe the effect of the intervention at a fixed interval \([{T}_{0},{T}_{2}]\). The probability of occurrence of a fixed interval can be determined by integrating the PDF. For the normal and lognormal distributions, we chose \((\mu 3\sigma ,\mu +3\sigma\)) and \(({e}^{\mu 3\sigma },{e}^{\mu +3\sigma })\), respectively, to truncate them such that the probability of occurrence in the fixed interval is up to 99.97%. Matching the truncated interval to our assumed time range \([{T}_{0},{T}_{2}]\), we have \(\left\{\begin{array}{c}{T}_{0}=\mu 3\sigma \\ {T}_{2}=\mu +3\sigma \end{array}\right.\) for the normal distribution and \(\left\{\begin{array}{c}{T}_{0}={e}^{\mu 3\sigma }\\ {T}_{2}={e}^{\mu +3\sigma }\end{array}\right.\) for the lognormal distribution. The intervention was essentially fully effective at \([{T}_{0},{T}_{2}]\). The truncated intervals of the normal and lognormal distributions are shown in Fig. 2. For the lognormal flip distribution, we only needed to apply an axisymmetric flip transformation to the truncated lognormal distribution. The lognormal and lognormal flip distributions represented the rightskewed and the leftskewed distributions, respectively, and accordingly indicated that intervention effects are concentrated in the front or the back part of the transition period \([{T}_{0},{T}_{2}]\).
Uniform distribution pattern (UD)
For a uniform distribution in the interval \([{T}_{0},{T}_{2}]\), its PDF and CDF are:
Then \({{\varvec{C}}{\varvec{D}}{\varvec{F}}}_{UD}=\left[{CDF}_{UD}\left(1\right),\dots ,{CDF}_{UD}\left(L\right)\right]=\left[\frac{1}{L},\dots ,1\right]\).
Normal distribution pattern (ND)
For a normal distribution, its PDF and CDF are:
The PDF of the normal distribution is an infinite integral; we truncated its PDF and calculated its mean \(\mu\) and standard deviation \(\sigma\), as \(\left\{\begin{array}{l}\sigma =\frac{L}{6}; \\ \mu ={T}_{0}+3\sigma .\end{array}\right.\) At one specific time point \(t\),
Then \({\varvec{CDF}}_{ND}=\left[{CDF}_{ND}\left(1\right),\dots ,{CDF}_{ND}\left(L\right)\right]=\left[{\int}_{{T}_{0}}^{{T}_{0}+1}{PDF}_{ND}\left(x\right)dx,\dots ,{\int }_{{T}_{0}}^{{T}_{2}}{PDF}_{ND}\left(x\right)dx\right]\).
Lognormal distribution pattern (LND)
For a lognormal distribution, in its definition domain \([0,+\infty ]\), its PDF and CDF are:
When an upper limit exists, this integral cannot be solved using algebraic operations; its integral is usually expressed in the form of an error function as follows.
Assuming that \({CDF}_{LND}\left(x\right)=\frac{1}{2}\left\{1+erf\left[\frac{(\ln x\mu )}{\sqrt{2}\sigma }\right]\right\}\) = 0.5, with the integral symmetry, the median coordinate of the lognormal distribution is \(x={e}^{\mu }\). The corresponding coordinate interval where the sample falls near the median with a distance of \(3\sigma\) standard deviation is \(({e}^{\mu 3\sigma },{e}^{\mu +3\sigma })\). Here, we used the same strategy as that for the truncated PDFs in the normal distribution. However, the lognormal distribution is skewed; thus, we additionally set its skewness ratio, which is defined by the ratio of the release time of the half effect of the intervention in a total transition period of intervention, i.e., \(Ratio=\frac{{e}^{\mu }{e}^{\mu 3\sigma }}{{e}^{\mu +3\sigma }  {e}^{\mu 3\sigma }}\). For instance, in the context of a 12session training course spanning three months, the parameter \(Ratio\)=\(\frac{1}{3}\) of the lognormal distribution implies that half of the training sessions were concluded within the initial month, specifically six sessions. The degree of skewness, denoted by the \(Ratio\), depends on the skewness of the actual intervention effect during the transition period \([{T}_{0},{T}_{2}]\). Correspondingly, we truncated its PDF and calculated its mean \(\mu\) and standard deviation \(\sigma\), as \(\left\{\begin{array}{l}{e}^{\mu }{e}^{\mu 3\sigma }=Ratio*L;\\ {e}^{\mu +3\sigma }{e}^{\mu 3\sigma }=L.\end{array}\right.\) At one specific time point \(t\),
Then \(\begin{aligned}{{\varvec{CDF}}}_{LND}&=\left[{CDF}_{LND}\left(1\right),\dots ,{CDF}_{LND}\left(L\right)\right]\\ &=\left[\frac{1}{2}\left\{1+\mathit{erf}\left[\frac{\left(\ln({T}_{0}+1)\mu \right)}{\sqrt{2}\sigma }\right]\right\},\dots ,\frac{1}{2}\left\{1+\mathit{erf}\left[\frac{\left(\ln{T}_{2}\mu \right)}{\sqrt{2}\sigma }\right]\right\}\right] \end{aligned}\).
Lognormal flip distribution pattern (LNFD)
For the lognormal flip distribution, we applied only an axisymmetric flip transformation to the truncated lognormal distribution. We chose the midpoint coordinates \(x={T}_{0}+\frac{L}{2}\) of the transition period as the axis of symmetry to perform the axisymmetric flip transformation of the lognormal distribution, allowing us to obtain the lognormal flip PDF and integrate it to obtain its CDF. The schematic diagram of the axisymmetric flip transformation is shown in Fig. 3.
According to the symmetry of the axisymmetric flip transformation, then we have
By modeling the four abovementioned distribution patterns of the intervention effect, we developed four OSR branching models: OSRUD, OSRND, OSRLND, and OSRLNFD.
Length of the transition period
In most cases, the length \(L\) of the transition period and the distribution pattern of the intervention effect are determined by the implementation process. When there was no information about the implementation process, we used a datadriven approach to select \(L\) for the above four distribution patterns of intervention effect and described the application process of the optimized model.
First, we set the maximum possible range for \(L\) selection, that is, the \({L}_{m}\) (\({L}_{m}=\max L\)). We then applied the optimized OSR model directly to all scenarios (\(L=0, 1, 2,\dots ,{L}_{m}\)), and \({L}_{m}+1\) scenarios for each OSR branching model for a total of \(4\times ({L}_{m}+1)\) scenarios. \(L=0\) corresponds to the CSR model; that is, there is no transition period. For the different distribution patterns of the intervention effect, we selected the value of \(L\) corresponding to the minimum MSE in all scenarios.
Application data analysis
Data description
In this study, we used raw data from a published research article [20] titled ‘Effect of Implementing a Free Delivery Service Policy on Women’s Utilization of FacilityBased Delivery in Central Ethiopia: An Interrupted Time Series Analysis’, to test and compare the CSR model and our optimized models. The raw data are provided in the supplementary file of the above research article and can be downloaded directly from the Journal of Pregnancy [20].
In Ethiopia, facility delivery services were not widely available or used. To encourage mothers to give birth in health facilities, the Ethiopian government implemented a policy of free delivery services in all public health facilities in July 2013. The government established a primary health care facility in the East Shewa administrative region where the national free delivery service intervention was implemented in all public health centers. Primarylevel care has been established by the government, which consists of health posts, health centers (HCs), and primary/district hospitals. Five HCs (Adama, Awashmelkasa, Bishoftu, Modjo, and Walinchity) with complete data from the previous nine years were chosen. For the nine years from July 2007 to June 2016, 108 data points were available, including facilitybased usage of delivery services (72 pre and 36 postintervention phases). The total number of monthly births in the five HCs mentioned above served as the outcome variable (Fig. 4).
The Ethiopian government implemented a national free delivery service. After the formal intervention implementation (2013/07), the Ethiopian government undertook a series of works to get this policy intervention fully off the ground, such as purchasing emergency vehicles, increasing the number of beds in health facilities, and related delivery equipment for women, and training relevant health care workers. Additionally, most pregnant women do not enjoy the benefits immediately if the gap between policy advocacy and public awareness is considered. Meanwhile, even if women in the Shewa region learned about the free delivery service policy when the intervention was formally implemented (2013/07) and became pregnant immediately, they only gave birth after nearly ten months. Therefore, the intervention effect cannot be fully interrupted at the time point when the intervention is implemented, as assumed by the CSR model. Thus, this intervention is considered an ideal application case for the optimized models.
Selection of the length of the transition period
Considering the gap between policy advocacy and public awareness and the length of a woman's pregnancy, we assumed that the maximum range of \(L\) was 10 months, that is, \({L}_{m}=10\). Among the possible scenario decision sets (\(L=0, 1, 2,\dots ,10\)), the \(L\) was selected based on the minimum MSE of the model application. For the LND pattern, we additionally assumed the \(Ratio=\frac{1}{3}\), namely, the halfeffect release time of the national free delivery service intervention accounted for \(\frac{1}{3}\) of the total transition period. Accordingly, for the LNFD pattern, the halfeffect release time of the national free delivery service intervention accounted for \(\frac{2}{3}=1\frac{1}{3}\) of the total transition period. The MSEs of the optimized model for all scenarios (all possible \(L\)) of the four intervention effect patterns are shown in Fig. 5.
From Fig. 5, we learned that the MSEs of the OSR models under different distribution patterns of the intervention effect were smaller than those of the CSR model (\(L=0\)), regardless of \(L\), indicating that the OSR models fit the data better. In the different OSR models, with the increase in \(L\), the change trajectories of the MSEs are different.
Taking the minimum MSE as the selection metric, different OSR models selected different transition lengths. The results of the selected \(L\) for the four distribution patterns of the intervention effect and the corresponding model statistics are presented in Table 1. As shown in Fig. 5 and Table 1, the selected \(L\) for the UD pattern of intervention effects was 8, that is, \({T}_{2}\) were 8 months after \({T}_{0}\). The selected \(L\) for the ND, LND and LNFD patterns were 5, 10, and 3, respectively. Among the four distribution patterns of the intervention effect, the OSRUD achieved the smallest MSE (408.5852).
In Additional file 1: Fig. S1, there were parameter estimation result planes and corresponding external studentized residuals for different distribution patterns of intervention effect, which indicated suitable fits.
In addition to MSE, the mean absolute error (MAE), mean absolute percentage error (MAPE), and median absolute deviation (MAD) can also be used as model fit metrics. The results of the \(L\) selection corresponding to the minimum of the other model fit metrics are shown in the Additional file 2: Table S1 and Additional file 3: Fig. S2, Additional file 4: Fig. S3, Additional file 5: Fig. S4. Different model fit metrics may lead to different selection results for \(L\).
Results of modeling
Results of models with selected \({\varvec{L}}\)
For intervention evaluation, the longterm impact \({\beta }_{3}\) is the most important evaluation indicator [21]. The results of the parameter estimation for the classic and four optimized models are listed in Table 2. We find heterogeneity in the parameter estimation results between the CSR and OSR models. The longterm impact estimate \({\widehat{\beta }}_{3}\) 1.4251 (95%CI: 0.6574, 2.1928) of the CSR model was higher than the estimates of the OSR model; specifically, the estimates were 0.1755 (− 0.6432, 0.9942), 0.7318 (− 0.0394, 1.5030), 0.4045 (− 0.4036, 1.2125) and 0.8751 (0.1241, 1.6260) for OSRUD, OSRND, OSRLND, and OSRLNFD models, respectively. Compared with the OSR models, the CSR model overestimated the longterm impact \({\widehat{\beta }}_{3}\). It is worth noting that OSRUD, OSRND, and OSRLND had longterm impacts estimates greater than zero, indicating positive longterm impacts of the interventions; however, these were not statistically significant.
\({\widehat{{\varvec{\beta}}}}_{3}\) estimates for all possible scenarios of \({\varvec{L}}\)
We estimated the longterm impact \({\widehat{\beta }}_{3}\) of intervention effects for all possible length scenarios with OSRUD, OSRND, OSRLND, and OSRLNFD models; the corresponding results are shown in Fig. 6. The estimates of \({\widehat{\beta }}_{3}\) were sensitive to the length of the transition period \(L\). With an increase in \(L\), the estimates of longterm impact \({\widehat{\beta }}_{3}\) kept decreasing for all four types of OSR models. There were slight differences between the estimates of different OSR models with the same \(L\). For the outcome time series analyzed in this study, OSRLND tended to provide the largest longterm impact estimates, whereas OSRLNFD did the opposite. When \(L\) was too large, some OSR models (OSRUD and OSRLNFD) estimated a negative longterm impact of the national free delivery service intervention, which was not convincing.
We presented the other coefficient estimate results of different OSR models under all possible length scenarios in the Additional file 6: Table S2, Additional file 7: Table S3, and Additional file 8: Table S4 correspond to \({\widehat{\beta }}_{0}\), \({\widehat{\beta }}_{1}\) and \({\widehat{\beta }}_{2}\), respectively. Additional file 6: Table S2 and Additional file 7: Table S3 showed that the estimates of \({\widehat{\beta }}_{0}\) and \({\widehat{\beta }}_{1}\) were almost the same for different choices of \(L\). While the estimates of \({\widehat{\beta }}_{2}\) were sensitive to the length \(L\) of the transition period, as shown in Additional file 8: Table S4, the estimated \({\widehat{\beta }}_{2}\) values of the different OSR models increased as \(L\) increased.
Discussion
In this study, to characterize different distribution patterns of intervention effects during the transition period, we introduced different CDFs to the CSR model and proposed the corresponding OSRUD, OSRND, OSRLND, and OSRLNFD models. Using the national free delivery service policy intervention in Ethiopia as an empirical study, the OSR models fit the outcome time series (the total number of births per month in the five Ethiopian health centers) better than the CSR model based on the model fit metric MSE. In this process, we suggest a possible datadriven approach to select the length of the transition period for OSR models by using MSE as a fitness metric.
The existence of a transition period between the pre and postintervention phases is common, especially for policy interventions to improve healthcare quality and equity [22, 23]. Regardless of the \(L\) choice, the MSE of the OSR models under different distribution patterns of the intervention effect was smaller than that of the CSR model, indicating that the assumption of the OSR model for the existence of the transition period was reasonable and the corresponding model optimization was more consistent with the actual characteristics of the real data.
Although OSR models fitted the data better than did the CSR model, there was heterogeneity in the longterm impact estimates \({\widehat{\beta }}_{3}\) of interventions when \(L\) and the distribution pattern of the intervention effects during the transition period (UD, ND, LND, and LNFD) varied. The modeling results were especially sensitive to change in \(L\). For example, in the current study, when the length of the transition period was too large, and some specific distribution patterns (UD and LNFD) were chosen, the OSR models might estimate nonconvincing results based on the empirical study. In addition, some estimates of longterm impacts may not be statistically significant. Therefore, the selection of \(L\) and distribution patterns of intervention effects are critical for the OSR models, which is also a difficulty in conducting the OSR analysis highlighted in this study. Notably, there could be two possible approaches for selecting \(L\) and the distribution patterns: the implementationdriven and the datadriven approach.
Under the implementationdriven approach, the length of the transition period and the distribution patterns of intervention effects can be determined by the researcher according to the implementation process. The intervention implementer mastered the exact information about the intervention process [24,25,26]. For example, in the Ethiopian free delivery service policy study, the transition period was defined as the duration between the beginning and end of the training of medical staff; however, this was not reported or considered in the original study [20]. For the four possible distribution patterns we assumed, the most appropriate one based on the training process, such as the frequency of training and number of people trained per session, could also be defined. From this sense, the parameters set by the implementationdriven approach are in line with the intervention process and have practical significance [27, 28].
Although it is better to select \(L\) and the distribution pattern of intervention effect in line with the intervention process, this information is not always accessible to the public. In this study, a datadriven approach was adopted. The choice of \(L\) should strike a balance between being datadriven with minimal metrics (MSE or other possible metrics) and being simple enough to interpret from the interventional perspective of epidemiology, medicine, and policy [29]. From a purely datadriven perspective, the results of the choice of \(L\) are not entirely consistent when different modelfitting metrics of goodness (e.g., MSE, MAE, MAPE, MAD) are used. Among the different metrics, confusion remains regarding their superiority. None of the indicators is inherently superior, and their relative superiority is conditional [30, 31]. The MAE is superior for Laplace errors, whereas the other metrics are preferable when the errors follow other distributions [32]. So far, the residual distribution type after applying the OSR models is unknown. Given this, we did not have a consensus on an “optimal’’ metric for selected \(L\) [33]. In addition, the excessive pursuit of a minimum of one metric is most likely to result in overfitting of the model [34]. Therefore, the datadriven approach for \(L\) is only one of the reference methods, and it is more appropriate to choose it according to the actual situation of the intervention.
Moreover, the uniqueness of the selected model parameters is not always necessary. Referring to the idea of the distributed lag model (DLM) [35, 36], we can describe the entire variance situation (\(L\)) of longterm impact estimates by specifying a series of transition lengths \(L\), as illustrated in Fig. 6. Specifying a range of \(L\) is one part of the sensitivity analysis [37], which is vital for judging the robustness of the corresponding estimates. In conclusion, for the selection of model parameters, our recommendation is, first, to conform to their practical meaning according to the implementation process; second, to take a possible datadriven approach; finally to calculate the estimated values of the model parameters for all possible cases and describe the entire variance situation of the estimated values.
In this study, we propose OSR models with four distribution patterns of intervention effects during the transition period and utilized them to estimate the specific values of the longterm impact of policy intervention. We examined real data and explained the modeling procedures in detail to provide practical insights into the impact of Ethiopia’s free delivery service policy intervention on the total number of births per month at the five health centers, providing new insights into the field of intervention evaluation.
A key strength and new aspect of this study is the inclusion of the intervention effects of the transition period into the model, rather than ignoring or cursorily removing the time points within the transition period. To correspond to the different distribution patterns of intervention effects during the transition period in practice, we abstracted four different forms of CDFs (UD, ND, LND, and LNFD), and proposed correspondingly different OSR models. In addition, for the selection of the length of the transition period, we suggested a datadriven selection approach, although it has some shortcomings.
Taking a specific policy intervention as an example, we conducted an empirical study to estimate the longterm impact of policy intervention. Simultaneously, we estimated and compared coefficients that reflected the longterm impacts of the intervention in the OSR model under 44 scenarios, considering the distribution patterns and \(L\). This study provides a comprehensive description of the estimated values of OSR models under different parameter settings and provides a reference for analyzing their sensitivity and subsequent research.
There are a few limitations of this study. First, we did not fundamentally offer a selected method for \(L\), although we suggested a possible datadriven approach. The choice of \(L\) still depends more on its practical significance without its fixed paradigm. Second, we examine our OSR model using a single policy intervention dataset with different parameter settings to check the sensitivity of the estimating results. In addition, more reexaminations of the OSR models using practical datasets are encouraged to be done. Thirdly, the current OSR models are only applicable for continuous outcome variables and have difficulty adapting to categorical data. We need to further improve the OSR model to accommodate such data types.
Conclusions
The CSR model may not always be sufficient when there is a transition period between the pre and postintervention phases. In such cases, our OSR models can be a potential alternative, because they use different CDFs to characterize the distribution patterns of the intervention effects during the transition period. To illustrate the effectiveness of the OSR models, we conducted an empirical study on national free delivery service policy intervention in Ethiopia. We estimated the longterm impact of policy intervention using both the CSR model and different OSR models and compared their estimated results. Our findings suggest that the OSR models, which are optimized to match the actual characteristics of the data, are powerful tools for accurately estimating the longterm impacts of interventions. These models demonstrate smaller MSEs and provide more accurate estimates of the longterm impacts than the CSR model. Our study emphasizes the importance of using appropriate statistical methods when evaluating intervention effects in the presence of a transition period.
Availability of data and materials
The data set used in the current study is completely public, and anyone can obtain it without any additional enquiry.
Abbreviations
 ITS:

Interrupted time series
 CSR:

Classic segmented regression
 OSR:

Optimized segmented regression
 CDF:

Cumulative distribution functions
 PDF:

Probability density functions
 UD:

Uniform distribution
 ND:

Normal distribution
 LND:

Lognormal distribution
 LNFD:

Lognormal flip distribution
 MSE:

Mean square error
 MAE:

Mean absolute error
 MAPE:

Mean absolute percentage error
 MAD:

Median absolute deviation
 DLM:

Distributed lag model
References
Harris AD, McGregor JC, Perencevich EN, Furuno JP, Zhu J, Peterson DE, et al. The use and interpretation of quasiexperimental studies in medical informatics. J Am Med Inform Assoc. 2006;13:16–23.
Lopez Bernal J, Soumerai S, Gasparrini A. A methodological framework for model selection in interrupted time series studies. J Clin Epidemiol. 2018;103:82–91.
Michielutte R. Use of an interrupted timeseries design to evaluate a cancer screening program. Health Educ Res. 2000;15:615–23.
Ewusie JE, Blondal E, Soobiah C, Beyene J, Thabane L, Straus SE, et al. Methods, applications, interpretations and challenges of interrupted time series (ITS) data: protocol for a scoping review. BMJ Open. 2017;7:e016018.
Kontopantelis E, Doran T, Springate DA, Buchan I, Reeves D. Regression based quasiexperimental approach when randomisation is not an option: interrupted time series analysis. BMJ. 2015. https://doi.org/10.1136/BMJ.H2750.
Taljaard M, McKenzie JE, Ramsay CR, Grimshaw JM. The use of segmented regression in analysing interrupted time series studies: an example in prehospital ambulance care. Implement Sci. 2014;9:77.
Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr. 2013;13:S38–44.
Linden A. Conducting interrupted timeseries analysis for single and multiplegroup comparisons. Stata J Promot Commun Stat Stata. 2015;15:480–500.
Cruz M, Bender M, Ombao H. A robust interrupted time series model for analyzing complex health care intervention data. Stat Med. 2017;36:4660–76.
Feldstein AC, Smith DH, Perrin N, Yang X, Simon SR, Krall M, et al. Reducing warfarin medication interactions. Arch Intern Med. 2006;166:1009.
Grijalva CG, Nuorti JP, Arbogast PG, Martin SW, Edwards KM, Griffin MR. Decline in pneumonia admissions after routine childhood immunisation with pneumococcal conjugate vaccine in the USA: a timeseries analysis. Lancet. 2007;369:1179–86.
Lopez Bernal JA, Lu CY, Gasparrini A, Cummins S, Wharham JF, Soumerai SB. Association between the 2012 Health and Social Care Act and specialist visits and hospitalisations in England: a controlled interrupted time series analysis. PLoS Med. 2017;14:e1002427.
Bender M, Williams M, Su W, Hites L. Refining and validating a conceptual model of clinical nurse leader integrated care delivery. J Adv Nurs. 2017;73:448–64.
Owuor H, Amolo AS. Interrupted time series analysis of free maternity services policy in Nyamira County, Western Kenya. PLoS ONE. 2019;14:e0216158.
Hategeka C, Ruton H, Karamouzian M, Lynd LD, Law MR. Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review. BMJ Glob Health. 2020;5:e003567.
Serumaga B, RossDegnan D, Avery AJ, Elliott RA, Majumdar SR, Zhang F, et al. Effect of pay for performance on the management and outcomes of hypertension in the United Kingdom: interrupted time series study. BMJ. 2011;342:d108–d108.
Cruz MF. Interrupted Time Series Models for Assessing Complex Health Care Interventions. 2019. https://escholarship.org/content/qt3p73v3j2/qt3p73v3j2.pdf?t=pyox9h&v=lg.
AcevesGonzález C, Cook S, May A. Bus use in a developing world city: Implications for the health and wellbeing of older passengers. J Transp Health. 2015;2:308–16.
Petrou S, Gray A. Economic evaluation alongside randomised controlled trials: design, conduct, analysis, and reporting. BMJ. 2011;342:d1548–d1548.
Demissie A, Worku A, Berhane Y. Effect of implementing a free delivery service policy on women’s utilization of facilitybased delivery in central Ethiopia: an interrupted time series analysis. J Pregnancy. 2020;2020:1–7.
Saldana L. The stages of implementation completion for evidencebased practice: protocol for a mixed methods study. Implement Sci. 2014;9:43.
Parmar D, Banerjee A. How do supply and demandside interventions influence equity in healthcare utilisation? Evidence from maternal healthcare in Senegal. Soc Sci Med. 2019;241:112582.
Hoque DME, Arifeen SE, Rahman M, Chowdhury EK, Haque TM, Begum K, et al. Improving and sustaining quality of child health care through IMCI training and supervision: experience from rural Bangladesh. Health Policy Plan. 2014;29:753–62.
Brereton L, Carroll C, Barnston S. Interventions for adult family carers of people who have had a stroke: a systematic review. Clin Rehabil. 2007;21:867–84.
BuljacSamardzic M, Dekkervan Doorn CM, van Wijngaarden JDH, van Wijk KP. Interventions to improve team effectiveness: a systematic review. Health Policy. 2010;94:183–95.
Hansen H, Metzl JM. New medicine for the U.S. health care system. Acad Med. 2017;92:279–81.
Sutherland WJ, Burgman M. Policy advice: use experts wisely. Nature. 2015;526:317–8.
Murphy JM, Sexton DMH, Barnett DN, Jones GS, Webb MJ, Collins M, et al. Quantification of modelling uncertainties in a large ensemble of climate change simulations. Nature. 2004;430:768–72.
Armstrong B. Models for the relationship between ambient temperature and daily mortality. Epidemiology. 2006. https://doi.org/10.1097/01.ede.0000239732.50999.8f.
Willmott C, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res. 2005;30:79–82.
Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)?—Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014;7:1247–50.
Hodson TO. Rootmeansquare error (RMSE) or mean absolute error (MAE): when to use them or not. Geosci Model Dev. 2022;15:5481–7.
Hossin M, Sulaiman MNA. Review on evaluation metrics for data classification evaluations. Int J Data Min Knowl Manag Process. 2015;5:01–11.
Bottou L, Curtis FE, Nocedal J. Optimization methods for largescale machine learning. SIAM Rev. 2018;60:223–311.
Gasparrini A, Armstrong B, Kenward MG. Distributed lag nonlinear models. Stat Med. 2010;29:2224–34.
Armstrong B. Models for the relationship between ambient temperature and daily mortality. Epidemiology. 2006;17:624–31.
Eyduran E, Ozdemir T, Alarslan E. Importance of diagnostics in multiple regression analysis. J Appl Sci. 2005;5:1792–6.
Acknowledgements
We thank Dr. Ayalneh Demissie (College of Health Sciences, Arsi University, Ethiopia) for granting permission for his research article to become an openaccess article under the Creative Commons Attribution License and for generously publishing the raw data in his corresponding research article. So that we can use the data as an example for an empirical study to verify pros and cons of optimized models, as well as their applicability.
Funding
This work is supported by the National Natural Science Foundation of China (72074229). This funding source had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data, or decision to submit results.
Author information
Authors and Affiliations
Contributions
XL Z designed the study and performed data analyses. XL Z wrote the first draft and developed subsequent manuscript. To be more specific, XL Z is in the charge of conceptualization, data curation, methodology, formal analysis, software, visualization and writingoriginal draft. KP W is responsible for supervision, validation and review & editing. RY, YP, YZ, DK, and QW participated in the writing and revision of the paper, and gave some constructive comments. As the corresponding author, WC participate all the above processes, guided the writing and critically revised the manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Supplementary Information
Additional file 1
. Table S1: L selected results with different model fit metrics.
Additional file 2
. Fig. S1: Parameter estimation result planes and corresponding external studentized residuals.
Additional file 3.
Fig. S2: MAEs under different distribution patterns of intervention effect.
Additional file 4
. Fig. S3: MAPEs under different distribution patterns of intervention effect.
Additional file 5
. Fig. S4: MADs under different distribution patterns of intervention effect.
Additional file 6
. Table S2: \({\widehat{\beta }}_{0}\) estimation results and corresponding 95% CIs.
Additional file 7
. Table S3: \({\widehat{\beta }}_{1}\) estimation results and corresponding 95% CIs.
Additional file 8
. Table S4: \({\widehat{\beta }}_{2}\) estimation results and corresponding 95% CIs.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Zhang, X., Wu, K., Pan, Y. et al. Optimized segmented regression models for the transition period of intervention effects. glob health res policy 8, 29 (2023). https://doi.org/10.1186/s41256023003123
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41256023003123