R E S E A R C H A R T I C L E Open Access
Quality of life analyses in patients with multiple myeloma: results from the
Selinexor (KPT-330) Treatment of Refractory Myeloma (STORM) phase 2b study
Gabriel Tremblay
1*, Patrick Daniele
1, Janis Breeze
1, Lingling Li
2, Jatin Shah
2, Sharon Shacham
2, Michael Kauffman
2, Monika Engelhardt
3, Ajaj Chari
4, Ajay Nooka
5, Dan Vogl
6, Maria Gavriatopoulou
7, Meletios-Athanasios Dimopoulos
8, Paul Richardson
9, Noa Biran
10, David Siegel
10, Philip Vlummens
11,
Chantal Doyen
12, Thierry Facon
13, Mohamad Mohty
14, Nathalie Meuleman
15, Moshe Levy
16, Luciano Costa
17, James E. Hoffman
18, Michel Delforge
19, David Kaminetzky
20, Katja Weisel
21, Marc Raab
22, David Dingli
23, Sascha Tuchman
24, Frenzel Laurent
25, Ravi Vij
26, Gary Schiller
27, Philippe Moreau
28, Joshua Richter
29,
Martin Schreder
30, Klaus Podar
31, Terri Parker
32, Robert Frank Cornell
33, Karlin Lionel
34, Sylvain Choquet
35and Jagannath Sundar
29Abstract
Background:Selinexor is an oral, selective nuclear export inhibitor. STORM was a phase 2b, single-arm, open-label, multicenter trial of selinexor with low dose dexamethasone in patients with penta-exposed relapsed/refractory multiple myeloma (RRMM) that met its primary endpoint, with overall response of 26% (95% confidence interval [CI], 19 to 35%). Health-related quality of life (HRQoL) was a secondary endpoint measured using the Functional Assessment of Cancer Therapy–Multiple Myeloma (FACT-MM). This study examines impact of selinexor treatment on HRQoL of patients treated in STORM and reports two approaches to calculate minimal clinically important differences for the FACT-MM.
Methods:FACT-MM data were collected at baseline, on day 1 of each 4-week treatment cycle, and at end of treatment (EOT). Changes from baseline were analyzed for the FACT-MM total score, FACT-trial outcome index (TOI), FACT-General (FACT-G), and the MM-specific domain using mixed-effects regression models. Two approaches for evaluating minimal clinically important differences were explored: the first defined as 10% of the instrument range, and the second based on estimated mean baseline differences between Eastern Cooperative Oncology Group performance status (ECOG PS) scores. Post-hoc difference analysis compared change in scores from baseline to EOT for treatment responders and non-responders.
© The Author(s). 2021Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visithttp://creativecommons.org/licenses/by/4.0/.
The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
* Correspondence:gabrieltremblay@pshta.com
1Purple Squirrel Economics, 1600 Notre Dame W, Suite 201, Montreal, QC H3J 1M1, Canada
Full list of author information is available at the end of the article
Results:Eighty patients were included in the analysis; the mean number of prior therapies was 7.9 (standard deviation [SD] 3.1), and mean duration of myeloma was 7.6 years (SD 3.4). Each exploratory minimal clinically important difference threshold yielded consistent results whereby most patients did not experience HRQoL decline during the first six cycles of treatment (range: 53.9 to 75.7% for the first approach; range: 52.6 to 72.9% for the second). Treatment responders experienced less decline in HRQoL from baseline to EOT than non-responders, which was significant for the FACT-G, but not for other scores.
Conclusion:The majority of patients did not experience decline in HRQoL based on minimal clinically important differences during early cycles of treatment with selinexor and dexamethasone in the STORM trial. An anchor-based approach utilizing patient-level data (ECOG PS score) to define minimal clinically important differences for the FACT-MM gave consistent results with a distribution-based approach.
Trial registration:This trial was registered on ClinicalTrials.gov under the trial-IDNCT02336815on January 8, 2015.
Keywords:Patient reported outcomes, Health-related quality of life, FACT-MM, Multiple myeloma, Selinexor
Background
Multiple myeloma (MM) is the second most common form of hematologic cancer in the United States (US), with an estimated 32,110 new cases in 2019 [1]. MM is characterized by the abnormal proliferation of clonal plasma cells in the bone marrow, alterations in the bone marrow microenvironment, and the production of mono- clonal protein and other bioactive molecules by malignant cells [2]. Patients with MM experience a burden of symp- toms due to clinical manifestations associated with end organ damage, including hypercalcemia, renal insuffi- ciency, renal failure, anemia, immune dysfunction, and bone destruction [3]. Current treatment modalities for MM include proteasome inhibitors, immunomodulatory agents, and monoclonal antibodies, which are often used in doublet or triplet drug regimens, as well as chemother- apy, bone marrow transplant, and radiation therapy [4,5].
According to the US Surveillance, Epidemiology, and End Results (SEER) database, the 5-year survival rate for patients diagnosed with MM from 2010 to 2016 was esti- mated to be 53.9% [1]. At present, MM remains generally incurable, and almost all patients relapse and develop re- fractory disease [6, 7]. With each relapse, patients face worsening clinical outcomes due to declining efficacy of treatment regimens, shorter duration of response, and in- creased refractoriness to therapeutic agents [6,8–10].
The refractory nature of MM and severity of symp- toms impact quality of life (QoL) and limit availability of treatment options for patients [11–13]. Previous studies have provided evidence of poor QoL among patients with relapsed/refractory MM (RRMM), who face a sig- nificant burden of disease and cumulative impacts of prior treatments and treatment-associated adverse events [12]. As patients progress through multiple lines of therapy and exhaust available treatment options with lessening clinical benefit, they may decide between ex- perimental therapy, retreatment strategies, and symp- tomatic care [14].
Selinexor is a first-in-class selective oral nuclear trans- port inhibitor that has been approved by the US Food and Drug Administration in combination with low-dose dexamethasone for the treatment of adults with RRMM who have received at least four prior therapies and whose disease is refractory to at least two proteasome inhibitors, two immunomodulatory agents, and an anti- CD38 monoclonal antibody [15]. Efficacy and safety of selinexor in RRMM were demonstrated in the STORM (Selinexor Treatment of Refractory Myeloma) phase 2b trial (NCT02336815; N = 122; n = 83 with penta- refractory myeloma i.e., refractory to bortezomib, carfil- zomib, lenalidomide, pomalidomide, and daratumumab [16]. Results of STORM have been previously published elsewhere [16]. Briefly, the primary endpoint was overall response, defined as a partial response or better, and was observed in 26% of patients (95% CI, 19 to 35%). Among all responders, the median duration of response was 4.4 months. In the modified intent-to-treat (mITT) popula- tion, the median overall survival was 8.6 months. The most common adverse events were thrombocytopenia, nausea, fatigue, anemia, decreased appetite, and de- creased weight, which were managed with supportive care and dose modifications. Patient-reported outcome (PRO) data were collected using the Functional Assess- ment of Cancer Therapy – Multiple Myeloma (FACT- MM) instrument at study screening, at each cycle, and at end of treatment. This analysis provides an assess- ment of patient-reported QoL with selinexor and low dose dexamethasone in the STORM trial. In addition, it aims to evaluate the proportion of patients with minimal clinically meaningful change in QoL from baseline.
Methods
Study design and quality of life assessment
The patient eligibility criteria and study design of STORM have been previously described [16]. Briefly, STORM was a phase 2b, multicenter, open-label study
of twice-weekly oral selinexor (80 mg) in combination with dexamethasone (20 mg) in patients with progressive MM [16]. The mITT population included 122 patients (median age 65.2 years), of whom 83 (68%) had penta- refractory MM. Patient-reported QoL was a secondary endpoint and was assessed at study screening and on day 1 of each 4-week treatment cycle beginning at cycle 2, and at end of treatment with the FACT-MM. The FACT-MM is a disease-specific instrument and has been previously applied in the assessment of health-related QoL (HRQoL) among patients with RRMM in investiga- tional studies [17–20]. FACT-MM combines the General version of the FACT (FACT-G; 27 items) with an MM- specific subscale (MM domain; 14 items). The MM do- main addresses symptomatic burden and disease-specific well-being [19]. The total FACT-MM score is obtained by adding individual subscale scores for physical well- being (7 items), social/family well-being (7 items), emo- tional well-being (6 items), and functional well-being do- mains (7 items) of the FACT-G and the MM domain [19]. The FACT-MM Trial Outcomes Index (TOI) is comprised of the physical and functional subscales and the MM domain [19].
Statistical analysis
The analysis methods have been previously presented in brief in a conference abstract by Breeze et al. [21]. The QoL analysis dataset consisted of 80 patients in the mITT population with FACT-MM data at baseline and at least one follow-up cycle or end of treatment. Baseline characteristics, including demographic information (e.g., age, gender, race) and clinical variables were summa- rized using means and standard deviations for continu- ous variables and counts and proportions for categorical variables. Race was collapsed as white, black, or other for inclusion in the analysis. Patients with missing demo- graphic or clinical variable data were omitted from the HRQoL analyses. Completeness was defined according to the FACT-MM Scoring Guidelines (Version 4), which allows subscales to be calculated if > 50% of items are present, and total scores if > 80% of items are present.
For each follow-up cycle, the magnitude of change from baseline was evaluated using mixed-effects regres- sion models, allowing for random slope and intercept terms for repeated measures for the FACT-MM total score, FACT-G, FACT-MM TOI, and the MM domain.
This type of regression model assumes data are missing at random (MAR).
Multivariable adjusted models were constructed which considered baseline scores and baseline characteristics including demographic data (age, gender, and race), Eastern Cooperative Oncology Group (ECOG PS) score (categorized as 0 or 1 to 2), and years since diagnosis as prognostic variables. During selection of variables for
model inclusion, each model was evaluated for robust- ness using model fit parameters including Akaike’s In- formation Criterion (AIC), Bayesian Information Criterion (BIC), and the model chi-square statistic [22, 23]. Final adjusted models which include baseline scores and specified baseline characteristics that improved fit are reported.
The minimal clinically important difference represents the smallest meaningful improvement in the score of a PRO domain, interpreted as a minimum level of per- ceived benefit by patients, and has been generally uti- lized in the translation of HRQoL outcomes to clinical practice and treatment choice [24, 25]. To our know- ledge, no minimal clinically important difference thresh- olds have been reported for the FACT-MM. In the current analysis, clinically meaningful changes were eval- uated by examination of minimal clinically important difference using two anchor-based approaches. In the first approach, minimal clinically important difference was defined as 10% of the instrument range, a threshold that has been associated with meaningful HRQoL change in patients using cancer-specific instruments such as the FACT-G [26]. An exploratory approach was developed for this analysis where HRQoL was‘anchored’
to differences in ECOG PS scores, which is a measure used by clinicians to assess and describe the clinical sta- tus and prognosis of patients and to guide treatment.
Previous anchor-based analyses have used clinical char- acteristics such as ECOG PS or laboratory findings such as hemoglobin levels to derive MCIDs for disease- specific FACT subscales [27,28]. In these analyses, adja- cent categories in selected characteristics were presumed to represent clinically distinguishable groups within the HRQoL dataset. Following these approaches, the current analysis used patient-level data to group patients into categories based on physician-assessed baseline ECOG PS. Due to the low number of patients included in the analysis, ECOG PS 1 and 2 categories were grouped to- gether. The minimal clinically important difference was thus defined as the difference in mean baseline scores between patients with ECOG PS of 0 compared with those with ECOG PS of 1 to 2, adjusted for significant baseline characteristics (race, age) to account for con- founding arising from the non-randomized nature of the ECOG groupings.
For either approach, patients with a minimal clinically important difference improvement were considered as having HRQoL improvement [24]. Patients with less than the minimal clinically important difference change were considered stable. Patients with a minimal clinically important difference decrease were considered as having HRQoL decline [29].
In addition, post-hoc testing was carried out to exam- ine HRQoL trends between treatment responders and
non-responders. Responders were defined as patients with overall response (partial response or better). Since these subgroups were not randomized, a difference ana- lysis (a quasi-experimental approach) was used to statis- tically compare the differences in HRQoL scores from baseline to end of treatment for treatment responders and non-responders. Estimated mean differences be- tween responders and non-responders were directly de- rived from the mixed effects model.
Results
The key results of the analysis were previously reported in a conference abstract by Breeze et al. [21]. Of 122 pa- tients in the STORM mITT population, 80 (66%) com- pleted the FACT-MM at baseline and at one or more follow-up cycle or at end of treatment (Fig. 1). In the QoL analysis population, 21 patients experienced partial response or better and were considered as treatment re- sponders (26%; n = 21/80). Baseline characteristics and clinical variables for the QoL analysis population are summarized in Table1.Patients were heavily pretreated, with a mean number of prior treatments of 7.9 (SD 3.1, range 3 to 18), and a mean duration of myeloma of 7.6 years (SD 3.4, range 1.2 to 18.6). Excluded patients were similar to the HRQoL analysis population with respect to mean age (65.0 vs 63.7 years), mean number of previ- ous regimens (7.2 vs 7.9), and mean time from initial diagnosis (6.6 vs 7.6 years). Minor differences were noted with respect to sex (62.5 vs 50% male), proportion of
patients with high-risk cytogenetics (59.5 vs 50%), and R-ISS risk score stage II (69.0 vs 61.2%). Given the lack of substantial differences in prognostic factors between the HRQoL analysis population and the excluded pa- tients, the impact of excluding these patients is unlikely to affect the results of the analysis.
Results from the mixed-effects regression analysis for the FACT-MM total score, FACT-G, FACT-MM TOI, and the MM domain are shown in Table2. Reported co- efficients represent the mean change from baseline as es- timated by the mixed effects model where a negative value indicates a relative decline from baseline, and a positive value indicates an improvement from baseline.
The number of patients who remained in the study de- clined with each successive cycle, reflecting the highly advanced nature of disease and the proportion of pa- tients who remained well-enough to continue treatment.
Most patients showed a monotonic pattern of missing- ness. Disregarding the end of treatment, only two pa- tients (2.5%) showed intermittent missingness. Scores for the FACT-MM, FACT-G and FACT-MM TOI de- creased from baseline at each cycle and at the end of treatment, with significant decreases observed in early cycles of treatment as well as end of treatment. The MM domain score did not change significantly at any cycle or at end of treatment.
Next, minimal clinically meaningful changes based on the FACT-MM were evaluated using two anchor-based approaches; the first based on a 10% difference in scale
Fig. 1Study flowchart. mITT: modified intent-to-treat; QoL = quality of life
range, and the second based on mean baseline differ- ences between ECOG groups, adjusted for baseline char- acteristics of age and race. The minimal clinically important difference thresholds calculated for the two approaches are shown in Table 3. These thresholds can be interpreted as the smallest clinically meaningful change for a particular domain. For example, a decrease of the FACT-MM total score by 13.5 points from base- line would represent a clinically meaningful decline as defined by the ECOG-based anchor.
The number and proportion of patients who im- proved, experienced no change, or declined in HRQoL based on the minimal clinically important difference an- chored by ECOG groups is shown in Table 4. The re- sults of the analysis based on minimal clinically important difference anchored by a 10% difference in scale range is shown in theSupplementary Information.
The combined proportions of patients who experienced no change in HRQoL or improvements compared to baseline through cycle 6 based on the FACT-MM total score, FACT-G, FACT-MM TOI, and the MM domain were generally greater than the proportions who experi- enced declines. Results of the analyses, according to the two minimal clinically important difference definitions, were consistent.
Post-hoc analysis evaluated trends in HRQoL change from baseline to end of treatment between treatment re- sponders and non-responders using a mixed-effects model. It should be noted that in the QoL dataset, there were 21 responders (26%; n= 21/80), suggesting that re- sponders were no more likely than non-responders to
Table 2Change in HRQoL as evaluated by a mixed-effects regression model Change in FACT-MM total
scorea
Change in FACT-G scorea Change in FACT-MM TOI total scoreb
Change in MM domain scorec
Max N
Coefficient (95% CI) p- value
Coefficient (95%
CI)
p- value
Coefficient (95%
CI)
p- value
Coefficient (95%
CI)
p- value Cycle 2 71d −3.94 (−8.03 to 0.16) 0.059 −3.87 (−6.66 to
−1.08)
0.007 −4.86 (−8.44 to
−1.29)
0.008 −0.10 (−2.11 to 1.92)
0.926
Cycle 3 42 −5.17 (−10.10 to
−0.25)
0.040 −5.69 (−9.05 to
−2.33)
0.001 −4.92 (−9.21 to
−0.64)
0.024 0.61 (−1.81 to 3.03) 0.619 Cycle 4 25 −5.47 (−11.38 to 0.45) 0.070 −4.00 (−8.04 to 0.04) 0.052 −5.65 (−10.79 to
−0.52)
0.031 −1.35 (−4.28 to 1.58)
0.367
Cycle 5 13 −3.97 (−11.58 to 3.65) 0.307 −3.62 (−8.82 to 1.59) 0.173 −4.60 (−11.22 to 2.03)
0.174 0.21 (−3.43 to 3.85) 0.911
Cycle 6 8e −3.54 (−13.59 to 6.51) 0.490 −4.62 (−11.49 to 2.25)
0.187 −3.35 (−12.10 to 5.41)
0.453 2.57 (−1.95 to 7.09) 0.265
End of treatment
39f −10.45 (−16.19 to
−4.70)
<
0.0001 −8.14 (−11.68 to
−4.60)
<
0.001 −9.39 (−14.22 to
−4.57)
<
0.001 −1.80 (−4.93 to 1.34)
0.261
AICAkaike’s Information Criterion,BICBayesian Information Criterion,FACT-GFunctional Assessment of Cancer Therapy–General,FACT-MMFunctional Assessment of Cancer Therapy–Multiple Myeloma,MMmultiple myeloma,TOITrial Outcomes Index,CIconfidence interval
aAdjusted for baseline score, race, sex, and years since diagnosis
bAdjusted for baseline score, race, sex, and number of prior regimens
cAdjusted for baseline score, race, sex, ECOG performance score, and number of prior regimens
dn= 70 for FACT-MM total score and FACT-G
en= 7 for FACT-MM total score, FACT-G, and FACT-TOI
fn= 38 for FACT-MM total score, FACT-G, and FACT-TOI
Table 1Patient baseline characteristics Patient baseline characteristics
N (%) unless otherwise indicated QoL analysis population (N= 80)
Male 50 (62.5)
Mean age (SD), range 63.7 (9.4), 40.4 to 85.9
Race
White 57 (71)
Black 8 (10)
Other 11 (14)
Missing 4 (5)
ECOG performance status
0 25 (31)
1 45 (56)
2 7 (9)
Missing 3 (4)
R-ISS
1 16 (20)
2 49 (61)
3 14 (18)
Missing 1 (1)
High-risk cytogenetic abnormalities (any of del(17p)/p53, t(14;16), t(4;14), or 1q21)
40 (50)
Mean number of previous regimens (SD), range 7.9 (3.1), 3 to 18 Mean years since diagnosis (SD), range 7.6 (3.4), 1.2 to 18.6 QoLquality of life,ECOGEastern Cooperative Oncology Group,R-ISSRevised International Staging System,SDstandard deviation
complete the FACT-MM assessment [16]. Results of the difference analysis are summarized in Table5. With the exception of the MM domain, the negative values for mean change indicate that HRQoL was decreasing for responders and non-responders. The FACT-MM, FACT-G, FACT-MM TOI, and the MM domain scores of non-responders showed a greater decrease from base- line to end of treatment. In contrast, responders had no change as evidenced by positive values of the mean dif- ference. This observed mean difference was significant for the FACT-G (p= 0.043), but not other scales.
Discussion
Despite advances made in the treatment of RRMM, the disease remains incurable and patients with RRMM face a significant burden due to symptoms, treatment- associated adverse events, and cumulative toxicities of prior therapies [11–13]. Inclusion of QoL evidence is an important factor in treatment decision-making that aims to balance clinical efficacy of newer therapies with the burden of adverse events, particularly among heavily pretreated patients with advanced disease [13]. Several studies have examined HRQoL in patients with RRMM receiving doublet or triplet therapies, daratumumab, or autologous stem cell transplantation with a variety of disease specific instruments [17,30–39]. Maintenance in HRQoL was observed in randomized controlled trials (RCTs) with pomalidomide and low-dose dexametha- sone [39], panobinostat, bortezomib, and dexamethasone [36], daratumumab [31], pomalidomide, bortezomib, and dexamethasone [38], and carfilzomib and dexametha- sone [34] in later lines of therapy (3 L+).
The current analysis examined HRQoL effects in pa- tients with penta-refractory MM, who received selinexor and low dose dexamethasone in the STORM phase 2b trial. FACT-MM, FACT-G, and FACT-MM TOI scores of patients declined significantly from baseline in the early cycles and at the end of treatment, while significant changes were not observed in the MM-specific domain at any point from baseline. Two anchors were utilized to estimate minimal clinically important difference and yielded consistent findings. Anchor-based approaches have been utilized to establish minimal clinically import- ant difference for other FACT instruments [40–44]. No
known, validated minimal clinically important difference has been reported for the FACT-MM. The exploratory approach utilizing patient-level data was based on the previously established relationship between QoL out- comes and ECOG PS scores [20]. This is in contrast to distribution-based minimal clinically important differ- ence evaluations, which do not consider clinical refer- ence points and are only statistical by nature. The observed consistency in findings with the previously ap- plied distribution-based approach serves as a validation of the novel anchor-based approach.
A key finding of the analysis is that, generally, the combined proportions of patients who experienced no change in HRQoL or improvements were higher com- pared to those who experienced declines in the early cy- cles of treatment with selinexor and dexamethasone according to minimal clinically important differences. In addition, the difference analysis identified that treatment responders had less HRQoL decline than non- responders. An association between HRQoL and re- sponse to treatment has been observed in previous RCTs. A significant improvement in HRQoL was ob- served in patients with partial or complete response with bortezomib in the SUMMIT phase 2 trial, while deteri- oration in HRQoL was observed among patients who did not respond and had progressive disease based on the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire C-30 (EORTC QLQ-C30) [45]. In the carfilzomib, lenalidomide, and dexamethasone arm of the phase 3 RCT ASPIRE, pa- tients achieving a partial response or better had signifi- cantly higher HRQoL over 18 cycles of treatment compared with patients who did not respond to treat- ment, according to the Global Health Status scale of the EORTC QLQ-C30 [37]. The limited impact on HRQoL, particularly among treatment responders, suggests a fa- vorable benefit-risk profile of selinexor, given its demon- strated efficacy and tolerability among patients with penta-refractory MM, and considering the unmet thera- peutic need in this patient population.
Limitations
An important limitation of the analysis is the single-arm study design of the STORM phase 2b trial, which did Table 3Minimal clinically important difference thresholds for two anchor-based approaches
10% difference in score range Mean baseline difference between ECOG performance scores 0 vs 1 to 2
Total FACT-MM 16.4 13.5
FACT-G 10.8 7.8
FACT-TOI 8.4 11.0
MM Domain 5.6 5.8
ECOGEastern Cooperative Oncology Group,FACT-GFunctional Assessment of Cancer Therapy–General,FACT-MMFunctional Assessment of Cancer Therapy– Multiple Myeloma,MMmultiple myeloma,TOITrial Outcomes Index
Table4Patientswithimprovement,nochange,ordeclineinHRQoLbasedonminimalclinicallyimportantdifferencesdefinedbyanECOG-basedanchor TotalFACT-MM;n(%)aFACT-G;n(%)bFACT-MMTOI;n(%)cMMDomain;n(%)d MaxNImprovementNochangeDeclineImprovementNochangeDeclineImprovementNochangeDeclineImprovementNochangeDecline Cycle271e9(12.9)42(60.0)19(27.1)9(12.9)38(54.3)23(32.9)10(14.1)36(50.7)25(35.2)14(19.7)39(54.9)18(25.4) Cycle3424(9.5)25(59.5)13(31.0)4(9.5)17(40.5)21(50.0)6(14.3)22(52.4)14(33.3)9(21.4)25(59.5)8(19.1) Cycle4252(8.0)14(56.0)9(36.0)4(16.0)10(40.0)11(44.0)1(4.0)14(56.0)10(40.0)1(4.0)17(68.0)7(28.0) Cycle5134(30.8)3(23.1)6(46.2)4(30.8)2(15.4)7(53.9)3(23.1)2(15.4)8(61.5)1(7.7)7(53.9)5(38.5) Cycle68f2(28.6)2(28.6)3(42.9)2(28.6)2(28.6)3(42.9)2(28.6)2(28.6)3(42.9)2(25.0)5(62.5)1(12.5) Endoftreatment383(7.9)17(44.7)18(47.4)3(7.9)11(29.0)24(63.2)3(7.9)14(36.8)21(55.3)10(25.6)13(33.3)16(41.0) FACT-GFunctionalAssessmentofCancerTherapy–General,FACT-MMFunctionalAssessmentofCancerTherapy–MultipleMyeloma,MMmultiplemyeloma,TOITrialOutcomesIndex aMinimalclinicallyimportantdifferencebasedonmeandifference(13.5points)inbaselineFACT-MMtotalscorebetweenpatientswithECOG0vsECOG1to2 bMinimalclinicallyimportantdifferencebasedonmeandifference(7.8points)inbaselineFACT-GscorebetweenpatientswithECOG0vsECOG1to2 cMinimalclinicallyimportantdifferencebasedonmeandifference(11.0points)inbaselineFACT-MMTOIscorebetweenpatientswithECOG0vsECOG1to2 dMinimalclinicallyimportantdifferencebasedonmeandifference(5.8)inbaselineMMdomainscorebetweenpatientswithECOG0vsECOG1to2 eForFACT-MMandFACT-G,n=70 fForFACT-MMandFACT-G,n=7
not enable comparison of HRQoL outcomes with a com- parator arm who received conventional medical manage- ment. As a result, treatment-associated changes in HRQoL cannot be directly extracted from the analysis.
HRQoL was a secondary endpoint, and all analyses and results should be considered as explorative. The analysis aimed to examine change in HRQoL during, and at end of treatment with selinexor and low dose dexametha- sone. The analysis did not investigate treatment- associated adverse events that may have been associated with changes in HRQoL due to the small sample size of the HRQoL dataset and high attrition rates seen in STORM; considering these factors, the study would be underpowered to detect significant differences.
Another limitation of the analysis was the small sam- ple size of patients with post-baseline HRQoL data. The sample size decreased over time, particularly in later cy- cles (i.e., cycle 7 or greater) that were omitted from the analysis due to sparse numbers. Observed differences in the composition of patients with minimal clinically im- portant differences may be attributed to attrition, par- ticularly among non-responders. Compliance rates were good in earlier cycles up to cycle 5, with ≥65% of pa- tients on treatment completing the FACT-MM, however a decline to 54% was observed in cycle 6. Combined with the small sample size, statistical power to detect changes in scores may be further reduced for each covariate added to the model.
The mixed-effects models assumed an MAR pattern of data, which presumes that all characteristics associated with missingness were adjusted for in the model. The MAR assumption was tested by examining patterns of missingness in the trial. Because different groups of pa- tients were observed at each cycle, baseline FACT-MM scores also varied across each cycle. The moving baseline values and the MAR assumption should be taken into consideration in the interpretation of observed HRQoL changes.
Lastly, treatment responders were not randomized. As a result, significant differences between responders and non-responders could be present between HRQoL at baseline or for other baseline characteristics. A differ- ence analysis was performed since subgroup analysis ac- cording to response was not well powered to perform statistical testing. It should be noted that the difference analysis is a quasi-experimental approach, which has been utilized in epidemiologic studies, but has not been commonly used for HRQoL analyses [46].
Conclusions
The current analysis examined patient-reported HRQoL in the STORM mITT population using the FACT-MM.
Minimal clinically important difference analyses demon- strated that most patients did not experience HRQoL decline during early cycles of treatment with selinexor and low dose dexamethasone. Exploratory minimal clin- ically important differences, defined as 10% of the in- strument range or as an ECOG-based anchor, yielded consistent results. Treatment responders were found to experience less decline in HRQoL from baseline to end of treatment than non-responders, which was significant only for the FACT-G. Important limitations of the ana- lysis were the single-arm study design and the limited sample size. Overall findings complement the demon- strated efficacy and tolerability of selinexor with low dose dexamethasone in patients with penta-refractory MM.
Abbreviations
AIC:Akaike’s Information Criterion; BIC: Bayesian Information Criterion;
CI: Confidence interval; ECOG: Eastern Cooperative Oncology Group; EORTC QLQ-C30: European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire C-30; FACT-G: Functional Assessment of Cancer Therapy–General; FACT-MM: Functional Assessment of Cancer Therapy– Multiple Myeloma; FACT-TOI: Functional Assessment of Cancer Therapy– Trial Outcomes Index; HRQoL: Health-related quality of life; MAR: Missing at random; mITT: Modified intent-to-treat; MM: Multiple myeloma; QoL: Quality of life; RCT: Randomized controlled trial; R-ISS: Revised International Staging Table 5Difference analysis results
Responders (N= 21)
Non-responders (N= 59)
Mean differencea
Mean change from baseline (95% CI) Mean change from baseline (95% CI) (95% CI) p-value
Total FACT-MM −1.72 (−13.63 to 10.20) −12.89 (−19.18 to−6.60) 11.17 (−2.30 to 24.65) p= 0.104
FACT-G −3.05 (−8.90 to 2.79) −10.28 (−14.13 to−6.42) 7.22 (0.22 to 14.22)
p= 0.043
FACT-MM TOI −4.33 (−14.11 to 5.44) −10.60 (−15.84 to−5.35) 6.26 (−4.82 to 17.35) p= 0.268
MM Domain 2.57 (−3.05 to 8.19) −2.16 (−5.60 to 1.27) 4.73 (−1.85 to 11.32)
p= 0.159
FACT-GFunctional Assessment of Cancer Therapy–General,FACT-MMFunctional Assessment of Cancer Therapy–Multiple Myeloma,MMmultiple myeloma,TOI Trial Outcomes Index,CIconfidence interval
aDifference analysis between baseline and end of treatment in responders compared to non-responders as estimated by a mixed effects model
System; RRMM: Relapsed/refractory multiple myeloma; SD: Standard deviation; STORM: Selinexor Treatment of Refractory Myeloma
Supplementary Information
The online version contains supplementary material available athttps://doi.
org/10.1186/s12885-021-08453-9.
Additional file 1 Table 1.Patients with improvement, no change, or decline in HRQoL based on minimal clinically important differences defined by≥10% of the instrument range.Table 1describes the number and proportion of patients with improvement, no change, or decline in HRQoL based on the minimal clinically important difference threshold defined as≥10% of the instrument range. Data are shown for the FACT-MM, FACT-G, FACT-MM TOI, and the MM domain at treatment cycle 2–6 and end of treatment.
Additional file 2.Provides a list of IECs and IRBs for the STORM trial.
Acknowledgements
KARE council members provided helpful comments on the manuscript. The authors acknowledge and thank the patients who participated in the NCT02336815 trial, their families, caregivers, and the study staff and healthcare providers at all trial sites. Writing and editorial support was provided by Nesrin Vurgun and Karen Sandman, who are employees of Purple Squirrel Economics.
Authors’contributions
G.T., J.B. and P.D. were responsible for data analyses and data interpretation and participated in drafting and revising of the manuscript. L.L., J.S., S.S., M.K., M.E., A.C., A.N., D.V., M.G., MA.D., P.R., N.B., D.S., P.V., C.D., T.F., M.M., N.M., M.L., L.C., JE.H., M.D., D.K., K.W., M.R., D.D., S.T., F.L., R.V., G.S., P. M, J.R., M.S., K.P., T.P., RF.C., K.L., S.C. and J.S. were responsible for study conception and design and participated in data interpretation and reviewed the manuscript. All authors have read and approved the manuscript.
Funding
This study and work were funded by Karyopharm Therapeutics Inc.
Availability of data and materials
The datasets analyzed in this work may be available from the corresponding author on reasonable request and permission of Karyopharm Therapeutics Inc.
Declarations
Ethics approval and consent to participate
This study was performed in compliance with the ethical principles that originate from the Declaration of Helsinki and are consistent with the International Council for Harmonisation (ICH) guidelines on Good Clinical Practice (GCP) and regulatory requirements as applicable. The study protocol was approved by the institutional review board or an independent ethics committee at each study center. Written informed consent in accordance with federal, local, and institutional guidelines was obtained from all patients.
A list of Institutional Ethics Committees (IECs) and Institutional Review Boards (IRBs) is provided as asupplement.
Consent for publication Not applicable.
Competing interests
Gabriel Tremblay, Janis Breeze and Patrick Daniele are employees of Purple Squirrel Economics, which received funding from Karyopharm Therapeutics Inc. to conduct the analysis. Lingling Li, Jatin Shah, Sharon Shacham, Michael Kauffman are employees of Karyopharm Therapeutics Inc. All other authors were investigators in the STORM trial.
Author details
1Purple Squirrel Economics, 1600 Notre Dame W, Suite 201, Montreal, QC H3J 1M1, Canada.2Karyopharm Therapeutics Inc., Newton, USA.3University of Freiburg, Freiburg im Breisgau, Germany.4Icahn School of Medicine at
Mount Sinai, New York, USA.5Winship Cancer Institute, Emory University, Atlanta, USA.6Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA.7Oncology Department, Alexandra Hospital, Athens, Greece.8School of Medicine, National and Kapodistrian University of Athens, Athens, Greece.9Harvard Cancer Center, Boston, USA.10Hackensack Meridian Health Hackensack University Medical Center, Hackensack, USA.11University Hospital Ghent, Ghent, Belgium.
12Université catholique de Louvain, Ottignies-Louvain-la-Neuve, Belgium.
13University Hospital, Lille, France.14Hopital Saint-Antoine, Paris, France.
15Institut Jules Bordet, Brussels, Belgium.16Baylor University Medical Center, Dallas, USA.17University of Alabama at Birmingham, Birmingham, USA.
18Sylvester Cancer Center, University of Miami, Miami, USA.19University of Leuven, Leuven, Belgium.20New York University Langone Medical Center, New York, USA.21University Medical Center Hamburg-Eppendorf, Hamburg, Germany.22University of Heidelberg, Heidelberg, Germany.23Mayo Clinic, Rochester, USA.24Lineberger Comprehensive Cancer Center at University of North Carolina-Chapel Hill, Chapel Hill, USA.25Hôpital Necker, Paris, France.
26Washington University School of Medicine, St. Louis, USA.27David Geffen School of Medicine at University of California, Los Angeles, USA.28University of Nantes, Nantes, France.29Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, USA.30University Hospital Würzburg, Würzburg, Germany.31University Hospital Krems, Karl Landsteiner University of Health Sciences, Krems an der Donau, Austria.32Yale School of Medicine, New Haven, USA.33Vanderbilt University Medical Center, Nashville, USA.34Centre Hospitalier Lyon Sud, Saint-Genis-Laval, France.35La Pitié Salpêtrière Hospital, Paris, France.
Received: 13 November 2020 Accepted: 7 June 2021
References
1. Surveillance, Epidemiology, and End Results (SEER) Program. Cancer stat facts: myeloma: National Cancer Institute (NCI); 2016. [cited 2018 Nov 27].
Available from:https://seer.cancer.gov/statfacts/html/mulmy.html 2. Rajkumar SV, Kumar S. Multiple myeloma: diagnosis and treatment. Mayo
Clin Proc. 2016;91(1):101–19.https://doi.org/10.1016/j.mayocp.2015.11.007.
3. Bazarbachi AH, Al Hamed R, Malard F, Harousseau J-L, Mohty M. Relapsed refractory multiple myeloma: a comprehensive overview. Leukemia. 2019;
33(10):2343–57.https://doi.org/10.1038/s41375-019-0561-2.
4. Sonneveld P, De Wit E, Moreau P. How have evolutions in strategies for the treatment of relapsed/refractory multiple myeloma translated into improved outcomes for patients? Crit Rev Oncol/Hematol. 2017;112:153–70.
5. Mikhael J. Treatment options for triple-class refractory multiple myeloma.
Clin Lymphoma Myeloma Leuk. 2020;20(1):1-7.https://doi.org/10.1016/j.
clml.2019.09.621.
6. Kumar SK, Dimopoulos MA, Kastritis E, Terpos E, Nahi H, Goldschmidt H, et al. Natural history of relapsed myeloma, refractory to immunomodulatory drugs and proteasome inhibitors: a multicenter IMWG study. Leukemia.
2017;31(11):2443–8.https://doi.org/10.1038/leu.2017.138.
7. Kumar S, Paiva B, Anderson KC, Durie B, Landgren O, Moreau P, et al.
International Myeloma Working Group consensus criteria for response and minimal residual disease assessment in multiple myeloma. Lancet Oncol.
2016;17(8):e328–46.https://doi.org/10.1016/S1470-2045(16)30206-6.
8. Cornell RF, Gandhi UH, Lakshman A, Gahvari Z, Jagosky MH, McGehee E, et al. Subsequent treatment outcomes of multiple myeloma refractory to CD38-monoclonal antibody therapy. Blood. 2018;132(Supplement 1):2015.
9. Gandhi UH, Cornell RF, Lakshman A, Gahvari ZJ, McGehee E, Jagosky MH, et al. Outcomes of patients with multiple myeloma refractory to CD38- targeted monoclonal antibody therapy. Leukemia. 2019;33(9):2266–75.
https://doi.org/10.1038/s41375-019-0435-7.
10. Pick M, Vainstein V, Goldschmidt N, Lavie D, Libster D, Gural A, et al.
Daratumumab resistance is frequent in advanced-stage multiple myeloma patients irrespective of CD38 expression and is related to dismal prognosis.
Eur J Haematol. 2018;100(5):494–501.https://doi.org/10.1111/ejh.13046.
11. Hulin C, Hansen T, Heron L, Pughe R, Streetly M, Plate A, et al. Living with the burden of relapse in multiple myeloma from the patient and physician perspective. Leuk Res. 2017;59:75–84.https://doi.org/10.1016/j.leukres.2017.05.019.
12. Ramsenthaler C, Osborne TR, Gao W, Siegert RJ, Edmonds PM, Schey SA, et al. The impact of disease-related symptoms and palliative care concerns on health-related quality of life in multiple myeloma: a multi-centre study.
BMC Cancer. 2016;16(1):427.https://doi.org/10.1186/s12885-016-2410-2.
13. Sonneveld P, Verelst SG, Lewis P, Gray-Schopfer V, Hutchings A, Nixon A, et al. Review of health-related quality of life data in multiple myeloma patients treated with novel agents. Leukemia. 2013;27(10):1959–69.https://
doi.org/10.1038/leu.2013.185.
14. NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines). Multiple myeloma: National Comprehensive Cancer Network; 2019. [cited 2020 Feb 14].
Available from:https://www.nccn.org/professionals/physician_gls/pdf/myeloma.pdf 15. XPOVIO (selinexor) prescribing information. [cited 2019 Jul 8]. Available from:
https://www.accessdata.fda.gov/drugsatfda_docs/label/2019/212306s000lbl.pdf.
16. Chari A, Vogl DT, Gavriatopoulou M, Nooka AK, Yee AJ, Huff CA, et al. Oral selinexor–dexamethasone for triple-class refractory multiple myeloma. N Engl J Med. 2019;381(8):727–38.https://doi.org/10.1056/NEJMoa1903455.
17. Abonour R, Wagner L, Durie BGM, Jagannath S, Narang M, Terebelo HR, et al. Impact of post-transplantation maintenance therapy on health-related quality of life in patients with multiple myeloma: data from the connect®
MM registry. Ann Hematol. 2018;97(12):2425–36.https://doi.org/10.1007/
s00277-018-3446-y.
18. Wagner LI, Robinson D, Weiss M, Katz M, Greipp P, Fonseca R, et al. Content development for the functional assessment of cancer therapy-multiple myeloma (FACT-MM): use of qualitative and quantitative methods for scale construction. J Pain Symptom Manag. 2012;43(6):1094–104.https://doi.org/1 0.1016/j.jpainsymman.2011.06.019.
19. Webster K, Cella D, Yost K. The functional assessment of chronic illness therapy (FACIT) measurement system: properties, applications, and interpretation. Health Qual Life Outcomes. 2003;1(1):79.https://doi.org/10.11 86/1477-7525-1-79.
20. Weiss M, Jacobus S, Wagner LI, Cella D, Katz MS, Rajkumar V, et al.
Development of the functional assessment of cancer therapy-multiple myeloma (FACT-MM) scale and validation in the eastern cooperative oncology group trial E1A05. Blood. 2011;118(21):4184.https://doi.org/10.11 82/blood.V118.21.4184.4184.
21. Breeze J, Mehta J, Tremblay G, Shacham S, Shah J, Li L, et al. PCN304 quality of LIFE analyses in patients with multiple myeloma: results from the selinexor (KPT-330) treatment of refractory myeloma (STORM) phase 2B study. Value Health. 2020;23:S77.https://doi.org/10.1016/j.jval.2020.04.1767.
22. Akaike H. A new look at the statistical model identification. IEEE Trans Automat Contr. 1974;19(6):716–23.https://doi.org/10.1109/TAC.1974.1100705.
23. Schwarz G. Estimating the dimension of a model. Ann Stat. 1978;6(2):461–4.
24. Jaeschke R, Singer J, Guyatt GH. Measurement of health status. Control Clin Trials. 1989;10(4):407–15.https://doi.org/10.1016/0197-2456(89)90005-6.
25. Jayadevappa R, Cook R, Chhatre S. Minimal important difference to infer changes in health-related quality of life—a systematic review. J Clin Epidemiol. 2017;89:188–98.https://doi.org/10.1016/j.jclinepi.2017.06.009.
26. Ringash J, O’Sullivan B, Bezjak A, Redelmeier DA. Interpreting clinically significant changes in patient-reported outcomes. Cancer. 2007;110(1):196– 202.https://doi.org/10.1002/cncr.22799.
27. Cella D, Eton DT, Lai J-S, Peterman AH, Merkel DE. Combining anchor and distribution-based methods to derive minimal clinically important differences on the functional assessment of cancer therapy (FACT) Anemia and fatigue scales. J Pain Symptom Manag. 2002;24(6):547–61.https://doi.
org/10.1016/S0885-3924(02)00529-8.
28. Pickard AS, Neary MP, Cella D. Estimation of minimally important differences in EQ-5D utility and VAS scores in cancer. Health Qual Life Outcomes. 2007 Dec;5(1):70.https://doi.org/10.1186/1477-7525-5-70.
29. Janelsins MC, Heckler CE, Peppone LJ, Kamen C, Mustian KM, Mohile SG, et al. Cognitive complaints in survivors of breast cancer after chemotherapy compared with age-matched controls: an analysis from a nationwide, multicenter, prospective longitudinal study. J Clin Oncol. 2017;35(5):506–14.
https://doi.org/10.1200/JCO.2016.68.5826.
30. Ahmedzai SH, Snowden JA, Ashcroft AJ, Cairns DA, Williams C, Hockaday A, et al. Patient-reported outcome results from the open-label, randomized phase III myeloma X trial evaluating salvage autologous stem-cell transplantation in relapsed multiple myeloma. JCO. 2019;37(19):1617–28.
https://doi.org/10.1200/JCO.18.01006.
31. Chari A, Lonial S, Mark TM, Krishnan AY, Stockerl-Goldstein KE, Usmani SZ, et al. Results of an early access treatment protocol of daratumumab in United States patients with relapsed or refractory multiple myeloma: early access treatment with daratumumab. Cancer. 2018;124(22):4342–9.https://
doi.org/10.1002/cncr.31706.
32. Kim SJ, Park Y, Mun Y-C, Kim JS, Kim K, Min C-K. The assessment of health- related quality of life in relapsed or refractory myeloma patients receiving
lenalidomide and low dose dexamethasone: a result of hola study. Blood.
2018;132(Supplement 1):4853.
33. Ludwig H, Pönisch W, Knop S, Egle A, Hinke A, Schreder M, et al. Quality of life in patients with relapsed/refractory multiple myeloma during ixazomib- thalidomide-dexamethasone induction and ixazomib maintenance therapy and comparison to the general population. Leuk Lymphoma. 2020;61(2):
377–86.https://doi.org/10.1080/10428194.2019.1666381.
34. Moreau P, Kumar S, Boccia R, Iida S, Goldschmidt H, Cocks K, et al.
Convenience, satisfaction, health-related quality of life of once-weekly 70 mg/
m2 vs. twice-weekly 27 mg/m2 carfilzomib (randomized A.R.R.O.W. study).
Leukemia. 2019;33(12):2934–46.https://doi.org/10.1038/s41375-019-0480-2.
35. Moreau P, Masszi T, Grzasko N, Bahlis NJ, Hansson M, Pour L, et al. Oral Ixazomib, lenalidomide, and dexamethasone for multiple myeloma. N Engl J Med. 2016;374(17):1621–34.https://doi.org/10.1056/NEJMoa1516282.
36. Richardson PG, Schlossman RL, Roy AN, Panneerselvam A, Acharyya S, Sopala M, et al. Patient-reported outcomes of multiple myeloma patients treated with panobinostat after≥2 lines of therapy based on the international phase 3, randomized, double-blind, placebo-controlled PANORAMA-1 trial. Br J Haematol. 2018;181(5):628–36.https://doi.org/1 0.1111/bjh.15248.
37. Stewart AK, Dimopoulos MA, Masszi T,Špička I, Oriol A, Hájek R, et al. Health- related quality-of-life results from the open-label, randomized, phase III ASPIRE trial evaluating carfilzomib, lenalidomide, and dexamethasone versus lenalidomide and dexamethasone in patients with relapsed multiple myeloma.
JCO. 2016;34(32):3921–30.https://doi.org/10.1200/JCO.2016.66.9648.
38. Weisel K, Paner A, Engelhardt M, Taylor F, Cocks K, Espensen A, et al.
Quality-of-life outcomes in patients with relapsed/refractory multiple myeloma treated with elotuzumab plus pomalidomide and
dexamethasone: results from the phase 2 randomized eloquent-3 study.
Blood. 2018;132(Supplement 1):2288.
39. Weisel K, Dimopoulos M, Song KW, Moreau P, Palumbo A, Belch A, et al.
Pomalidomide and low-dose dexamethasone improves health-related quality of life and prolongs time to worsening in relapsed/refractory patients with multiple myeloma enrolled in the MM-003 randomized phase III trial. Clin Lymphoma Myeloma Leuk. 2015;15(9):519–30.https://doi.org/1 0.1016/j.clml.2015.05.007.
40. Askew RL, Xing Y, Palmer JL, Cella D, Moye LA, Cormier JN. Evaluating minimal important differences for the FACT-melanoma quality of life questionnaire. Value Health. 2009;12(8):1144–50.https://doi.org/10.1111/j.1 524-4733.2009.00570.x.
41. Cella D, Nichol MB, Eton D, Nelson JB, Mulani P. Estimating clinically meaningful changes for the functional assessment of cancer
therapy—prostate: results from a clinical trial of patients with metastatic hormone-refractory prostate cancer. Value Health. 2009;12(1):124–9.https://
doi.org/10.1111/j.1524-4733.2008.00409.x.
42. Cheung YT, Foo YL, Shwe M, Tan YP, Fan G, Yong WS, et al. Minimal clinically important difference (MCID) for the functional assessment of cancer therapy:
cognitive function (FACT-Cog) in breast cancer patients. J Clin Epidemiol. 2014;
67(7):811–20.https://doi.org/10.1016/j.jclinepi.2013.12.011.
43. Yost K, Cella D, Chawla A, Holmgren E, Eton D, Ayanian J, et al. Minimally important differences were estimated for the functional assessment of cancer therapy–colorectal (FACT-C) instrument using a combination of distribution- and anchor-based approaches. J Clin Epidemiol. 2005;58(12):
1241–51.https://doi.org/10.1016/j.jclinepi.2005.07.008.
44. Yost KJ, Sorensen MV, Hahn EA, Glendenning GA, Gnanasakthy A, Cella D.
Using multiple anchor- and distribution-based estimates to evaluate clinically meaningful change on the functional assessment of cancer therapy-biologic response modifiers (FACT-BRM) instrument. Value Health.
2005;8(2):117–27.https://doi.org/10.1111/j.1524-4733.2005.08202.x.
45. Dubois D, Dhawan R, van de Velde H, Esseltine D, Gupta S, Viala M, et al.
Descriptive and prognostic value of patient-reported outcomes: the Bortezomib experience in relapsed and refractory multiple myeloma. JCO.
2006;24(6):976–82.https://doi.org/10.1200/JCO.2005.04.0824.
46. Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies:
best practices for public health policy research. Annu Rev Public Health. 2018;
39(1):453–69.https://doi.org/10.1146/annurev-publhealth-040617-013507.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
