III. Application
Acceptance of the definitions proposed brings us to the comparison of two computational frameworks for modelling real-life disease evolution for in- and outpatients with major unipolar depression over a period of 10 years.
The final health outcomes of interest were time spent without depressive symptoms (i.e. time in remission and full recovery) and the number of relapses and recurrences occurring over the study period. The simulation models described hereafter illustrate disease progression over time regardless of therapeutic strategy and take into consideration realistic patient behaviour patterns as well as important prognostic factors. We provide additional technical details on both Markov and Discrete Event Simulation models, together with a practical example of both modelling methods. The software used to implement the simulation models was TreeAge Pro 2006 Healthcare Software, release 0.1 by TreeAge Software Inc., Williamstown, MA 01267 USA.
III.1 Markov models
Markov modelling is a decision-analytic technique that characterizes the prognosis of a cohort of patients by assigning them to a fixed number of health states and then models transitions among health states [15]. Markov models (typically Markov chains) assume transition probabilities to be constant over time. However, it's possible to bypass this strict assumption by modelling non-homogeneous (i.e. time-dependent) Markovian stochastic processes. Markov models are particularly suited to modelling events of interest that occur repeatedly over a long period of time [5,15]. However, an important limitation of Markov models is that they lack "memory". This means that the probability of moving from one state to another does not take into account the history of the patient before he or she arrived in that state. This is also referred to as the Markovian assumption.
In our illustrative case (picture in Figure 5), health states were divided into three levels of risk -low, moderate, high (reflecting patient's number of previous episodes), each being divided into multiple temporary states associated with varying probabilities of remission according to the time elapsed in the disease state (in order to handle illness persistence issues). Therefore, for each level of risk, on the basis of a 1-week cycle (the accepted time span in MDD before observing any potential health transitions), we defined 24 temporary depressed states, i.e. 24 weekly remission probabilities adjusted for the duration of the disease. If the patient was still depressed at week 24, a constant probability of remission was applied. The number of temporary states was chosen according to the accepted management of an episode (i.e. a continuation period of 6 months) [16,17].
This resulted in 24*3 = 72 (temporary) health states accounting for risk levels and duration of the illness.
To differentiate remission periods from recovery periods (for increased precision when assessing the ability of a given strategy to further delay development of depressive symptoms), it was necessary to divide the "well" state into two separate temporary states. The first state was remission, i.e. the first 24 weeks following the disappearance of symptoms (in accordance with clinical guidelines which define a minimum of 24 symptom-free weeks before concluding that the patient has achieved full recovery). The second state was the full recovery period (i.e. a period of remission longer than 24 weeks).
The data required to specify this simulation model (in terms of clinical data exclusively) are survival distributions of remission and relapse at each cycle (i.e. one-week transition probabilities), conditional on the number of previous depression events. The time spent in the any health state may be summed over the period (i.e. 10 years = 520 cycles) and eventually discounted according to applicable rates.
This somewhat "simple" model (i.e. in terms of the number of risk factors taken into account) demonstrates the suitability of Markov models in addressing the key features of importance when modelling depression evolution over time. First, they handle the problem of patients' history of the disease by splitting health states according to different risk levels (low, moderate, high). This is computationally acceptable. Second, the chronic nature of the disease (for approximately 20% of patients, as mentioned previously) was managed at the expense of defining multiple health states (i.e. 72 states encoded as "tunnel" variables), making it possible to assign varying transition probabilities according to the time spent in the "depressed" state. Lastly, Markov models can distinguish between remission and recovery periods by using temporary states (i.e. 6 more health states). Therefore, a Markov representation of the problem requires defining at least 72+6 = 78 health states to properly take into consideration primary relevant risk factors (i.e. severity and duration of the disease).
The efficiency of such a method in more complicated scenarios, however, is questionable. For example, what if the analyst would like to take into consideration an important factor of prognosis such as patients' attitude towards treatment? This would necessitate further splitting each state in two more states. With every additional factor, the model becomes increasingly more difficult to handle properly. Would using a Markovian representation make it possible to efficiently consider the key factor of suicidal behaviour? The same reasoning applies: the integration of all relevant factors into a Markov model may render it too complex and prone to bias. Markov models have previously been used to model the cost-effectiveness of relapse prevention interventions for recurrent depression [18-23]. They have also been used as a tool to portray the epidemiology of depression [24-29]. However, we were unable to find any Markov model that simultaneously took into consideration all of the confounding factors just mentioned.
These factors are of great interest to researchers and decision makers alike and, naturally, may merit a more flexible simulation method. Discrete event simulation models may be an opportunity to adequately address the limitations of Markov models, and our intention was to assess the benefits and drawbacks of DES compared with Markov models.
III.2 Discrete Event Simulation models
Discrete event simulation (DES) is one way of observing the time- dependent (or dynamic) behaviour of a system [30-32]. As a cost-effectiveness tool, DES models have been widely used in various disease areas, including laparoscopic surgery [33], gastric cancer [34], renal diseases [35], drug abuse [36], HIV transmission [37], early breast cancer [38,39] and liver transplants [40]. To our knowledge, DES models have not yet been used within the field of major depression.
Recently, J.J Caro proposed further examination of DES models as a computational tool for cost-effectiveness analyses. In doing so he reiterated the key principles of the method [6]:
Entities
Entities are the items that evolve through the simulation. In the clinical simulation of a disease, patients are the entities. The patient is an explicit element of a discrete event simulation model. In DES models, patients are assigned attributes (e.g., age, sex, duration of the disease) with a specific value (distribution) for each. These values are defined at the start of the simulation and may be updated as required: age increases, disease severity levels rise and fall, the number of depressive events increases, etc. Other model specifications such as time horizon and discount rate are encoded in variables. These values may change during the simulation.
Events
An event is defined as anything that can happen during the simulation. This can include occurrence of depressive symptoms, remission from depressive symptoms, patients stopping treatment, a suicide attempt, an adverse event, etc. This concept extends well beyond the transitions in a Markov model, because the event need not imply a change in the patient's state. Events can occur sequentially and/or even simultaneously. They can recur – if this corresponds to clinical reality – and they can change the course of a given patient's experience by influencing that patient's attributes and the occurrence of future events. The rates at which events occur can take on any functional distribution supported by the data. They can be dependent on any attributes or variables and these functions can change over time as appropriate.
Time
The third fundamental component of a DES is time itself. An explicit simulation clock keeps track of the passage of time. This makes it possible for the analysts to clearly signal the start and end of the simulation and to create secondary clocks that track interim periods such as depression episode duration or remission periods. By making time explicit, a DES enables handling time much more flexibly compared with Markov models because there is no need to define cycle length.
The model described here belongs to the class of models that have been described elsewhere as individual sampling models [41,42]. Rather than following an entire cohort through a model by assigning proportions to different states, discrete event simulation models the pathway of an individual by sampling probabilities from an a priori distribution. This results in greater realism in describing a patient's evolution through the healthcare system and offers more flexibility in the data requirements needed to feed the model. DES models provide an alternative tool capable of considering multiple risk factors and non-Markovian structures (i.e. non memory-less stochastic processes). Peter W. Glynn describes a mathematical formalism for the underlying stochastic process [43,44], named "Generalized Semi-Markov Process" (GSMP). A GSMP is an established formalism for modeling continuous-time stochastic discrete event systems.
Throughout the entire simulation, new information (depending on the triggered events) can be tracked and stored into a temporary variable, so that future events' probabilities can be changed to reflect a patient's new clinical and socio-demographic profile. Patients may then acquire attributes (e.g., a higher risk of relapse) as certain events occur within the model. The attributes of a particular patient influence his/her pathway through the simulation, as well as the economic outcomes associated with the events experienced (e.g., hospitalisation resulting from a suicide attempt).
Thanks to the strength of the assumptions the technique offers, and by modelling individual patient pathways, DES provides a greater degree of flexibility which, when supplied with adequate data, may allow greater confidence in the results [45].
To examine the properties of DES under practical circumstances, we chose to apply them to the dynamics of depression. The algorithm associated with our problem is depicted in Figure 6. There are no cycle lengths to declare and no health states to define. Disease evolution is pictured using events that will trigger a change of health state. The method chosen to select the next occurring event was that used by Barton et al. [46], the underlying idea being "sample times for each possible event and use the minimum" (the rationale being that once the first event has happened, times to other events may need to be resampled). For each event, therefore, survival distributions were required assuming that no other event was possible. A time was sampled for each event and the earliest time determined which event happened. This is implemented by considering other events as censored events and the other times are discarded.
The information required was survival data conditioned on the number of prior depressive events. The time spent in the "well" state was obtained by summing the tracked sampled times leading to relapse. The time spent in the depressed state was obtained by summing the tracked sampled times leading to remission. Other tracker variables make it possible to count the number of relapses or recurrences occurring during the study period. The patient started the model with a first depressive event (i.e. no prior depressive events). The only event that would remove a patient from the state of depression would be achieving remission of symptoms. Therefore, a "time to remission" was sampled based on the patient's history. The clock advances of the simulated "time to remission" and a test is performed to check if there is time left to continue the simulation (according to the fixed time horizon) or not. Once the patient is symptom-free, he is still subject to relapse. Therefore, a "time to depression" is sampled and the clock is advanced to this sampled "time to depression". If the sampled time was inferior to 24 weeks then the patient was in "simple remission"(i.e., relapse). If the sampled time was superior to 24 weeks, then the patient was in full recovery. Based on these tests, tracker variables count the number of events that occur, and patient's attributes are updated accordingly. Figure 6 displays a graphical representation of the algorithm.
The present DES model reflects a pathway with a very limited list of possible events: there are no competing events. This renders the analysis quite simple and as a practical example makes it possible to visualize the flexibility with which DES models cope with multiple competing events. DES models appear to be a powerful means to address both the problem of a patient's history and the risk for an illness to persist by using survival distributions conditioned by tracker values. Similarly, remission and recovery periods can be easily distinguished by tracking the sampled "time to depression" and see if this sampled time (24 weeks, per consensus definitions) is inferior or superior to a threshold value. If the sampled time to depression was less than 24 weeks, then the patient could not be in full recovery and was, therefore, only in a "remission phase". By implementing various queries, it is possible to define new trackers that will remember whether the patient was completely well or not.
DES models are as efficient as Markov models, with perhaps, slightly more flexibility regarding their implementation. If the analyst wishes to tackle the problem of adherence to treatment (a key factor of interest in modelling depression), DES models are flexible enough to manage this by adding an event to the list of possibilities a patient is likely to experience (along with its own survival distribution) and run the model. The sequence of events experienced by the patient will be randomly generated according to the event selection method previously outlined (i.e. sample "time to events", the first event being the one selected). This means that key events in depression such as attempted suicides, adverse events or any other event for which there is adequate data, can be easily taken into account.
DES models seem to be a promising simulation technique, very flexible and easy to follow for any analyst who may not be familiar either with the key aspects of the disease or with simulation tools in general. DES models are able to overcome Markov model limitations particularly in their ability to take into account multiple events, which can be crucial when trying to depict disease progression as close to reality as possible.
Answers to all your Biology Questions
