Predicting information about human physiology and pathophysiology from genomic data is a …

• Summary
• Introduction
• The development of integrated cardiac models
• Criteria for model assessment
• Objective criteria
• Discussion
• References
• Figures
• Table

Biology Articles » Biomathematics » Computational biology of cardiac myocytes: proposed standards for the physiome project » Criteria for model assessment

Criteria for model assessment

- Computational biology of cardiac myocytes: proposed standards for the physiome project

Criteria for model assessment  

Systematic validation against experimental data of models linking detailed cellular biophysics to tissue function remains challenging. As outlined above this is, in part, due to the technical difficulties associated with managing and maintaining links to experimental data required for each mechanism in the excitation–contraction metabolism process. Nonetheless, validation is essential before these promising simulation techniques can provide real value to the clinician.

The specific difficulties outlined above are as follows. (1) Models are rarely implemented and tested as part of the peer-review process for journal publications, meaning the published manuscript may contain errors. (2) The connection between model parameters and data is often ambiguous. Making this link transparent is fundamental to building large-scale models that integrate different physiological subsystems. (3) The functional limitations of a model do not become apparent until significant time and effort has been put into model implementation, application and coupling. (4) There are few public forums where feedback, experiences and critique of existing published models can be shared. (5) The experimental data used to parameterize and validate computational models are rarely available to the community in convenient useable formats.

Each of these issues undermines confidence and impairs the application and extension of models by people other than the developers, or those with specific expertise in model development. As discussed above, a number of cell modelling mark-up languages have been developed (CellML, SBML, Jsim) and using these, and other established computing languages, cell models can be made freely available. Furthermore, there is on-going discussion of the development of FieldML a mark-up language that will enable the representation of structural and continuum information about biological and physical entities. This will allow the unambiguous machine-readable representation of structural and tissue-based models. Running versions of models provided by model authors using these codes provides a significant step in overcoming issue 1. Furthermore, a model that is compliant against the MIRIAM rules guarantees machine readability, an unambiguous description of the model, consistency with the published model, and consistency between published results and simulation output.

To address issues 2–5 will require the community to build on these initiatives, and the development of openly available resources to disseminate models linked to the data sets used to parameterize them. We suggest that the following two types of entities should be collected and published online in a physiome database: published models, including complete codes for simulation, and peer-reviewed published data sets in accessible electronic formats. The first of these is the domain of the MIRIAM standard. Model entries in the database will be annotated using established ontologies, and include working and executable codes, using freely available tools, or computational code in an established language (C, Matab, Fortran, Pascal). These marked up executables with the addition of digitized data sets (see point 1 below) will ideally be available as part of the review process. This will enable the reviewer and user community to curate entries in the database with the following tools and criteria:

1. Explicit links will be established between data sets and models. Specifically, each model will link to: (i) the data that were used to parameterize the model; (ii) additional data that are used to verify or demonstrate the scope and physiological application of the model; and (iii) known relevant data sets that the model does not satisfactorily fit. In addition each data set will link to: (i) model(s) that use the data set as part of the parameterization of those models; (ii) models that fit and/or help to explain the data set; and (iii) models that are not able to fit the data. These links will be edited by the authors.
   
2. Classification of the model according to the objective criteria listed below. The authors will be invited to provide this classification. The ultimate goal is to have submission of a model to the Physiome resource with classification according to these criteria as part of the review process for major journals. Reviewers may be expected to verify the initial classification entered by the authors.
   
3. A user feedback and review section where people can post non-anonymous style comments on their experiences. In each case the authors will be invited to provide a response and, if necessary, update their work.