2. Scoring Fits

  1. Global scores
  2. Scoring ensemble
  3. Local scores

2.1. Global scores

_images/global_scores.png

Fig: (A) Mutual information between binned densities (B) Normal vectors (C) Surface exposed and buried voxels.

Difference metrics for assessing model fits in density (or for comparing maps) are implemented in TEMPy. Depending on the map resolution, extent of overlap or shape features, one scoring function may be more useful than others. In a recent study we found mutual information score better discriminatory than cross correlation coefficient especially at intermediate-low resolutions or when the density distributions differ significantly. We have also included normalized mutual information score (Studholme et al. 1999) and scores that combines evaluation of different features (e.g. density mutual information and fraction of overlap).


For more information on the performance of the difference scoring functions please read:

  • Joseph et al (2017) Improved metrics for comparing structures of macromolecular assemblies determined by 3D electron-microscopy. J Struct Biol 99(1): 12–26
  • Vasishtan and Topf (2011) Scoring functions for cryoEM density fitting. J Struct Biol 174:333-343.

To calculate scores using TEMPy, see Global score calculation

2.2. Scoring ensemble

_images/clusterplot1.png

Fig: Cluster of high scoring ensemble.

Different scores in TEMPy can be also used to evaluate an ensemble of models and identify one or a few best scoring models (best scoring cluster). A model can be also assessed against an ensemble of models. TEMPy has useful routines to generate ensembles:


To generate ensembles using TEMPy, see Ensemble generation

2.3. Local scores

2.3.1. Segment Based cross-correlation score (SCCC)

_images/sccc.png

Fig: SCCC score based coloring.

Used to quantify and compare the local quality of fits. The segment selection is defined from a rigid body list file where each line is made of start and end residue numbers of the fragments comprising the segment. For example, a line in the rigid body file can be: 1 10 15 20 for a segment made of residues 1 to 10 and 15 to 20. For pdbs with multiple chains, 1:A 10:A 15:A 20:A includes the chain ID for the segment. A tutorial file is available.

For more information:

  • Pandurangan AP, Shakeel S, Butcher SJ, Topf M. (2014) Combined approaches to flexible fitting and assessment in virus capsids undergoing conformational change. J Struct Biol. 185, 427-439.

An example script is provided (get_sccc.py). Inputs: [-m mapfile] [-p pdbfile] [-r map_resolution] [-rf rigid_body file] [–help help]


_images/1ake45_softmask_smoc.png

Fig: SMOC score based coloring.

2.3.2. Segment Based Manders’ Overlap Coefficient (SMOC)

Used to quantify and compare the local quality of fits. The script to calculate SMOC scores is added in Examples (score_smoc.py). A tutorial is available.

SMOC score is similar to the SCCC score but the mean deviation is not accounted in the calculation of cross correlation. Also instead of segments, the score is calculated on overlapping residue windows so that each (central) residue in the chain gets a score for the local fit.

_images/smocplot.png

Fig: SMOC score plot.

For more information:

  • Joseph AP, Malhotra S, Burnley T, Wood C, Clare DK, Winn M, Topf M. (2016) Refinement of atomic models in high resolution EM reconstructions using Flex-EM and local assessment. Methods.

An example script is provided (score_smoc.py). Inputs: [-m mapfile] [-p pdbfile] [-r map_resolution] [-win window_size (optional)][–help help]