Reading Against a Model of Experimental Evidence

Here we describe a possible approach to evaluating the process of extracting experimental evidence from pathway papers automatically based on the Big Mechanism principle of 'reading against a model'.

June 2016

  1. Dry Run II Papers from MITRE

July 2015

  1. Two-Hop Ras BioPax Model

May 2015


April 2015

  1. Open Access Pathway Logic Papers and Figures
  2. Extended Coprecipitation Frames v2

January 2015

  1. Coprecipitation Frames v2
  2. List of Experimental Motif Types + Definitions
  3. KEfED Database Construction

December 2014

  1. KEfED Modeling of Coprecipitation Ras Papers
  2. Initial Extraction Study of Results-Based Epistemics

October 2014

  1. Pathway Logic Experiment Types
  2. Building a Database of Observations from Result Text
  3. Deploying the BioScholar System
  4. Reading Against a Model of Experimental Evidence

August 2014

  1. Developing NL Annotations for KEfED Elements
  2. Epistemics and Fragments
  3. Generating the Big Mechanisms Evaluation Corpus
  4. A Generative Story for Scientific Text from Experimental Data
  1. How to Extend Existing Evaluation To Include Evidence

Following the underlying premise of conducting reading evaluation within the Big Mechanism program as ‘reading against the model’, we examine how the structures being proposed for reading against an interpretive model (such as BioPax) might also apply to reading against a model of experimental observations. The relationship between interpretative assertions (such as those generated against a BioPax model) and text describing experimental results is relevant here:

KEfED Staged Modeling Elements

As a representation, BioPax and other formal representations of pathways do not explicitly represent the intermediate elements required to represent experimental design and data. Thus to evaluate this multi-stage process, simply focussing only on the ability of the system to generate final-stage interpretive assertions is likely to direct focus too heavily on the end result (and disempower staged approaches that explicitly deal with experimental evidence).

Elements that require explicit representation within the model to be evaluated against include:

Within the L2K2R2 reading evaluation model for reading against BioPax, evaluation takes place by representing the model M as a list of assertions. Each reading system produces lists of evidence with a relation for each assertion pertaining to the model (new; confirm; refute).

BioPax does provide a framework for attaching evidence to models which does not currently extend to support details to the extent required above. We will characterize evidence within BioPax models in a way that is consistent with the modeling approach described above so that BioPax assertions of evidence may be included in the model under evaluation. This will permit the extension of phase 1 evaluation to include experimental evidence at later phases.

An Example of Reading Against a KEfED Model

Given a model (from Montero et al. 2013, Figure 5B) such as the one shown below:

With this text from the figure legend:

B, 8505C cells were treated with increasing concentrations of AZD6244. Lysates were extracted at 24 h post-treatment and immunoblotted with the indicated antibodies.

And the following text from the results section to assert data:

The effects of the MEK inhibitor on total HER2, HER3 protein and on pHER3 were dose dependent, and inversely associated with the degree of inhibition of pERK (Fig. 5B)

This model generates this KEfED data structure:

	[ ?cell-type ][  ?reagent ][ ?time ][ ?primary-antibody ] 

The statement basically makes four statements.

	[ 8505C ][ AZD52ff ][ 24h ][ tHER2 ] = 'positive'

	[ 8505C ][ AZD52ff ][ 24h ][ tHER3 ] = 'positive'
	[ 8505C ][ AZD52ff ][ 24h ][ pHER3 ] = 'positive'
	[ 8505C ][ AZD52ff ][ 24h ][ pERK ] = 'negative'

How does this information inform a BioPAX-style model?

The simple answer to this is that “it doesn’t”