Not all of these papers are available as *.nxml. All of them are available as PDF.

Running Preprocessing

Note that FRIES preprocessing does not seem to work currently. May need to update REACH.

Running SciDP on Soweto

The embeddings are here /usr1/shared/projects/bigmech/data/embeddings/pyysalo_et_al

Pradeep’s version of the code is here: /usr1/home/pdasigi/workspace/sciDP/

Importantly, this includes the model files: model_att=*, which we copy into a shared
version of sciDP here: /usr1/shared/projects/bigmech/tools/sciDP

Training

python nn_passage_tagger.py /usr1/shared/projects/bigmech/data/embeddings/pyysalo_et_al/PubMed-and-PMC-w2
v.txt.gz --train_file /usr1/shared/projects/bigmech/data/discourse_tagging/train+test_data/passage_train.txt --use_attention

Testing

Setting up dependencies locally so that I can run sciDP:

• Use Anaconda2-2.5.0-Linux-x86_64.sh
• conda install theano
• conda install --channel https://conda.anaconda.org/kundajelab keras

Running the classifier:

   python nn_passage_tagger.py 
      /usr1/shared/projects/bigmech/data/embeddings/pyysalo_et_al/PubMed-and-PMC-w2v.txt.gz 
      --test_files /usr1/shared/projects/bigmech/corpora/2016-06-01-DryRun2MITRE/ --use_attention