## Asking Dialogue Systems for an Opinion

### Details about the contents

- `(convai1|convai2|cornell)_preprocess.py`
    - these scripts are used for downloading and preparing training and testing dialogue data
        - `python -u (convai1|convai2|cornell)_preprocess.py`
- `src/train_4models.sh`
    - trains the four encoder-decoder approaches on the Cornell dialogues
- `src/eval_4models_convai.sh`
    - evaluates the four encoder-decoder approaches on the ConvAI data
- `src/conversation`
    - contains the pretrained encoder-decoder models and also their probability outputs on ConvAI data for calculating the correlation scores
- `src/gpt_convai_eval.py`
    - scripts to perform opinion based evalation using GPT2 and DialoGPT on ConvAI dialogues
- `src/convai(1|2)_results.pickle.bz2` - we provide the already computed probability scores from the language-model-based approaches (GPT2 and DialoGPT) as a shortcut for the correlation analysis
- `src/correlations.ipynb` - Jupyter notebooks that compute the correlation scores


### Instructions

Python 3.6 is used to run the scripts. We recommend using a virtual environment like (Ana|Mini)conda. Steps:

1. Install dependencies
    - `conda env create -f env.yml`
    - `conda activate dialogue_opinion`
2. Train encoder-decoder models
    - `bash train_4models.sh`
2. Evaluate encoder-decoder models
    - `bash eval_4models_convai.sh`
2. Evaluate language-model-based approaches
    - `python -u gpt_convai_eval.py`
3. Start Jupyter:
    - `jupyter notebook`
    - used to calculate/view correlation results in `src/correlations.ipynb`

### Notice

Parts of the code are based on the following work: ["A Hierarchical Latent Structure for Variational Conversation Modeling"](https://arxiv.org/abs/1804.03424)

Original code is located here: [https://github.com/ctr4si/A-Hierarchical-Latent-Structure-for-Variational-Conversation-Modeling](https://github.com/ctr4si/A-Hierarchical-Latent-Structure-for-Variational-Conversation-Modeling)