An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detection (doi:10.60507/FK2/MAVB6H)

View:

Part 1: Document Description
Part 2: Study Description
Part 5: Other Study-Related Materials
Entire Codebook

An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detection

doi:10.60507/FK2/MAVB6H

bonndata

2023-06-05

1

Nedelchev, Rostislav, 2023, "An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detection", https://doi.org/10.60507/FK2/MAVB6H, bonndata, V1

An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detection

doi:10.60507/FK2/MAVB6H

Nedelchev, Rostislav

Python

virtual environment like (Ana|Mini)conda

bonndata

Nedelchev, Rostislav

Nedelchev, Rostislav

2023-05-20

https://doi.org/10.60507/FK2/MAVB6H

Computer and Information Science

For the last several decades, a focus of artificial intelligence work has been computer-based systems for interacting with humans. Such a system must ensure smooth communication to provide thoughtful answers by interacting with the system. A major critical issue is they can create an unwanted statement that results in a breakdown of dialogue, which degrades the overall interaction quality. Thus, it is incredibly beneficial to employ a system to detect whether a breakdown hampers the conversation flow. In this work, we propose an unsupervised dialogue breakdown detection technique that can be trained using only dialogue data and no other supervision. Our experiments show that using out-of-distribution detection methods can perform on par with competing supervised methods that use labeled English data. On the DBDC4 benchmark, it is the only unsupervised approach to our knowledge.

Nedelchev, Rostislav: Automatic Evaluation of Dialogue-Systems Using Neural-Network Methods. - Bonn, 2023. - Dissertation, Rheinische Friedrich-Wilhelms-Universität Bonn. Online-Ausgabe in bonndoc: https://nbn-resolving.org/urn:nbn:de:hbz:5-70983

urn:nbn:de:hbz:5-70983

Nedelchev, Rostislav: Automatic Evaluation of Dialogue-Systems Using Neural-Network Methods. - Bonn, 2023. - Dissertation, Rheinische Friedrich-Wilhelms-Universität Bonn. Online-Ausgabe in bonndoc: https://nbn-resolving.org/urn:nbn:de:hbz:5-70983

Label:

environment.yml

application/octet-stream

Label:

readme.md

text/markdown

Label:

evaluation.py

text/x-python

Label:

gpt2background_0.pth

application/octet-stream

Label:

gpt2odin_0.pth

application/octet-stream

Label:

gpt2_0.pth

application/octet-stream

Label:

main.py

text/x-python

Label:

metrics.py

text/x-python

Label:

nohup.out

application/octet-stream

Label:

prepare_dataset.py

text/x-python

Label:

run_eval_all.sh

application/x-sh

Label:

run_training_all.sh

application/x-sh

Label:

train.py

text/x-python

Label:

gpt2background_eval.json

application/json

Label:

gpt2odin_eval.json

application/json

Label:

gpt2_eval.json

application/json

Label:

00README

text/plain; charset=US-ASCII

Label:

Bot001_007.log.json

application/json

Label:

Bot001_010.log.json

application/json

Label:

Bot001_018.log.json

application/json

Label:

Bot001_022.log.json

application/json

Label:

Bot001_023.log.json

application/json

Label:

Bot001_025.log.json

application/json

Label:

Bot001_027.log.json

application/json

Label:

Bot001_031.log.json

application/json

Label:

Bot001_032.log.json

application/json

Label:

Bot001_034.log.json

application/json

Label:

Bot001_035.log.json

application/json

Label:

Bot001_037.log.json

application/json

Label:

Bot001_040.log.json

application/json

Label:

Bot001_041.log.json

application/json

Label:

Bot001_043.log.json

application/json

Label:

Bot001_046.log.json

application/json

Label:

Bot001_049.log.json

application/json

Label:

Bot001_054.log.json

application/json

Label:

Bot001_059.log.json

application/json

Label:

Bot001_060.log.json

application/json

Label:

Bot001_062.log.json

application/json

Label:

Bot001_065.log.json

application/json

Label:

Bot001_066.log.json

application/json

Label:

Bot001_068.log.json

application/json

Label:

Bot001_069.log.json

application/json

Label:

Bot001_072.log.json

application/json

Label:

Bot001_075.log.json

application/json

Label:

Bot001_076.log.json

application/json

Label:

Bot001_077.log.json

application/json

Label:

Bot001_079.log.json

application/json

Label:

Bot001_082.log.json

application/json

Label:

Bot001_085.log.json

application/json

Label:

Bot001_088.log.json

application/json

Label:

Bot001_090.log.json

application/json

Label:

Bot001_091.log.json

application/json

Label:

Bot001_092.log.json

application/json

Label:

Bot001_094.log.json

application/json

Label:

Bot001_095.log.json

application/json

Label:

Bot001_096.log.json

application/json

Label:

Bot001_097.log.json

application/json

Label:

Bot001_099.log.json

application/json

Label:

Bot001_100.log.json

application/json

Label:

Bot001_102.log.json

application/json

Label:

Bot001_110.log.json

application/json

Label:

Bot001_111.log.json

application/json

Label:

Bot001_113.log.json

application/json

Label:

Bot002_001.log.json

application/json

Label:

Bot002_003.log.json

application/json

Label:

Bot002_005.log.json

application/json

Label:

Bot002_010.log.json

application/json

Label:

Bot002_012.log.json

application/json

Label:

Bot002_013.log.json

application/json

Label:

Bot002_015.log.json

application/json

Label:

Bot002_016.log.json

application/json

Label:

Bot002_020.log.json

application/json

Label:

Bot002_023.log.json

application/json

Label:

Bot002_024.log.json

application/json

Label:

Bot002_027.log.json

application/json

Label:

Bot002_028.log.json

application/json

Label:

Bot002_030.log.json

application/json

Label:

Bot002_040.log.json

application/json

Label:

Bot002_041.log.json

application/json

Label:

Bot002_043.log.json

application/json

Label:

Bot002_047.log.json

application/json

Label:

Bot002_048.log.json

application/json

Label:

Bot002_049.log.json

application/json

Label:

Bot002_052.log.json

application/json

Label:

Bot002_053.log.json

application/json

Label:

Bot002_055.log.json

application/json

Label:

Bot002_056.log.json

application/json

Label:

Bot002_061.log.json

application/json

Label:

Bot002_069.log.json

application/json

Label:

Bot002_072.log.json

application/json

Label:

Bot002_074.log.json

application/json

Label:

Bot002_081.log.json

application/json

Label:

Bot002_082.log.json

application/json

Label:

Bot002_084.log.json

application/json

Label:

Bot002_093.log.json

application/json

Label:

Bot002_099.log.json

application/json

Label:

Bot003_003.log.json

application/json

Label:

Bot003_004.log.json

application/json

Label:

Bot003_009.log.json

application/json

Label:

Bot003_017.log.json

application/json

Label:

Bot003_021.log.json

application/json

Label:

Bot003_023.log.json

application/json

Label:

Bot003_024.log.json

application/json

Label:

Bot003_025.log.json

application/json

Label:

Bot003_035.log.json

application/json

Label:

Bot003_038.log.json

application/json

Label:

Bot003_041.log.json

application/json

Label:

Bot003_045.log.json

application/json

Label:

Bot003_046.log.json

application/json

Label:

Bot003_047.log.json

application/json

Label:

Bot003_050.log.json

application/json

Label:

Bot003_054.log.json

application/json

Label:

Bot003_061.log.json

application/json

Label:

Bot003_062.log.json

application/json

Label:

Bot003_067.log.json

application/json

Label:

Bot003_068.log.json

application/json

Label:

Bot003_070.log.json

application/json

Label:

Bot003_072.log.json

application/json

Label:

Bot003_073.log.json

application/json

Label:

Bot003_074.log.json

application/json

Label:

Bot003_075.log.json

application/json

Label:

Bot003_079.log.json

application/json

Label:

Bot003_080.log.json

application/json

Label:

Bot003_085.log.json

application/json

Label:

Bot003_087.log.json

application/json

Label:

Bot003_090.log.json

application/json

Label:

Bot003_091.log.json

application/json

Label:

Bot003_093.log.json

application/json

Label:

Bot003_094.log.json

application/json

Label:

Bot003_095.log.json

application/json

Label:

Bot003_096.log.json

application/json

Label:

Bot003_097.log.json

application/json

Label:

Bot003_099.log.json

application/json

Label:

Bot003_102.log.json

application/json

Label:

Bot003_105.log.json

application/json

Label:

Bot003_109.log.json

application/json

Label:

Bot003_110.log.json

application/json

Label:

Bot003_115.log.json

application/json

Label:

Bot003_116.log.json

application/json

Label:

Bot003_119.log.json

application/json

Label:

Bot003_120.log.json

application/json

Label:

Bot003_121.log.json

application/json

Label:

Bot003_124.log.json

application/json

Label:

Bot004_007.log.json

application/json

Label:

Bot004_008.log.json

application/json

Label:

Bot004_012.log.json

application/json

Label:

Bot004_013.log.json

application/json

Label:

Bot004_014.log.json

application/json

Label:

Bot004_016.log.json

application/json

Label:

Bot004_025.log.json

application/json

Label:

Bot004_026.log.json

application/json

Label:

Bot004_030.log.json

application/json

Label:

Bot004_034.log.json

application/json

Label:

Bot004_036.log.json

application/json

Label:

Bot004_039.log.json

application/json

Label:

Bot004_043.log.json

application/json

Label:

Bot004_047.log.json

application/json

Label:

Bot004_051.log.json

application/json

Label:

Bot004_052.log.json

application/json

Label:

Bot004_065.log.json

application/json

Label:

Bot004_067.log.json

application/json

Label:

Bot004_068.log.json

application/json

Label:

Bot004_070.log.json

application/json

Label:

Bot004_072.log.json

application/json

Label:

Bot004_081.log.json

application/json

Label:

Bot004_084.log.json

application/json

Label:

Bot004_086.log.json

application/json

Label:

Bot004_089.log.json

application/json

Label:

Bot004_090.log.json

application/json

Label:

Bot004_091.log.json

application/json

Label:

Bot004_092.log.json

application/json

Label:

Bot004_095.log.json

application/json

Label:

Bot004_097.log.json

application/json

Label:

Bot004_104.log.json

application/json

Label:

Bot004_109.log.json

application/json

Label:

Bot004_111.log.json

application/json

Label:

Bot004_115.log.json

application/json

Label:

Bot004_119.log.json

application/json

Label:

Bot004_120.log.json

application/json

Label:

Bot004_121.log.json

application/json

Label:

Bot004_124.log.json

application/json

Label:

Bot005_002.log.json

application/json

Label:

Bot005_006.log.json

application/json

Label:

Bot006_000.log.json

application/json

Label:

Bot006_003.log.json

application/json

Label:

Bot006_005.log.json

application/json

Label:

Bot006_009.log.json

application/json

Label:

Bot006_013.log.json

application/json

Label:

Bot006_015.log.json

application/json

Label:

Bot006_017.log.json

application/json

Label:

iris_DBDC4_003.log.json

application/json

Label:

iris_DBDC4_009.log.json

application/json

Label:

iris_DBDC4_013.log.json

application/json

Label:

iris_DBDC4_020.log.json

application/json

Label:

iris_DBDC4_023.log.json

application/json

Label:

iris_DBDC4_025.log.json

application/json

Label:

iris_DBDC4_030.log.json

application/json

Label:

iris_DBDC4_033.log.json

application/json

Label:

iris_DBDC4_042.log.json

application/json

Label:

iris_DBDC4_048.log.json

application/json

Label:

iris_DBDC4_051.log.json

application/json

Label:

iris_DBDC4_054.log.json

application/json

Label:

iris_DBDC4_056.log.json

application/json

Label:

iris_DBDC4_064.log.json

application/json

Label:

iris_DBDC4_065.log.json

application/json

Label:

iris_DBDC4_066.log.json

application/json

Label:

iris_DBDC4_077.log.json

application/json

Label:

iris_DBDC4_080.log.json

application/json

Label:

iris_DBDC4_083.log.json

application/json

Label:

iris_DBDC4_085.log.json

application/json

Label:

iris_DBDC4_091.log.json

application/json

Label:

iris_DBDC4_098.log.json

application/json

Label:

iris_DBDC4_103.log.json

application/json

Label:

iris_DBDC4_104.log.json

application/json

Label:

iris_DBDC4_105.log.json

application/json

Label:

iris_DBDC4_111.log.json

application/json

Label:

iris_DBDC4_114.log.json

application/json

Label:

messages.html-10.log.json

application/json

Label:

messages.html-12.log.json

application/json

Label:

messages.html-13.log.json

application/json

Label:

messages.html-15.log.json

application/json

Label:

messages.html-16.log.json

application/json

Label:

messages.html-19.log.json

application/json

Label:

messages.html-2.log.json

application/json

Label:

messages.html-20.log.json

application/json

Label:

messages.html-21.log.json

application/json

Label:

messages.html-23.log.json

application/json

Label:

messages.html-26.log.json

application/json

Label:

messages.html-28.log.json

application/json

Label:

messages.html-3.log.json

application/json

Label:

messages.html-5.log.json

application/json

Label:

messages.html-6.log.json

application/json

Label:

messages.html-10.log.json

application/json

Label:

messages.html-11.log.json

application/json

Label:

messages.html-13.log.json

application/json

Label:

messages.html-14.log.json

application/json

Label:

messages.html-15.log.json

application/json

Label:

messages.html-2.log.json

application/json

Label:

messages.html-20.log.json

application/json

Label:

messages.html-21.log.json

application/json

Label:

messages.html-23.log.json

application/json

Label:

messages.html-24.log.json

application/json

Label:

messages.html-29.log.json

application/json

Label:

messages.html-4.log.json

application/json

Label:

messages.html-6.log.json

application/json

Label:

messages.html-8.log.json

application/json

Label:

messages.html-10.log.json

application/json

Label:

messages.html-13.log.json

application/json

Label:

messages.html-15.log.json

application/json

Label:

messages.html-17.log.json

application/json

Label:

messages.html-2.log.json

application/json

Label:

messages.html-20.log.json

application/json

Label:

messages.html-21.log.json

application/json

Label:

messages.html-22.log.json

application/json

Label:

messages.html-23.log.json

application/json

Label:

messages.html-25.log.json

application/json

Label:

messages.html-27.log.json

application/json

Label:

messages.html-28.log.json

application/json

Label:

messages.html-6.log.json

application/json

Label:

messages.html-8.log.json

application/json

Label:

messages.html-1.log.json

application/json

Label:

messages.html-12.log.json

application/json

Label:

messages.html-13.log.json

application/json

Label:

messages.html-14.log.json

application/json

Label:

messages.html-17.log.json

application/json

Label:

messages.html-18.log.json

application/json

Label:

messages.html-2.log.json

application/json

Label:

messages.html-20.log.json

application/json

Label:

messages.html-25.log.json

application/json

Label:

messages.html-26.log.json

application/json

Label:

messages.html-27.log.json

application/json

Label:

messages.html-28.log.json

application/json

Label:

messages.html-4.log.json

application/json

Label:

messages.html-5.log.json

application/json

Label:

messages.html-6.log.json

application/json

Label:

messages.html-8.log.json

application/json

Label:

messages.html-12.log.json

application/json

Label:

messages.html-13.log.json

application/json

Label:

messages.html-14.log.json

application/json

Label:

messages.html-2.log.json

application/json

Label:

messages.html-22.log.json

application/json

Label:

messages.html-23.log.json

application/json

Label:

messages.html-24.log.json

application/json

Label:

messages.html-25.log.json

application/json

Label:

messages.html-26.log.json

application/json

Label:

messages.html-29.log.json

application/json

Label:

messages.html-4.log.json

application/json

Label:

messages.html-5.log.json

application/json

Label:

messages.html-7.log.json

application/json

Label:

messages.html-9.log.json

application/json

Label:

1550628075.log.json

application/json

Label:

1550628190.log.json

application/json

Label:

1550628223.log.json

application/json

Label:

1550628251.log.json

application/json

Label:

1550628280.log.json

application/json

Label:

1550628300.log.json

application/json

Label:

1550628303.log.json

application/json

Label:

1550628325.log.json

application/json

Label:

1550628334.log.json

application/json

Label:

1550628344.log.json

application/json

Label:

1550628345.log.json

application/json

Label:

1550628374.log.json

application/json

Label:

1550628376.log.json

application/json

Label:

1550628377.log.json

application/json

Label:

1550628388.log.json

application/json

Label:

1550628390.log.json

application/json

Label:

1550628396.log.json

application/json

Label:

1550628406.log.json

application/json

Label:

1550628407.log.json

application/json

Label:

1550628426.log.json

application/json

Label:

1550628445.log.json

application/json

Label:

1550628465.log.json

application/json

Label:

1550628547.log.json

application/json

Label:

1550628594.log.json

application/json

Label:

1550628606.log.json

application/json

Label:

1550628612.log.json

application/json

Label:

1550628619.log.json

application/json

Label:

1550628679.log.json

application/json

Label:

1550628703.log.json

application/json

Label:

1550628708.log.json

application/json

Label:

1550628728.log.json

application/json

Label:

1550628738.log.json

application/json

Label:

1550628739.log.json

application/json

Label:

1550628747.log.json

application/json

Label:

1550628752.log.json

application/json

Label:

1550628794.log.json

application/json

Label:

1550628807.log.json

application/json

Label:

1550628862.log.json

application/json

Label:

1550628868.log.json

application/json

Label:

1550628911.log.json

application/json

Label:

1550628927.log.json

application/json

Label:

1550629012.log.json

application/json

Label:

1550629091.log.json

application/json

Label:

1550629256.log.json

application/json

Label:

1550629275.log.json

application/json

Label:

1550629405.log.json

application/json

Label:

1550629759.log.json

application/json

Label:

1550629796.log.json

application/json

Label:

1550632049.log.json

application/json

Label:

1550646797.log.json

application/json

Label:

1550793218.log.json

application/json

Label:

1550793261.log.json

application/json

Label:

1550793278.log.json

application/json

Label:

1550793279.log.json

application/json

Label:

1550793280.log.json

application/json

Label:

1550793283.log.json

application/json

Label:

1550793289.log.json

application/json

Label:

1550793364.log.json

application/json

Label:

1550793396.log.json

application/json

Label:

1550793420.log.json

application/json

Label:

1550793521.log.json

application/json

Label:

1550793545.log.json

application/json

Label:

1550793561.log.json

application/json

Label:

1550793568.log.json

application/json

Label:

1550793579.log.json

application/json

Label:

1550793652.log.json

application/json

Label:

1550793659.log.json

application/json

Label:

1550793661.log.json

application/json

Label:

1550793681.log.json

application/json

Label:

1550793685.log.json

application/json

Label:

1550793737.log.json

application/json

Label:

1550793753.log.json

application/json

Label:

1550793770.log.json

application/json

Label:

1550793800.log.json

application/json

Label:

1550793816.log.json

application/json

Label:

1550793826.log.json

application/json

Label:

1550793863.log.json

application/json

Label:

1550793927.log.json

application/json

Label:

1550793960.log.json

application/json

Label:

1550793972.log.json

application/json

Label:

1550794037.log.json

application/json

Label:

1550794115.log.json

application/json

Label:

1550794156.log.json

application/json

Label:

1550794173.log.json

application/json

Label:

1550794197.log.json

application/json

Label:

1550794204.log.json

application/json

Label:

1550794243.log.json

application/json

Label:

1550794256.log.json

application/json

Label:

1550794378.log.json

application/json

Label:

1550794588.log.json

application/json

Label:

1550795036.log.json

application/json

Label:

1550795192.log.json

application/json

Label:

1550795615.log.json

application/json

Label:

1550796630.log.json

application/json

Label:

1550796777.log.json

application/json

Label:

1550797119.log.json

application/json

Label:

1550797330.log.json

application/json

Label:

1550798218.log.json

application/json

Label:

1550814141.log.json

application/json

Label:

1550814305.log.json

application/json

Label:

1550565796.log.json

application/json

Label:

1550565841.log.json

application/json

Label:

1550565855.log.json

application/json

Label:

1550565938.log.json

application/json

Label:

1550565947.log.json

application/json

Label:

1550565978.log.json

application/json

Label:

1550565981.log.json

application/json

Label:

1550565995.log.json

application/json

Label:

1550566015.log.json

application/json

Label:

1550566035.log.json

application/json

Label:

1550566067.log.json

application/json

Label:

1550566073.log.json

application/json

Label:

1550566129.log.json

application/json

Label:

1550566180.log.json

application/json

Label:

1550566190.log.json

application/json

Label:

1550566203.log.json

application/json

Label:

1550566238.log.json

application/json

Label:

1550566252.log.json

application/json

Label:

1550566272.log.json

application/json

Label:

1550566284.log.json

application/json

Label:

1550566326.log.json

application/json

Label:

1550566364.log.json

application/json

Label:

1550566390.log.json

application/json

Label:

1550566427.log.json

application/json

Label:

1550566432.log.json

application/json

Label:

1550566495.log.json

application/json

Label:

1550566516.log.json

application/json

Label:

1550566517.log.json

application/json

Label:

1550566524.log.json

application/json

Label:

1550566663.log.json

application/json

Label:

1550566677.log.json

application/json

Label:

1550566691.log.json

application/json

Label:

1550566872.log.json

application/json

Label:

1550567014.log.json

application/json

Label:

1550567117.log.json

application/json

Label:

1550567148.log.json

application/json

Label:

1550567155.log.json

application/json

Label:

1550567234.log.json

application/json

Label:

1550567255.log.json

application/json

Label:

1550567288.log.json

application/json

Label:

1550567437.log.json

application/json

Label:

1550567523.log.json

application/json

Label:

1550567546.log.json

application/json

Label:

1550567651.log.json

application/json

Label:

1550568195.log.json

application/json

Label:

1550568432.log.json

application/json

Label:

1550568472.log.json

application/json

Label:

1550569424.log.json

application/json

Label:

1550571274.log.json

application/json

Label:

1550577973.log.json

application/json

Label:

gpt2.py

text/x-python

Label:

gpt2_background.py

text/x-python

Label:

gpt2_odin.py

text/x-python

Label:

__init__.py

text/x-python

Label:

gpt2.cpython-38.pyc

application/octet-stream

Label:

gpt2_background.cpython-38.pyc

application/octet-stream

Label:

gpt2_odin.cpython-38.pyc

application/octet-stream

Label:

__init__.cpython-38.pyc

application/octet-stream