An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detectiondoi:10.60507/FK2/MAVB6Hbonndata2023-06-051Nedelchev, Rostislav, 2023, "An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detection", https://doi.org/10.60507/FK2/MAVB6H, bonndata, V1An Unsupervised Baseline For Dialogue Breakdown Detection Using Ouf-of-distribution Detectiondoi:10.60507/FK2/MAVB6HNedelchev, RostislavPythonvirtual environment like (Ana|Mini)condabonndataNedelchev, RostislavNedelchev, Rostislav2023-05-20Computer and Information ScienceFor the last several decades, a focus of artificial intelligence work has been computer-based systems for interacting with humans. Such a system must ensure smooth communication to provide thoughtful answers by interacting with the system. A major critical issue is they can create an unwanted statement that results in a breakdown of dialogue, which degrades the overall interaction quality. Thus, it is incredibly beneficial to employ a system to detect whether a breakdown hampers the conversation flow. In this work, we propose an unsupervised dialogue breakdown detection technique that can be trained using only dialogue data and no other supervision. Our experiments show that using out-of-distribution detection methods can perform on par with competing supervised methods that use labeled English data. On the DBDC4 benchmark, it is the only unsupervised approach to our knowledge.<a href="http://creativecommons.org/publicdomain/zero/1.0">CC0 1.0</a>Nedelchev, Rostislav: Automatic Evaluation of Dialogue-Systems Using Neural-Network Methods. - Bonn, 2023. - Dissertation, Rheinische Friedrich-Wilhelms-Universität Bonn. Online-Ausgabe in bonndoc: https://nbn-resolving.org/urn:nbn:de:hbz:5-70983urn:nbn:de:hbz:5-70983Nedelchev, Rostislav: Automatic Evaluation of Dialogue-Systems Using Neural-Network Methods. - Bonn, 2023. - Dissertation, Rheinische Friedrich-Wilhelms-Universität Bonn. Online-Ausgabe in bonndoc: https://nbn-resolving.org/urn:nbn:de:hbz:5-70983environment.ymlapplication/octet-streamreadme.mdtext/markdownevaluation.pytext/x-pythongpt2background_0.pthapplication/octet-streamgpt2odin_0.pthapplication/octet-streamgpt2_0.pthapplication/octet-streammain.pytext/x-pythonmetrics.pytext/x-pythonnohup.outapplication/octet-streamprepare_dataset.pytext/x-pythonrun_eval_all.shapplication/x-shrun_training_all.shapplication/x-shtrain.pytext/x-pythongpt2background_eval.jsonapplication/jsongpt2odin_eval.jsonapplication/jsongpt2_eval.jsonapplication/json00READMEtext/plain; charset=US-ASCIIBot001_007.log.jsonapplication/jsonBot001_010.log.jsonapplication/jsonBot001_018.log.jsonapplication/jsonBot001_022.log.jsonapplication/jsonBot001_023.log.jsonapplication/jsonBot001_025.log.jsonapplication/jsonBot001_027.log.jsonapplication/jsonBot001_031.log.jsonapplication/jsonBot001_032.log.jsonapplication/jsonBot001_034.log.jsonapplication/jsonBot001_035.log.jsonapplication/jsonBot001_037.log.jsonapplication/jsonBot001_040.log.jsonapplication/jsonBot001_041.log.jsonapplication/jsonBot001_043.log.jsonapplication/jsonBot001_046.log.jsonapplication/jsonBot001_049.log.jsonapplication/jsonBot001_054.log.jsonapplication/jsonBot001_059.log.jsonapplication/jsonBot001_060.log.jsonapplication/jsonBot001_062.log.jsonapplication/jsonBot001_065.log.jsonapplication/jsonBot001_066.log.jsonapplication/jsonBot001_068.log.jsonapplication/jsonBot001_069.log.jsonapplication/jsonBot001_072.log.jsonapplication/jsonBot001_075.log.jsonapplication/jsonBot001_076.log.jsonapplication/jsonBot001_077.log.jsonapplication/jsonBot001_079.log.jsonapplication/jsonBot001_082.log.jsonapplication/jsonBot001_085.log.jsonapplication/jsonBot001_088.log.jsonapplication/jsonBot001_090.log.jsonapplication/jsonBot001_091.log.jsonapplication/jsonBot001_092.log.jsonapplication/jsonBot001_094.log.jsonapplication/jsonBot001_095.log.jsonapplication/jsonBot001_096.log.jsonapplication/jsonBot001_097.log.jsonapplication/jsonBot001_099.log.jsonapplication/jsonBot001_100.log.jsonapplication/jsonBot001_102.log.jsonapplication/jsonBot001_110.log.jsonapplication/jsonBot001_111.log.jsonapplication/jsonBot001_113.log.jsonapplication/jsonBot002_001.log.jsonapplication/jsonBot002_003.log.jsonapplication/jsonBot002_005.log.jsonapplication/jsonBot002_010.log.jsonapplication/jsonBot002_012.log.jsonapplication/jsonBot002_013.log.jsonapplication/jsonBot002_015.log.jsonapplication/jsonBot002_016.log.jsonapplication/jsonBot002_020.log.jsonapplication/jsonBot002_023.log.jsonapplication/jsonBot002_024.log.jsonapplication/jsonBot002_027.log.jsonapplication/jsonBot002_028.log.jsonapplication/jsonBot002_030.log.jsonapplication/jsonBot002_040.log.jsonapplication/jsonBot002_041.log.jsonapplication/jsonBot002_043.log.jsonapplication/jsonBot002_047.log.jsonapplication/jsonBot002_048.log.jsonapplication/jsonBot002_049.log.jsonapplication/jsonBot002_052.log.jsonapplication/jsonBot002_053.log.jsonapplication/jsonBot002_055.log.jsonapplication/jsonBot002_056.log.jsonapplication/jsonBot002_061.log.jsonapplication/jsonBot002_069.log.jsonapplication/jsonBot002_072.log.jsonapplication/jsonBot002_074.log.jsonapplication/jsonBot002_081.log.jsonapplication/jsonBot002_082.log.jsonapplication/jsonBot002_084.log.jsonapplication/jsonBot002_093.log.jsonapplication/jsonBot002_099.log.jsonapplication/jsonBot003_003.log.jsonapplication/jsonBot003_004.log.jsonapplication/jsonBot003_009.log.jsonapplication/jsonBot003_017.log.jsonapplication/jsonBot003_021.log.jsonapplication/jsonBot003_023.log.jsonapplication/jsonBot003_024.log.jsonapplication/jsonBot003_025.log.jsonapplication/jsonBot003_035.log.jsonapplication/jsonBot003_038.log.jsonapplication/jsonBot003_041.log.jsonapplication/jsonBot003_045.log.jsonapplication/jsonBot003_046.log.jsonapplication/jsonBot003_047.log.jsonapplication/jsonBot003_050.log.jsonapplication/jsonBot003_054.log.jsonapplication/jsonBot003_061.log.jsonapplication/jsonBot003_062.log.jsonapplication/jsonBot003_067.log.jsonapplication/jsonBot003_068.log.jsonapplication/jsonBot003_070.log.jsonapplication/jsonBot003_072.log.jsonapplication/jsonBot003_073.log.jsonapplication/jsonBot003_074.log.jsonapplication/jsonBot003_075.log.jsonapplication/jsonBot003_079.log.jsonapplication/jsonBot003_080.log.jsonapplication/jsonBot003_085.log.jsonapplication/jsonBot003_087.log.jsonapplication/jsonBot003_090.log.jsonapplication/jsonBot003_091.log.jsonapplication/jsonBot003_093.log.jsonapplication/jsonBot003_094.log.jsonapplication/jsonBot003_095.log.jsonapplication/jsonBot003_096.log.jsonapplication/jsonBot003_097.log.jsonapplication/jsonBot003_099.log.jsonapplication/jsonBot003_102.log.jsonapplication/jsonBot003_105.log.jsonapplication/jsonBot003_109.log.jsonapplication/jsonBot003_110.log.jsonapplication/jsonBot003_115.log.jsonapplication/jsonBot003_116.log.jsonapplication/jsonBot003_119.log.jsonapplication/jsonBot003_120.log.jsonapplication/jsonBot003_121.log.jsonapplication/jsonBot003_124.log.jsonapplication/jsonBot004_007.log.jsonapplication/jsonBot004_008.log.jsonapplication/jsonBot004_012.log.jsonapplication/jsonBot004_013.log.jsonapplication/jsonBot004_014.log.jsonapplication/jsonBot004_016.log.jsonapplication/jsonBot004_025.log.jsonapplication/jsonBot004_026.log.jsonapplication/jsonBot004_030.log.jsonapplication/jsonBot004_034.log.jsonapplication/jsonBot004_036.log.jsonapplication/jsonBot004_039.log.jsonapplication/jsonBot004_043.log.jsonapplication/jsonBot004_047.log.jsonapplication/jsonBot004_051.log.jsonapplication/jsonBot004_052.log.jsonapplication/jsonBot004_065.log.jsonapplication/jsonBot004_067.log.jsonapplication/jsonBot004_068.log.jsonapplication/jsonBot004_070.log.jsonapplication/jsonBot004_072.log.jsonapplication/jsonBot004_081.log.jsonapplication/jsonBot004_084.log.jsonapplication/jsonBot004_086.log.jsonapplication/jsonBot004_089.log.jsonapplication/jsonBot004_090.log.jsonapplication/jsonBot004_091.log.jsonapplication/jsonBot004_092.log.jsonapplication/jsonBot004_095.log.jsonapplication/jsonBot004_097.log.jsonapplication/jsonBot004_104.log.jsonapplication/jsonBot004_109.log.jsonapplication/jsonBot004_111.log.jsonapplication/jsonBot004_115.log.jsonapplication/jsonBot004_119.log.jsonapplication/jsonBot004_120.log.jsonapplication/jsonBot004_121.log.jsonapplication/jsonBot004_124.log.jsonapplication/jsonBot005_002.log.jsonapplication/jsonBot005_006.log.jsonapplication/jsonBot006_000.log.jsonapplication/jsonBot006_003.log.jsonapplication/jsonBot006_005.log.jsonapplication/jsonBot006_009.log.jsonapplication/jsonBot006_013.log.jsonapplication/jsonBot006_015.log.jsonapplication/jsonBot006_017.log.jsonapplication/jsoniris_DBDC4_003.log.jsonapplication/jsoniris_DBDC4_009.log.jsonapplication/jsoniris_DBDC4_013.log.jsonapplication/jsoniris_DBDC4_020.log.jsonapplication/jsoniris_DBDC4_023.log.jsonapplication/jsoniris_DBDC4_025.log.jsonapplication/jsoniris_DBDC4_030.log.jsonapplication/jsoniris_DBDC4_033.log.jsonapplication/jsoniris_DBDC4_042.log.jsonapplication/jsoniris_DBDC4_048.log.jsonapplication/jsoniris_DBDC4_051.log.jsonapplication/jsoniris_DBDC4_054.log.jsonapplication/jsoniris_DBDC4_056.log.jsonapplication/jsoniris_DBDC4_064.log.jsonapplication/jsoniris_DBDC4_065.log.jsonapplication/jsoniris_DBDC4_066.log.jsonapplication/jsoniris_DBDC4_077.log.jsonapplication/jsoniris_DBDC4_080.log.jsonapplication/jsoniris_DBDC4_083.log.jsonapplication/jsoniris_DBDC4_085.log.jsonapplication/jsoniris_DBDC4_091.log.jsonapplication/jsoniris_DBDC4_098.log.jsonapplication/jsoniris_DBDC4_103.log.jsonapplication/jsoniris_DBDC4_104.log.jsonapplication/jsoniris_DBDC4_105.log.jsonapplication/jsoniris_DBDC4_111.log.jsonapplication/jsoniris_DBDC4_114.log.jsonapplication/jsonmessages.html-10.log.jsonapplication/jsonmessages.html-12.log.jsonapplication/jsonmessages.html-13.log.jsonapplication/jsonmessages.html-15.log.jsonapplication/jsonmessages.html-16.log.jsonapplication/jsonmessages.html-19.log.jsonapplication/jsonmessages.html-2.log.jsonapplication/jsonmessages.html-20.log.jsonapplication/jsonmessages.html-21.log.jsonapplication/jsonmessages.html-23.log.jsonapplication/jsonmessages.html-26.log.jsonapplication/jsonmessages.html-28.log.jsonapplication/jsonmessages.html-3.log.jsonapplication/jsonmessages.html-5.log.jsonapplication/jsonmessages.html-6.log.jsonapplication/jsonmessages.html-10.log.jsonapplication/jsonmessages.html-11.log.jsonapplication/jsonmessages.html-13.log.jsonapplication/jsonmessages.html-14.log.jsonapplication/jsonmessages.html-15.log.jsonapplication/jsonmessages.html-2.log.jsonapplication/jsonmessages.html-20.log.jsonapplication/jsonmessages.html-21.log.jsonapplication/jsonmessages.html-23.log.jsonapplication/jsonmessages.html-24.log.jsonapplication/jsonmessages.html-29.log.jsonapplication/jsonmessages.html-4.log.jsonapplication/jsonmessages.html-6.log.jsonapplication/jsonmessages.html-8.log.jsonapplication/jsonmessages.html-10.log.jsonapplication/jsonmessages.html-13.log.jsonapplication/jsonmessages.html-15.log.jsonapplication/jsonmessages.html-17.log.jsonapplication/jsonmessages.html-2.log.jsonapplication/jsonmessages.html-20.log.jsonapplication/jsonmessages.html-21.log.jsonapplication/jsonmessages.html-22.log.jsonapplication/jsonmessages.html-23.log.jsonapplication/jsonmessages.html-25.log.jsonapplication/jsonmessages.html-27.log.jsonapplication/jsonmessages.html-28.log.jsonapplication/jsonmessages.html-6.log.jsonapplication/jsonmessages.html-8.log.jsonapplication/jsonmessages.html-1.log.jsonapplication/jsonmessages.html-12.log.jsonapplication/jsonmessages.html-13.log.jsonapplication/jsonmessages.html-14.log.jsonapplication/jsonmessages.html-17.log.jsonapplication/jsonmessages.html-18.log.jsonapplication/jsonmessages.html-2.log.jsonapplication/jsonmessages.html-20.log.jsonapplication/jsonmessages.html-25.log.jsonapplication/jsonmessages.html-26.log.jsonapplication/jsonmessages.html-27.log.jsonapplication/jsonmessages.html-28.log.jsonapplication/jsonmessages.html-4.log.jsonapplication/jsonmessages.html-5.log.jsonapplication/jsonmessages.html-6.log.jsonapplication/jsonmessages.html-8.log.jsonapplication/jsonmessages.html-12.log.jsonapplication/jsonmessages.html-13.log.jsonapplication/jsonmessages.html-14.log.jsonapplication/jsonmessages.html-2.log.jsonapplication/jsonmessages.html-22.log.jsonapplication/jsonmessages.html-23.log.jsonapplication/jsonmessages.html-24.log.jsonapplication/jsonmessages.html-25.log.jsonapplication/jsonmessages.html-26.log.jsonapplication/jsonmessages.html-29.log.jsonapplication/jsonmessages.html-4.log.jsonapplication/jsonmessages.html-5.log.jsonapplication/jsonmessages.html-7.log.jsonapplication/jsonmessages.html-9.log.jsonapplication/json1550628075.log.jsonapplication/json1550628190.log.jsonapplication/json1550628223.log.jsonapplication/json1550628251.log.jsonapplication/json1550628280.log.jsonapplication/json1550628300.log.jsonapplication/json1550628303.log.jsonapplication/json1550628325.log.jsonapplication/json1550628334.log.jsonapplication/json1550628344.log.jsonapplication/json1550628345.log.jsonapplication/json1550628374.log.jsonapplication/json1550628376.log.jsonapplication/json1550628377.log.jsonapplication/json1550628388.log.jsonapplication/json1550628390.log.jsonapplication/json1550628396.log.jsonapplication/json1550628406.log.jsonapplication/json1550628407.log.jsonapplication/json1550628426.log.jsonapplication/json1550628445.log.jsonapplication/json1550628465.log.jsonapplication/json1550628547.log.jsonapplication/json1550628594.log.jsonapplication/json1550628606.log.jsonapplication/json1550628612.log.jsonapplication/json1550628619.log.jsonapplication/json1550628679.log.jsonapplication/json1550628703.log.jsonapplication/json1550628708.log.jsonapplication/json1550628728.log.jsonapplication/json1550628738.log.jsonapplication/json1550628739.log.jsonapplication/json1550628747.log.jsonapplication/json1550628752.log.jsonapplication/json1550628794.log.jsonapplication/json1550628807.log.jsonapplication/json1550628862.log.jsonapplication/json1550628868.log.jsonapplication/json1550628911.log.jsonapplication/json1550628927.log.jsonapplication/json1550629012.log.jsonapplication/json1550629091.log.jsonapplication/json1550629256.log.jsonapplication/json1550629275.log.jsonapplication/json1550629405.log.jsonapplication/json1550629759.log.jsonapplication/json1550629796.log.jsonapplication/json1550632049.log.jsonapplication/json1550646797.log.jsonapplication/json1550793218.log.jsonapplication/json1550793261.log.jsonapplication/json1550793278.log.jsonapplication/json1550793279.log.jsonapplication/json1550793280.log.jsonapplication/json1550793283.log.jsonapplication/json1550793289.log.jsonapplication/json1550793364.log.jsonapplication/json1550793396.log.jsonapplication/json1550793420.log.jsonapplication/json1550793521.log.jsonapplication/json1550793545.log.jsonapplication/json1550793561.log.jsonapplication/json1550793568.log.jsonapplication/json1550793579.log.jsonapplication/json1550793652.log.jsonapplication/json1550793659.log.jsonapplication/json1550793661.log.jsonapplication/json1550793681.log.jsonapplication/json1550793685.log.jsonapplication/json1550793737.log.jsonapplication/json1550793753.log.jsonapplication/json1550793770.log.jsonapplication/json1550793800.log.jsonapplication/json1550793816.log.jsonapplication/json1550793826.log.jsonapplication/json1550793863.log.jsonapplication/json1550793927.log.jsonapplication/json1550793960.log.jsonapplication/json1550793972.log.jsonapplication/json1550794037.log.jsonapplication/json1550794115.log.jsonapplication/json1550794156.log.jsonapplication/json1550794173.log.jsonapplication/json1550794197.log.jsonapplication/json1550794204.log.jsonapplication/json1550794243.log.jsonapplication/json1550794256.log.jsonapplication/json1550794378.log.jsonapplication/json1550794588.log.jsonapplication/json1550795036.log.jsonapplication/json1550795192.log.jsonapplication/json1550795615.log.jsonapplication/json1550796630.log.jsonapplication/json1550796777.log.jsonapplication/json1550797119.log.jsonapplication/json1550797330.log.jsonapplication/json1550798218.log.jsonapplication/json1550814141.log.jsonapplication/json1550814305.log.jsonapplication/json1550565796.log.jsonapplication/json1550565841.log.jsonapplication/json1550565855.log.jsonapplication/json1550565938.log.jsonapplication/json1550565947.log.jsonapplication/json1550565978.log.jsonapplication/json1550565981.log.jsonapplication/json1550565995.log.jsonapplication/json1550566015.log.jsonapplication/json1550566035.log.jsonapplication/json1550566067.log.jsonapplication/json1550566073.log.jsonapplication/json1550566129.log.jsonapplication/json1550566180.log.jsonapplication/json1550566190.log.jsonapplication/json1550566203.log.jsonapplication/json1550566238.log.jsonapplication/json1550566252.log.jsonapplication/json1550566272.log.jsonapplication/json1550566284.log.jsonapplication/json1550566326.log.jsonapplication/json1550566364.log.jsonapplication/json1550566390.log.jsonapplication/json1550566427.log.jsonapplication/json1550566432.log.jsonapplication/json1550566495.log.jsonapplication/json1550566516.log.jsonapplication/json1550566517.log.jsonapplication/json1550566524.log.jsonapplication/json1550566663.log.jsonapplication/json1550566677.log.jsonapplication/json1550566691.log.jsonapplication/json1550566872.log.jsonapplication/json1550567014.log.jsonapplication/json1550567117.log.jsonapplication/json1550567148.log.jsonapplication/json1550567155.log.jsonapplication/json1550567234.log.jsonapplication/json1550567255.log.jsonapplication/json1550567288.log.jsonapplication/json1550567437.log.jsonapplication/json1550567523.log.jsonapplication/json1550567546.log.jsonapplication/json1550567651.log.jsonapplication/json1550568195.log.jsonapplication/json1550568432.log.jsonapplication/json1550568472.log.jsonapplication/json1550569424.log.jsonapplication/json1550571274.log.jsonapplication/json1550577973.log.jsonapplication/jsongpt2.pytext/x-pythongpt2_background.pytext/x-pythongpt2_odin.pytext/x-python__init__.pytext/x-pythongpt2.cpython-38.pycapplication/octet-streamgpt2_background.cpython-38.pycapplication/octet-streamgpt2_odin.cpython-38.pycapplication/octet-stream__init__.cpython-38.pycapplication/octet-stream