10.60507/FK2/FX37GD
Nedelchev, Rostislav
Rostislav
Nedelchev
Evaluating Dialogue Systems via an Opinion
bonndata
2023
Computer and Information Science
Nedelchev, Rostislav
Rostislav
Nedelchev
2023-05-20
2023-06-05
urn:nbn:de:hbz:5-70983
10.60507/FK2/FX37GD/8VBSOR
10.60507/FK2/FX37GD/N2IXSR
10.60507/FK2/FX37GD/BOU3CD
10.60507/FK2/FX37GD/QBLRDJ
10.60507/FK2/FX37GD/9UBFBQ
10.60507/FK2/FX37GD/2TKD2I
10.60507/FK2/FX37GD/TUDRHR
10.60507/FK2/FX37GD/BOWWOZ
10.60507/FK2/FX37GD/S9LRT6
10.60507/FK2/FX37GD/ABNMFD
10.60507/FK2/FX37GD/LV43N2
10.60507/FK2/FX37GD/XJZFUV
10.60507/FK2/FX37GD/PBRQNB
10.60507/FK2/FX37GD/ZLHDCD
10.60507/FK2/FX37GD/UNY5NS
10.60507/FK2/FX37GD/Y3UOPJ
10.60507/FK2/FX37GD/KTOF9E
10.60507/FK2/FX37GD/JZQ4N0
10.60507/FK2/FX37GD/NUHNDS
10.60507/FK2/FX37GD/S1QAQO
10.60507/FK2/FX37GD/LCF1CF
10.60507/FK2/FX37GD/SRPBOJ
10.60507/FK2/FX37GD/JDG14V
10.60507/FK2/FX37GD/AJDHXG
10.60507/FK2/FX37GD/DOU0LY
10.60507/FK2/FX37GD/AG0AW5
10.60507/FK2/FX37GD/FLF5LX
10.60507/FK2/FX37GD/06CHLJ
10.60507/FK2/FX37GD/I55ZJT
10.60507/FK2/FX37GD/D3CEME
10.60507/FK2/FX37GD/HVJY9H
10.60507/FK2/FX37GD/T8RWKF
10.60507/FK2/FX37GD/AUY6FT
10.60507/FK2/FX37GD/1JAD5Q
10.60507/FK2/FX37GD/DUUM6T
10.60507/FK2/FX37GD/E33T34
10.60507/FK2/FX37GD/BV4FZO
10.60507/FK2/FX37GD/B6DJJE
10.60507/FK2/FX37GD/CSSL6N
10.60507/FK2/FX37GD/ZUN0AC
10.60507/FK2/FX37GD/VWLVS5
10.60507/FK2/FX37GD/JHHWBG
10.60507/FK2/FX37GD/24WFAL
10.60507/FK2/FX37GD/LWACD9
10.60507/FK2/FX37GD/S1R7NF
10.60507/FK2/FX37GD/0XCISU
10.60507/FK2/FX37GD/I6UXVJ
10.60507/FK2/FX37GD/0BVKZ2
10.60507/FK2/FX37GD/CJY8A1
10.60507/FK2/FX37GD/HBUADY
10.60507/FK2/FX37GD/D3LAO6
10.60507/FK2/FX37GD/EORT5G
10.60507/FK2/FX37GD/KH577K
10.60507/FK2/FX37GD/2JVDXD
10.60507/FK2/FX37GD/ZGYMCH
10.60507/FK2/FX37GD/7XPDZS
10.60507/FK2/FX37GD/XOFVG3
10.60507/FK2/FX37GD/A0CJNY
10.60507/FK2/FX37GD/DZ2WEO
10.60507/FK2/FX37GD/IB9HAA
10.60507/FK2/FX37GD/F5201G
10.60507/FK2/FX37GD/I1QPLV
10.60507/FK2/FX37GD/KLM3C8
10.60507/FK2/FX37GD/6VMWKO
10.60507/FK2/FX37GD/ZSBDMS
10.60507/FK2/FX37GD/BS8JN6
10.60507/FK2/FX37GD/VK5ZUE
10.60507/FK2/FX37GD/4ZQWNC
10.60507/FK2/FX37GD/2MES1N
10.60507/FK2/FX37GD/NGNQD9
10.60507/FK2/FX37GD/DORAXW
10.60507/FK2/FX37GD/NH9AGY
10.60507/FK2/FX37GD/SARIEN
10.60507/FK2/FX37GD/PROR8G
10.60507/FK2/FX37GD/A8J3VU
10.60507/FK2/FX37GD/JU47HH
10.60507/FK2/FX37GD/GAETYB
10.60507/FK2/FX37GD/6MBV9V
10.60507/FK2/FX37GD/GJDI3F
10.60507/FK2/FX37GD/5DR8V3
10.60507/FK2/FX37GD/MICAJI
10.60507/FK2/FX37GD/GC2ZMQ
10.60507/FK2/FX37GD/ZTRS2H
10.60507/FK2/FX37GD/TAPTXE
10.60507/FK2/FX37GD/HHVQWO
10.60507/FK2/FX37GD/S6XG7G
10.60507/FK2/FX37GD/UGTWQO
10.60507/FK2/FX37GD/VUEJB4
10.60507/FK2/FX37GD/PEGSXI
10.60507/FK2/FX37GD/QHSX4M
10.60507/FK2/FX37GD/1MZCAQ
10.60507/FK2/FX37GD/UIFJPZ
10.60507/FK2/FX37GD/659PYV
10.60507/FK2/FX37GD/T5ZKDW
10.60507/FK2/FX37GD/1C1WP2
5804611
16644
7938
856064
71458
4954
2080
299497
128405303
3070
6460
2542718
6015
1207
223
56177
123603
3668
7540
2077
4320
186459282
1138
860
6760930
212482594
296152508
290691
1168
5279906
1665
7540
12223
38317
273
34641919
124157
997403
868352
52383
2943221
886
1102
953
860160
1688
1146880
16644
8097
1095517
133094
0
13323619
5250859
1130496
21374451
6036
1774
4795
147
6445
6636788
1017054
299497
2086
2127
2743
1179
8548
8853
1134592
4182
2526
42099789
986246
7666
1094130
7745
2077
1132
210
60871
1134592
705695
259
1922
856064
19884
6870
12544
3070
67289
895
1680
2000
application/json
application/octet-stream
text/x-python
application/octet-stream
application/x-ipynb+json
application/octet-stream
text/plain
application/octet-stream
application/octet-stream
application/octet-stream
text/x-python
application/octet-stream
text/x-python
text/x-python
text/x-python
text/plain
application/octet-stream
text/x-python
application/octet-stream
text/plain
application/octet-stream
application/octet-stream
text/x-python
text/x-python
text/plain
application/octet-stream
application/octet-stream
application/pdf
text/x-python
application/octet-stream
text/x-python
application/octet-stream
text/x-python
text/x-python
text/x-python
text/plain
application/octet-stream
application/octet-stream
application/octet-stream
application/octet-stream
application/octet-stream
text/x-python
text/x-python
text/x-python
application/octet-stream
text/x-python
application/octet-stream
application/octet-stream
text/x-python
text/plain
application/octet-stream
text/x-python
application/x-bzip2
application/octet-stream
application/octet-stream
application/x-bzip2
text/x-python
text/x-python
text/x-python
text/x-python
text/x-python
application/json
text/plain
application/octet-stream
text/plain
text/x-python
text/x-python
application/x-sh
text/x-python
text/x-python
application/octet-stream
text/plain
text/x-python
application/octet-stream
text/plain
text/x-python
text/plain
application/octet-stream
text/plain
text/x-python
application/x-sh
application/octet-stream
application/octet-stream
text/plain
text/x-python
text/x-python
application/octet-stream
text/x-python
text/x-python
text/x-python
application/octet-stream
text/plain
text/x-python
text/x-python
text/markdown
1.0
Creative Commons CC0 1.0 Universal Public Domain Dedication.
Dialogue systems are a significant field of research and development in artificial intelligence. Until today, the evaluation of such algorithms happens in one fundamental way. They solve "hypothetical problems," i.e., dialog systems are tested by being asked to respond in specific scenarios and provide a "solution", i.e. a reply, to a "problem". The replies are either explicitly compared to reference responses using overlap-based metrics (e.g., BLEU) or are evaluated by human annotators, which can also be seen as compared to (implicit) references. Instead, we propose to ask dialogue systems to tell whether a sample "solution" to a sample "problem" is good or bad. In other words, we ask another dialog system whether a conversation is fluent and coherent or not, and to what degree. In our experiments, we show how to evaluate dialogue systems by "asking for an opinion" and that it indeed offers an additional perspective on assessing these methods.
Python, 3.6
virtual environment like (Ana|Mini)conda