|
2 | 2 |
|
3 | 3 | Dialogue is notoriously hard to evaluate. Past approaches have used human evaluation.
|
4 | 4 |
|
| 5 | +## Dialogue act classification |
| 6 | + |
| 7 | +Dialogue act classification is the task of classifying an utterance with respect to the fuction it serves in a dialogue, i.e. the act the speaker is performing. Dialogue acts are a type of speech acts (for Speech Act Theory, see [Austin (1975)](http://www.hup.harvard.edu/catalog.php?isbn=9780674411524) and [Searle (1969)](https://www.cambridge.org/core/books/speech-acts/D2D7B03E472C8A390ED60B86E08640E7)). |
| 8 | + |
| 9 | +### Switchboard corpus |
| 10 | +The [Switchboard-1 corpus](https://catalog.ldc.upenn.edu/ldc97s62) is a telephone speech corpus, consisting of about 2,400 two-sided telephone conversation among 543 speakers with about 70 provided conversation topics. The dataset includes the audio files and the transcription files, as well as information about the speakers and the calls. |
| 11 | + |
| 12 | +The Switchboard Dialogue Act Corpus (SwDA) [[download](https://web.stanford.edu/~jurafsky/swb1_dialogact_annot.tar.gz)] extends the Switchboard-1 corpus with tags from the [SWBD-DAMSL tagset](https://web.stanford.edu/~jurafsky/ws97/manual.august1.html), which is an augmentation to the Discourse Annotation and Markup System of Labeling (DAMSL) tagset. The 220 tags were reduced to 42 tags by clustering in order to improve the language model on the Switchboard corpus. A subset of the Switchboard-1 corpus consisting of 1155 conversations was used. The resulting tags include dialogue acts like statement-non-opinion, acknowledge, statement-opinion, agree/accept, etc. |
| 13 | +Annotated example: |
| 14 | +*Speaker:* A, *Dialogue Act:* Yes-No-Question, *Utterance:* So do you go to college right now? |
| 15 | + |
| 16 | +| Model | Accuracy | Paper / Source | Code | |
| 17 | +| ------------- | :-----:| --- | --- | |
| 18 | +| CRF-ASN (Chen et al., 2018) | 81.3 | [Dialogue Act Recognition via CRF-Attentive Structured Network](https://arxiv.org/abs/1711.05568) | | |
| 19 | +| Bi-LSTM-CRF (Kumar et al., 2017) | 79.2 | [Dialogue Act Sequence Labeling using Hierarchical encoder with CRF](https://arxiv.org/abs/1709.04250) | [Link](https://github.com/YanWenqiang/HBLSTM-CRF) | |
| 20 | +| RNN with 3 utterances in context (Bothe et al., 2018) | 77.34 | [A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural Networks](https://arxiv.org/abs/1805.06280) | | |
| 21 | + |
| 22 | + |
| 23 | +### ICSI Meeting Recorder Dialog Act (MRDA) corpus |
| 24 | +The [MRDA corpus](http://www1.icsi.berkeley.edu/Speech/mr/) [[download](http://www.icsi.berkeley.edu/~ees/dadb/icsi_mrda+hs_corpus_050512.tar.gz)] consists of about 75 hours of speech from 75 naturally-occurring meetings among 53 speakers. The tagset used for labeling is a modified version of the SWBD-DAMSL tagset. It is annotated with three types of information: marking of the dialogue act segment boundaries, marking of the dialogue acts and marking of correspondences between dialogue acts. |
| 25 | +Annotated example: |
| 26 | +*Time:* 2804-2810, *Speaker:* c6, *Dialogue Act:* s^bd, *Transcript:* i mean these are just discriminative. |
| 27 | +Multiple dialogue acts are separated by "^". |
| 28 | + |
| 29 | +| Model | Accuracy | Paper / Source | Code | |
| 30 | +| ------------- | :-----:| --- | --- | |
| 31 | +| CRF-ASN (Chen et al., 2018) | 91.7 | [Dialogue Act Recognition via CRF-Attentive Structured Network](https://arxiv.org/abs/1711.05568) | | |
| 32 | +| Bi-LSTM-CRF (Kumar et al., 2017) | 90.9 | [Dialogue Act Sequence Labeling using Hierarchical encoder with CRF](https://arxiv.org/abs/1709.04250) | [Link](https://github.com/YanWenqiang/HBLSTM-CRF) | |
| 33 | + |
5 | 34 | ## Dialogue state tracking
|
6 | 35 |
|
7 | 36 | Dialogue state tacking consists of determining at each turn of a dialogue the
|
|
0 commit comments