Speech Interface for controlling an Hi-fi Audio System Based

Speech Interface for controlling an Hi-fi Audio System Based on

a Bayesian Belief Networks Approach for Dialog Modeling

Fernando Fernández, Javier Ferreiros, Valentín Sama,

Juan Manuel Montero, Rubén San Segundo, Javier Macías-Guarasa, Rafael García Speech Technology Group, Dept. of Electronic Engineering, Universidad Politécnica de Madrid

E.T.S.I. Telecomunicación, Ciudad Universitaria s/n, 28040-Madrid, Spain

{efhes, jfl, vsama,juancho, lapiz, macias, rgarcia}@die.upm.es

Abstract

This paper presents the development of a speech interface for controlling a high fidelity system from natural language sentences. A Bayesian Belief Network approach is proposed for dialog modeling. This solution is applied to infer the user’s goals corresponding to the processed utterances. Subsequently, from the inferred goals, missing or spurious concepts are automatically detected. This is used to drive the dialog prompting for missing concepts and clarifying for spurious concepts allowing more flexible and natural dialogs.

A dialog strategy which makes use of the dialog history and the system’s state is also presented.

1.Introduction

We assume dialogue as a communicative process aimed to satisfy some objective. In the context of controlling electronic domestic devices, such goals could be the execution of some commands. In this sense dialogue modeling plays a fundamental role in helping users to reach their dialogue goals efficiently.

Typically, rule based solutions are used. In those solutions the dialog is programmed as a script coded by a set of execution rules. Each of these rules specifies an interaction with the system, [1][2]. Specifically each rule is associated to a specific goal and establishes a set of conditions under which that rule should “fire”. When a rule fires, an execution module handles the operation/action. Of course, the rules may trigger the system to ask for more information in order to be able to execute a command or action. However, this is a closed solution since the commands executed are only those ones that exactly match with the conditions of each rule. Therefore this solution allows a limited dialog only in those cases where the user is talking about some goal for which we have designed specific clarification rules.

On the other hand, the Belief Networks (BN) is a stochastic solution that takes several advantages respect to that approach. BN can be applied for mixed-initiative dialog modeling where BN infer the user’s informational goals [3]. Subsequently, using the backward inference technique, missing or spurious concepts can be automatically detected based on the inferred goals. This is used to drive the dialog prompting for missing concepts and clarifying for spurious concepts allowing more flexible and natural dialogs. Additionally, the automatic learning of the dialog manager is possible from training data. Moreover, BN allow us to include external knowledge into the BN models from which we manage the dialog. This allows us to take advantage of an expert’s knowledge in the application domain.

2.System architecture

This is a conversational interface that allows users to drive the Hi-fi system from natural language sentences, differentially from other typical control systems based on simple commands. Thus, users can feel free to give several complex commands from a single sentence. Moreover, they don’t have to memorize any command list neither use a closed specific phraseology in order to control the system successfully. The hi-fi audio system we are controlling is a commercial system constituted by a compact disc (with a charger of three discs), two tapes and a radio receiver. This system can be controlled by an infrared (IR) remote control. Instead, users are going to control the Hi-fi system from a microphone. Our interface translates the speech into IR commands in order to carry out some operation or action over the system. This translation is done so that the appropriate IR commands are sent according to the user’s intention. Figure 1 shows a block diagram of our interface.

Figure 1: Block diagram of the interface

2.1.Speech Recognition Module

This module translates the speech into a sequence of words from a continuous speech recognizer based on CD-HMMs.

166ee1e8172ded630b1cb6a8nguage Understanding Module

This module extracts the relevant semantic concepts from the sequence of words output by the recognition module.

First, we have to assign the possible semantic categories to each word present in the sentence according to a semantic dictionary. The dictionary has been defined by an expert in the application domain trying to cover all the semantic categories regarding the control of a Hi-fi system. The semantic categories can be classified into the following groups: actions (to be carried out with the hi-fi system i.e. to play), parameters (that can be configured in the system, i.e. the volume), and values (that can be assigned to the system’s parameters i.e. a number).

Last, we apply a set of context dependent rules, designed and handcrafted by the expert, in order to tag each word with the appropriate semantic attributes according to the specific context of the processed sentence. Rules are not eliminatory and their application order goes from the specific to the general. This way we extract the relevant concepts from the sentence as a list of pairs: attribute-value. 2.3. Dialog Manager Module

The dialog manager is based on BN. From the list of concepts output by the language understanding module it infers the dialog goals applying Bayesian inference. Then, based on the inferred goals, it applies the backward inference technique to automatically detect the missing and the spurious concepts, prompting the user for the first ones and trying to clarify the latter. This process will be explained in detail later.

From the inferred goals and the confirmed concepts, the dialog manager fills an execution frame for each present action in the user’s command. Each action is coded by a frame with three fields: the selected source, the specified parameter and the assigned value. 2.4. Execution Module

From the execution frames the execution module determines the set of IR remote commands to be sent to the Hi-fi. This module has to check the system’s state before the sending of the IR commands in order to check if it is possible or not to execute the specified action, i.e. it makes no sense to try to switch-off of the system when the system is already off.

Figure 2 represents the system’s state.

HIFI

RADIO

BAND TUNE MEMORY STEREO

STATE(ON/OFF)

VOLUME

EQUALIZATION

SOURCE

CD

STATE PLAY MODE SELECTED CD

MEMORY

TRACK TAPE

TAPE 2 STATE

TAPE 1 STATE TAPE

Figure 2: Hi-fi system’s state

2.5. Response Generation Module

This module provides some feedback to the users about the performed actions by the system and the system’s interpretation about the user’s intention. Moreover, this module defines the appropriate questions from which the system asks the user about some needed information in order to perform a requested action.

As each specific action is associated to a particular dialog goal, this module is constituted by a set of goal specific templates. From these templates the response generation module selects the appropriate text messages to be prompted according to the inferred goal. In order to make the dialogs more natural we select the text message randomly each time. 2.6. Text-to-Speech Module

This module synthesizes the speech from the sentence proposed by the generation module in order to give useful feedback to users.

3. BN based Dialog Modelling

3.1. Overview of the Belief Network Approach

From concepts extracted from semantic parsing, as well as those retrieved from the dialog history, we are going to infer the goal(s) of the user’s query. A goal is considered as a specific action over the Hi-fi system, i.e. to set the volume to a specific value. A set of 20 goals and a set of 70 concepts have been defined by an expert in the application domain according to a relevance criterion for the control task.

We are going to develop one BN per goal. A BN is a directed acyclic graph with nodes and arcs where the direction of the arcs represents the probabilistic dependency between two nodes. The arrows of the acyclic graph are drawn from cause to effect. Assuming the basic topology depicted in Figure 3 we are modelling the causal relation between the goal and the concepts.

Figure 3: Basic Topology for a BN

This topology assumes conditional independence among concepts. Each BN is defined by a specific goal G i and a set of input concepts C j . We have assumed that the goals and the concepts are all binary, so the concept C j is true (C j =1) when it is observed in the sentence. In order to avoid too complex models, the expert has selected the concepts with the strongest dependency for each goal as its inputs.

Hence, we have to make N (N=20) binary decisions, with N BN, on the presence or the absence for each goal. From observations extracted from the user’s sentence, i.e. ={C 1=0, C 2=1,…, C M =1}, we apply Bayesian Inference to obtain the a posteriori probability P(G i |C) for each goal (see Equation 1, it simply applies Bayes’ Theorem assuming marginal and conditional independence, which is equivalent to a na?ve Bayes formulation; M is the number of input evidences). Then we make a binary decision by comparing this probability with a defined threshold.

()()()()

{}

M M M

k k k i k k i i c C c C c C C where c C P G c C P G P C G P ==========∏

=,...,,11121111

Equation 1. Bayesian Inference.

We are going to assume that the goal is present or active if its posteriori probability is greater than the threshold; otherwise the goal is absent. For simplicity, the confidence threshold may be set to 0.5 since (1):

搜索“diyifanwen.net”或“第一范文网”即可找到本站免费阅读全部范文。收藏本站方便下次阅读，第一范文网，提供最新实用文档Speech Interface for controlling an Hi-fi Audio System Based全文阅读和word下载服务。