Speech Transcript - More than meets the ear: Processes that shape dialogue

0:00:16	thank you just that paper that kinda and introduction i'm so excited that you invited
0:00:20	me it's been several years since i've been to this conference
0:00:23	and in looking over the paper is at the conference i was really excited just
0:00:28	see that some of them are
0:00:29	on topics that some of us of an issue signal long time but we have
0:00:32	been unable to wrestle and or
0:00:34	control and up to study in some cases and some of us have managed to
0:00:37	make some have a way there
0:00:39	so things like flexible interest and interaction in interruptions reference resolution
0:00:44	entrainment in p incremental understanding nonverbal behaviors sarcasm social strategies
0:00:51	non-native speakers and also nontask-oriented dialog
0:00:54	so the for those of us we've been studying dialogue for a while he's really
0:00:58	gratifying to see that social in cognitive question that of things so interesting at so
0:01:02	ill formed a couple of decades ago
0:01:04	are now making their way into real systems at least
0:01:07	ask racially not functionally
0:01:10	so i want to ask you guys a question
0:01:14	i meant is that it would be okay if i did a little history of
0:01:17	some early findings in this talk
0:01:20	and i want to ask you
0:01:22	when you first start studying dialogue and wide you study dialogue
0:01:27	i wanted to know did you choose this topic or did it choose you
0:01:32	when you first started working on dialogue where you fortunate enough to be in an
0:01:35	environment where everyone around you with also interest dialogue or did you have to swim
0:01:39	upstream
0:01:41	i have the feeling that so you had to swim upstream and you just kind
0:01:44	of
0:01:44	meta the topic id whatever live you were working in
0:01:48	i one is just a the back in the mid eighties when lynn walker or
0:01:51	not you were working at hewlett packard natural language group
0:01:55	we had a manager well at least you tried to managers but we were pretty
0:01:58	much unmanageable given that then you're you know we're talking about and
0:02:04	i need to really appreciate why were so interested in pronouns okay
0:02:10	we are really interested in private so what if you start of basically trying to
0:02:15	study connected discourse because we were shallow people who wanted to become famous okay so
0:02:21	that that's why this computational guy one that we were starting from its okay and
0:02:26	so basically will work hewlett packard
0:02:29	and
0:02:31	waffling about whether to go back to grad school and whether get my phd in
0:02:34	psychology or linguistics or computer science
0:02:37	i discovered can designer and barbara grosses work in a high on discourse that i
0:02:42	just found it incredibly exciting i thought it was very exciting that they were actually
0:02:46	looking at language and trying to explain the structure of language via rules or whatever
0:02:50	okay
0:02:51	and then i when i went to my first little workshop on discourse the dialogue
0:02:55	i and it is incredibly exciting that so many of the big names in the
0:02:59	field where women a barber growth can decide our julie a bunny karen's marginal and
0:03:04	all of these people i found really inspiring so i knew that my field and
0:03:09	i wound up in psychology
0:03:10	not quite a flip of the three sided coin but close to it okay
0:03:14	knowing very much a psychologist
0:03:17	though this i want to show you some early example of the kind of under
0:03:23	later kinda psychology experiments like to later on for this is comes from
0:03:28	back in the days when i was working at hewlett packard in the natural language
0:03:31	group
0:03:32	and there was the system that came out and all database query system are called
0:03:37	human a from them and tech and it enable people to do type dialogues like
0:03:42	this one where the user could type in about a little database
0:03:45	who have the computer you imagine database the
0:03:49	equipment employees managers et cetera
0:03:51	the system responds with shelley do the following create a report showing full name a
0:03:55	manager in equipment from all forms of which equipment include computer
0:03:59	now back in those days on the remaining people who just love this little system
0:04:04	at that there is nothing at all wrong with the dialogue like that okay
0:04:07	so one of the first thing that i did one i got there was this
0:04:11	just seems kind of atrocious so they put me in charge of working on the
0:04:14	articulator
0:04:15	for awhile and so
0:04:17	i basically i thought the most obvious hack to fix that was just provide answers
0:04:24	that parallel the forms of the question that a list of them so
0:04:28	i just to the parse trees and i random backwards and i stuck the answer
0:04:31	into the wh position the parse tree
0:04:34	this solution with not endorsed or popular with
0:04:38	the linguists on the project recounted unprincipled and
0:04:41	yes it didn't work all the time they got it was kind of cheesy but
0:04:44	it worked ninety five percent of the time and so that's good enough for demo
0:04:47	is you will know and so that's what we had to be for a while
0:04:53	so basically noticing this problem early on and the obvious peaks
0:04:58	began to kind of fire of an interest in what i don't think of is
0:05:02	entrainment in dialogue which is this connectedness between
0:05:05	utterances where people are using information that the other person may have endorsed are introduced
0:05:12	and creating
0:05:13	together something that i think of is the conceptual pact to proceed in this way
0:05:17	and of course this is a flexible pattern can be adjusted very rapidly when the
0:05:21	situation changes
0:05:24	so back to a this dialogue system the other things we did
0:05:28	where
0:05:29	basically when one conversation when something goes wrong usually the kind of feedback you get
0:05:34	from your partners than up to
0:05:36	set you wanna course of initiating repair that will fix things
0:05:40	so the system at that time
0:05:42	i didn't display answers in any understandable way it just kind of
0:05:46	brought a little representations the database objects in response to queries
0:05:49	this particular one with the
0:05:51	we don't board managers and equipment and employee so we switched of an goes painting
0:05:56	that was more fun to work although there is just a simple
0:05:59	and so basically depending on you know this is very modular system you know good
0:06:04	a software engineering design of all that and modularity made a very interest a very
0:06:09	a straightforward to implement
0:06:11	i mean so if the break the system experience response to user query like
0:06:15	where van gogh paint starry night which is missing
0:06:18	are where we do rather
0:06:21	then it would get a break in the syntactic module and gives you an error
0:06:25	message call basically in the form of please try rephrasing that
0:06:29	if the break within the lexical module where is and then something and interpretable as
0:06:34	the word it would repeat the word and then say that's an unknown a word
0:06:39	or if it was just an outside of domain error or how large is window
0:06:42	x a it could interpret how large a starry night and give you the dimensions
0:06:46	of the painting but it couldn't tell you how large ringo was
0:06:49	it would come back with sorry that's not the database so again this was an
0:06:53	attempt to early on we tried to get what basic rudimentary dialogue capabilities of the
0:06:58	system that the system could possibly have to reflect what people might expect now
0:07:04	this is way back in the days before we had a dialogue manager that something
0:07:07	that linen others worked on
0:07:09	once we got going and eliza that was important
0:07:14	so i know that there is no anthropomorphic is them in these messages that with
0:07:18	the debate that was active back then and so i reported assiduously the use of
0:07:22	having the system
0:07:23	we refer to itself as i
0:07:25	at the time
0:07:28	and so that to pronouns on so back in those days when we're working on
0:07:33	the natural language processing system
0:07:35	i was all this talk about what will people do when they talk to compute
0:07:38	are able to use all the english that they are used to using where they
0:07:41	use basic english or something called tiny english of the time i think with called
0:07:46	and will they just kind of avoid using anything but a restricted subset will take
0:07:50	you can buy that the computers are restricted partner
0:07:53	and so faq all just thing real remote been done in nineteen eighty seven did
0:07:57	what i thought of what i think of is the first wizard-of-oz study or the
0:08:00	first one i became aware of my that was a really exciting technique when she
0:08:04	did this like i was
0:08:05	i thought well that's great you don't have to put people through a system that
0:08:09	doesn't work very well you can simulated on the other hand
0:08:11	and she found in da to support this hypothesis the prediction that people did not
0:08:16	use pronouns in they
0:08:17	when they were
0:08:21	talking to a system that provided advice about statistics now when you think about that
0:08:25	again she was looking at things like personal pronouns like you see in it
0:08:29	well there's not very many chances you have to talk about you she in it
0:08:32	when you're asking
0:08:33	about your in nova or to explain a t-test or something like
0:08:37	so that struck me as a little bit you know a premature conclusion i didn't
0:08:41	apply
0:08:43	so basically when i and others were that we worked with including hold with their
0:08:49	the time and decided not to take that idea to seriously
0:08:53	and so we forged ahead to spiderman adjourned we start working on pronounced so then
0:08:57	and i added to do a really wonderful classes at stanford type i part we're
0:09:01	grows and rape arrow and bill cullen
0:09:04	and we got a hold of this wonderful draft paper like roosters you mind sitting
0:09:08	on entering
0:09:09	occult scored the computational theory of discourse
0:09:12	back in nineteen eighty six this was and it with actually later published much later
0:09:16	and you probably seen that but we have the old version that
0:09:19	that very prominently across the top do not slight of course we added
0:09:23	with barbara splicing eventually decided it anyway
0:09:26	i and so basically we just to some of the ideas and that paper and
0:09:30	we use them to interpret pronouns in each you know so i'm not gonna go
0:09:33	into the details but this little box represents some of the rules between
0:09:37	transitions between sentences and the attentional shifts associated with these intent with these transitions and
0:09:43	so
0:09:43	i this is revolutionary because
0:09:46	at the time i was really interested in what people to win and the cognition
0:09:49	that's going around when you're interacting with the system and
0:09:53	and many of the people around us were very interested just in the formal representations
0:09:57	and just trying to parse sentences to begin with then what they were doing was
0:10:00	very wonderful and l there as well but
0:10:04	we were interested in the fact that you're really thinking about the psychology of a
0:10:07	user when you're parsing syntax and interpreting referring expressions
0:10:12	so
0:10:13	so this box represents the algorithm and we were doing really simple kinds of sentences
0:10:18	like dan works with derek at each p supervises derek there is a programmer he
0:10:23	answer the question is which
0:10:25	who does the he represent in these various situations
0:10:29	now i start working on centering i still get a few papers every year to
0:10:34	review but i usually turn them down if i'm the longer
0:10:37	doing that kind of research and part the reason for that is that it became
0:10:41	obvious to me that there is much more going on
0:10:46	in pronoun interpretation and the interpretation of referring expressions that just a simple algorithm so
0:10:51	i you know i think the centering approach from christchurch and white is wonderful and
0:10:56	was groundbreaking and taking account of the yours the speaker centers of attention
0:11:01	but there's it's much messier than that so i decided to go back to grad
0:11:05	school and get my phd in psychology with her park and then it entered the
0:11:09	messi world of human behaviour
0:11:12	which we all live at least one were not work
0:11:16	i and so my very first experiment uses a language game in which i tested
0:11:21	some of the predictions that we had to write from centering theory namely that mentioning
0:11:25	something in subject position as opposed to as the object of a sentence
0:11:30	me to say only and thus able to be pre-normalized
0:11:33	so i was thinking of a pronoun as a few that picks out the most
0:11:38	saintly representation in your partner is mental model of the situation
0:11:43	not as
0:11:44	something to trigger research on among all the possible reference for that pronoun which is
0:11:49	how the algorithm worked and that works well for a computer but it
0:11:53	is certainly not help people do it okay
0:11:55	so from the hearer's perspective i saw the interpretation of the pronoun just as the
0:11:59	selection of the best out of all possible interpretations
0:12:02	that's because it was most salient not because of the search
0:12:06	and so it basically i recorded pairs the stanford students to were naive subjects they
0:12:12	both the word basketball day and time and i had to do something that they
0:12:16	found engaging one of them watched a video of a basketball game without the silent
0:12:20	i gave a running description play by play
0:12:24	all the other one behind a screen had to keep track of who had the
0:12:27	ball at any given moment
0:12:29	and they had to write down with the ball but a like one on which
0:12:32	is kind of random
0:12:33	and they could speak to each other as much as they liked so this language
0:12:36	game got people to generate chains of referring expressions to the same object
0:12:41	but lots and lots of third person singular mail
0:12:45	entities in the discourse which is just what i was after
0:12:48	and so that may generated things like and now we'll train set of all they're
0:12:52	going down number thirty you passes it up to forty one forty one goes up
0:12:56	a shot emails
0:12:57	now what you're eight grade english
0:12:59	ugh teacher would probably have taught us that
0:13:03	a pronoun refers of the most recent thing that agrees a gender number we all
0:13:06	know that's not true and so you can see with this pattern they repeat the
0:13:11	speaker repeats of the noun phrase forty one rather than problem i think off of
0:13:15	that
0:13:17	that
0:13:19	referring expression and so
0:13:21	basically this task was it worked really well because the semantics of the task or
0:13:26	biased against the centering prediction
0:13:29	so the centering projection based well the semantics of the task basically say that you
0:13:33	can't shoot unless you have a ball that obvious right so you should be able
0:13:37	to get away of the pronoun here word forty one is underlined but you don't
0:13:41	you follow the predictions of centering
0:13:43	on average not always the can people don't always
0:13:46	don't ever do something all the time they do it with some probability okay
0:13:51	and so again the action ones very fast paced and so the speaker had to
0:13:55	try very hard to keep up
0:13:57	and so it always shorter to use a pronoun that of all noun phrase sometimes
0:14:01	they would usable noun phrase like
0:14:03	you know
0:14:03	number forty one the degree chi force whatever you know they really we're getting into
0:14:07	this task in providing colourful descriptions
0:14:10	so basically
0:14:13	just as the centering l two algorithm projected many people referred to in handy
0:14:18	of the people and he object
0:14:20	and then they refer to it there are much more likely to read refer to
0:14:24	it
0:14:24	by repeating it verbatim
0:14:27	as a volunteer then in instead of pro normalizing it and so they would move
0:14:30	in a to subject position and then they would problem lies that was on the
0:14:33	pro one of the predictions we derive from sorry
0:14:37	and so the other thing is if pronoun was used in that position
0:14:43	with forty one so number thirty passes of up to forty one he goes up
0:14:47	for the shot any misses the problem would get stressed
0:14:50	and that also an interesting discovery so
0:14:52	i have several different techniques at their disposal to maintaining a good focus of attention
0:14:58	with her address these and we found evidence for both those things
0:15:03	so at this point it was pretty clear that the algorithm was not psychologically possible
0:15:07	okay her park with kind of horrified that i was even work there's a what
0:15:11	i said like paper is ready just do you wanna be michael what is that
0:15:16	no that's okay go ahead submitted and i was horribly offended i think you take
0:15:20	me a favour in retrospect but you know i with
0:15:22	kind of cross but i was eager to move on to the world where the
0:15:25	world of psychology than just entering
0:15:28	so clearly cognitive and perceptual accessibility
0:15:31	it is important in both speech planning and reference resolution
0:15:35	but since entering your the entities
0:15:37	are not allowed to decay the discourse context are pretty much the whole sentence at
0:15:41	least the way our algorithm worked
0:15:43	and so it really require segmented discourse in order to pull off the centering algorithm
0:15:49	and so i was learning is the students like a linguistics that on language planning
0:15:52	and interpretation are really incremental huntley incremental more incremental i could've imagined
0:15:58	not even word by word but as soon as you hear two hundred milliseconds of
0:16:01	a word
0:16:01	you're you start to work on it as a listener
0:16:04	and so on that certainly that information was just coming on the scene around nineteen
0:16:10	ninety five and mike townhouse and his lab
0:16:12	publish their really important early work on visual worlds and
0:16:16	and so i was very eager are
0:16:19	back in grad school to
0:16:22	move on to the world of psychology and do something that was plausible and yet
0:16:25	computationally interest
0:16:27	so
0:16:29	back to you guys
0:16:32	i don't know if you thought about what cause you to
0:16:36	a start working on dialogue make we can talk about this at the end of
0:16:40	question period there's on but i also what you think about what i think dialogue
0:16:44	is
0:16:45	what is dialogue will kinda obvious right
0:16:48	the rest of this talk is really about what is dialogue and the assumptions that
0:16:53	you make about what dialogue is what the essences and what it's what simplifying assumptions
0:16:58	are safe to make
0:16:59	and don't destroy the phenomena of interest
0:17:02	and a what things are okay to control in your experiment
0:17:06	and will destroy the thing that you're trying to study okay
0:17:10	so the question is what we need to preserve in our research in order to
0:17:14	model dialogue appropriately
0:17:17	so
0:17:18	so if you think about the way we approach dialogue with respect both to machines
0:17:22	and humans
0:17:25	let's start with some kind of data
0:17:27	these might be data from previous experiments or examples that we find compelling and we
0:17:32	wish to implement or embody in a system or in an experiment
0:17:36	maybe the storyboard of have someone interacts with the an intelligent personal assistant
0:17:41	or maybe it's the corpora were looking at were looking at distributions of behavior over
0:17:45	lots of lots of people aggregated
0:17:47	or maybe a description of some product that somebody think should be built okay and
0:17:52	that we take those that those data those examples and we do something with them
0:17:56	and on the left what we do i guess like point with my cursor
0:18:02	can i can't one i can put the microphone on the left we have engineering
0:18:07	where we're trying to create a computational formalisms for dialogue processing in management on the
0:18:13	right we actually have reverse engineering what we're doing we're trying to figure out how
0:18:16	human processing works
0:18:18	with all of its cognitive social and neural constraints
0:18:22	so that if you think about the very different tasks
0:18:26	these two different
0:18:27	things involve okay
0:18:29	so when we think about how dialogue is implemented in dialogue systems we have no
0:18:35	limits on working memory you know if we want to remember the past and
0:18:39	create a space for which you can search for the referent of a referring expression
0:18:43	go back here it can cover thousands of users it doesn't necessarily cover that individual
0:18:49	that came up yesterday the attentional focus doesn't need to be modeled like human machines
0:18:56	don't have the same kinds of interruptions by the fact that we're now trying to
0:18:59	have them i'll do more than one thing and a type in these personal assistant
0:19:03	okay
0:19:04	but they don't their performance need not to k on any one of these things
0:19:08	while they're doing it okay
0:19:10	and the inferences are represented logically and their computed no matter what okay where is
0:19:15	the people you know often people here pronoun they don't even bother to resolve it
0:19:19	if they don't need to if it's difficult if it doesn't just pick something out
0:19:22	of their standard attention easily okay
0:19:25	so people don't always make the inferences that you think that maybe they should be
0:19:28	making in the hike
0:19:31	the architecture because we are
0:19:33	some of us are then software engineers at various points that still are maybe
0:19:37	you know that modularity makes things a lot more elegant one okay
0:19:41	so that tends to be the architectural choice when you're modeling dialogue systems
0:19:47	and there's also we're very limited perceptual ability for monitoring now there's work presented a
0:19:52	conference on reading people space of facial expressions and looking at these kinds of wonderful
0:19:56	nonverbal
0:19:57	things that will talk about a lot in a few minutes
0:20:00	and that's really important if you're really going to be a full dialogue partner and
0:20:04	deal with pragmatics and way that is easy for people to deal
0:20:08	so in mind brain we have limited working memory okay
0:20:12	we have and attentional focus that emerges from biological constraints
0:20:17	okay so
0:20:18	and it's probably evolutionary really good that we forget things that we don't always have
0:20:23	things active in working memory so forgetting is an important skill it turns out
0:20:28	inferences are often associated and they're not always made in some of my talk will
0:20:32	be about how certain important kinds of inferences are made and how they are deployed
0:20:37	in just in dialogue processing
0:20:40	and whether it's done really immediately in easily and automatically or later as a kind
0:20:45	of laborious work here okay
0:20:48	and then the architecture has to admit incremental processing now but first time
0:20:54	i thought of the first time that a man stand was what i went to
0:20:56	rochester to work with her and a team that included my tandem house
0:21:01	we were trying to write a and nsf grant
0:21:04	that would enable parsers to be incremental head pose we didn't get finding but
0:21:08	it was a wonderful thing to do because i'm at amanda step
0:21:11	so i
0:21:13	okay no bit or comments on that okay lots of other good things to get
0:21:18	a fixed iq and stuff
0:21:21	and so in that architecture of the difficult one to implement and i'm delighted that
0:21:26	many the people in this conference are really acknowledging that it's not necessary to always
0:21:31	do that with
0:21:32	spoken dialog systems but sometimes it might be desirable especially if you really wanna make
0:21:36	something human like
0:21:38	and again you can have abundant monitoring of perceptual information and of planned
0:21:43	you know people monitor their own upcoming speech errors that their speaking they monitor all
0:21:47	kinds of feedback coming in from the world and so
0:21:50	that kind of monitoring isn't part of most spoken dialogue systems just
0:21:54	so that's our question if you're a computational linguist or an engineer you make various
0:21:58	sorts of simplifying assumption okay and all of these assumptions move our research for it
0:22:03	but i think it's important not to lose track of what we had to set
0:22:06	aside in order to proceed because it might come back to hunt is
0:22:11	so it's good to make these assumptions explicit so the way in which you station
0:22:15	experiment often depends on your implicit theory of what a dialogue is
0:22:21	here's what i think a dialogue is here's a good example from my collection now
0:22:25	it to use the same example over and over in different parts are making different
0:22:29	points so apologies of using some of my examples before
0:22:32	i'm presenting this is a different context right now
0:22:35	so this i think a good example of what i think a spoken dialogue is
0:22:38	that this one comes out a bit lower so if the
0:22:42	if someone's adjusting the audio or maybe actually just like computers
0:22:46	now this was collected by trying to crawl g who works that you want now
0:22:50	with one of my early grad students and
0:22:54	we were trying to collect examples of a spontaneous getting to know you dialogue dialogues
0:23:00	from
0:23:01	a bilingual through didn't know each other were bilingual and they were recruited to the
0:23:05	last of these are two strangers
0:23:06	i
0:23:11	ordering
0:23:13	right
0:23:22	what
0:23:27	what
0:23:32	what and why i
0:23:37	right
0:23:42	i love this example it has so much in it and i went when my
0:23:46	i p a can model this then i will be happy
0:23:49	i will retire then okay so what i'm about this is that you know there's
0:23:55	all this really interesting stuff there's code switching but you can see that little can
0:24:01	the little constituents a little increments that each speaker presents what they're doing their face
0:24:06	to face that can see each other
0:24:08	their grounding where each other since they're trying to get you know each other they're
0:24:11	giving each other constant can see what's feedback about how some things than interpreted and
0:24:16	that's kind of manifested in the simultaneous speech around the asterisks where they both say
0:24:22	something at the exact same time
0:24:24	and one jumps in to define the spanish term at the other one is presented
0:24:28	so it's very clear that they very quickly establish this common ground
0:24:32	and they use that's very as a foundational part of their conversation
0:24:38	and so you know we can
0:24:40	okay we can observe abundant examples of referring expressions in any given task dialogue but
0:24:46	it's really important to think about the language game in which people are finding themselves
0:24:50	in this particular language game
0:24:52	they have the ability to fully established common ground with each other and there's nothing
0:24:56	restricted from doing that
0:24:58	on
0:24:59	and so in psychology experiment as you know there are often very we're they're very
0:25:03	weird language games you know
0:25:05	students come at the level paranoid what about today
0:25:08	trying to read my minded you know there are they all have this notion of
0:25:12	social psychology experiments which often have a large with them and then list of their
0:25:15	board to that in a kind of experiment where they
0:25:18	or just getting a cube right now
0:25:20	you are very different language case of so
0:25:22	a language game here is nothing like that one
0:25:25	but in most language games that we set of the lab we're trying to get
0:25:28	many observations from someone so we can have enough power to draw a conclusion so
0:25:34	that we can find out something
0:25:36	create new knowledge about dialogue
0:25:37	and you statistics on it so in a typical rep referential communication experiment we have
0:25:43	two people coming to lab signed consent
0:25:45	and then we initiate this mysterious language games and then they meet each other they're
0:25:49	see that with the barrier in between them
0:25:51	they're given identical to that the picture cards something like this perhaps
0:25:55	and that they need to get matching to the same order
0:25:57	and then they have these like the conversations are not gonna belabour this next example
0:26:01	it's only here if you're someone who has seen this kind of stuff before which
0:26:05	i think you probably are not you would be this room but
0:26:08	"'cause" you know what dialogue is but
0:26:09	here's the kind of dialogue the two subjects might reduced about a particular card you
0:26:13	can see it's very length in that's all of disfluencies and provisional
0:26:18	utterances
0:26:20	like a for this one are it looks kind of like the top their squares
0:26:23	that the looks i know and then be goes a
0:26:28	meeting i don't not quite sure yet i'm trying and you have sort of another
0:26:32	like rectangle shape and then like rectangle angle than on the bottom it's are under
0:26:36	what that is clash eight already i think i got it
0:26:40	it's almost like a person kind of in a weird way like a much prettier
0:26:43	something which is interesting here because be doesn't know what it is an europeans are
0:26:48	proposing the perspective that they end up taking throughout the rest of the experiment
0:26:52	and so we have them refer to this over and over again okay
0:26:56	and so later on you know about eleven cards later after we scramble them and
0:27:00	put them out again be gets to be the director this time and b goes
0:27:03	right to that unite a number nine is that one training and it goes you
0:27:07	open about eleven cards later a now is the director and that's number three michael
0:27:12	case the disentrainment so what these people are done is they have proposed
0:27:16	and kind of training a weighted in on an agreeable perspective to both of them
0:27:20	and then they both use that
0:27:22	so this is i found very striking and even more striking was what people do
0:27:26	in different carers talking about the same object
0:27:30	people come up with very different perspectives
0:27:34	this one you problem you've seen in other types are just as an example
0:27:37	you know you might call the anchor the candle
0:27:40	the symmetrical one shapes on top of shakes are my favourite them and jumping in
0:27:44	the air with bell bottoms on
0:27:45	and they continue to refer to what throughout the experiment as
0:27:48	i sh slightly shortened version of that okay
0:27:51	so it's really amazing to me there so much variation language that's probably one of
0:27:56	the things that attracts us to study that's good you love trying to explain that
0:27:59	variation
0:28:00	but there's very little variation it turns out when people have had a chance to
0:28:03	in trained on something okay so as the system designers you can exploit that
0:28:08	in terms of your intelligent personal assistant you can constrain
0:28:11	the set of things people state not because of tiny english or anything like that
0:28:15	the because people coming trained on these things
0:28:18	and so we view this as people setting up a conceptual pact that was the
0:28:22	term that you've clark suggested referred me one night when we were casting about for
0:28:26	the right term to capture what it was people ended up with a during training
0:28:30	on something okay
0:28:32	so and our first set of experiments we used both tang rooms and
0:28:37	these common objects
0:28:38	and you know we use the tigers just to throw them in there so people
0:28:42	would get distracted because you know system is a language game people are gonna try
0:28:46	to reverse engineer what you're doing to them and you don't want them focusing in
0:28:49	trying to guess i'd guess what you're hypothesis as
0:28:52	so what we were interested in what people would call things like used are dogs
0:28:55	cars and finish
0:28:57	they that we're setting the ten groups that they focused entirely on that
0:29:01	and so basically what we found in this a conceptual pacts experiment was that
0:29:10	people
0:29:12	don't just follow the expected gracie in the thing of saying as much information as
0:29:18	is necessary to distinguish an object from a set of objects which you find it
0:29:22	it so they would start calling this something like the really cool red car the
0:29:26	cork article right powerful right particle red car
0:29:29	and then what it was the only car in this that they didn't go right
0:29:31	to car they continue calling a typical rank are so that was our main finding
0:29:35	we also found that the extent to which they did this was probabilistic and depended
0:29:39	on how many chances they had gotten in the entrainment a part of the experiment
0:29:43	before
0:29:43	the critical trial
0:29:46	i'm such as urban i thought we were done we show something that we you
0:29:50	know that was pretty tangible in useful a controversy erupted okay
0:29:56	so to be here we are a little bit over reaching and the conclusions we're
0:30:00	drawing from these data
0:30:01	so one thing we did in the three experiments and that really paper was we
0:30:05	how to partners which try to the and for the last exterior experiment
0:30:09	and we found that people who switch partners
0:30:11	we often go back to the basic level term and just start calling at a
0:30:15	car when they do you do
0:30:17	consider the conceptual pact that they had established with a particular part or
0:30:20	what if they did that same trial with the
0:30:22	the old partner then they would
0:30:24	continue to use you know the correct are okay
0:30:27	so we were arguing that audience design or this kind of a entrainment thing with
0:30:32	partners this effect that was what we thought we had shown
0:30:36	but it turns out that
0:30:39	we don't really show it in terms of an online demonstration that you're taking this
0:30:43	information really into account with your partner okay
0:30:46	so basically
0:30:48	again this is the summary of our findings which i just covered
0:30:52	speakers were not just as the mormon of as possible and they to continue to
0:30:57	follow the conceptual pact the data samples with a particular partner
0:31:00	but they did not when they were working with someone else
0:31:04	okay
0:31:06	so
0:31:08	i one just briefly presented with a plane five acts so
0:31:11	this is the series of experiments
0:31:13	the talks about a little light we had in the literature and what i learned
0:31:16	was once the stomach that you know i was a young assistant professor back then
0:31:20	and of course the stomach acid that you get when someone attacks you're working
0:31:24	considerable right but then what it turns out is it can really be a wonderful
0:31:28	thing you can engage collegial e with your
0:31:31	where the opponent and you can both improve your research which is what i'm happy
0:31:35	to say is the ending of displaying five act or at least i think it
0:31:40	is i'm not sure my pointed agree to probably anyway they probably
0:31:44	so
0:31:46	so the first question after a verb and i published or paper
0:31:51	was the question is in train it really partner specific queries that just based on
0:31:55	which is just a simple association of memory okay
0:31:59	and so basically demonstrate that something really is partner specific to an individual
0:32:05	and not to just any old individual not just to the priming
0:32:08	in memory simple association with the between an object in the term and maybe a
0:32:12	link to that person
0:32:14	in each are really show the two people with different perspectives are knowledge so the
0:32:18	speaker the here
0:32:19	can adapt to each other from the earliest moments of processing this is hard because
0:32:23	most the time when you're in a conversation
0:32:25	you're really similar you're sharing the same context and you may i just happened to
0:32:30	get it right by chance and that's probably happens a lot of the time right
0:32:34	so dealer in both cases are publish this paper called angry comprehension linguistic precedent
0:32:42	and basically they were inspired by an anecdote that but was had where
0:32:47	you just happen to interpret something egocentric leah not gonna go into the details
0:32:52	but his proposal was that listeners expect presidents in it doesn't matter who the speaker
0:32:56	is
0:32:57	and then if you do just to speaker huge laboriously afterwards
0:33:01	inferential e as a late occurring here
0:33:04	and certainly to be fair some of the data that we presented in the re
0:33:08	original branding part paper i'm had little simple the dialogue where people would say the
0:33:13	first one is the car kind of read where red and strategy or something like
0:33:17	that so
0:33:18	so you can see sometimes it is presented after-the-fact and others other times it would
0:33:22	be the first one is that right but rowdy and
0:33:25	so you can see evidence of the syntax for early adjusting to the partner and
0:33:29	late adjusting
0:33:30	so but that's gave a talk about this at coney two thousand one in philadelphia
0:33:35	and trolls messing it i want my graduate students work in the audience that the
0:33:38	time
0:33:39	and basically i'm gonna go through this quickly
0:33:47	basically boas and they'll found no evidence for partner specific processing so
0:33:54	they had people in these somewhat unnatural situations where they're talking to someone but then
0:33:58	the subjects wearing headphones that are also getting things in there you're from some disembodied
0:34:01	voice somewhere else
0:34:02	that was pre-recorded okay
0:34:04	and so some of the time they found interference between these two things okay
0:34:09	and so
0:34:11	basically hearing the president expression the other expression that they then trained on with the
0:34:16	interactive partner
0:34:18	was no faster than hearing it from the new partner
0:34:21	and so a bar indicates that is the evidence that in train it was not
0:34:26	very specific okay
0:34:28	i mean so what's wrong with this picture
0:34:30	would be that if let's say you and i talk about something we call it
0:34:34	provides that's nancy read moderately and then i'll then walks and then she says i
0:34:39	that's that read matter it out side that would mean that we should be slower
0:34:44	to interpret it probably in
0:34:45	because she wasn't there on the entrainment phrase that doesn't make any sense it doesn't
0:34:49	preamps when using the same phrase that we've talked about just because
0:34:53	she wasn't there when we introduced and that's what this
0:34:57	argument was based on
0:34:58	so basically the criticism i had and i raised my hand during the talk and
0:35:03	i said
0:35:04	okay you fill this l b all partner using the right term again
0:35:08	you've got the new partner using the same old term in a new partner using
0:35:12	a new term
0:35:13	and you're finding that use two are both faster than this one
0:35:16	but what cell so that just one was really interesting but when you have the
0:35:20	old partner committed inexplicably break the conceptual pact what about that does not take any
0:35:25	longer and if you compare that to that's
0:35:28	if it's not part of specific then you should be the same if it is
0:35:31	partner specific and they should take much longer okay
0:35:34	so but was said in response my question well that's not an interesting cell so
0:35:39	we didn't bother with that one okay fine so childlike jump to the train and
0:35:42	ratio for training and with that of use that of his experiment we are you
0:35:46	have the set of objects are still in from by then young child story boxes
0:35:51	note when you're still but
0:35:53	little things that don't really have lexicalized expressions for them and we put them in
0:35:58	an array and we basically
0:36:00	i had a confederate speaker referred to what he object is either the shiny silver
0:36:05	that's shine use cilantro this over high whether these are equally good for that expression
0:36:10	and so basically a naive confederate a naive matcher and a confederate director repeatedly match
0:36:16	the objects and the director have the spoken use kind of show the object what
0:36:20	he was doing you know i have to tell you to get it into this
0:36:23	arrangement but they subjected know that of you the utterances were highly scripted the rest
0:36:28	raw completely natural
0:36:29	so after the in trained on one of these words then the director ago okay
0:36:34	it's time for me to get a get up and leave the room
0:36:37	subjects have been told this experiment is about how you follow directions from different people
0:36:41	so they were given the appropriate cover story this was not too weird in that
0:36:45	language game
0:36:46	and so the directory getup income in it either the same person would come back
0:36:50	in or different person would come back again so we had to confederates
0:36:53	so here is our lab manager darren and then the lowest joy hannah a was
0:36:58	also my collaborative which is serving as the second better
0:37:02	so some kind what we have is the same partner using the original expression on
0:37:07	this critical trial the new partner happening to use the same expression
0:37:12	then you partner happening to use the new expression which we are they were there
0:37:16	during the interim thing
0:37:17	or the original part or index what we breaking a conceptual pact okay
0:37:22	and so i might be interested time i won't up ladies but what you would
0:37:26	see is this one is much lower world just play it quickly
0:37:31	in the next one
0:37:32	to reach into the frame and follow the instructions to look like this comes out
0:37:36	kind of low i think sound
0:37:39	so
0:37:48	okay so i don't know useful work
0:37:52	still
0:37:57	so
0:37:59	okay but you can imagine so basically what's happening in this one is we're recording
0:38:03	the eye gaze of the subject and
0:38:05	we find that a lot all around the array when they hear a new term
0:38:09	problem and all partner but i doubt when they hear the new term from the
0:38:13	other partner
0:38:14	and so if you look at the time that takes in that one broken conceptual
0:38:18	pacts l
0:38:19	it takes significantly longer okay
0:38:22	well as the price and thus was that the somewhat so fast basically
0:38:25	when the
0:38:27	when the new partner use the new expression if you just looked at a bar
0:38:30	in case hours argument that anything that's new should take longer than something that was
0:38:35	already primed that's all
0:38:37	you would expect this to be a bit higher but it wasn't and it turns
0:38:40	out that we had norm both of these expressions they were equally good for the
0:38:43	object that's probably why that happened
0:38:45	so at three okay
0:38:48	so what kind of language so basically we ought without to we have shown that
0:38:55	you know there is evidence that you take a partners
0:38:58	identity and you're in train with them into account really on
0:39:02	so at three came along and now every young professor works hard on their data
0:39:07	they would rather die don't publish something that wasn't true
0:39:09	now we are all concert the applicability or if you're not should be
0:39:13	and it's really important that you do something that's replicable but i always be here
0:39:17	that while the things like ten to do is experiments are so
0:39:20	complicated in we're that who wants to try to replicate someone's time-consuming complicated where experiment
0:39:25	what i was really delighted when somebody did so this is a three so let
0:39:30	me just say about are
0:39:32	experiment which is act to
0:39:34	that we only had a critical observations for the whole
0:39:38	session for each pair acceptable for each subject and confederates
0:39:42	so basically we had to old expressions by new speakers to hold expressions by all
0:39:47	speakers to new expressions when you speakers
0:39:49	and two new expressions by will be speakers we only had two instances out of
0:39:53	the a critical trials
0:39:55	where the conceptual pact was broken before that the experiment
0:39:58	was taken up by all the entrainment faces because the chip quite awhile for people
0:40:02	to in train on these objects before you wanted to be natural
0:40:05	so a lot so basically what's interesting is in that last case the broken conceptual
0:40:10	pact is in full listed as
0:40:12	so when you're part just something in fullest that is once or twice
0:40:16	it's not a big deal maybe they just their attention wandered or whatever but what
0:40:20	to do what over and over again
0:40:23	are you playing the same language game or not i mean this is a psychology
0:40:26	experiment okay so map useful even in thomas l o with little kids range three
0:40:32	and five
0:40:33	replicated our experiment data you sidetracking but they emulated the design otherwise exactly
0:40:39	they had only
0:40:40	these eight critical trials and only two of them were broken conceptual pacts
0:40:45	okay
0:40:47	and so basically you know there was the experimenter present who told the children to
0:40:51	movies objects around
0:40:54	and then they just videotape the children they could code basically how quickly they were
0:40:58	able to position they were looking there are gays okay
0:41:01	and so basically what you see here is the at a critical trials in those
0:41:06	for different conditions okay and so
0:41:10	here we have the original partner
0:41:13	and here we're the new partner in the darker colour
0:41:17	okay
0:41:18	so what we see here is that i'm when the original partner break the conceptual
0:41:22	pact it takes a long time to process when the new partner
0:41:29	uses a different crafted in a different term that the brakes were prevalent
0:41:32	it's find it is much faster
0:41:34	but you effect really diminishes on the second occurrence you with a three year old
0:41:39	and also to some extent with a five year old
0:41:42	so this suggests that even a little children are exquisitely sensitive to implicitly in dialogue
0:41:48	okay
0:41:49	and you know that i'm charles and i had been sad we couldn't got more
0:41:53	power we're happy with the effect came out but if we had done what we
0:41:56	would have if we could have done anything we would've had like a hundred broken
0:42:00	cpus and we probably wouldn't of gotten or effect
0:42:03	and so in retrospect were very glad and so basically putting people in implicit situations
0:42:09	too often is unwise
0:42:12	so
0:42:14	basically after that act for
0:42:17	is a crime miller a and one problem really adamant crime miller and elbow are
0:42:21	one of use one deal students
0:42:25	tried it more detailed eye tracking experiment they argue that we were not as methodological
0:42:30	e sophisticated as we should abandon our analysis and so what we should have looked
0:42:33	at early in the trial was not just people's first look to the
0:42:37	object
0:42:38	that was the target object but you look around things as well
0:42:42	and they argue that if we don't that we would have found evidence that precedent
0:42:45	or using the old expression regardless of who is that it
0:42:50	what is important early on and then only later did the partners specific part kick
0:42:54	in
0:42:55	so we thought okay will try that they try they able to a speaker specific
0:43:00	effects and found them but only later on okay
0:43:04	and so i joined i came along and we analyze the matching a brown and
0:43:08	data and we actually early in the experiment did not find any effective precedent that
0:43:13	would be the black and blue lines here and the higher the winds the more
0:43:18	likely they are to get to the correct target but this is a noise in
0:43:22	there are no difference between and you please
0:43:26	lines right here
0:43:26	so we didn't find any evidence of the old from its timing people but we
0:43:29	did find this evidence of the broken conceptual pact here is the rise to the
0:43:34	looks at the correct target object when the cp is broken and the other lines
0:43:39	are essentially indistinguishable
0:43:41	right
0:43:42	and so this still supported our conclusion now note that in all the bar experiments
0:43:49	they have these pre-recorded partners going on in the crimea wherein bar experiment they had
0:43:53	a pre-recorded partner we had an interacting partner
0:43:57	and so acts five finally really quickly i dealt are did this with new calling
0:44:02	in the scanner so he's doing this and mpeg study okay very similar designed to
0:44:08	matching in brandon with to live confederates out by the scanner and one person in
0:44:13	scanner and so again he's now is looking for
0:44:17	evidence of mental arising in the theory of my network so called
0:44:21	which consists almost accounts of three different areas one of them's frontal one separately as
0:44:26	the ones that are on right temporal profile bridal region
0:44:31	and so you found no evidence for
0:44:35	mental i think in his experiment basically
0:44:38	but the problem is that's subjects in the scanner experience broken conceptual pacts at times
0:44:45	and they that was twice as many times as they experience maintain or follows it
0:44:50	is conceptual pacts so that's another issue
0:44:52	so basically my taken from this is that
0:44:57	then the language gain you put people into matters accurately dramatically change your results
0:45:03	and so and a cool and i wrote a little position paper on this
0:45:08	and basically
0:45:10	but ways in which confederates are deployed
0:45:13	can make a big difference in the results that you get
0:45:16	and also the ways in which experimenters choose to deploy confederates differs depending on what
0:45:21	they think the essence of dialogue is
0:45:25	okay what they choose to control what features to make explicit what they choose to
0:45:28	let the confederate just run with without instructing them what to do okay
0:45:32	okay right
0:45:34	so i'm gonna take you through the argument pretty quickly here so we use confederates
0:45:40	because we want a conversational partners who show up to the lab you know it's
0:45:45	harder to get two subjects to show up than one subject so if you have
0:45:48	one subject i see some that are not even going i'm my heart results you
0:45:53	if you want if you have four people coming in which i've done that then
0:45:56	that's even worse right and so we really big this research difficult
0:45:59	so that's one thing that people can do to solve that problem it maximizes the
0:46:03	efficiency in your data collection and it gives you a lot of experimental control because
0:46:07	as k bach once noticed people say whatever they want to say should call this
0:46:11	exuberant response thing which is one of my favourite
0:46:13	noun phrases of the whole world and the editors always try to correct if i
0:46:17	corpus in the paper
0:46:18	but it's called exuberant funding
0:46:21	and so if you to the extent that you can control one partners behaviour then
0:46:24	you can
0:46:25	reduce the variance and maybe get more powerful to conclude but the other subject is
0:46:29	doing okay
0:46:30	so maybe basically a lot of dialog experiments involve while deception and that's okay every
0:46:35	experiment
0:46:35	has some deception and that we don't tell you exactly what the hypotheses are before
0:46:39	you're in it okay
0:46:40	so i think there often not as they appear you might be interacting with the
0:46:44	computerised dialogue system
0:46:48	or with the person who provides rulebased responses and sometimes it can be unclear
0:46:53	you can be interacting with over an intercom with another student in the next room
0:46:56	or maybe that's pre-recorded you don't know
0:46:58	if you're not allowed to interact with them
0:47:00	well you can be interacting with another student or with an experimenter and so studies
0:47:04	do these different things depending on what they think dialogue is okay
0:47:09	and so on the questions when might using a confederate really threatened
0:47:14	your conclusions in the dialogue experiment
0:47:18	and again this depends on the purpose and on what you think dialogue is
0:47:23	so if you think dialogue is just like language processing by yourself only more engaging
0:47:28	that's one possibility or you might think it's a set of expect alternating monologues that's
0:47:33	kind of the way
0:47:35	the message model assumes dialogue is in a lot of spoken dialogue systems assume this
0:47:40	where they're just looking
0:47:41	at your move and then my moving and where you're the computer and i user
0:47:44	your movement mine moving your movement mine move
0:47:46	we're just doing these are alternating monologue sometimes
0:47:49	or maybe a little more sophisticated comprehension production about activate one
0:47:55	and
0:47:56	or maybe it's really shaped continuously by the interaction between partners
0:48:00	okay
0:48:02	now in this first one
0:48:04	the mere presence is what makes a partner make dialogue real and engaging if you
0:48:09	think just having someone their the audience is what does it
0:48:13	then this is you can see this is really just social facilitation theory okay
0:48:18	and so basically that having a partner just
0:48:21	after the projection space for the user to produce more natural
0:48:25	dialogue okay
0:48:27	okay
0:48:28	and so that had a long and distinguished history and social psychology ever sensible gram
0:48:31	a nash all of those other experiments
0:48:34	if you think it's alternating monologues again
0:48:36	this is all of you that is widely used in by many people who do
0:48:39	a i research computer science linguists psychologists you don't actually do research on dialogue people
0:48:44	like that
0:48:46	and it may be fine for some purposes right comprehension production about it but once
0:48:51	this is a few popularised by martin pickering and simon garrett in their interactive alignment
0:48:56	model
0:48:57	and basically this is interesting "'cause" it leads to parody meaning the speaker and hearer
0:49:03	using the same representations and acting on them
0:49:06	and that could be what you think of is common ground
0:49:10	but they argue that it's really just to priming they try to explain the whole
0:49:13	thing because of the simple association
0:49:16	and they are also argue on the same
0:49:18	kind of logic that bar in his are we're using that priming really will explain
0:49:23	all of this
0:49:23	so called partner adaptation
0:49:25	okay unless the late repair
0:49:27	and then finally if you basically i could go into the pickering everything which i'm
0:49:32	not going to
0:49:33	really brought about five used in this wonderful picture and the bbn thing which that
0:49:37	fall do the priming
0:49:38	right here we see the
0:49:40	one partner as partner a on the side partner be on the side and you
0:49:44	know my semantics just primes yours somehow through the air and not quite sure how
0:49:48	that happens but
0:49:49	you know and this is highly modular to but
0:49:51	the problem is that if you assume that a and b are carbon copies of
0:49:55	each other's interlocutors we do not that's not the case
0:49:58	my semantic network differs from yours if i hear the word
0:50:02	eunice i think mother because my mother's name is unit and she's going to be
0:50:07	ninety in we were so
0:50:09	and you think something else right you might think will eat on one is the
0:50:13	old telephone operator on t v comedy
0:50:16	whereas you know your mother might be named it'll travel and you know that will
0:50:19	think you have in your network so people are different
0:50:21	partners are not carbon copies of each other
0:50:24	priming is not an explanation for this i are you okay
0:50:28	and so just to get naturalist if it shaped by the correlation between partners
0:50:35	then this is a different you okay and you might decide to use partners differently
0:50:40	if you believe that likes you think confederates differently if you believe that
0:50:44	so these general concerns that you have in place when you use a confederate that's
0:50:50	you know basically a confederates can be biased if they
0:50:57	it is well let me just of overview of the concerns right now that an
0:51:01	and i talked about in our paper there's the bias confederate the covert confederate done
0:51:06	in secret the know what all confederate who knows too much about the experiment in
0:51:11	terms of the task that they're doing at that moment
0:51:14	as opposed to the first one who knows about the hypotheses
0:51:17	and the script a confederate
0:51:19	user for concerns that we go over
0:51:23	so basically ideally to deal with the bias confederate ideally you're confederate should be blinded
0:51:29	the experimental hypotheses and to the conditions
0:51:31	that can always be the case that would be ideal
0:51:34	and alternatively you can you can script the confederate behaviour in a few critical places
0:51:39	and not in other places
0:51:41	with the culvert confederate on this is we never use this in my lab we
0:51:45	never fool people into thinking that this is a real subject
0:51:48	other experiments that use confederates i'm this
0:51:51	vary dramatically stage managed thing where the confederate pretends to arrive late of a stress
0:51:56	pretends to be a subject need extra instruction "'cause" they're clueless so there
0:52:01	they're trying to kind of pretend should be not a confederate
0:52:05	but during the experiment itself they just behave however they are usually not given instructions
0:52:09	for how to behave and so that role is sometimes concealed a great length but
0:52:14	then neglected
0:52:17	see
0:52:19	so i want to just say these are examples of two different studies one problem
0:52:24	though as a slap in one that from a hannah and townhouses lab where they
0:52:29	basically deal with these concerns very differently so
0:52:34	with the experiment on the left which found no evidence for audience design or partner
0:52:39	specific processing and concluded that language comprehension is egocentric okay versus the one of the
0:52:45	right found audience to find that language comprehension takes the partners knowledge into account they
0:52:50	don't with these concerns very differently so on the right
0:52:53	the confederate was blind to the condition they did not have hidden knowledge during the
0:52:58	task okay and they were told of subjects were told that the confederate with someone
0:53:04	from the lab it was night the you know
0:53:06	i was gonna play this game along with them okay
0:53:10	but they didn't hide the status of the confederate and it's really the opposite on
0:53:13	the side interplay how this stage det and so basically those found to very different
0:53:18	results okay
0:53:20	with these other concerns
0:53:24	basically
0:53:26	an overall confederate this is when someone knowledge doesn't match on what they're supposed to
0:53:31	be so if you're confederates than sitting there as a listener forty times in the
0:53:36	experiment and knows the story that the value subject is telling them better than the
0:53:39	subject as
0:53:40	their feedback is going to indicate that unless there exists an extremely good actor
0:53:45	the problem is that when you're using a confederate as a speaker in experiment you
0:53:49	and script that if you want and
0:53:51	we know what speaking involves for the most part most don't know very much about
0:53:55	what listening involves or no formal models of what backchannels people given any given moment
0:54:00	really
0:54:00	and so
0:54:02	what are the experimenters are more likely to let that one run wild and so
0:54:05	therefore if the confederate addressee has too much knowledge table display it to the subject
0:54:10	and that's problem
0:54:12	so that try to speed up a bit five one
0:54:16	allowed time for questions but we'll see about actually happens
0:54:21	so it is important for the addressee not to have too much knowledge about the
0:54:25	experiment
0:54:26	and then finally i i'm gonna skip over the scripted one okay but if you
0:54:30	want to take a look at
0:54:31	are examples for
0:54:33	how different
0:54:35	experiments come up with different results depending on how the can better it is deployed
0:54:40	you can take a look at the paper
0:54:41	okay
0:54:42	so it turns out that even it and addressee is distracted
0:54:49	and
0:54:50	the speaker will tell a story differently depending on if the addressee shows that they're
0:54:55	distracted or not
0:54:56	and but interacts with that is it the speaker expects the addressee to be distracted
0:55:00	that also interacts with what the addressee does so
0:55:04	i'm gonna skip over this pretty quickly
0:55:07	but in s as study with this kind of design that an equal in an
0:55:10	idea and that's no longer in private banal for yourself skip that basically if a
0:55:15	speaker extracts an address the user is a tender then they get one that's good
0:55:19	if they expect an addressee whose attended but they get one who's counting the number
0:55:24	and in everything they say and pressing a button to the chair secretly whenever they
0:55:28	do that
0:55:28	but the speaker doesn't know exactly why the distracted
0:55:32	then a
0:55:33	or they don't even know that there are distracted they're expecting them to be attentive
0:55:37	then they have that interesting condition then if you tell the speaker the address is
0:55:41	going to be doing some secret task
0:55:43	here the are they're getting an attentive addressee but here they're not know getting what
0:55:48	they expect
0:55:48	so you get different results depending on all of these different cells
0:55:53	okay so it's not only the feedback that matters but the expectation that the speaker
0:55:57	and use the experiment with
0:55:59	so i'm gonna jump ahead
0:56:02	two or recommendations right confederates no i guess i've already covered most of those basically
0:56:08	try not to have the confederate have
0:56:11	information they shouldn't have at that point in the task and take into account both
0:56:15	what the subject is expecting and what they're actually see
0:56:24	just two
0:56:26	if you another example of audience design and partner specific processing you know
0:56:33	in things like just your gestures are a little more ambiguous than words people can
0:56:38	project all kinds of things onto a gesture but you would gestures people just your
0:56:42	differently when they're talking to someone who already knows what they're talking about versus when
0:56:46	someone doesn't okay so in a this next study
0:56:52	a lexical it who is now weight are said right now suppose start working with
0:56:56	retail
0:56:57	did this need study
0:57:00	by having people describe roadrunner cartoons she comes from the manual average in chicago and
0:57:05	so she is all about gesture
0:57:07	and so you have people
0:57:10	watching these roadrunner cartoons and describing the either telling it telling the story to
0:57:16	a new partner okay and then retelling it to that same new partner or retelling
0:57:21	into a different partner
0:57:22	so you have a preconditions but the two partners were counterbalanced for order
0:57:27	now i don't know if this will play the video
0:57:31	no one okay so basically the idea is that
0:57:36	this person i telling this to
0:57:39	a new partner versus and all partner
0:57:42	basically
0:57:43	the gesture space that she uses is much smaller the second time around is used
0:57:48	in this kind of diminishing of information that should provide the partners who already have
0:57:53	the information
0:57:53	whereas when there's a new partner it goes right back up again and the gestures
0:57:57	are large and
0:57:58	demonstrative since you give that the speaker the right cover story so this isn't a
0:58:02	weird language game for them
0:58:03	okay
0:58:05	and so
0:58:07	let's see
0:58:08	i'm going to jump ahead and since the videos are working i'm going to
0:58:13	jump ahead a little bit
0:58:16	and just say that computationally again you can either model adaptation as a slowly inferential
0:58:23	process or an immediate nimble process that if it's activated in memory and you don't
0:58:29	have to make inference "'cause" you've already made it
0:58:31	thank you can use it just like any other information in memory it's not modular
0:58:35	you not stuck
0:58:37	with using partner specific information like
0:58:40	in these situations here
0:58:42	what it was i looked at the numbers of our experiments that had shown clear
0:58:46	evidence for partner specific processing sometimes very early in the interaction and they were all
0:58:51	simple situations that didn't have any we were done natural
0:58:55	recordkeeping that a naive subject would have to do but things are very perceptually clear
0:59:00	like does my partner speak english or not does my partners speech this particular dialect
0:59:05	or not is the partner looking at the thing we're talking about or not and
0:59:09	when you have that simple and a binary situation
0:59:13	then you can think of this is very simple partner model so obviously that's a
0:59:16	lower part about it is computationally expensive to keep track of you your i p
0:59:21	a can do all the keeping track at once because it has all in a
0:59:24	list computing power but with a human if it simple enough they to can keep
0:59:29	track of the information show partner specific processing
0:59:33	so if you just stop acknowledge that you know these situations are quite different and
0:59:39	then and i'll be aware of when humans can keep track of partner specific information
0:59:44	then that could provide insight into what you wanted to as if you don't always
0:59:48	want to emulate computers sometimes you can do it better but
0:59:51	may want to take that into account
0:59:53	and then very much take into account what language game you're playing
0:59:57	so i'm getting near the end i know i'm running a little bit late but
1:00:01	just to wind up i wanna just say that i agree with some of the
1:00:05	discussion yesterday that we are only at the tip of the iceberg
1:00:08	concerning our understanding of the pragmatics around dialogue
1:00:13	but it still really important to better understand have system should perform the role of
1:00:17	dialogue partner and how best they should adapt to a human dialogue partner and i
1:00:24	have some concern about using the wonderful successful applications we already have like calendar management
1:00:30	information access
1:00:31	and try to use that to project ahead everything because in other socially interesting complex
1:00:37	pragmatic situation there's a lot more going on and that is
1:00:40	many of us to comments from the audience nuclear
1:00:43	but i just wanna amplitude very short clips from the internet i just put on
1:00:48	this morning
1:00:49	because i think there are relevant when we think about what a conversational partner really
1:00:53	is okay so first of all i call this the chance to garner effect and
1:00:58	basically i think people using these an intelligent personal assistant
1:01:02	they're projecting a lot of relevance and sensibility and things that are not so sensible
1:01:06	when there's ambiguity people do their best to make it what they think something sensible
1:01:11	would be and so there's a movie if you back away used back with peter
1:01:15	sellers playing this
1:01:16	so one type of character
1:01:18	and the it's described in the clip as a simple sheltered are near becomes an
1:01:22	unlikely trusted adviser to a powerful business man and fighter in washington politics okay
1:01:30	and so what i wanna do quickly as just
1:01:33	if you that this will cooperate
1:01:36	see
1:01:42	if i can make full screen
1:01:46	okay people are seen this movie
1:01:48	being there okay so you youngsters have not okay good score
1:01:52	alright so here we have chancy gardner walking with the l important items the a
1:01:57	trusted advisers the present
1:01:59	who eventually chancy gets promoted to being the trusted adviser suppressed
1:02:03	we want a which present it is but you kernel have your own fears okay
1:02:11	then going to know this is done by a single between is television was to
1:02:18	present them
1:02:19	you are much smaller
1:02:22	but i guess what
1:02:26	alright so basically you know this is somebody who really is very simple but everything
1:02:32	used as is taken as a
1:02:34	and word is the rooms do not suffer
1:02:39	well as well known we will
1:02:44	and we got a unicorn
1:02:57	i know and the related to brazil would be to
1:03:02	okay so that's that so basically you know things people will try their best to
1:03:07	make sense out of whatever they're experiencing okay and
1:03:13	we just get five two
1:03:21	okay
1:03:23	so basically even though we try to make sense of the main message that we're
1:03:28	hearing even when we have evidence so the contrary or ambiguous
1:03:32	we are really exquisitely sensitive when the non-verbal signals are wrong okay we may not
1:03:38	even be aware of what it is where
1:03:40	reacting to okay
1:03:41	and so if you don't take that into work then you might as well just
1:03:45	be on a date with contractually still some of you remember this clip from a
1:03:49	you they years back any one thing the scope for
1:03:53	okay about again less than half of you so you know what enter actually assistant
1:03:57	really highly successful dialogue system from years ago that
1:04:01	from what you may even have been involved with i'm not sure
1:04:04	and i just wanna play that could really quickly and then we'll
1:04:10	see
1:04:16	alright so make it big
1:04:27	right here it can tell
1:04:30	okay
1:04:33	here
1:04:49	of course i
1:05:07	i
1:05:11	i
1:05:31	all right next to the map to the lack there which i
1:05:38	i think there is no shot i
1:05:42	since i last i that's why i had to i
1:05:50	i just wanted to the two
1:05:56	it's a much
1:05:59	so we got them i want to know i
1:06:07	sure that i
1:06:13	and that of that
1:06:14	okay
1:06:17	so
1:06:18	back to the
1:06:19	and doing
1:06:21	to be of course faster i were able to that things properly
1:06:30	so the point of all that is that there are these little implicit right
1:06:34	do you
1:06:36	verbal and nonverbal collateral signal that's her pocket call them on
1:06:40	to which people or implicitly and exquisitely sensitive and when you get is wrong
1:06:45	it really shifts you into a different language game and of course the funny part
1:06:49	of this is that you doesn't get that he's
1:06:51	he's believing her but we all get at that point
1:06:54	so basically the language game in any experiment in any kind of application varies quite
1:06:59	a bit into assume that it doesn't matter is to really miss out on this
1:07:04	i think it's something really important to take into account when you're designing a personal
1:07:08	assistant but switching applications the something that has very different pragmatics okay
1:07:13	and so
1:07:15	it's better to acknowledge what you're assumptions are what you really think dialogue is and
1:07:19	how you've constrain the language game and what you've sacrificed basically and so basically
1:07:26	language processing in general with humans is extremely flexible
1:07:30	and yet it's extremely important you get these its right and i think there's a
1:07:34	lot more to learn and
1:07:36	i thank you for your being a wonderful audiences making audience design used easier by
1:07:42	didn't have so much too much material to present
1:07:45	and i just want to thank my collaborators and my home institution and stuff thank
1:07:49	you
1:07:59	sorry to run like yes when
1:08:51	we could hear you
1:08:52	okay
1:08:54	well try to summarise
1:08:55	right
1:09:02	right so the questions about crowdsourcing and whether you can just lived responses out of
1:09:07	a crowd and stick them in your application
1:09:10	which i love the idea of crowdsourcing many things right
1:09:16	right exactly right
1:09:23	right so the point with entrainment is it your restricting the particular packed the content
1:09:28	of the particular perspective that you take in that indexed by this lexical item that
1:09:31	using
1:09:32	so basically if you have the domain where that's not important where the domain is
1:09:37	choppy and people are referring over and over to the same thing then crowdsourcing might
1:09:42	well work
1:09:42	if you gotta domain where you in your i p a have had some preliminary
1:09:46	discourse about something and you're agreed on calling something a term which is also in
1:09:51	big you know it would be ambitious to have a spoken dialogue system that can
1:09:54	train i would love that
1:09:56	but most the time people end up adopting the terms of their computers use of
1:10:00	their systems use "'cause" they have no other choice
1:10:02	but if there were such a thing that we're flexible then this would be a
1:10:06	very incoherent dialog if it just the dropped in
1:10:09	utterances from a crowd because they wouldn't be lexically constrained in the same way to
1:10:14	indexes joint perspective that we think dialogue is about if you don't think dialogue is
1:10:18	about
1:10:19	working off the joint perspective that you've achieved with a particular individual
1:10:23	then you then crowdsourcing will work if your application fits that assumption
1:10:27	if it doesn't then it won't
1:10:29	i think
1:10:30	thank you for raising that i think that
1:10:32	that really
1:10:33	makes it clear
1:10:35	justine
1:10:40	only to thank you
1:11:29	or fact
1:11:51	right
1:13:06	no i don't think it we have i don't think
1:13:10	we should abdicate that responsibility i think sometimes
1:13:12	you have a different grounding criterion depending on the situation if you're entertain yourself with
1:13:16	theory
1:13:17	then it doesn't matter
1:13:18	and you are fine attributing all kinds of bizarre responses to be in contingent on
1:13:23	your own when they may not be
1:13:24	but when it something important
1:13:27	like referring to an object in you want the right object
1:13:29	then it does become important and so
1:13:33	you have to use the right term or else your partner things you mean something
1:13:36	else even if it's a perfectly good crowdsourced term that many people will like
1:13:41	and so it requires i don't think this is contradictory dog i'm sorry i didn't
1:13:45	summarise your question because it was impossible but
1:13:48	but i think everyone the room probably hurt it i'm not sure that people on
1:13:51	the weapon are converted
1:13:52	but you know the question is you know if you can basically take the partner
1:13:56	into account is just in just said you know in the micro sense moment-by-moment depending
1:14:01	on the feedback in the mapper sense what you want about them i would also
1:14:03	say at the beginning you start with the expectation about the partner we don't a
1:14:07	lot of work with that
1:14:08	and so all of those things it becomes very powerful the evidence you get moment-by-moment
1:14:11	you revise the initial model of the partner and by model can be a labrador
1:14:15	simple depending on how computationally expensive you wanna get
1:14:18	and then at the end you have some information long term memory that you take
1:14:22	with you in the next time you talk to them
1:14:24	then some of that gets downloaded right
1:14:26	so it takes a little bit to download apple once it's in working memory it's
1:14:29	fast rate
1:14:30	and so i think i don't think these things are contradictory tall sometimes meeting matters
1:14:35	and needs to be achieved painfully and other times it doesn't and it depends on
1:14:40	the joint purpose the two people presume in a conversation that's not always the same
1:14:44	purpose but it often is
1:14:48	any
1:14:49	are we done
1:14:51	have to stop i think you are

More than meets the ear: Processes that shape dialogue

Keynotes

Susan Brennan (Stony Brook University)