Speech Transcript - Jack Godfrey and the Art of Evaluation and Benchmarking

0:00:02	good morning this is joe campbell from mit lincoln laboratory
0:00:06	and i'm joining by tell me canyon in from university of eastern finland
0:00:12	were here this morning with a set of distinguished panelists
0:00:18	to honour and represent and reflect on our dear friend i jack godfrey who passed
0:00:26	away last one
0:00:29	we have a very full agenda today so
0:00:34	i think what will i wanna just remind to remind you of a couple things
0:00:41	there were manning many people wanted to enjoying the panel but on the virtual stage
0:00:48	door only room for six
0:00:50	so if you could send messages in the chat
0:00:55	we will be happy to share them with the godfrey family as a memento from
0:01:01	this event
0:01:03	now let me handed over to tell me
0:01:07	totally that's a moment of silence
0:01:11	okay so probable system also might be so we will go moment was on as
0:01:18	norm so
0:01:20	based on
0:01:55	okay the criminals
0:01:59	so i think you know angel
0:02:02	thank you if i can have the next slide please
0:02:09	so
0:02:11	doctor john j godfrey led a very distinguished career
0:02:16	main so many friend friends along the way
0:02:19	and is a pillar of our community
0:02:23	he is largely the reason why this workshop exist
0:02:28	through is
0:02:29	efforts in leading corpus space research
0:02:34	is you know phd from georgetown in linguistics
0:02:39	his early career work focused on psycholinguistics at wright patterson air force base that the
0:02:46	air force research laboratory
0:02:48	and the university of texas at dallas is collier centre that some of the earliest
0:02:55	papers in human speaker recognition where published by jack in those days
0:03:03	he was a contributor to the revolutionary speak and spell troika texas instruments
0:03:11	this was
0:03:13	the first that i know of spelling computer for children very effective
0:03:19	is served as the first executive director of the linguistic data consortium
0:03:25	and
0:03:26	this enabled him to drive a whole corpus based already community
0:03:33	back any all day is
0:03:36	people will publish results and whatever datasets they would collect
0:03:41	usually ad hoc
0:03:44	without rigour in it was impossible for people to compare performance across systems
0:03:51	so this was a big contribution
0:03:54	as the chief of the h l t group it and the say you provided
0:03:59	national direction in litres and brought together this international community promoting an evaluation culture as
0:04:07	whereabout to see in a few minutes
0:04:10	is a dedicated has been father friend mentor
0:04:14	and loves to fish and hold great stories we learn a lesson
0:04:20	i just have another minute or so the
0:04:26	but what i'll do you now is let tony at a few remarks as the
0:04:31	is gonna s p l c signature
0:04:35	thank you so his or her or something the special interest group of a speaker
0:04:40	role recognition community and so we are going to some this is the remainder of
0:04:47	this on is a common is able to so are you over to work well
0:04:53	known
0:04:54	really wanting to introduce bills
0:04:58	with control
0:05:01	thank you tell me
0:05:04	i'll just quickly say the that's jack with his wife nina the aim at wright
0:05:10	patterson air force base
0:05:12	he loved to fish
0:05:15	that was is real big thing and then where we've got a huge smile there
0:05:20	is a holding my bell i'm two hundred centimetres tall so he was upon is
0:05:27	tells for that great picture
0:05:31	now i'd like to introduce a movie where jack will be able to tell us
0:05:36	in its own words
0:05:38	the importance of evaluation and the scientific method and its quest for understanding
0:05:46	this is courtesy of the johns hopkins university c lsp animal lecture he gave in
0:05:53	two thousand twelve
0:05:56	but i have the video please
0:06:32	i
0:06:35	thus
0:06:40	she
0:06:47	it is
0:06:49	you
0:06:54	i
0:06:57	sorry
0:07:03	i
0:07:04	i
0:07:07	i
0:07:16	i
0:07:23	there
0:07:37	right
0:07:39	we
0:07:46	i
0:07:47	see
0:07:51	really
0:08:01	so
0:08:03	my
0:08:07	i
0:08:21	my
0:08:27	i
0:08:40	i
0:08:43	it is
0:08:46	once i models
0:08:50	i
0:08:52	right
0:08:58	i
0:09:02	it is
0:09:19	this is
0:09:22	s
0:09:28	s
0:09:39	three
0:09:43	so
0:09:46	like
0:09:50	so
0:09:52	so
0:09:57	e
0:10:00	e
0:10:09	so
0:10:22	for so
0:10:36	i
0:10:40	well
0:10:53	so
0:11:04	here
0:11:06	okay
0:11:13	so problem
0:11:23	landline
0:11:29	yes
0:11:34	so
0:11:39	so
0:12:02	s
0:12:05	so
0:12:17	although
0:12:19	i
0:12:20	no
0:12:26	e
0:12:34	right
0:12:35	one
0:13:15	so it is
0:13:17	right
0:13:27	so
0:13:36	why
0:13:40	right
0:13:43	okay
0:13:46	or
0:13:48	i
0:13:51	so
0:14:02	i
0:14:04	one
0:14:15	use
0:14:22	or
0:14:27	say
0:14:31	this
0:14:38	also
0:14:43	these
0:14:47	e
0:14:49	that was
0:14:51	then
0:14:52	i
0:14:57	or
0:15:06	right
0:15:12	so
0:15:33	so
0:15:56	so
0:16:06	yes
0:16:07	thus
0:16:15	yes
0:16:27	more than
0:16:31	right
0:16:35	e
0:16:44	i
0:16:47	okay
0:16:52	why the
0:16:56	the only ones
0:17:01	yes
0:17:27	seven video really captures the essence of jack godfrey is methods in driving an international
0:17:36	research community
0:17:38	and
0:17:39	finding results there are surprises and wanting to understand why
0:17:45	and this was a key the johns six jack godfrey six s
0:17:50	and largely to our community six s
0:17:54	those evaluation curves over the years
0:17:58	showing performance the problem getting harder making performance gains again changing the problem to make
0:18:07	an even more realistic was this of virtuous cycle
0:18:11	the jack brought to the government and in the us
0:18:17	and also internationally and help the number of government partners and research t means
0:18:24	there was instrumental in the advancement of or feel
0:18:29	the things that have been made by the community here at odyssey over the last
0:18:34	two decades have been remarkable and i congratulate you offer that and we can think
0:18:40	jack for supporting of the datasets to drive all this and is this insistence and
0:18:46	sign
0:18:49	now will transition into the upper panel session
0:18:54	we have sticks distinguished colleagues in dear friends joining us
0:19:02	and the first one is craig greenberg
0:19:06	he was actually in that video with jack that you just a the whole video
0:19:11	is over an hour but we didn't have time to play the whole thing
0:19:15	so let me handed over to craig take the of a few minutes four minutes
0:19:22	worth the of remarks or show a slight evaluations
0:19:26	great
0:19:30	and it was a degree privilege and you're to get to know and we're project
0:19:35	jack was instrumental in having various human language technology evaluations of news
0:19:40	including of automatic speech recognition spoken dialog systems
0:19:45	and of course of speaker and language recognition evaluations the median and ninety six and
0:19:50	can do you want to their
0:19:52	definitely involved in the nist evaluations offering his vision an insight sometimes aligner french
0:19:59	and all along with a joke in a kind word
0:20:02	and resulted in two thousand and i started working the nist on the speaker and
0:20:07	language recognition evaluations which we typically referred to as sre and lre
0:20:12	actually all the direction of all aspects of the sre use another use
0:20:17	in the beginning evaluation training data collection is all the weighted including workshop
0:20:23	there is a leadership dedication
0:20:25	generate householder real of the sre series and the first evaluation with ten or so
0:20:30	systems and around four thousand trials
0:20:33	into a series that commonly includes hundreds of systems could be millions of miles
0:20:38	the other easy reason to and similar groups that are is all sources
0:20:42	actually
0:20:43	i understand the timit corpus the jet worked on was originally set around one male
0:20:48	and one t
0:20:49	before he retired we began running evaluations entirely online
0:20:54	surely this tremendous contributions and it was remarkable progress of the evaluations and the technology
0:20:59	and so
0:21:01	part of whitney objects contribution was brilliance as a linguist
0:21:06	just to give a sense is
0:21:07	sometimes i feel f voicemail he'd come in here office and right on your whiteboard
0:21:12	the phonetic transcription of your message
0:21:14	where we used that's what i was told by some of those of her colleagues
0:21:18	personally i really think of and all that if you didn't answer
0:21:21	i didn't that didn't get to see the sort of thing lyman action
0:21:26	they're in one been you notice my use of the non-lexical conversation also
0:21:32	the sound was a
0:21:34	we just found that using able to correctly infer that i was the simpsons fan
0:21:39	furthermore
0:21:40	i'll i'm not sure of me is that only we also really like the simpsons
0:21:46	i'm so grateful that in addition to working with great i got a known person
0:21:50	we don't think what we were years after evaluation maybe millions and at conferences
0:21:55	where of course we talk about language and language technology
0:21:59	but also politics history philosophy economics education
0:22:03	just to name a few topics about which is usually knowledgeable
0:22:07	another so much
0:22:10	you reminded me not only professional managers but also personal items that will continue to
0:22:14	carry with me all my
0:22:16	and so like to take this opportunity to take two injectors mentor shipping guidance as
0:22:22	well as for is tremendous contributions the human language technologies
0:22:25	and to the nist evaluations
0:22:28	after such monumental where
0:22:29	i don't we all agree that he rest in peace
0:22:32	i'm
0:22:36	thank you craig that was very nice
0:22:41	next we have douglas reynolds
0:22:44	who
0:22:46	is an assignment in the us government and the is in my
0:22:54	don't can i join us in person but he kindly sent this video the we
0:23:00	will play now
0:23:08	recording could happen or weekly where p e
0:23:12	three would like to select keywords and i'm your friend happy jack godfrey
0:23:18	but my earliest recollections of jack
0:23:20	with the precursor to this whole odyssey workshop series ninety nine core margin use what's
0:23:25	one
0:23:27	is relatively not you researcher having some of the jack stature come to my coaster
0:23:32	listen to my work asking questions make me feel that what i was doing with
0:23:37	the most important thing but the major motivator might really career
0:23:41	but jack kind of way of doing that with whomever speaking with
0:23:44	you matrix will since they were the most important thing but is warm apple nature
0:23:51	one of jack's
0:23:52	legacies to the speech community is being a high here in strong component of common
0:23:57	corpus evaluation driven research and development
0:24:01	in today's your numbers benchmark datasets and challenges
0:24:06	it's harder to imagine a time when even small data sets were expensive and harder
0:24:11	to manipulate
0:24:13	and there were numerous speakers were published and cried datasets using a tricks
0:24:19	and tracking enough everybody had about a hundred percent accuracy
0:24:24	this around the mid nineteen edus the darpa was one of the first organisations to
0:24:28	really push common corpus evaluations so the technical progress could be meaningfully measure
0:24:35	jack along with this texas instrument colleagues which is george doddington you can strong advocates
0:24:40	of the value of common corpora evaluations
0:24:44	jack was instrumental in recreation in collection of the first conversational telephone corpus
0:24:50	call switchboard
0:24:52	but in a standard is pretty small twenty five hundred conversations with five hundred speakers
0:24:56	but the time it was one of the data is that really started should drive
0:25:01	the fundamentals in speech recognition and speaker recognition
0:25:09	jack and became the first executive director and linguistic data consortium
0:25:14	where you continued his dry to have a common corpora for human language technology tasks
0:25:20	in expanded their catalogue with numerous amounts of
0:25:24	novel weighting the collecting speech corpora including text into that mix and other types of
0:25:30	data sets
0:25:32	after ldc
0:25:34	jack went on to become the each individual t research the dod
0:25:39	here you continue to abdicate can drive you called virtuous circle research development that allowed
0:25:46	him to leverage the worldwide research community
0:25:50	but each antony open speaker and language recognition evaluations run by nist
0:25:55	these evaluations have been
0:25:58	the driver from much of the core technology that we'll be hearing about obviously this
0:26:02	week
0:26:04	jack was always a fierce contractor the people in his great wherever you worked and
0:26:09	not much one to deal with bureaucracy very well
0:26:12	in fact idiot is good at here is always the skunk at the garden already
0:26:15	saying we need to be standard and not trying to put on the niceties pursuit
0:26:19	circuitry
0:26:22	in addition
0:26:24	jack's professional accomplishments he was also warm
0:26:29	affable charming mare
0:26:31	and i wonder person around
0:26:35	in the problem family man friend mentor in a heck of a storyteller
0:26:41	is it your soul this greatly by discriminately and based family
0:26:46	thank you joe
0:26:52	therefore
0:26:53	thank you do that was really nice integrity is one of the words the
0:27:01	there really stands out
0:27:05	so yes i want to introduce john for us all born this
0:27:11	to speak about jack's work mostly in their intersection in the world of four and
0:27:19	six
0:27:20	and then all hand over the introduction of the following speakers to tell me
0:27:26	john first law
0:27:29	and stroll
0:27:31	good use and my slide or died we
0:27:36	if you can do it that would be great
0:27:39	in less they have them at the odyssey team
0:27:45	there's actually go have your slides
0:27:49	you go is it possible
0:27:51	okay sure
0:27:54	you
0:28:00	well
0:28:07	we next
0:28:19	well i i'm very are known to be here and to say some walls of
0:28:23	the
0:28:24	but to dye my story reading
0:28:28	be began to be in a nineteen and ninety eight even if i saw him
0:28:32	in matching you for years a the before
0:28:36	nineteen eighty eight that was my first time in the usa
0:28:39	my first time in the nist
0:28:42	it's very
0:28:44	the first five actually
0:28:46	trying to do speaker verification and more exactly the first time i
0:28:52	the idea of the evaluation with this war
0:28:58	and from my own to go to public
0:29:02	a common the vocal
0:29:04	and
0:29:05	to be able to
0:29:07	and the wheel beginning my first the discussion we draw
0:29:13	where was
0:29:15	very successful for you to my
0:29:18	english ability to two hundred and speakers it possible to and it can also but
0:29:27	also i will the dark object that and
0:29:32	a little bit rate and very great by both
0:29:37	was already a big i in the community and also by be context
0:29:43	it was not at all
0:29:46	easy
0:29:48	the for this for during this first the meeting i
0:29:53	have a strong memories about discussion within a but what means that you identification and
0:29:59	that i and
0:30:01	i
0:30:02	i have in mind if i but i agree on the table problem of via
0:30:07	a cost to
0:30:08	no one runs we can try to say what is happening you in france at
0:30:14	this moment
0:30:16	and a duck
0:30:18	this then to us
0:30:20	not only but refundable
0:30:23	and the and the conclusion of the meeting
0:30:26	and nineteen ninety eight meeting came back and i and say okay maybe we have
0:30:31	more on this removal
0:30:33	currently convert remote quite three you and whatnot
0:30:38	the mainstream or what we walk
0:30:41	next slide please
0:30:45	so
0:30:47	i'm
0:30:48	very happy because after each time i had the problem but we need to use
0:30:53	the to meet and
0:30:55	there are more rules interaction with him
0:31:00	i'm not sure because my initiation improved a lot but stephanie i
0:31:08	i was closer to him and more able to
0:31:12	to discuss the without fear of with e
0:31:15	i remember particularly discussion about
0:31:19	the speaker or reason for forensic an investigative it application of the freedom to be
0:31:27	that is we started with the
0:31:29	do so drawing me many
0:31:32	and also very
0:31:34	no i discussion in two fifteen sixteen when he
0:31:39	came to me after the presentation of my paper with the way the g
0:31:45	the ball really video speaker characteristics and he say go
0:31:49	i would like to continue all are not going to start the phd unto the
0:31:54	able with the
0:31:56	what we things we have now be able to start with the shah really mixing
0:32:00	linguistics aspect and the speaker recognition so
0:32:05	this is a very nice
0:32:07	moment in my brain no
0:32:12	so
0:32:13	i will be of individual to after on the maybe
0:32:19	we can extract the and of my
0:32:24	mostly consist
0:32:26	because when they look at the at the video of jack had to the you
0:32:31	know restore i realised
0:32:34	how much necessary and i
0:32:38	had a huge influence on my career and the on the what
0:32:43	you why you research
0:32:46	so i we just
0:32:50	say i but
0:32:52	for to what i really don't from him what they share with limited to about
0:32:58	some more best and i don't know stuff like a wheelchair would be we will
0:33:03	stand me and we show me for a long time and remind conferences and my
0:33:08	which were
0:33:11	a and i mean you could give to me you okay to say that will
0:33:15	be but
0:33:16	i
0:33:22	okay like to some friends well so well we use the or to general solution
0:33:29	to propose something about perhaps the previous or some of what is an okay mister
0:33:35	thank you can you show lights lies please the
0:33:47	i
0:33:49	i was the organiser of that workshop in nothing is that the dog and of
0:33:55	course of ready for the two and it was my first meeting these with jack
0:34:01	and the
0:34:04	wanted to
0:34:07	to say
0:34:10	what i feel about the influence for his work on the things we've done in
0:34:18	europe so that score to the next slide please
0:34:22	where you will see the next slide we will see the party seep than
0:34:28	and
0:34:30	many people that you know that contributed to these workshop be nodding e
0:34:37	and
0:34:39	what i want to stress is really what is on the next slide
0:34:44	so if you go to the next slide
0:34:49	basically
0:34:51	and the times there was a the caucuses the which is a quality needing only
0:34:56	t on the speech database
0:34:59	and speech assessment
0:35:01	and
0:35:03	that was before this work shall mean id for whatever swearing that jack that
0:35:09	at the same time that me conclude that any he was already
0:35:16	the british in this a knuckle full database
0:35:20	which was the first the only for speech database that was produced
0:35:26	and
0:35:27	i mean in this
0:35:28	this database is a five thousand speech is
0:35:32	that the
0:35:35	say the control the sentences on the
0:35:39	an and the sun the all sorts of items of fifty items i guess all
0:35:46	the telephone so it's it was the first the green the database that inspired the
0:35:56	some of the word i was doing in a in switzerland that the time where
0:36:01	we record the swiss french polyphone
0:36:05	database which was equivalent to the microphone
0:36:09	and another database that we call clearly well she's
0:36:14	for a speaker variability so that we had the about the fifty speeches that were
0:36:21	able to already called several times the of the polyphone at a
0:36:29	so that we had the intra speaker variability as well
0:36:33	in the meantime jack actually are produced also lizzy voice of cross is tiny can
0:36:39	eric which was the at nazareth the database
0:36:45	and dior ideal that is the key lingual effort
0:36:49	was really is the start of the european project called speechdat
0:36:54	which was
0:36:57	the started after that
0:37:00	actually in nineteen ninety five variability
0:37:03	and
0:37:04	which are created the
0:37:06	databases for
0:37:09	most of the european
0:37:12	other languages
0:37:13	and also
0:37:17	i guess and
0:37:20	database for
0:37:21	speech over a that in america so there was a
0:37:28	these so that project
0:37:30	and of course that was
0:37:33	all the work that jack has been has started
0:37:38	with the switchboard was taken over in the
0:37:43	ldc for and down that for recording
0:37:48	but as a date that this is like fisher and mixer and many and that
0:37:54	was also the in pure initial the inspiration for the not data collection which was
0:37:59	the non dairy in than entering polyphone version
0:38:03	so that was really the start of the or the work he has done from
0:38:08	the for the microphone and the voiceover hispanic and eric out was really spread all
0:38:14	over the world
0:38:16	for these collection all the polyphone databases so i think and like to
0:38:22	to recognize or the work that has done in the in pushing for all of
0:38:28	those these the data collection and very good and the price
0:38:33	so that the next slide is not i know there is an ex like but
0:38:37	it's not very
0:38:38	it's and
0:38:39	it's just the picture from the microphone
0:38:43	data collection process but it's not very interesting
0:38:47	so that's that was my conclusion
0:38:50	okay
0:38:51	thank you very much
0:38:53	okay you're a monster
0:38:55	so do you might wanna be a really global or anymore is able to texas
0:39:01	instruments on performance sensible things
0:39:05	list model
0:39:07	you you're for only one session together and part of it
0:39:13	i'm gonna
0:39:16	a couple of other memories for a second one what everyone right contribution
0:39:23	and evaluation dataset to remind right
0:39:29	i wanna from the other side or
0:39:33	formerly on and more recently as well and you know
0:39:40	i have privilege working with amount of money are really far i see your out
0:39:46	there somewhere
0:39:48	and you only other directions are i like
0:39:53	is he's
0:39:58	and what i remember work in different areas of course and
0:40:06	i remember being test the artifact them for right i guess in a straight into
0:40:12	your all incidents down
0:40:14	now the neighbours and
0:40:17	so anyhow the songs and i think many of us have clearly so
0:40:22	finally in which are commonly used and all of us engineering or c
0:40:29	interest in our culture you in line with one
0:40:34	and that of my first experience that
0:40:38	i also had privilege of the in one of the corporate he recorded and that's
0:40:42	not
0:40:45	but what i remember that is the one for restaurant on the unit was and
0:40:51	i think if it is needed in the southwestern style right now
0:40:56	i don't know everything analysis mathematics and so question here
0:41:01	and then we enjoy the moment in many years that about a fantasy effectively now
0:41:12	at sort of early on where i had no and then a more recently proceed
0:41:17	with very or a and
0:41:21	so i'm gonna start using and not getting having means and is the low say
0:41:27	might expect that he was a little bit from the writing it is not someone
0:41:32	who really
0:41:34	and i think it's time in industry and government
0:41:38	influence
0:41:40	i
0:41:43	no the since the disaster the problem is the really i what he'll say is
0:41:50	this is the only thing we have for one
0:41:53	and had a strong interaction with that
0:41:58	and you would say some through the l is the okay for another center agents
0:42:03	easier
0:42:05	i think the prefive a little bit differently fancy expectations about the really wanna make
0:42:12	it anything we system
0:42:14	towards a shot the problem
0:42:16	we had a zero eer from last the listening for us and hang out with
0:42:22	the solver
0:42:24	so
0:42:25	we are very grateful for what you percent has a and b and sort of
0:42:29	driving
0:42:31	evaluations and matrix and is a fast
0:42:35	for the
0:42:38	and then added to retire the on our seamless the list and z and with
0:42:44	higher bic your after the somewhere
0:42:48	and sort of went back is true and channel four and it's each time and
0:42:53	you are zero one myself
0:42:56	i to have them with that but we're so much humour these days we did
0:43:02	what we call this the science thing with a
0:43:05	so that sort of with the poles are comfort obviously start it is true academic
0:43:11	in the data and as a science than that in
0:43:14	you the and you don't these models and for us and
0:43:21	yes i second what everybody saying was it is a realistic in two
0:43:25	where in different
0:43:31	i think you're emotional
0:43:33	so maybe scores in an somehow some and also something remotely sensed on the influence
0:43:42	with two
0:43:47	exactly
0:43:48	and they're in general form
0:43:51	for organising the session
0:43:55	to say that seen me so we started session here
0:44:01	i don't like i was sitting across from jack daniels
0:44:04	a unique words in building a small words of in my understanding the other understanding
0:44:11	of problems in jacket this way you are kind of talking with people
0:44:16	that was very
0:44:18	but it's following but also in a collaborative way never and i really am stay
0:44:24	on the you know is kind of understood that this was kind of a room
0:44:28	and a collaborative effort to train two million four
0:44:32	so i'm just a few words in step two gently for students in for the
0:44:38	academic field i think time i pointed out he's done in the nn exploration move
0:44:45	into the government industry in i think he's perspective on the academic field
0:44:52	where is kind of somehow influenced after going to industry into and into the government
0:44:57	no set yours a so
0:45:00	you know no it just started to exactly mimicry of georgetown is usually inverse text
0:45:06	a to do a postdoctoral position of the you are still for star airspace medical
0:45:13	research lab and that actually i think about a lot more on physiology aspects so
0:45:19	objects work inside into there's linguistics background
0:45:23	and after that he joins in a year is in texas at dallas here and
0:45:29	extension here's your you g d in slope maybe it'll sciences a most of is
0:45:34	that what was really focused in the carter centre color centre was found nineteen forties
0:45:39	actually by e
0:45:42	are three founders a ti johnson examining bringing this river system for one for johnson
0:45:48	which actually the person is found colours ten and they are just one for is
0:45:53	non speech production him psycholinguistics
0:45:57	i talked many quotes is there a he did we eventually real eh jean of
0:46:02	researchers
0:46:05	including a number of errors and i know just actually used a question medical centre
0:46:11	and the dollar sign an age project that show that just actual ear as essential
0:46:18	has extent to actually what it also is effective utility to discriminate in class prior
0:46:25	phonetic features in speech this is
0:46:28	well it is the m zero is gonna work understanding the science and how the
0:46:33	signs actually motivated maybe somebody technology advances in should feel so it's balance or something
0:46:40	that which really i think that justin but also something good okay and it gave
0:46:44	people there are more the technology side better understanding science
0:46:48	the forensic science to better understand in technology
0:46:52	well as you but jealousy worked there quite a bit with a lot still min
0:46:57	ross perot's ram or it would be
0:47:01	a good move a variety of the that researchers in also supervised belongs to a
0:47:09	joy still which measures usually student actually to follow him to g i would be
0:47:15	shifted or gi
0:47:17	a i thought we were still min it's this week a one of is what
0:47:21	collections is that usually there is to a most systems everybody will be more clinically
0:47:27	oriented towards a phonetic speech science
0:47:31	well to begin well we use asking questions and variance fiery by jack's was the
0:47:39	man is one of your
0:47:41	i when the engine is related you gaussian random scroll celebrating is twenty fifth anniversary
0:47:47	we had set up distinguished lecture
0:47:53	process and be projected the you distinguished lecture for twenty fifth anniversary okay
0:48:01	and subject you know being however
0:48:04	having a good sense if you my estimated i need a picture you chair
0:48:09	and in suggest that well you know the only actually that could you know really
0:48:12	in that room and there was a picture of him holding no more the largest
0:48:17	fish i've ever seen any insisted you might and you use this picture you know
0:48:22	for the for the keynote and i
0:48:25	we had with him as i felt you know it's probably something that i was
0:48:29	gonna get five from what might be in my question so i meant that a
0:48:33	mormon and you know you really like to use this picture again that are using
0:48:38	a better than you see here is reached i and
0:48:43	just gave expensive two days with us okay
0:48:47	in october two thousand eleven is connotes on which i think backwards i think we
0:48:54	can buy light of the five minutes you i auditorium the engineerings for we based
0:48:59	on and you met many students in one of things that their students are is
0:49:05	appreciated was
0:49:08	both this knowledge is worse than this way of balancing
0:49:12	i guess so it
0:49:13	say that
0:49:14	just understood forms of workout not
0:49:19	say
0:49:20	something were words and say always
0:49:24	five value or whatever it's a
0:49:27	then it was it is gonna of one of the digits are you had news
0:49:33	meeting with very well one of our are all stock staff members just probably and
0:49:38	we've been with us for many years and i got it should holding this one
0:49:43	sets for
0:49:45	cover mechanisms of speech by german by name alexander graham bell and just and where
0:49:52	you initiated or its science and technology so maybe my common to be jot notes
0:49:58	committed to helping students in the academic community
0:50:02	you know listing and yes reason lr use as well as i cast in interspeech
0:50:07	is gonna talking be collecting gerard injury have also pointing out that you know you
0:50:15	make it make a way to go to those jurors to a one on one
0:50:20	language and you know it's fell
0:50:24	you had his undivided attention your attention
0:50:28	we seven interactions there now see you really understood have to a inspires to actually
0:50:35	motivates to understand that they were great opportunities for them in your rear and
0:50:43	a new helping thing one many students are two very productive right
0:50:50	in one of us to be used as you got can
0:50:53	it plays and gender probably and i had the weather spending for they were generated
0:51:00	by
0:51:02	four hours that numbers i just two minutes for our own and we found this
0:51:08	one
0:51:10	i think
0:51:12	well kitchen that was kind of like the whole mirrors in jack a we do
0:51:17	we show in a look at things from different perspectives and you can see a
0:51:23	lots of comments there but i think doing that the a
0:51:29	pretty much everywhere we went with me
0:51:32	given some historical knowledge of a few you know but i can and if knowledge
0:51:39	of just
0:51:42	everything historical
0:51:44	the motivation for the online specs were just not robust one something jack in every
0:51:50	aspect
0:51:52	so maybe just in closing house data
0:51:55	well if you look at the nist and so is now we competitions it certainly
0:51:59	challenge speech community
0:52:02	really motivating from many researchers and teams and feel
0:52:06	a competitions also offer extensive opportunities for common speech used to just very speech speaker
0:52:14	and language or your wrist can so i think one of the looking at evaluation
0:52:19	and benchmarking
0:52:21	gender effort seems to have also used
0:52:24	challenges and stories now read
0:52:27	really one three inspirational wine for maybe students feel
0:52:32	i'll say he of course need strategic fusion of speech science and technology it is
0:52:39	also inspired motivated in mention countless young researchers to
0:52:44	in the linking enhanced or where you know someone there really just wanted to make
0:52:50	sure where
0:52:52	if you want away from conversation jack one layer a better person your value understanding
0:52:58	in more
0:53:00	suggest a with a little snow you for so many years
0:53:04	and you would be remiss fire mainly are probably weeks
0:53:08	well most importantly all the friends and family if we can contact with
0:53:13	i hear someone okay
0:53:14	what we wanna
0:53:16	it remains for everything you contribute to the fuel and since the site
0:53:23	apparently remotes some it's been really inspiring to are used or is one of the
0:53:28	analysis
0:53:29	something to remotes for one on my husband's to margin
0:53:41	this
0:53:42	so this work we will go over the main point
0:53:51	i don't see people have baseline well you know
0:53:56	and war conditioning
0:53:59	only
0:54:01	so
0:54:12	so as you for your decision for my only a single file and you probably
0:54:17	all whereas if you john godfrey contribution size and sufficiency argument
0:54:25	you can even
0:54:27	his personality was already okay a few
0:54:34	an amazing what you vector and one more community and we all
0:54:40	i've got you can partly by one or more one issue as an are you
0:54:48	gonna
0:54:51	for each iteration fisher
0:54:54	where q our are actually in a more efficiently
0:54:58	especially if you expressed if you have some experience here fishing
0:55:03	you know as a few us what are you last year
0:55:09	are contribution a legacy our fear
0:55:12	well i
0:55:14	wow
0:55:15	it will be naming policy option bass or more
0:55:20	after
0:55:21	younger of three this year
0:55:25	you know go free so that or exactly strong i mean or feel and how
0:55:32	she by one norm asking you about
0:55:36	and what jack on a and
0:55:40	what action
0:55:42	and it would be and he will be greatly missed
0:55:45	well this all one would allow someone okay be one or i mean
0:55:52	i don't we all remember
0:55:54	john asks more well assuming you see the first winter we see john best or
0:56:03	a one-best or
0:56:06	and all that if initial something infinite all five k i don't know that what
0:56:12	really
0:56:13	it won't one question okay you
0:56:21	thank you very much machine appreciate the language too
0:56:27	finally and we would like to invite brian godfrey
0:56:32	to give some words and the half of the godfrey family and what are not
0:56:39	there it is to have brian representing the family today right
0:56:44	based on or something everybody
0:56:49	from a sitting
0:56:51	this to you put this together and fusion the on the in our family accessing
0:56:59	alone
0:57:02	fusion
0:57:08	as my doesn't mind noted and an outstanding memorial on jack was a renaissance man
0:57:15	i and different on about on that were results just randomly
0:57:18	but i guess i can i losing everyone standard but multiple languages
0:57:25	you know have a creation and technology good as we talked about here you talked
0:57:29	about his stand academia industry
0:57:32	all snotty little excursion appliances buildings and outdoors
0:57:39	comprehension literature history economics along the successor
0:57:47	listed is balanced analysis on
0:57:52	unrelated reasons you could be easily areas but sometimes you can be
0:57:57	smart analysis and within the to unit this study
0:58:03	not of everything adjustments is that it
0:58:06	and then sawdust only
0:58:09	outstanding
0:58:10	so
0:58:11	well brothers not master
0:58:13	the background model
0:58:17	as was ordinance no rates on integrity
0:58:20	a generalist really fair and open minded readers
0:58:25	all things it will for reverb discussed
0:58:29	already
0:58:31	thank goodness of my brother sometimes also instantly stations
0:58:37	right along all about that stuff also
0:58:40	he's into an unlimited amount of energy
0:58:43	and the decision was done on doing twirl one something
0:58:49	you data manager originally you don't and others
0:58:53	which embrace from this time
0:58:55	regions and store managers and a right
0:59:00	really and still an intense desire to one and then do things with the knowledge
0:59:07	three task one of the common
0:59:10	it seems i noticed and the nodes are received one or more of the people
0:59:13	posted
0:59:14	was only focus on the mentor
0:59:18	it only once the best comedy the received one i one time student and years
0:59:23	later
0:59:24	note that you don't and then the best teacher
0:59:29	well after reading
0:59:30	no memorials and you're not everybody set here
0:59:34	i realise if we see that in a long history stuff
0:59:38	teaching be a professor so i'll be use the influence you have also values calling
0:59:42	so
0:59:45	yes i was bruno was applied in the real us to really implementing was
0:59:50	in that also struck me
0:59:52	the speakers and politics not all the
0:59:56	so many names of people that i never had the opportunity need in detail about
1:00:01	how incredibly smart people how it was addressed as far
1:00:06	but what they were able to do
1:00:08	and speaking with you several million over the past months replenisher
1:00:15	so no the feelings mutual
1:00:17	most a stalling so
1:00:19	also you know i held everybody them also saw perfect embodiment model professor
1:00:25	no surprise i can be
1:00:27	first renew noisily that's where you know where he is waiters or was that was
1:00:32	is wrong
1:00:35	so now on the side of the challenge
1:00:39	and right you know use an and just you know
1:00:45	it's only many times you were used for the next question else
1:00:49	and since all about what else is a year
1:00:53	unless personally i don't know anybody more
1:00:57	and well i don't know the specifics
1:01:00	i do not exhaustion
1:01:03	i know the speakers
1:01:05	usually always closest friends intolerance it only
1:01:08	what he brought of the field of human language technology will make a little better
1:01:12	place was no less
1:01:13	so
1:01:15	enclosing and not accumulating better
1:01:19	almost close to the minimum orders of that's where in the streets enslavement
1:01:24	just as a simple sentence somewhere for describing then
1:01:27	and the loss is the last time
1:01:30	you know personally and professionally
1:01:33	so again thank you guys so much
1:01:36	for including
1:01:38	myself and my family in this and
1:01:41	really means a lot so
1:01:43	you
1:01:47	thank you very much brighter
1:01:49	that's very touching the here from the family and
1:01:53	your father was an amazing
1:01:57	i'd also like to thank the panelists and the attendees
1:02:02	and the odyssey teen ugh on a
1:02:05	key go and talk a for me for helping us put this together and my
1:02:12	colleague tell me can in for
1:02:17	all the organising their you did together with me
1:02:22	i wanna remind everybody to
1:02:26	at end of the upcoming virtual tore of tokyo that starts at nine fifteen uses
1:02:35	a different link
1:02:36	so be sure to
1:02:40	this came from this and go to the page to see that
1:02:44	thank you that looks really nice and it'll be
1:02:49	there appear after this session i also want to point out the
1:02:56	tell me what's gonna point out the
1:02:58	the other session go ahead tell me
1:03:02	no nor is true
1:03:05	also going to one and also show your work has been really on a normal
1:03:11	to remove organs in this isn't so is reasonably
1:03:15	suppose you are
1:03:17	so
1:03:19	yes so are very well as well as on the eval or someone benchmark a
1:03:25	two minutes or something so on sore everyone so the turks influence on the on
1:03:31	the on benchmarking
1:03:34	so based on this one to one
1:03:37	so this summer
1:03:41	okay maybe this is what channel
1:03:46	yes thank you can only i think
1:03:50	the other thing i wanna dimension is
1:03:53	there is a session and evaluation in benchmarking chaired by omit from nist
1:04:03	the duh directly follows
1:04:06	in the frame of jack's leadership
1:04:11	and guiding the community that's a and the man and them and session so you
1:04:17	can view that at your leisure
1:04:21	do you please joining the tokyo tore and that's all we have affected a i'm
1:04:27	sorry were four minutes overtime
1:04:29	thank you everybody i really appreciated
1:04:33	well
1:04:35	thank you
1:04:38	and smell on anyone

Jack Godfrey and the Art of Evaluation and Benchmarking

Live sessions