Speech Transcript - APPLICATIONS OF SHORT SPACE-TIME FOURIER ANALYSIS IN DIGITAL ACOUSTICS

0:00:13	"'kay" thank you all for coming
0:00:14	um my talks about applications of short space time for analysis
0:00:18	in digital acoustics
0:00:20	john to work uh with mark actually
0:00:24	so what is digital still acoustics
0:00:26	so what we call the field of a dimensional a a signal processing
0:00:31	that involve
0:00:32	a a sampling and reconstructing the weight
0:00:35	uh a typical using
0:00:37	uh a of microphones and well course
0:00:41	so these are race they act
0:00:43	as a a a a D computers
0:00:44	and the egg cover
0:00:46	respect of
0:00:47	so it means that
0:00:49	uh i after sampling between we have
0:00:51	a matrix
0:00:52	of a samples
0:00:54	of course
0:00:55	a temporal and spatial samples
0:00:57	that we can process using
0:00:58	two S P
0:01:00	and my michael
0:01:04	oh
0:01:05	cool
0:01:09	um so for example
0:01:11	if there's a source
0:01:12	um
0:01:13	in the acoustic scene
0:01:15	that we want to get rid of we could apply to the mission filter
0:01:18	and uh we could get rid of this source and have only the one that manners
0:01:23	oh we could for example
0:01:25	and these are to to applications gonna talk about
0:01:27	um
0:01:29	a in coal
0:01:30	the information that is sampled with your array of microphones
0:01:33	and all star it in an i pod for example
0:01:35	so then later we can reconstruct it with
0:01:38	a a for example way feel to
0:01:41	um but sort order this kind of um processing to the mission signal processing we need
0:01:47	to mission signal processing tools
0:01:50	and the first to i want to talk about is the spatial temporal free a transform
0:01:55	and this involves taking the if transform across
0:01:58	you're right X
0:02:00	so
0:02:01	why would you want to do this
0:02:03	well essentially actually
0:02:05	uh the wave equation tells us that
0:02:07	um
0:02:08	if the way feel this harmonic in time
0:02:10	then necessarily certainly it's harmonic in space
0:02:12	so there's a the here and the fourier transform a
0:02:15	a the array access use went to exploit this
0:02:17	how it in space
0:02:20	for example if
0:02:21	if you contrast a bit uh looking at each microphone individually
0:02:25	i this case you have three sources
0:02:27	i it's not very clear
0:02:29	uh where the information from each source is going to appear so
0:02:33	um all this doesn't give you a clear visual impression of what is it "'cause" sixteen
0:02:38	what is if you take the free transform across and
0:02:41	not all these three sources
0:02:43	are nice place
0:02:44	in this uh
0:02:45	a two dimensional frequency domain in one acts you have the regular frequency
0:02:50	um to improve frequency
0:02:52	and any the other acts you have the spatial frequency
0:02:54	or the wave number
0:02:57	and you see that um
0:03:00	the displacement of source uh with respect to to the every X is going to place
0:03:06	um
0:03:07	this lines where all the energies constant
0:03:11	uh so for example we can even complicated bit more if there's a source like can schools or to your
0:03:16	rate
0:03:17	you also see a nice pat an where where these lines open up as a triangle so we're gonna see
0:03:21	more detail
0:03:22	what is mean
0:03:24	um and so for example in this situation
0:03:27	oh where have two sources that matter and you have this
0:03:30	this two in the far field
0:03:32	and you have this near field in the for we wanna
0:03:34	uh i get rid of
0:03:36	uh
0:03:37	it is very clear what we have to do here
0:03:39	uh we the play field they're that to get rid of these components
0:03:43	so in the end
0:03:43	we'll have a the two components that my
0:03:48	so for for doing this kind of uh applications we need to understand
0:03:53	um
0:03:54	one simple scenario which is the the point source in there
0:03:58	so what happens the spectral representation when you have a point source this you
0:04:03	oh a this point sources
0:04:04	i is a driven can by uh a signal S of T
0:04:07	and the angle with respect to X is this off X
0:04:10	i has a minimum angle and i some angle depends on the length of the right
0:04:14	and if you look at to to the mission spectrum
0:04:17	you actually see that
0:04:18	um
0:04:19	the special pattern is uh this trying to region
0:04:22	with a few ripples on the outside
0:04:25	um
0:04:26	so this trying to region
0:04:28	is the limited by the by the cosine sign of the two angles
0:04:32	"'cause" of these two angles
0:04:34	a a a completely define what is the aperture of this trying to reach
0:04:38	and the ripples on the outside
0:04:40	which come from the effect from doing
0:04:42	so the thing function fact
0:04:44	they're oriented toward
0:04:45	the continuous average
0:04:47	uh of the angle of course
0:04:49	the the entire
0:04:51	so not one the the whole mathematical michael expression because the be dense
0:04:56	uh but i'm gonna tell you that's
0:04:58	um this is actually means
0:05:00	um
0:05:01	that you have
0:05:03	is the free transform of the source signal
0:05:06	but multiplied
0:05:07	by a combination of max operation
0:05:10	of the sinc function E fact caused by window
0:05:13	and this trying the region
0:05:15	that depends on the distance
0:05:16	of the source
0:05:18	you're right
0:05:19	and this represents presents are information and this one presents a spatial information so there set
0:05:26	and so using this result how how could we uh for example filter to source
0:05:30	in the way feel
0:05:32	so
0:05:32	we all have to go where the energy
0:05:34	we have to define a filter
0:05:37	that takes
0:05:38	um at this trying to that preserved this trying to the region
0:05:42	a where all the energies contain
0:05:44	and um this all the rest
0:05:47	so this would be like as a simple two dimensional filter design problem
0:05:50	okay so you define what are
0:05:53	defined where of the transition region
0:05:56	um you could be fine if it's linear phase are you can find the
0:06:00	the bandpass ripples and the
0:06:02	um
0:06:03	the best top people's
0:06:06	and then you could use a to design technique
0:06:08	uh i two dimensions
0:06:09	uh to obtain the realizable filter
0:06:12	oh these are a few examples
0:06:14	for example using the win me method or the
0:06:16	part some
0:06:18	um
0:06:20	but that is very easy one you have only one source
0:06:23	so what happens when you have more than one or
0:06:27	don't have as as the following it's
0:06:28	each source
0:06:30	uh each point source
0:06:31	has uh what a called this chat the reason behind it
0:06:35	uh that if if it's going to overlap
0:06:37	with
0:06:37	the shot a region of another source
0:06:40	that here
0:06:41	for sure you gonna have spectral overlapping
0:06:43	okay so this these two trying to regions
0:06:45	um which chris want which source
0:06:48	i want to overlap in just something
0:06:49	and so with menu filtering you cannot for you uh week cover
0:06:53	um
0:06:54	uh each source
0:06:56	the one thing you can do to to read this problem
0:06:59	is we split up there are the array into
0:07:01	um
0:07:02	equal parts
0:07:04	and then we're gonna see that
0:07:05	for example and the left block
0:07:07	the shall a reason is
0:07:08	he's reduce for the two sources
0:07:10	i don't what we're gonna have
0:07:11	much less overlapping
0:07:13	in this in the spectral domain
0:07:14	and the same thing gonna happen for
0:07:17	the block on the right
0:07:20	and so you can do this um you can keep uh dividing
0:07:24	you're a into smaller parts
0:07:26	and you gonna see that's the these trying to reason is gonna get smaller and smaller
0:07:30	row the other hand you wanna have
0:07:32	uh other if effect
0:07:34	are harmful which for example if you reduced to mean the size
0:07:38	you're gonna have a much more sing function effects so there's a balance
0:07:41	um there's like a limit
0:07:43	uh
0:07:44	a limited number of times
0:07:46	that you can divide you can split up to three
0:07:48	i and smaller parts
0:07:50	so you might have
0:07:50	you might have already noticed that
0:07:52	this is
0:07:53	analogous to a short time fourier transform
0:07:55	except you're doing it in
0:07:58	a to do this
0:07:59	um in in this space and time
0:08:02	uh it's essentially you can do it essentially with a a a a dimensional
0:08:05	uh filter bank
0:08:06	oh which implements a a a a a lapped transform
0:08:11	so it's the in this example for you have us
0:08:13	a source and a new field
0:08:15	a be closer to the right near rate
0:08:17	and you see that you have
0:08:18	a a lot more curvature
0:08:20	in the space time representation
0:08:22	you have a lot more curvature in a region that is close to the source
0:08:27	um so of course is the filter bank is but reconstruction than the input is gonna be the same as
0:08:32	the output
0:08:32	and in the middle you see the the actual composition
0:08:36	you have a in one direction you have the the special blocks
0:08:40	and in the vertical axis you have the temporal blocks
0:08:42	if you look close or
0:08:44	to the spatial uh the dimension
0:08:46	you see that
0:08:47	a for each of these blocks
0:08:49	this trying to reason is much more narrow
0:08:52	and it to fall O uh the location of the source
0:08:56	okay
0:08:56	as the as a block was close or two
0:08:59	to word the source
0:09:03	um here's one example this is one issue in the paper
0:09:06	um
0:09:07	it's a it's a it's a nice worst case scenario
0:09:11	oh where and you for source
0:09:13	is cool line with a with a far source of the far for source actually
0:09:17	with respect to the rate is behind the near field source
0:09:20	and you can see the spectra representation that
0:09:23	the far field source completely immersive
0:09:25	um in the spectrum of of the new
0:09:29	so if we do this decomposition
0:09:31	uh let's see one iteration you have
0:09:34	you you start seeing that the the new two source
0:09:37	yeah the sure of the trying to gets reduced
0:09:40	so they start getting separate
0:09:42	and you can do a with a larger when the size
0:09:45	um and then in the end if you want to filter to one of those
0:09:49	um
0:09:51	you apply it to each of the blocks able apply filter
0:09:55	that's isolates
0:09:56	source
0:09:58	so this is a a a result the one we we got um
0:10:02	the best results in the mean sense
0:10:04	was for uh
0:10:05	a window size of thirty two
0:10:07	K and you can see that
0:10:09	in the space time representation
0:10:10	there's a there's a source in the new field
0:10:12	mixed up with one in the far field
0:10:14	and here to get this if you string to get clues
0:10:19	a the second application wanna to talk about this
0:10:22	coding
0:10:23	so how do we code
0:10:24	the wave field in this domain
0:10:26	uh the structure we use
0:10:28	is is very similar to what's
0:10:30	um state of art you call there's to let's say in P three or a C
0:10:35	so you actually take it to my mission filter bank
0:10:37	um
0:10:39	that is the the fourier transform across cross
0:10:41	time and space
0:10:44	and then in this domain
0:10:45	you're going to
0:10:46	quantise code
0:10:48	um
0:10:49	all this uh coefficients
0:10:51	so this is the bit allocation problem
0:10:54	and that for comparing the results
0:10:55	uh we do the inverse uh
0:10:57	feel the the inverse quantization and then
0:11:00	comparing the mean scores
0:11:03	um
0:11:05	and so if you look at our of worst case scenario
0:11:09	uh we see that
0:11:10	so this is this is the plots
0:11:12	uh the rate distortion plot
0:11:14	uh i the mean square sense
0:11:16	you have in this is the the rate we use we used for
0:11:19	encoding the spectrum and the distortion
0:11:21	we get
0:11:22	yeah
0:11:25	a so a first thing you see here
0:11:27	is that
0:11:28	the worst
0:11:28	the worst result to can possibly have
0:11:31	is actually if you code each each channel independently
0:11:35	okay so this would be
0:11:36	this
0:11:37	a a line a on the uh the outer line would be
0:11:40	for a window size of one
0:11:41	so that is equivalent to coding each microphone signal in the pen
0:11:46	um but
0:11:48	the interesting thing here is that
0:11:51	if you look at
0:11:52	if you take the fourier transform across the entire array
0:11:55	you don't actually get the best results
0:11:57	you get an need to results so this would be like
0:12:00	be quite D correlating this
0:12:01	or a signals
0:12:03	wouldn't give you the the best result
0:12:05	you see that the actual best results
0:12:07	comes again for a a a window size
0:12:10	uh of thirty two
0:12:11	okay like like seen before
0:12:14	so just by um
0:12:16	just by changing the window size
0:12:18	um you can get a much clear improvement
0:12:21	uh compared to either called in each one individually or D correlating your a
0:12:26	i
0:12:26	and this point here you C is the um these the operating point of M P three
0:12:31	um if we take into account the the um
0:12:34	the average bit-rate
0:12:36	uh we're not using psychoacoustics your
0:12:38	and so from here to here
0:12:40	you have a a about uh uh seven
0:12:43	a factor of seven of compression
0:12:48	so in conclusion i and still acoustics of the wave fields are discrete eyes and process
0:12:53	as to the mission signal
0:12:55	a short space time free analysis that like presented
0:12:58	um
0:12:59	improves the performance of sound field processing operations
0:13:02	such a spatial filtering coding
0:13:05	and these experiments with the with a case scenario they suggest that
0:13:09	um
0:13:10	have you a window that
0:13:12	uh a complete complete in close as the entire rate
0:13:15	it's not the best result nor is to have uh
0:13:18	only one microphone
0:13:20	is actually about the fourth
0:13:21	of the length of your
0:13:23	that's
0:13:30	we have time for a few questions
0:13:39	oh
0:13:41	i
0:13:45	i
0:13:47	oh
0:14:04	but before
0:14:05	maybe form or
0:14:17	um
0:14:18	what that was a study um
0:14:21	from by i swear
0:14:23	that
0:14:23	yeah you know i how how this a representation
0:14:27	the domain
0:14:28	um
0:14:29	how it changes with sources
0:14:31	a there's are five
0:14:34	uh
0:14:36	and
0:14:36	here
0:14:37	a
0:14:38	trying
0:14:40	you're
0:14:41	oh condition
0:14:42	you can uh
0:14:45	uh
0:14:46	say that
0:14:47	well if the source
0:14:48	yeah
0:14:49	they
0:14:51	i
0:14:52	and you can
0:14:53	i
0:14:54	oh my for example the spectrum
0:14:56	of
0:14:57	a
0:15:01	great
0:15:03	a
0:15:04	right
0:15:05	and
0:15:06	you
0:15:09	you can
0:15:10	um you a much right than
0:15:11	uniform
0:15:13	i
0:15:14	can design
0:15:16	so
0:15:17	it it has an
0:15:23	i
0:15:24	yeah
0:15:26	yeah
0:15:40	i
0:15:41	i
0:15:43	okay that i
0:15:46	i
0:15:50	so nine
0:15:51	the space so that would be if you if you if it trace the vertical profile L you one mike
0:15:56	the signal you in one
0:15:59	that's
0:16:01	i
0:16:07	alright right oh you mean i
0:16:10	i
0:16:11	i
0:16:12	yeah
0:16:13	so if you are a representation here
0:16:16	a four dimensional
0:16:18	i
0:16:20	which which we can that here but
0:16:21	of course there
0:16:28	you mean need to consider it as and the non-separable space
0:16:31	i
0:16:33	right
0:16:34	one know i
0:16:36	you
0:16:38	a
0:16:38	right
0:16:39	or not
0:16:40	and efficient that
0:16:43	oh
0:16:43	right
0:16:44	source
0:16:45	so that makes sense to to use not separable
0:16:51	the
0:16:58	yeah yeah
0:17:00	i'm not not aware of but we work with uh
0:17:03	where i is from which
0:17:12	uh_huh
0:17:19	no we actually a um
0:17:22	we we don't have of the the rate the actual reconstruction
0:17:25	of the wave
0:17:26	uh we just a which is go got to how to process it you know
0:17:34	no we assume that is possible to reconstruct
0:17:37	you know it's just
0:17:39	not or not
0:17:39	i just one in the samples of the old
0:17:41	and and the interpolation space
0:17:43	that's for the which also
0:17:45	and
0:17:45	or a
0:18:08	no where
0:18:08	actually the the purpose here is not to do beamforming is just a present
0:18:12	uh
0:18:13	no know tool that is
0:18:15	visually appealing and it can be used for uh
0:18:18	uh a design
0:18:20	i feel there's
0:18:21	uh
0:18:21	very effective
0:18:22	a what actually trying to compare it we
0:18:24	beamforming algorithms or
0:18:27	uh a source separation of
0:18:29	you know we doesn't yeah there's a lot of analogy
0:18:33	when
0:18:34	the problem
0:18:35	domain main at we it in the spatial domain or
0:18:38	a time space to domain
0:18:44	i just have shot question
0:18:46	in the conclusion about the uh uh a optimal side
0:18:50	of the we of is it's in and dependent
0:18:52	particular i is you P now fact that
0:18:55	in use scenario you yep
0:18:57	new field source and a lot so she just for
0:19:00	i could be funny uh where
0:19:02	are
0:19:03	and the
0:19:05	right
0:19:06	because that's where
0:19:07	i
0:19:08	a problem when
0:19:09	one spectrum
0:19:12	like
0:19:13	i
0:19:13	uh
0:19:17	and you the question
0:19:20	okay so on that
0:19:21	that's that's that is gone

APPLICATIONS OF SHORT SPACE-TIME FOURIER ANALYSIS IN DIGITAL ACOUSTICS

Detection and Estimation

Presented by: Francisco Pinto, Author(s): Francisco Pinto, Martin Vetterli, Ecole Polytechnique Fédérale de Lausanne, Switzerland