Speech Transcript - FAST DAMPED GAUSS-NEWTON ALGORITHM FOR SPARSE AND NONNEGATIVE TENSOR FACTORIZATION

0:00:14	yeah
0:00:15	a risk is switch to the second
0:00:17	present a sent in this in this you in this a we we you present the fast they might those
0:00:22	new with them
0:00:24	yes
0:00:26	and is
0:00:27	this is a but you on what of well with a bit that's caff be and i C okay
0:00:32	and uh
0:00:34	oh
0:00:34	so a
0:00:35	i you already know that the um
0:00:38	this
0:00:38	yeah are another
0:00:40	kind of but with some for at the end and for C be that
0:00:43	the the on a the on S one egg with um
0:00:45	and one as the only one in some these
0:00:48	yeah the dam goes you than it them
0:00:50	but usually for them most new data can you we we on we need to compute the that be beyond
0:00:55	oh the gradient and the hessian and and usually do they has and very very few already last
0:01:00	and
0:01:01	uh it basically that's
0:01:03	we
0:01:04	was up we need to in but very hot
0:01:06	but a at that possibly has an and in in this
0:01:10	in this present is that we we we to do how to
0:01:12	avoid this problem
0:01:14	how to how to avoid
0:01:15	how to
0:01:16	a lot but enough that has an and all so how to a the
0:01:21	the lower the last
0:01:22	in but up last
0:01:23	has yeah
0:01:25	and now is the a life
0:01:26	oh
0:01:27	might talk that we we you
0:01:29	resent deeply side it press for the good
0:01:32	the copy of method
0:01:33	and we will find the
0:01:35	the fast way to compare
0:01:36	to compute looks
0:01:38	the gradient
0:01:39	i also to
0:01:40	compute a process as N
0:01:42	and it here we we
0:01:45	with an
0:01:46	the low rain and that's one for the a possibly has and
0:01:49	and then by on this
0:01:52	but you by on this low when it just when we we fight
0:01:55	we we i
0:01:56	we we we present
0:01:58	a very it's so that need to
0:02:00	to avoid
0:02:01	the input of last has
0:02:05	i skip this require already meant something
0:02:08	for
0:02:08	reverse light
0:02:10	okay
0:02:11	a you know the to derive with that
0:02:12	the the atlas electrics um we can see that the same the similar conversant robot whereabouts roy
0:02:18	in nineteen seventy nine
0:02:20	but here you see here this is a conference on
0:02:22	two we to approximate the ten so by
0:02:25	but extract the
0:02:27	with
0:02:28	this the two
0:02:29	to write really says that um is the
0:02:31	okay this term
0:02:34	is rest and it's right here and this time we present here
0:02:36	this time is to inform the nonnegative trends on the front end
0:02:40	i N
0:02:41	and this time to an problems
0:02:43	spice you can train
0:02:45	and the car far and bit eyes
0:02:48	zero are uh plus they've
0:02:52	and is a see here
0:02:54	to that
0:02:55	the on as one of the lead that we have best
0:02:57	all that on the fact that i the same time
0:03:00	and he'll to be two two we before all this we use this
0:03:05	but that's room
0:03:06	with
0:03:08	with i we've fit i is that that's
0:03:10	okay you C
0:03:12	this the best the is a set up fact the i one
0:03:15	and then you
0:03:16	code for i on the on the but is a seven
0:03:19	to very long for that
0:03:21	but long list that and we employ this
0:03:23	a back room
0:03:25	do you best
0:03:26	the fact
0:03:28	but
0:03:29	you see here is we we need to compute the the copy and here i'm node that has sent the
0:03:34	possibly may has sent here
0:03:35	and also the gradient here
0:03:39	and the
0:03:40	a call we the also we of go we need to need to compute you
0:03:44	the the cool and
0:03:46	from the be and we compute the gradient
0:03:49	for a for in this fall
0:03:50	and also so be that has an approach me has an
0:03:53	for in this fall
0:03:55	and
0:03:56	a probably C come men
0:03:58	i i a high computational cost
0:04:00	and of course the of this but might is side
0:04:03	you one here
0:04:06	and a lot of time and also
0:04:08	yes
0:04:10	and
0:04:11	we we we present here than
0:04:12	the way to avoid this problem
0:04:16	it this line we'll
0:04:18	so that how to
0:04:19	compute the will be and matters
0:04:22	this do though it the bustle
0:04:24	the river a of the approximate ten so
0:04:27	which search but only one come but
0:04:30	and
0:04:31	we saw this this
0:04:34	we be is spread in the very
0:04:36	come come
0:04:37	vol
0:04:39	with
0:04:39	this one is that correct up
0:04:41	correct upper up all the
0:04:44	common a
0:04:45	in in on the fact that it's that the the fact that and
0:04:49	and B and is that the rotation matrix
0:04:52	it better
0:04:53	okay to say here this this breath the relation between
0:04:56	the vector is a stand up
0:04:58	okay so
0:04:59	and vector is a
0:05:00	and vector is a set of the most and
0:05:05	my matrices a send up to time so
0:05:08	quite complicated here
0:05:10	you see if you were to i don't the data and
0:05:13	and this one leader
0:05:14	you where the wide the mode and
0:05:17	methods and of the ten so
0:05:19	and this italy's relation between
0:05:21	two batteries and
0:05:25	and we fine
0:05:26	the compact form for that the the copy be and map so here
0:05:30	and with few we we from this result we
0:05:33	find the weighted method
0:05:38	this one is the uh
0:05:40	the final
0:05:41	result for the gradient vectors
0:05:44	and it with you see here this one is the gradient of the tensor with respect to the fact though
0:05:48	and
0:05:49	the fact that one
0:05:50	and this one is the right
0:05:52	the great and
0:05:53	of the
0:05:54	a cross and tensor
0:05:56	which are but to the fact the and
0:06:00	and the got my mike this is press here the that the how to map product up
0:06:04	on
0:06:05	this time
0:06:06	what is that the fact the and
0:06:08	is that the the something
0:06:11	and of some this this crime matter
0:06:14	at the nest
0:06:15	slide
0:06:16	we we present
0:06:17	how to compute the
0:06:18	approximate
0:06:19	yes
0:06:22	the idea is this we we've split
0:06:25	we can see that the the possibly has and that though
0:06:28	and by and
0:06:30	block might be
0:06:31	and we we fight the plug i that is the press for for every block here
0:06:38	is
0:06:38	this dist
0:06:39	to and you with
0:06:41	how to you
0:06:43	that was sub just
0:06:44	has us a but just
0:06:46	like did you have a you
0:06:48	if and by it
0:06:49	for
0:06:52	say this is them
0:06:54	the diagonal block this
0:06:56	is and equal to M you you have to um the diagonal plot mattress and C
0:07:01	compute by
0:07:02	yeah my
0:07:03	and
0:07:04	i am and go product up a got my
0:07:07	with the and it it matters
0:07:09	a a um of sci ice ice of N
0:07:13	and is it that the diagonal vector
0:07:15	and for that off diagonal my
0:07:17	for the up
0:07:18	diagonal diagonal method
0:07:19	when
0:07:20	and you know in a equal to N
0:07:22	we we we can decompose
0:07:24	this
0:07:24	supp might in into
0:07:26	this four
0:07:27	this it is a diagonal matrix
0:07:29	this the that block diagonal matrix enough for
0:07:31	i i know what it
0:07:32	is it problem
0:07:33	i i matter and is also the block diagonal matter
0:07:36	we be spread here
0:07:38	in the next line
0:07:40	so
0:07:42	a very in i property of that of possibly has an that that we can
0:07:45	it's spread
0:07:46	the approximate has an and that the low end
0:07:49	and that one
0:07:51	you see year
0:07:52	so i say yeah that a i all method
0:07:55	we we already defined though
0:07:57	re slide
0:07:58	and is that the point
0:07:59	but but and method
0:08:01	and
0:08:02	see here here that the right block low matter
0:08:06	or within this fall
0:08:08	i K yeah A the
0:08:10	square matrix
0:08:10	oh side and
0:08:13	and ask way by and ask where
0:08:16	and this well
0:08:17	in the is the mac of the K method here even in this fall
0:08:22	be here
0:08:24	okay
0:08:25	ah
0:08:28	it in S like we you see here how to
0:08:31	a which to like the case and
0:08:32	and how
0:08:34	how we can
0:08:34	and to spend a little way that's one
0:08:36	i just one for the prosody has and yeah
0:08:39	yeah
0:08:39	the the has the a to my has an
0:08:42	and it is a metric
0:08:43	see you the plot like no mapping
0:08:45	see you that the also plot that all method
0:08:48	and case have where
0:08:49	particular
0:08:51	such that
0:08:53	and of car yeah if we employ
0:08:56	deep if we employ the by no
0:08:59	binomial
0:08:59	it but to your we can it
0:09:01	we can but we can of one
0:09:03	the in but a blast scale
0:09:06	a little the last has in method
0:09:08	by this
0:09:09	for
0:09:10	and if we compute here is that a compute the in but uh
0:09:13	the approximate
0:09:14	has and we
0:09:16	all only compute it looked at the you might rate the came at rate and also in lot of this
0:09:20	style
0:09:21	and this time have very
0:09:22	small
0:09:24	the by some because this is this term is
0:09:26	has i'll up and by K and
0:09:29	and K
0:09:31	and a
0:09:31	and a square by and ask square
0:09:36	and in but
0:09:37	the same her
0:09:41	we can quickly compute
0:09:43	by
0:09:44	a because it brought no matrix
0:09:46	so is slide up problem
0:09:47	is that of compute the in of the whole that the whole take we compute in but it's
0:09:52	sub method along the diagonal
0:09:54	so do you have here
0:09:57	and we also had how to compute the in but at that a least
0:10:01	is have very is so uh
0:10:03	nine
0:10:04	it press here
0:10:07	and finally from that them "'cause" new done of that room we fight with that
0:10:11	we we formulate the fast them let's be learned with some that to we reply here
0:10:16	i the approximation has has
0:10:18	and we define you
0:10:20	this
0:10:21	the fast them book use like
0:10:23	which
0:10:24	the with uh okay this
0:10:26	this term
0:10:27	this term we that
0:10:29	no this um
0:10:32	i you see
0:10:33	we so
0:10:34	this them with that this this
0:10:36	this a
0:10:41	and
0:10:43	and the L make
0:10:45	it
0:10:45	this border
0:10:47	with uh this the
0:10:49	okay also it's spread by the
0:10:51	block diagonal i dunno method
0:10:54	so finally
0:10:55	you see here the the the true this
0:10:57	most one
0:10:58	with the of the
0:11:00	the the five but to the much smaller than the this the approximation
0:11:03	uh has and then and these K we can we do the at the clear do the commit
0:11:07	so the of the
0:11:09	in a of this method
0:11:11	in the very don't to the in but that this
0:11:13	the possibly has and
0:11:15	and a call we don't need to control the to go beyond method
0:11:18	we not only to compute the
0:11:20	a possibly has an
0:11:24	maybe a nice
0:11:27	okay okay
0:11:29	so in this snow the next slide you see here
0:11:32	how to to the power with a how do to the the but but that in in the
0:11:37	you know an increase um
0:11:39	a very simple it and also very if if you soon that's a proposal were about rowing two thousand nine
0:11:44	that's of and
0:11:44	you nine
0:11:45	uh that's but more like that it we it in line
0:11:49	on the
0:11:50	the low power by real met um but power what's so by the la
0:11:53	last
0:11:55	enough a value and then we greater only
0:11:57	we do
0:11:59	you do the bar with and then we can
0:12:01	after the fire ten iteration we can
0:12:03	yeah
0:12:04	the can was and that the that's solution
0:12:06	and
0:12:07	we
0:12:08	we don't use this approach
0:12:09	we you are not a
0:12:10	and this you this one T by on the cake Q T condition
0:12:14	and this list to
0:12:16	the medium i of this problem
0:12:18	this for that's of that too
0:12:19	this condition
0:12:21	okay
0:12:23	and as to the simulation
0:12:27	is assume a some place and you see here for three dimensional tensor a with side what one and right
0:12:32	by one hundred by one hundred combo by ten component
0:12:36	yeah
0:12:37	but
0:12:37	the common
0:12:39	we for to be collinear we the auto by this form
0:12:43	and we see here
0:12:44	with the
0:12:45	multiplicative kl K with them what what deeply get the L
0:12:50	the green light
0:12:52	and that the i
0:12:54	is from a up the
0:12:56	or
0:12:57	one one right it ways that no one that fred
0:13:01	i not yet and
0:13:02	one ten held and B
0:13:04	that's how the in twice and then the and with only
0:13:07	get a convert to the solution and
0:13:09	and i like that on the L
0:13:11	the fast lm level and is them with quickly can what up the only
0:13:16	one at right right
0:13:22	but not case and not a that it we one that we
0:13:25	test with one tell than by one column
0:13:27	i want how than i with the common
0:13:29	you to here one attract recommended
0:13:31	it's really he'll
0:13:32	and the
0:13:34	fast and where make on is also quickly
0:13:36	fit the data up the
0:13:38	fifteen no sitting it weights and
0:13:41	i and all right with some can of fit the data
0:13:45	i is to to much but maybe i skip this like
0:13:49	is used for to this prior to
0:13:51	for of the method but maybe there with
0:13:54	and
0:13:54	or
0:13:56	this you know uh an application with the leave at the faded the way that we
0:14:02	we classified the data
0:14:06	that's we from the data that we have forty four interest for a and that we put from the fight
0:14:10	label
0:14:11	by the cable with flat it to this just kind of ten so C to by sitting
0:14:16	by
0:14:17	that T two to T two because we have yeah i
0:14:20	i orientation and for scale
0:14:22	and of call we have yet the last of and is that number of plays
0:14:26	for right
0:14:27	and
0:14:28	we
0:14:29	it truck of from this
0:14:30	ten so
0:14:33	for the first
0:14:34	first we track by
0:14:36	thirty two
0:14:37	and T F that we have here
0:14:40	and the S like that we employ an an map to talk dove
0:14:43	three too
0:14:44	from the last
0:14:45	one
0:14:46	the life factor
0:14:47	and the but form we
0:14:48	of and here you see
0:14:50	but the K and it with on is only
0:14:52	i was N
0:14:53	with the us
0:14:54	i T i percent
0:14:56	is
0:14:57	maybe maybe though with the
0:14:58	are you less
0:14:59	with night night it's three was and but our and with need
0:15:02	not for was a and if we employ this was can trying the ten i
0:15:06	one as that was sent with ten
0:15:08	face and all is not enough ten fate take class
0:15:11	that's me we have one and right
0:15:13	for
0:15:17	and we
0:15:19	it to the good clothes and
0:15:21	a you see here we already but both a fast and with them too
0:15:25	for for et and a map the fast that was knew that for N
0:15:28	and the M
0:15:29	and that was don't we
0:15:31	a you don't
0:15:32	we to compare you are the last scale that will be and and also that scale
0:15:36	has has C and
0:15:37	the process may has them
0:15:39	and we avoid
0:15:41	they vote of the the last
0:15:43	has
0:15:46	and it
0:15:47	and you are virtually you see here would have the a into trend
0:15:50	the fast them was new than
0:15:52	we we the fast lemme i rammed
0:15:54	and we term for canonical only
0:15:56	um only to decompose and
0:15:59	that these these two D dizzy
0:16:01	similar to the one the one of the wooden and we can where you but row and
0:16:05	and the mostly but
0:16:06	oh and we is further
0:16:09	thank you
0:16:26	i
0:16:38	uh yeah think to the
0:16:40	okay we we we we may the the net and also is similar to we man that the relative error
0:16:46	two
0:16:47	you see here
0:16:48	this is the right of the arrow
0:16:51	but we we do is read how to compute this error of this is very simple that's we
0:16:55	we
0:16:56	okay
0:16:57	from the conference and you
0:17:06	this so we compute this error and divide by norm up the so
0:17:12	yeah
0:17:13	this your L T you ever
0:17:17	yes
0:17:26	no i i you these
0:17:27	for from one is known so we don't you are not the kind of um is a one
0:17:34	i

FAST DAMPED GAUSS-NEWTON ALGORITHM FOR SPARSE AND NONNEGATIVE TENSOR FACTORIZATION

Non-negative Tensor Factorization and Blind Separation

Presented by: Anh-Huy Phan, Author(s): Anh-Huy Phan, Brain Science Institue, Japan; Petr Tichavský, Institute of Information Theory and Automation, Czech Republic; Andrzej Cichocki, Brain Science Institue, Japan