Speech Transcript - STRETCHING, COMPRESSION AND SHEARING DISPARITY COMPENSATED PREDICTION TECHNIQUES FOR STEREO AND MULTIVIEW VIDEO CODING

0:00:13	yeah
0:00:14	thank
0:00:23	uh morning
0:00:24	uh to they are like to send a paper
0:00:27	a stretching
0:00:28	an you're in this party
0:00:30	compensated prediction
0:00:32	and he's for multi view video coding
0:00:35	so um
0:00:39	he's the content
0:00:40	or
0:00:40	uh my presentation
0:00:42	uh uh make a joint action of the disparity compensated in section
0:00:46	especially that's the techniques used in the
0:00:48	curve
0:00:49	uh multi we the coding standard
0:00:52	and
0:00:52	you propose a lost yeah
0:00:55	i forty for S E S
0:00:57	for shot
0:00:58	for the S yes six technique
0:01:00	and the related uh consideration
0:01:02	and some experimental sound would be given
0:01:05	and
0:01:06	the conclusion
0:01:07	joint final
0:01:08	so um
0:01:10	P is the introduction
0:01:12	for the current uh a multi view video coding
0:01:15	a a standard
0:01:16	that at to
0:01:18	it is then that they exploit
0:01:20	correlation between the real
0:01:22	okay
0:01:22	as a this fight
0:01:24	previous uh
0:01:25	H
0:01:26	so um
0:01:28	it is a a a a a a a extension of the explore
0:01:32	as one a V C
0:01:33	so uh it is
0:01:35	is this
0:01:36	uh
0:01:38	in and
0:01:40	so so
0:01:41	the that this party
0:01:43	uh information
0:01:44	recall or that by the disparity compensated prediction
0:01:47	because uh
0:01:49	this technique
0:01:50	station
0:01:50	uh exposed to
0:01:52	what you see they
0:01:53	don't
0:01:53	same coding instead
0:01:55	all
0:01:55	spot to the cool
0:01:56	so
0:01:57	um
0:01:59	also so
0:02:00	the the the coding to is just the same as they used in the
0:02:04	and C P
0:02:06	so um
0:02:08	so uh if they use the same set of coding tools
0:02:11	a they
0:02:13	the only sport the pure translation uh um most a back to this party act
0:02:19	so it is a broad based um
0:02:22	um
0:02:23	motion compensated prediction technique
0:02:27	so that this party back to K
0:02:29	send the
0:02:30	the should all the blocks
0:02:33	where
0:02:34	uh but i
0:02:35	not indicate the act
0:02:37	uh
0:02:39	a to that
0:02:45	so um
0:02:46	i'll met what is to put holes a a a technique
0:02:49	to improve the this
0:02:51	particle can see text
0:02:52	for the and B C
0:02:54	ah
0:02:55	and the effect that um
0:02:57	oh we want to cheap
0:02:58	is to um
0:03:00	buy by a a a a a new technique
0:03:02	a proposed the course of a sub sample the block matching
0:03:05	it does not imply a uh
0:03:07	additional memory for
0:03:09	storing the test
0:03:10	additional
0:03:11	we're frames
0:03:12	and
0:03:13	it
0:03:14	you know
0:03:14	need to perform a this to though uh and the polish
0:03:19	so um all propose at any will be uh
0:03:22	able to integrate
0:03:24	to the actual to exploit
0:03:25	is it
0:03:27	so uh the S
0:03:28	talk out the um a the
0:03:31	this party fact
0:03:33	uh first
0:03:34	for four we propose
0:03:36	that
0:03:37	so
0:03:40	a a uh i here we show a a a a a real world example or all a a a
0:03:44	this party
0:03:45	that
0:03:46	it is a a real picture taken from the
0:03:48	i
0:03:49	a still can
0:03:50	two can
0:03:52	so it is obvious that um
0:03:54	the the difference
0:03:55	between the frames
0:03:57	a are not just
0:03:58	translation
0:03:59	okay that's that
0:04:00	uh
0:04:00	to what you close to
0:04:03	so for the uh
0:04:05	oh example there is a wall
0:04:07	uh uh on the on the right side
0:04:09	and in many find that we find that uh
0:04:12	for the last one
0:04:13	but
0:04:14	oh we use this one as the reference
0:04:16	you can see that a on the wall the plate on the war
0:04:19	it's up here to be a can
0:04:21	press
0:04:22	in the right view
0:04:24	but
0:04:25	so we find that a a there is a
0:04:27	a compression in that
0:04:29	if you a um not turning around
0:04:31	you will be a stretching that
0:04:33	so i here use the a vertical option
0:04:36	for and and example
0:04:38	uh
0:04:39	for example in the C
0:04:41	where some some horizontal objects
0:04:44	on a if you look at two
0:04:46	let me
0:04:47	you can of fun this
0:04:49	the texture on the
0:04:50	see
0:04:51	and
0:04:51	you a that a a on the right real the the texture it uh appear to be
0:04:56	uh a two a a i
0:04:59	um um we said to have it is a a a a a a
0:05:02	a year
0:05:04	fact
0:05:04	although though um
0:05:05	you may not be a a a a um
0:05:07	oh perfectly uh a processing uh more model that but
0:05:11	we believe that um
0:05:13	uh using the sheer in if can cross to me this that
0:05:17	in this case
0:05:18	so
0:05:20	um
0:05:21	so he he is the the uh uh uh S A six if that are we we want to a
0:05:25	sprawl
0:05:26	so it should be noted that a a a you the camera
0:05:30	set
0:05:30	this up and down
0:05:32	is that the uh uh that right
0:05:34	the the S S six if should be horse so yeah for
0:05:38	should be a vertical instead or something
0:05:42	so
0:05:43	oh how do we provide a um a the S E X fig
0:05:46	if fact
0:05:47	and the uh uh this partly compensated prediction
0:05:50	what in into what you
0:05:51	i i that
0:05:53	we can prove i
0:05:54	a some uh uh are transformed the frame
0:05:57	vol
0:05:58	for the the the
0:06:00	this part to text
0:06:04	a a for example we can provide a some uh a she had that
0:06:07	a a frame
0:06:08	and also uh
0:06:10	a strange uh a stretch and compress the frame
0:06:14	and we
0:06:14	can make use of these and frame to at all different uh uh i is sick
0:06:19	fact
0:06:20	to to improve the that
0:06:22	the this on the
0:06:24	text
0:06:26	but
0:06:26	ah
0:06:28	uh this
0:06:29	uh a what if that we we'll have a a a a a very serious uh
0:06:33	but not for
0:06:34	it major problem
0:06:36	uh
0:06:37	for for practical use
0:06:40	because uh and this frame based on require a some um memory space
0:06:45	uh a in the and code
0:06:47	because you you need to provide a
0:06:50	a a number all
0:06:51	reference frame
0:06:53	a a for you can you can
0:06:55	for for the problem actually has one
0:06:58	so
0:06:59	um the number uh the
0:07:01	then we we we'll be role
0:07:03	a a a a a with the number all
0:07:05	it's is actual
0:07:06	uh can
0:07:07	provide
0:07:08	and also
0:07:09	uh the the computing line and for providing this uh S A six frames
0:07:14	is also a complex because you be two
0:07:17	uh a a you the poll at of the reference frame that
0:07:21	performing transformation
0:07:23	for for the S is section five
0:07:25	the frame based approach is not
0:07:27	right
0:07:29	so
0:07:30	in this research we will
0:07:32	uh a post match what
0:07:34	uh
0:07:35	to achieve the S A six
0:07:37	this C P
0:07:38	i i a new technology at a new to a uh and i call
0:07:42	a a sample the block matching
0:07:44	that's i set it to you know it does not require additional memory and
0:07:49	additional look
0:07:50	the
0:07:52	so um
0:07:55	uh i here is all on
0:07:58	the S A six this if you
0:08:01	a a in the extra two six for U V C
0:08:04	are we have a a what a pixel uh a of us to use it
0:08:09	and
0:08:10	a in a typical uh echo that
0:08:12	we will have a uh some four times in the for it to that and frame
0:08:17	for for as to um
0:08:19	form the problem matching
0:08:21	and and uh the
0:08:22	and the uh motion compensated prediction works
0:08:27	so um
0:08:30	in the in the and C P and this if you process
0:08:33	um
0:08:35	uh the echo the we sub same the the frame
0:08:39	to taking the uh uh
0:08:42	problem
0:08:44	in the original scale
0:08:45	because you you you
0:08:47	in in with that
0:08:48	in four times
0:08:49	and subset wave by what times
0:08:51	so that the scale
0:08:53	change
0:08:54	but
0:08:54	you can a the location
0:08:56	the a pixel location
0:08:58	so
0:08:59	we define a
0:09:00	oh we first define this uh uh is
0:09:03	a a a subsampling sampling rate five
0:09:05	question
0:09:06	okay
0:09:07	it should be
0:09:08	for simple
0:09:10	um
0:09:10	to achieve the the S A six
0:09:12	uh effect
0:09:13	we can modify this um
0:09:16	a subsampling sampling re
0:09:18	for um
0:09:20	for for for different probably test
0:09:22	to achieve the if that you like to
0:09:24	sport
0:09:26	so
0:09:27	yeah use the um
0:09:29	um
0:09:29	um stress
0:09:31	and compression
0:09:32	um
0:09:34	uh
0:09:35	if X that we we decide
0:09:38	so that we use
0:09:39	okay we use
0:09:40	the the a that's same we we define by this
0:09:43	creation
0:09:45	but that and you can see that and the
0:09:47	um top left corner
0:09:49	all of the the the matrix data
0:09:52	um
0:09:53	this is and compression
0:09:55	a a problem
0:09:57	we pacing the original
0:09:59	for
0:10:00	part to
0:10:01	i in the corner
0:10:03	so i we will achieve a a a
0:10:06	strange and compression horizontal horizontal
0:10:08	for example we do uh we we this up to do the that the S C problem at a with
0:10:14	three
0:10:14	we you have a
0:10:15	the
0:10:16	the subsampling grid shown in
0:10:18	uh the left of figure
0:10:20	you can see that um this
0:10:22	some and one where you all the horizontal september we all the
0:10:26	all
0:10:27	i the reference frame
0:10:28	three
0:10:28	we go
0:10:30	you can do
0:10:33	this uh pixels
0:10:35	for example if we
0:10:37	we we are subject to you of the S C problem to i five
0:10:41	right you have a is trash problem
0:10:45	so here is the
0:10:47	the
0:10:47	stress and press
0:10:50	so for this we
0:10:52	we um
0:10:53	uh you use this
0:10:55	uh are also send bring re
0:10:57	well
0:10:57	by a S problem to
0:11:00	so
0:11:01	a by such doing by one or a we will have a
0:11:05	a a a a a a right view that
0:11:07	uh problem
0:11:09	for
0:11:10	to
0:11:12	the row by the the you if at the left
0:11:14	that that if that we
0:11:16	does
0:11:16	such a use some on
0:11:18	on minus one
0:11:20	i
0:11:21	this is uh
0:11:22	example of the search
0:11:25	of course a to achieve a more a stretch and compression
0:11:30	it she hearing um
0:11:31	if
0:11:32	we can be more uh meet a in the
0:11:35	S C A S
0:11:36	from
0:11:40	so so there's a how we uh cheap that S E is six five
0:11:44	the uh C P
0:11:46	oh uh you can uh we like to all would post some my experimental result
0:11:51	i here is the that
0:11:53	a general condition that
0:11:55	can pick a the the
0:11:57	uh
0:11:58	in the be people in the
0:12:00	G M Z
0:12:01	seventeen
0:12:02	encoder
0:12:03	so
0:12:03	there a high profile
0:12:05	we use
0:12:06	some a video sequence that is quite a used a
0:12:10	paul room X that what's a right now
0:12:13	so uh a a the
0:12:14	prime time
0:12:15	thus does and for your information
0:12:19	so for the um
0:12:20	for the
0:12:22	prime we can we use the for our uh and also
0:12:26	we use
0:12:26	uh X
0:12:28	where we use
0:12:28	H
0:12:29	it and
0:12:31	session
0:12:31	okay
0:12:32	let
0:12:33	we the horse on those us a ten or in way from one
0:12:36	to not
0:12:38	and
0:12:38	X
0:12:39	as she ring levels will be used at
0:12:41	but minus four to four
0:12:44	i i should of that uh our for is currently only have high
0:12:47	P
0:12:49	well for for the uh
0:12:53	also so that uh um and a is on the uh uh coding efficiency
0:12:58	we we also sell to set
0:13:00	the and
0:13:01	one is um with all you frames
0:13:03	uh
0:13:04	i in this set um
0:13:06	the let
0:13:09	and that sequence
0:13:10	the base will will be a i frame
0:13:12	and the right real is all P frame
0:13:15	the P frame only um
0:13:17	a predicted by
0:13:18	uh this party it
0:13:21	and i set is a a hierarchical be frame with uh
0:13:24	save and B frames that is by company
0:13:27	in
0:13:28	in uh a new
0:13:30	techniques
0:13:32	uh this is
0:13:33	used to uh explore the effect
0:13:36	all the
0:13:37	all our proposed idea
0:13:40	in
0:13:40	some practical group
0:13:43	so for the
0:13:45	at first
0:13:45	set all the uh um
0:13:48	uh encoding we sell
0:13:50	um
0:13:51	we find that uh we have uh improvement in all case
0:13:55	and uh uh F H B that's and it's all one point six nine
0:14:01	send
0:14:02	so uh
0:14:04	this is
0:14:05	yeah that this one point
0:14:08	no
0:14:09	so
0:14:10	vol
0:14:11	the the the coding structure we've a seven hierarchical B thing
0:14:15	we
0:14:17	we found that uh we still have improvement in
0:14:19	in
0:14:20	all this case
0:14:21	but um
0:14:22	but the improvement is uh uh a little bits
0:14:25	a
0:14:26	i it is possible that a the improvement
0:14:29	i do
0:14:29	by was beeping because we did up i
0:14:32	uh
0:14:33	no one B frame
0:14:34	but the F H B and
0:14:37	is you are out of the
0:14:39	so you want to two
0:14:40	two one to five
0:14:42	then
0:14:44	so ah a
0:14:45	a he is
0:14:47	close and this
0:14:48	oh
0:14:49	take
0:14:50	so that we have
0:14:51	uh a proposed post that in that one two
0:14:54	all
0:14:54	to that that the the S A S six back
0:14:57	for the disparity compensated prediction
0:15:00	we use the subset for the wrong match
0:15:03	technique
0:15:05	this is a um that's yes S brought i'll taking the from the interpolated frame
0:15:10	so that we don the two
0:15:12	two
0:15:14	and and the memory and
0:15:16	notation full
0:15:18	uh taking in
0:15:19	uh
0:15:20	blocks
0:15:21	and somehow T
0:15:22	spot
0:15:24	so a all for future work
0:15:27	we find more um
0:15:29	a and we we
0:15:30	for
0:15:31	both uh D C P A and C P
0:15:33	for for or five to with the the proof
0:15:36	the um
0:15:37	formant
0:15:38	a a a a a general video you to
0:15:41	and
0:15:42	you also find some possible that you know were from
0:15:45	subsampled sample you
0:15:48	oh
0:15:49	this is
0:15:50	one one
0:15:50	is that they
0:15:51	think the very much
0:15:59	we
0:16:00	maybe one or two
0:16:01	which
0:16:08	yeah why
0:16:09	but that that is the most for things you already an estimation
0:16:14	on
0:16:15	as you a in our analysis is found that the p-frame frame use of the P right
0:16:20	we
0:16:21	and use only a uh in the in the and
0:16:24	at the and we C and a moment
0:16:26	the P frame is used to be uh predicted by
0:16:29	the uh i i i I detection from the same view
0:16:33	it is usually a out there are very uh well to
0:16:36	so so the room for whole
0:16:38	using the in the in the we detection may not be very very good
0:16:43	and is not use that and i do you
0:16:45	yeah use the don't frames
0:16:47	of course not
0:16:48	that i frame is
0:16:50	in
0:16:50	i intra prediction
0:16:52	because you produce
0:16:54	even though it terms of motion
0:16:56	still do in did you
0:16:58	all
0:17:02	the intra frame
0:17:03	we not use of
0:17:05	that's the ah
0:17:06	and i'm not that from but the standard itself
0:17:08	but
0:17:10	five a okay i guess it's a but i think is just a meeting a homography
0:17:17	i as opposed to sharing
0:17:19	yeah estimation
0:17:20	actually is a a a and that what is
0:17:23	try to provide some
0:17:24	uh approximation all this uh
0:17:27	a so called more to try formation by a very simple way
0:17:32	so so
0:17:34	that the if at all in the the transmission
0:17:37	may not be uh uh
0:17:40	or a to the uh
0:17:42	uh uh generated
0:17:44	but
0:17:44	would be that that uh this is a a quite good
0:17:47	approximation
0:17:49	and questions
0:17:52	i

STRETCHING, COMPRESSION AND SHEARING DISPARITY COMPENSATED PREDICTION TECHNIQUES FOR STEREO AND MULTIVIEW VIDEO CODING

Stereoscopic and 3-D Coding

Presented by: Ka-Man Wong, Author(s): Ka-Man Wong, Lai-Man Po, City University of Hong Kong, Hong Kong SAR of China; Kwok-Wai Cheung, Chu Hai College of Higher Education, Hong Kong SAR of China; Ka-Ho Ng, Xuyuan Xu, City University of Hong Kong, Hong Kong SAR of China