Speech Transcript - RIG: Advanced GPU visuals for UI design

0:00:10	us so thanks a lot for coming along is i'm right am reusing some slides
0:00:16	i gave
0:00:17	values for different presentation an internal sort of some it we have intel i hadn't
0:00:23	planned ahead to do this talk what they opportunity came up since vision a pistol
0:00:28	guess today to present this is
0:00:31	apologies about the branding and they didn't tell confidential as ensemble size you know that
0:00:42	okay so
0:00:44	my name's rubber private can i work for yet entails open source technology centre in
0:00:49	london
0:00:50	on a graphics engine enjoyed graphics for a long time
0:00:56	and just have the opportunity to work on a number of players in the graphics
0:01:01	to a linux ideas
0:01:04	one of the first companies i work so was imagination technologies right at the option
0:01:10	introduce silicon enabling
0:01:13	some about where and work through the open G L driver and some of the
0:01:18	kernel driver and that kind of low level work and
0:01:21	and i moved to
0:01:24	sort of a small company product and swear we'd it's discarded toolkit and gain some
0:01:29	experience with high level you are technologies and through into stress composites as and
0:01:37	or you can
0:01:39	the work in firefox
0:01:42	but so
0:01:45	as well as having this
0:01:47	passion for graphics
0:01:48	i've been fortunate to work in an office which is got cut indicated design is
0:01:55	visual design is interaction design as well as engine is the have been
0:02:00	involved in creating applications the design by these guys
0:02:04	and so i've seen the process by which we come up with ideas and we
0:02:10	create applications and so i hope they have some insight into where that process is
0:02:16	not as effective as i think it could be
0:02:20	and
0:02:21	so i'm here to talk about this project who rate which i started about
0:02:27	that you're going to be working on sort of part time which is this is
0:02:31	owed sort of markup
0:02:33	as you people missing from this and you know rubber's the is a guy i
0:02:38	work with maintaining project look always been helping with this time you must be a
0:02:42	he's on the graphics team at that in D C
0:02:46	i mean a is an intent have been working with that visit ago i could
0:02:50	chris cummings he's been helping out there as well actually we cows a visual design
0:02:54	it did this markup and help with the visuals of this project
0:02:58	the change rate has been helping out as well so
0:03:04	and so
0:03:06	i want to give a bit of background to why i started this project
0:03:11	and so the focus a good friend so i mean this the body aim of
0:03:17	this project is the to
0:03:21	to drive innovation in the space of user interface is we're looking to explore for
0:03:27	the use of the G P because
0:03:30	so
0:03:31	well actually just about because i i'm just gonna be a little bit rougher many
0:03:35	"'cause" i haven't run through this but so
0:03:40	during that experience of say five years of me working and you i technologies i
0:03:45	guess i've
0:03:46	built up this
0:03:47	kind of feeling that
0:03:50	you do we are completely you will not we were really wasting this gpu hardware
0:03:55	which is in our devices like mobile phones and a not tablets in a laptops
0:04:00	the state of you i technologies today we used a gpu for basically shifting tech
0:04:06	should rectangles around and so i've had this mounting feeling that is gonna be something
0:04:10	that we can do to
0:04:12	expose more creativity to the visual design braces an expose that gpu out with a
0:04:17	fully utilising the G P S as what i want to go through a bit
0:04:21	more
0:04:22	okay so and this and that through that experience of working with visual design is
0:04:28	what can we do to optimize the website from coming up with an idea to
0:04:32	create an application something just quickly recap state-of-the-art them
0:04:37	state-of-the-art
0:04:39	toolkits like G T K G T S waiting list plus the you pass and
0:04:45	even web technologies the that with familiar with that web okay
0:04:51	all of these are actually extremely similar technically speaking if you look at how they
0:04:55	work
0:04:57	and it's
0:04:59	they so they basically implement some form of the postscript or pdf rendering model is
0:05:05	the foundation of the vocabulary by which all of these technologies work
0:05:09	with and
0:05:11	so that vocabulary
0:05:14	in essence is these
0:05:16	three things it is just you have to D shapes you can fill them with
0:05:21	images there's two D shapes a basically just always rectangles i rounded rectangles if you
0:05:26	getting pretty fancy and you know theoretically you can use basic us and all that
0:05:31	fancy stuff but
0:05:32	who actually doesn't
0:05:35	you fill these two D shapes with images and you have text that's your vocabulary
0:05:40	i don't these are all built up you build you why up using the paint
0:05:44	is everything which just means that store the background and
0:05:48	will basically familiar with this a really a background press you met grant and then
0:05:53	your labels and foreground on top of that that's the model
0:05:57	and
0:05:58	you have all they we employ some form of seeing graph to organise the primitives
0:06:03	the which it's the components of the you why so you have a hierarchy with
0:06:09	the window the top and this is what defines
0:06:12	how you implement the paint is that within the stuff the top of the profits
0:06:16	of is the background
0:06:17	and the leaves a of the foreground just pretty simple it works to some extent
0:06:24	i mean the cornerstone of that model is the quality of those static images that
0:06:29	you get from your visual design that's
0:06:32	so these technologies
0:06:35	pretty much all i mean to engine is so that's part of the problem now
0:06:39	the come as an imperative A P I C python and such things
0:06:44	they have a markup language so
0:06:47	will familiar with H T M L with but there are things that you amount
0:06:50	as well
0:06:52	and it's
0:06:54	yes they all have this of sort of this contract and this promise that they
0:06:59	will as you develop these technologies you will to find a an interface for engineers
0:07:03	to work to and you must keep that's tables so
0:07:07	you know there's a button involved in maintaining a stable interface say that all of
0:07:12	your existing applications don't break as you of all that technology so is pretty darn
0:07:17	hard to think ahead of time what interface am i gonna be have to stick
0:07:21	to for the for the rest of time that i'm not gonna pay myself into
0:07:25	a corner as to how we can improve the flexibility of this
0:07:29	this technology visually speaking and enable more creativity for designers
0:07:34	it's that's difficult
0:07:38	so i sort of said this year this overwhelming failure of through this experience that
0:07:43	we could do so much more with the
0:07:46	more creative use of the gpu
0:07:49	so
0:07:51	this is spelling out of the be a the problem so we've got this the
0:07:57	first one is this limited vocabulary is the P D F postscript
0:08:01	rendering model it is like twenty five years so it's
0:08:07	and hasn't changed
0:08:09	so we've got the middleman toolkits this is the this sort of
0:08:14	liar where it's a means to an end these toolkit so just there for an
0:08:19	engineer to recreate the vision of the designer and this is sort of a language
0:08:24	barrier between that vision and the technology the designers tend to have very little insight
0:08:30	into the working of these technologies so the really obvious middleman and maybe we can
0:08:35	think about ways of not having quite the same
0:08:39	approach and avoid that middleman sort of
0:08:42	translation that we that we deal with today
0:08:46	and when we think about it from the designer's perspective as well
0:08:51	they're using tools like photoshop well in escape
0:08:57	and things like aftereffects
0:09:00	these are the intended for creating static images
0:09:05	and they have all kinds of fancy effects available to them by the end of
0:09:08	that you get a beta emission and we slice up and the rectangles and or
0:09:12	you are with that all the made for offline video
0:09:17	pricing like aftereffects and that uses things like ray tracing to come up with this
0:09:21	affects you know there's no way those kinds of effects are gonna run in real
0:09:25	time on a firing anytime see
0:09:30	and so
0:09:33	well these designers are actually trying to do is create interfaces that can run on
0:09:37	a results constrained device and be responsive to in the
0:09:42	and just you well within a power sort of constrained and it's
0:09:52	so they really don't at the moment have tools are actually geared towards that specific
0:09:57	problem so maybe we can think about
0:10:01	inventing new tools that right
0:10:03	improve that part
0:10:05	so
0:10:07	that is just the actual what fly
0:10:10	the very plastic sort of outline of how design goes of them i mean is
0:10:16	you have the special design a
0:10:19	creates sort of sketches out is vision in something like photoshop to indicate
0:10:26	and they had over the wall if you like to the engineer who has to
0:10:31	reinterpret that vision and translate that into and i pi a sort of fashion so
0:10:39	that the this station totally disconnected from the sort of visual creative side of things
0:10:44	that are opening up your reference manuals and implementing this and that implement is typically
0:10:48	on that desktop machine a laptop or so even if they are trying to create
0:10:51	something for fun you can initially priced at the stuff on your laptop
0:10:55	and once you kind of got it working then you're gonna started so that's integrate
0:11:01	this with appeal system to be able to cross compiler whatever it takes to get
0:11:04	it on T of fighting and then start playing around with that will how does
0:11:08	this before madison's response
0:11:11	and
0:11:12	at that point that way you're actually starting to test on the device
0:11:17	is a pretty high chance that you're gonna realise that this like some problem like
0:11:23	it doesn't the design doesn't fit the form factor like the designer which nist back
0:11:27	expected it to it doesn't respond as of as of quickly is it needs to
0:11:32	this like so many possibilities that could just not be quite right and you're gonna
0:11:37	have to go all the way back to square one where the designer has to
0:11:41	rethink the idea in the interaction model to be able to work with the technology
0:11:46	that they have
0:11:47	because the alternative is that you go back to engineering transfixed the technology but that's
0:11:52	the and you know an unknown quantity as to whether or not you're gonna be
0:11:55	have to fix a technology in time if you got the time so your options
0:12:00	are pretty slim about point to have to redesign things
0:12:05	and
0:12:06	another aspect of this is the ability to objectively measure certain
0:12:14	quantities of quality a that there are important for user interfaces
0:12:20	you a consumer expects that they're fine it's gonna respond in a timely manner when
0:12:25	they when they touch it lays we've around the you why their expected not to
0:12:30	redo that battery but i expect a certain level of visual quality and right now
0:12:35	we don't have the integration in a tooling and the
0:12:41	the processes by which to really make sure that there is
0:12:45	that we're on top of those quantities right through the whole design development process those
0:12:50	tend to just be a and often will not so much an article but they're
0:12:54	the end of the line
0:12:56	in the pipe line of
0:12:58	how we develop these things
0:13:02	so
0:13:03	i guess i kind of into the but some of the solutions to these as
0:13:07	i was going through the problems but
0:13:11	obviously the
0:13:12	one of the main things that we're interested in here is
0:13:16	throwing away that limited price good model and saying if we can explore way more
0:13:21	features of the gpu to
0:13:25	there's a lot of flexibility in a gpu what can we are not there and
0:13:28	give to the visual design of so allow them to just be more creative
0:13:34	the model of the price good model is actually really difficult to implement efficiently on
0:13:40	a gpu just given the way the gpus is kind of difficult to avoid the
0:13:43	over draw in the price good model because you going from background or foreground and
0:13:47	that's the model but you build up to see your it's hard to avoid colouring
0:13:52	in all of those redundant fragments and pixels behind the stuff environment of it that's
0:13:59	just quite a tricky problem
0:14:03	and so as you can probably guess from the screenshot of the beginning of this
0:14:07	talk the a big aspects
0:14:10	to how i've sort of see as being able to so many of those problems
0:14:15	is to develop a spec tooling
0:14:18	which is what the rig projects as is looking at doing
0:14:22	so what we're doing is we're creating we were actually
0:14:26	trying to learn from how game developers work here because
0:14:30	game developers face so many of the same problems that we face as you i
0:14:35	developers except that they have
0:14:39	done a much better job of taking advantage of gpu hardware than we have they
0:14:43	have to create interface they have to create something that can run in real time
0:14:47	response that you know interactively with the user interface with the user and we need
0:14:53	to do the same kinds of things
0:14:56	and
0:14:58	the weighted various guys work is that they have a rendering engine
0:15:02	and they often build up a lot of this but tools to help them create
0:15:07	their wells create that they games
0:15:11	and
0:15:12	they have to work as well with design is an engine is and if you
0:15:17	look at projects like unity three D where they have really quite so you know
0:15:22	capable tooling to be able to connect
0:15:25	that whole works like between
0:15:28	game design is and the engine is the created this logic and you game itself
0:15:37	so we're looking to sort of reports their ideas as much possible "'cause" we can
0:15:41	build on their experience especially they call of that on a lot of work on
0:15:44	into researching algorithms the less is the really interesting visual things of the G P
0:15:50	A
0:15:51	so
0:15:52	we want to create technology which is the you are rendering engine that we can
0:15:56	share between something that's gonna deploy onto a device
0:16:00	the route to run a U I and this and that the tool itself is
0:16:03	gonna be using so the design process is actually constrained by the capabilities of the
0:16:09	technology
0:16:10	so the visual designer was and then
0:16:15	the beginning and specialised tools during the visual design process
0:16:19	we want them to also the really connecting that be issue does development stage with
0:16:24	the hardware that they want to design for so we have
0:16:28	the i what we have in implemented even the ability to have a network connection
0:16:33	between the design so and the device this
0:16:37	that your you you're targeting so that while you're working in the design so
0:16:42	it's easy play around with animations of things that's gonna immediately a date
0:16:47	what's on the device so you can pick it up and check what's the responsiveness
0:16:51	like what's the quality of this like say that you're connecting testing right so that
0:16:57	early prototyping stage which there's just no substitute for testing your ideas on real hardware
0:17:03	so we see that as a pretty compelling way of improving how works like
0:17:09	and optimising the time it takes to develop
0:17:13	applications
0:17:16	the
0:17:17	the last one is necessarily can it will be done with within the same so
0:17:23	what we need to develop tools
0:17:26	for measuring save the power usage while you're running you so you testing your you
0:17:33	why on the device are descended asking you wanna be able to measure so what's
0:17:37	the power usage what i'm doing certain things or measuring the latency by which
0:17:43	you when use what
0:17:45	how long does it take for the interface to stop responding to that so even
0:17:50	we're even thinking of things like will high-speed be at video cameras and a processing
0:17:56	recordings of that kind of thing
0:17:59	we need to think through that problem as well
0:18:02	so
0:18:04	i guess at this point time to just
0:18:07	maybe but
0:18:09	if it was to just jump in and ask questions especially now that i'm gonna
0:18:13	basically demo a demo a then
0:18:17	just raise and stuff so
0:18:20	and
0:18:22	there's i wasn't really expecting to be there and then i guess and i've been
0:18:26	hacking on this the chances that this is
0:18:31	so let's this gonna go through a bit in order to this area on
0:18:38	on the last year is
0:18:40	well so that we're aiming at work can asset you can work load here so
0:18:44	we are aiming for
0:18:47	this area is where you import say things that we created that about does it
0:18:51	could be images made by design is impaired e-shop we could be models made introduced
0:18:55	your max it could be logic was created by engine is something that needs to
0:18:59	be integrated could be audio and so these kinds of things
0:19:05	and is this the sort of search area so we can search the images of
0:19:11	videos or whatever
0:19:14	whatever we need and then
0:19:16	people there is assets into the this centre area which is where we actually build
0:19:23	up the user interface itself we say we can we get immediate visual feedback about
0:19:28	the scenes that we're trying to create
0:19:32	on the right hand side here which is public to the moment but this bottle
0:19:36	would be where we would expect a more tooling capabilities and it's put there so
0:19:41	you can to the trends in the expand into the space of the interface to
0:19:46	give more detailed information about tutoring and when you're not using very spending can sort
0:19:51	of shift out of the way
0:19:55	and then down button here this is where we would so
0:19:58	deal with controlling of the problem is actually it is the best
0:20:03	just as the actually a where this is some of the properties
0:20:10	for the one of the objects investing a this idea of entities in components and
0:20:18	entity is like something ins is essentially just relative to a parent in space and
0:20:24	it has a bunch of components that can be associated that adding more semantic information
0:20:29	to the object the more components year at the most semantic meaning that thing has
0:20:34	and
0:20:36	these red buttons here about being able to control the particular property so if you
0:20:43	click one of those and it brings the ads it that this area down here
0:20:47	which is cooled what is to control is the initially when we were first that
0:20:51	this is just but online management but that's in the process of evolving beyond just
0:20:56	key frame based animation at the moment this is this example is just built around
0:21:00	key frame animations and it's just a
0:21:04	a toy them or
0:21:07	having in the idea i guess in this case is a fire an application that
0:21:11	we've got notifications bubbling up and something that's more recently people problem and it's gonna
0:21:18	ten years excellency can see a bit
0:21:23	visual stuff
0:21:24	so you've got down the field of facts which shows the something better way look
0:21:28	soft it's as modelling out real well camera works
0:21:32	and we're also playing around with shattered mapping in this example which is way of
0:21:38	having real time that shot as much
0:21:42	and we've also got real time lighting so
0:21:50	and you have a standard a lot of the lights based on and if you
0:22:00	which is a
0:22:01	so the
0:22:02	is a light source
0:22:06	well see we're ripping off at the moment it's
0:22:08	very traditional with actually seeing daniel engines this is largely a as a practical
0:22:15	the this
0:22:16	the practicality because
0:22:18	we have to bring up a whole so it's
0:22:22	there's
0:22:23	we didn't have the time to do the research and you visual algorithms this one's
0:22:28	a well understood and they work well together
0:22:30	but we'd be looking to do research on that's the
0:22:35	if you a stylised ratio that would be which is not something you'd necessarily expect
0:22:41	and again but would make more sense in a you why
0:22:46	but even these ones i think that there is scope being
0:22:50	so for having a design and it's have to use that a so say to
0:22:54	know go overboard because this does give you a lot of power to
0:23:00	you know
0:23:01	create a you why that would be and in massive three D virtual kind of
0:23:06	the you could easily go for the problem is a three D environment that you're
0:23:11	modelling within and but
0:23:14	you don't have to go overboard
0:23:18	so let me just the same time show some of the effects that
0:23:25	that
0:23:29	this one this one doesn't have i have got assets that really geared towards this
0:23:33	but it's a
0:23:36	a technique that as well uses the G P a bit more than where
0:23:40	doing them i wanted to bump mapping
0:23:43	but it's
0:23:44	your meeting so that that's twenty ten should have that image but that represents a
0:23:49	modulation in the shape of the a surface so it makes so it represents a
0:23:54	high essentially and
0:23:56	and you can really see the when i click that the service that it's the
0:24:01	it changed visually but what's kind of interesting is when you select the light
0:24:08	and you change the direction of the light goes out
0:24:15	didn't last too
0:24:25	so that's just to stop i found nothing special about that and the bum not
0:24:31	the case with this was just generated by running it through a program that uses
0:24:35	essentially so profile to that says there where there's contrast in the image that they're
0:24:41	edges best it automatically generates
0:24:44	so idea the edges you're moving up there
0:24:48	so if you take the last of the questions again you see that this like
0:24:53	a detail in this every stick gets changed the changes we make a lot of
0:24:57	this that this is not some is craft it's
0:24:59	what it's about this but i think with a bit of have a visual design
0:25:03	and knowing what they wanted to G fan with us dedicated assets then this room
0:25:09	to make something just bit more dynamic an interesting that light source might be doesn't
0:25:13	have to be moving know every time you move your finger around it could look
0:25:17	a little bit over the top but maybe it moves is
0:25:21	as you sort of time between spaces on your fine
0:25:27	another there's another this one is a fine example of just using the G P
0:25:31	to style as something the
0:25:36	show is
0:25:38	is the video on this
0:25:41	just to show the videos just to show really that is something that can be
0:25:44	done in real time and this point is an is court appointed is in the
0:25:50	fact that and all this thing is
0:25:53	great something on the original images scale this write up so you can see what's
0:25:57	going on
0:25:59	so as well
0:25:59	snapped and things to be have to
0:26:01	to control
0:26:04	and
0:26:08	we can
0:26:10	stylised something and this point is an effect
0:26:15	i guess used quite a bit in static images and prices of things
0:26:19	but we can do this dynamically and depending on the lightness of the pixel is
0:26:24	a it within the image then we're changing the size of some geometry to
0:26:30	to just create is a like nice looking at back to the
0:26:35	might want to have this in on a background of some aspect of your hard
0:26:40	merrier of a of a device interface is something that again have to design either
0:26:46	i get to leave it out to them but i'd like to be up to
0:26:49	provide them tools and freedom and but constraint as well two things i know to
0:26:56	run efficiently on the G P and it's almost cheaper to run this video effects
0:27:02	then it would be to actually show you the video normally just because in this
0:27:06	case okay ish
0:27:09	there will be in this case
0:27:13	a the i only points where we do like colour space conversion that why U
0:27:17	V to rgb of the double points which we are showing a secretary doing in
0:27:23	some ways less work than you would do to sure the emission and that's kind
0:27:29	of the but
0:27:31	so that's kind of the state of the two of them i expect i actually
0:27:38	they way in which the design so to next to the slave device actually which
0:27:45	nice
0:27:46	so this represents a device sitting on your desk which might be if i you
0:27:49	know say
0:27:55	well i it i don't have anything that's got this like running on a real
0:27:59	fine but this is the that we had an hundred back and that we created
0:28:04	a while ago and but it's like some updating also
0:28:19	so that there's connecting this
0:28:22	design to that device
0:28:25	celebrate from civilisation
0:28:28	so if i scrub through this animation then you see we have the effects are
0:28:33	enabled on this one well the not in the editing area so you can get
0:28:37	a feel for what it's like on a device and in here
0:28:40	but then
0:28:42	here if i gotta say to the end of the time line and grab this
0:28:47	change the animation at the end
0:28:49	about my device so anyway immediately outdated and it's see going to in czech what's
0:28:55	the timing of that change the key frame it's right into you constantly you're happy
0:29:00	with the work that while seeing it for real on the actual
0:29:05	and this done using of all here at the moment so ago zero come so
0:29:09	that it can just discover the device
0:29:12	this is possible
0:29:14	okay so that's basically will content about the moment to be have to introduce great
0:29:20	but be happy to take any questions and to
0:29:25	okay thanks to
0:29:46	question to you
0:29:50	the point to do so i want to run right well actually is the that's
0:29:57	not just i if
0:29:59	save for many with the times fragment pricing in fact expressing in terms of gpu
0:30:04	something like the future as well so often video effects i just done as fragment
0:30:10	programs like say you are doing just colour rising or you have to de saturate
0:30:16	or contrast or brightness is kind of thing you just running over every single bit
0:30:20	pixel in the image in this case where actually we have geometry for the cells
0:30:27	if you like in that affect
0:30:30	and
0:30:31	it starts off as basically a tiny unit rectangle for each cell which texture maps
0:30:37	a circle and when it reads through the videos data i need something on these
0:30:45	course great points so i really does like that why you be colour space compression
0:30:50	on a specific points looks at the lightness of those and then depending on the
0:30:55	like this is essentially scaling out that geometry so i've rule you're doing way less
0:31:01	fragment processing then you would often differ a bit effect a and you're moving a
0:31:05	bit to the you know the but there was really interesting to what i'm what
0:31:13	is you some that's in the tool designers can use
0:31:20	so someone to explore
0:31:25	i was so you wouldn't be able to see yes is so what are the
0:31:35	particular a design decisions made a is the it doesn't really doesn't really scale to
0:31:42	try to come up with them really modularising rendering system like at the extreme where
0:31:47	we have class to well i mean it's changed now but we you know with
0:31:51	to use to be able to have a so that
0:31:54	every single actor could write all kinds of in narrative card to do alighted roaring
0:32:00	and it's very flexible but if you want to optimize how use a gpu you
0:32:06	tend to have to do really classy with things and make various assumptions and it
0:32:12	works a lot better if you just make that piece of rendering card a lot
0:32:16	more monolithic so enrage it's much more sort of like a game engine monolithic rendering
0:32:22	engine and if you wanna extend a then that
0:32:28	the then you basically
0:32:31	the engine is to have this specialised experience to know how to program a gpu
0:32:35	efficiently they should come and should be every project and at their affects another features
0:32:41	that because the number of people that have specialising gpu programming
0:32:46	is barry lighter in terms of the number of people the ones right applications so
0:32:51	we can expect application developers to be writing G S L and we're you know
0:32:57	sort of shade ascends integrating their rendering ideas it a into a modular system and
0:33:04	also get the performance we will amount is that
0:33:15	so a couple questions the first is
0:33:20	what do you actually sending to the devices it just is just display list basically
0:33:24	with animation descriptions and i didn't have the network to device but well the whole
0:33:30	the rendering engine itself is running on a device or the description of the scene
0:33:34	which is we have this seen trough of
0:33:38	entities and entities just
0:33:42	there they just have a position in three D which can be write a sin
0:33:46	transform relative to a parent and potentially we could be sending multiple would be seeing
0:33:51	grass and each component has each entity has a bunch of components which say ads
0:33:57	the idea that you have material which says how should be lit or some sort
0:34:03	of video source image source it could be some logic associated with that entity that
0:34:10	triggers when you have input all this what kind of high level
0:34:14	sort of semantic description it's what's gonna get sent over the wire to the device
0:34:19	and it's gonna like that
0:34:24	second of which in the my second question you talked about it kind of like
0:34:29	already against are they created a their scene with reagan so like a game engine
0:34:35	so when it comes time to actually like should the application of a shipping the
0:34:40	maybe so what you know where they shipping the all the assets and then this
0:34:44	description it's interpreted
0:34:47	well that's because the way the than the network writable has to be able to
0:34:51	serialise everything that relates to be the description of the scene including bss themselves and
0:34:59	we also use this so we using part school buses which quite widely used way
0:35:05	of civilised later and they give you a nice markup for describing messages for stabilising
0:35:10	data and we use the same particle we use the same mechanism for being a
0:35:16	to stabilise over the network as we do for stabilising to this when we is
0:35:20	at the moment when we see realised that this we're keeping references to external assets
0:35:25	but if you're gonna make it for distribution on device then we would also be
0:35:30	see realising bss themselves in the same way that we do for the network and
0:35:34	you would just have this one top rate file which you just
0:35:39	and put on the device along with the actual read
0:35:45	sort of engine itself built in three ways the i the bills with all of
0:35:49	the editing capabilities around it or it builds
0:35:53	in as a slave the sort of it as a slave so the one on
0:35:57	a device which has that network transparency or just
0:36:01	for deployment of device it doesn't need to network transparency doesn't need any day editing
0:36:05	capabilities it's quite got basically no dependencies exact G L at that point and
0:36:13	huge ship that device might plus your top rate file which has everything sort of
0:36:18	sailors together
0:36:21	so one final question have you considered a
0:36:28	a web device so i probably but it actually say this so we we're interested
0:36:35	in supporting a bunch of platforms with this and that includes web development in fact
0:36:40	so this way we have got the fold them are working but i'm the i
0:36:46	at least we want to do the proof of concept is
0:36:48	see if it's feasible and i'm kind of optimistic that is because we have the
0:36:52	rendering side working with web G L and so you know this depends on G
0:36:57	L to do to have access to the G P we trying to projects the
0:37:00	gpu the only
0:37:02	weighted we can do that in the context of the web is web geo and
0:37:05	it looks like that's gonna basically be
0:37:08	across all major browsers that is soon including i and so
0:37:13	that's
0:37:14	that's where we're looking to go
0:37:19	so you mentioned that the friendly model we should try to not a bill with
0:37:27	like you know the we that anymore so you have a specific a specific examples
0:37:33	of how we
0:37:38	so apply principles that re well probably uses to get rid of the P D
0:37:44	F really model like well run vision even this initial example but the way that
0:37:50	you build this particular scene out it's got intermediate rent is where you have to
0:37:55	first determine that if this geometry up before you're able to come up with the
0:38:01	blurring this based on that distance and the there are multiple stages involved in coming
0:38:09	up with that final is so you have to do it is a it is
0:38:12	a step where you look from the perspective of the like for instance just say
0:38:17	well from the lights perspective what geometry can i see said i can determine what
0:38:21	geometries in shadow and you know this there's a whole bunch more involved in that
0:38:29	model of rendering which is completely different
0:38:33	the postscript model of you know you just draw your background you draw other stuff
0:38:36	on top of it in the middle and you draw your labels and stuff the
0:38:40	is just a that's a completely different model you couldn't achieve
0:38:45	this kind of thing with the postscript model in real time
0:38:49	but is the other things like i'd be interested it's at least exploring some of
0:38:53	the stylised to painterly rendering or so where do we can make something look a
0:38:59	bit like it was painted with oil paints or the like it was sketch just
0:39:04	something aesthetically
0:39:08	you know interesting there is a
0:39:11	that would that would take your completely different approaches to rendering as well then even
0:39:18	what i just described there
0:39:20	and so these are the really quite far removed from that
0:39:24	painter's algorithm hands and the vocabulary of two D shapes text and images
0:39:32	is that
0:39:33	that makes sense
0:39:35	i'm thinking of something like the broadsword browser for example yes you know very complex
0:39:39	you have to support a lot of different techniques for throwing a so probably we
0:39:45	could not completely get rid of all P D F and the remote but maybe
0:39:49	some of the principles that you a scribe know could be applied
0:39:54	so probably you have some specific you know experience with their potentially be way so
0:40:00	you could
0:40:01	other ad rendering a new a rendering model capabilities within the browser but it works
0:40:11	i to be sort of
0:40:14	sandbox to some region and then you get this new model to use in a
0:40:18	certain area it would be challenging to fit the postscript model within a different model
0:40:25	in that
0:40:26	this in these in a lot of these cases you're using that to determine a
0:40:30	something is behind something else so you have to other
0:40:36	trying to fight the idea of the postscript tiring in terms of shifting things that
0:40:42	you might be able to sort of rhetoric that's on this about stuff into the
0:40:45	and
0:40:46	a some browsers which has
0:40:49	but
0:40:50	maybe we can chat afterwards of that's different is if you interested
0:41:01	go thanks
0:41:03	so

RIG: Advanced GPU visuals for UI design

User Experience Track

Robert Bragg