| 0:00:13 | but i well they |
|---|
| 0:00:14 | first off thirty is my to than would to silver of a |
|---|
| 0:00:17 | and the other authors have from johns hopkins jobs at kings into more S |
|---|
| 0:00:21 | and so the idea is that we use hearing and that's to convey a graphical and pictorial information and so |
|---|
| 0:00:28 | i'm |
|---|
| 0:00:28 | so the in that X and here is |
|---|
| 0:00:30 | i it's more of a translation between a a video and an audio |
|---|
| 0:00:34 | and and what using that is the pointing device you'll see |
|---|
| 0:00:38 | so the idea is to to be able to do the display just display information or maybe eventually use it |
|---|
| 0:00:44 | for navigation |
|---|
| 0:00:45 | a a and and a key thing is that the user that explores with the finger |
|---|
| 0:00:50 | a if usually a |
|---|
| 0:00:51 | and of course i i mean if you want to to give visual information to people were visually impaired our |
|---|
| 0:00:57 | main target is visually impaired |
|---|
| 0:00:59 | i mean there are really a invasive procedures like visual usual T is you you in since F the cheap |
|---|
| 0:01:04 | the in that a team at that |
|---|
| 0:01:06 | no quite quite invasive |
|---|
| 0:01:08 | that have less invasive but still |
|---|
| 0:01:10 | not not too pleasant approach is like the time display view for example |
|---|
| 0:01:15 | where you in of um it on the on the time can and that |
|---|
| 0:01:18 | stimulates the time according some visual input |
|---|
| 0:01:21 | okay and and there have the remotes for doing this like a line drawings for example |
|---|
| 0:01:26 | a a and and the that been some side is on those |
|---|
| 0:01:29 | of course i mean a a our results can also benefits i the people in cases where |
|---|
| 0:01:33 | that is not like i mean file fighter operations and smoke military missions at night and so |
|---|
| 0:01:39 | so a little bit of you study how we arrived that these |
|---|
| 0:01:43 | i i i i one of our goals "'cause" been to i mean he's been to press load visual to |
|---|
| 0:01:47 | tactilely me silly |
|---|
| 0:01:49 | a so we have any mud uh we segment |
|---|
| 0:01:57 | we it we have any much which to segment |
|---|
| 0:02:00 | and and then we present segment by different fact that X two |
|---|
| 0:02:03 | and and it's important to have perceptually distinct act i'll text |
|---|
| 0:02:07 | so one thought wise and several people have suggested this |
|---|
| 0:02:10 | that to enhance the differences between the tactile textures you may want to large sums so different sounds |
|---|
| 0:02:16 | okay |
|---|
| 0:02:17 | a a and that would work very well if you have a display that can both modify the self us |
|---|
| 0:02:22 | and and and |
|---|
| 0:02:23 | a it i i acoustic feedback is is out of that in the set |
|---|
| 0:02:28 | but but existing displays is the you cannot do that i mean we have a i pods and and and |
|---|
| 0:02:32 | i thought is and also of |
|---|
| 0:02:34 | devices |
|---|
| 0:02:35 | and |
|---|
| 0:02:35 | it that it that let okay |
|---|
| 0:02:37 | a a that they can at it's point with the to that's but they can change the sense for so |
|---|
| 0:02:42 | we thought it would be interesting to do or the fact that display |
|---|
| 0:02:46 | used that's |
|---|
| 0:02:47 | just as the pointing device |
|---|
| 0:02:49 | and and and this and use the i was the display |
|---|
| 0:02:52 | so the idea is a very simple idea |
|---|
| 0:02:54 | we have a i mean we simplified very simple shapes |
|---|
| 0:02:57 | okay and again as you explore with your finger |
|---|
| 0:03:00 | you hear a different sounds is you is you moving to different it saves and you can that you can |
|---|
| 0:03:05 | be you can try to to to to to identify the saves as you pray |
|---|
| 0:03:09 | okay |
|---|
| 0:03:10 | so |
|---|
| 0:03:11 | a and of course uh it's you're moving or fink get and you hearing the audio your and how you |
|---|
| 0:03:15 | moving a finger is what really gives you the information so that's they can can a static feed |
|---|
| 0:03:21 | so i'm just to give you an idea |
|---|
| 0:03:24 | it's or i one one in one of our configurations |
|---|
| 0:03:27 | you have a sound for the background so |
|---|
| 0:03:30 | wait a you |
|---|
| 0:03:32 | getting |
|---|
| 0:03:35 | yeah and and they have a complete the it |
|---|
| 0:03:37 | this thing it |
|---|
| 0:03:38 | a a sound for for the |
|---|
| 0:03:40 | inside you |
|---|
| 0:03:41 | so as you move your get you hear the different sounds and you form an opinion of quite are the |
|---|
| 0:03:46 | object |
|---|
| 0:03:47 | now |
|---|
| 0:03:48 | a a |
|---|
| 0:03:49 | we we chose to to have one sound inside one sound outside |
|---|
| 0:03:53 | because uh there is evidence in the literature that it's easier to |
|---|
| 0:03:57 | i mean i can from from i from uh back that perception |
|---|
| 0:04:01 | that a days a raised line drawing is more difficult to perceive then you've you have a strong |
|---|
| 0:04:06 | object |
|---|
| 0:04:08 | so we tried to do that |
|---|
| 0:04:09 | the same in sound |
|---|
| 0:04:11 | but what we found easy it's important for people to identify the object when that inside the object but they |
|---|
| 0:04:16 | also tried to trace the board |
|---|
| 0:04:18 | so probably we need |
|---|
| 0:04:19 | we conclude that the probably we need a combination of both |
|---|
| 0:04:22 | so then we tried another configuration what do we have one sound inside one outside |
|---|
| 0:04:27 | and the distinct sound for that for the |
|---|
| 0:04:29 | for the ball |
|---|
| 0:04:31 | is the same sounds here |
|---|
| 0:04:33 | okay |
|---|
| 0:04:34 | but uh and N is an inside the the other |
|---|
| 0:04:37 | but |
|---|
| 0:04:38 | but hope |
|---|
| 0:04:39 | well i have to |
|---|
| 0:04:40 | but then you have a very different sound for the background |
|---|
| 0:04:44 | for the |
|---|
| 0:04:45 | for the board |
|---|
| 0:04:46 | and then the |
|---|
| 0:04:47 | sound sound that you have before four |
|---|
| 0:04:50 | in P |
|---|
| 0:04:50 | so we have three |
|---|
| 0:04:52 | this thing cost on |
|---|
| 0:04:55 | no |
|---|
| 0:04:55 | that is a problem with the racing the the the the bound E |
|---|
| 0:04:59 | in that if you make the bound that it to not your fink it can sleep outside and it's it's |
|---|
| 0:05:03 | kind of annoying |
|---|
| 0:05:04 | going on and off the bound E |
|---|
| 0:05:06 | if you make it to broad |
|---|
| 0:05:07 | then then you don't necessarily for the so you might fall another project and you may get that own perception |
|---|
| 0:05:13 | about that the pair that the the bond it looks like |
|---|
| 0:05:16 | so we thought that we need that some proximity feedback |
|---|
| 0:05:19 | inside the board there |
|---|
| 0:05:20 | okay and for that we try to use them as melissa |
|---|
| 0:05:24 | so that i'm only describe |
|---|
| 0:05:25 | is a low frequency a it this guy by that and rate but i at that and we use those |
|---|
| 0:05:30 | to to define the object |
|---|
| 0:05:32 | and then we say the carrier frequency |
|---|
| 0:05:34 | to so it |
|---|
| 0:05:36 | how half we out of a proximity hundred a new out of the bar |
|---|
| 0:05:39 | so just so you with the picture |
|---|
| 0:05:41 | okay so |
|---|
| 0:05:42 | before a like the more detail |
|---|
| 0:05:44 | again that at |
|---|
| 0:05:46 | two major segments but it's segment "'cause" the board is done and yellow is the is the board that between |
|---|
| 0:05:51 | the inside and the outside |
|---|
| 0:05:52 | and and |
|---|
| 0:05:54 | uh the |
|---|
| 0:05:55 | the F R it is constantly image segment but increases as the fink that our approach is the bank E |
|---|
| 0:06:00 | and here is how it very |
|---|
| 0:06:02 | so we have |
|---|
| 0:06:03 | one sound |
|---|
| 0:06:04 | for the inside |
|---|
| 0:06:07 | okay |
|---|
| 0:06:07 | and another sound |
|---|
| 0:06:09 | for the outside |
|---|
| 0:06:11 | can he this |
|---|
| 0:06:13 | and then we have a different sound at the border there |
|---|
| 0:06:16 | for example like these and they five play then type thing |
|---|
| 0:06:20 | it's it's constant the the F R and then ink is this one you are on the board there |
|---|
| 0:06:26 | and then |
|---|
| 0:06:27 | decreases and then eventually been one and big now there was a problem with |
|---|
| 0:06:35 | the problem is in principle it's fine |
|---|
| 0:06:38 | but in practice the inside and outside |
|---|
| 0:06:41 | sounds are not very easy to define say are not is perceptually distinct is the other signals it i |
|---|
| 0:06:46 | and also that i a lot eight depending on how you move your finger made sense to fast |
|---|
| 0:06:51 | in the small area to be not simple so this didn't quite work very well |
|---|
| 0:06:56 | well |
|---|
| 0:06:57 | the next step was to use the X and also |
|---|
| 0:07:00 | so we wanted to use a a directional acoustic feedback in the back for example when your finger is down |
|---|
| 0:07:05 | here |
|---|
| 0:07:06 | the sound you can hear the sound coming from a and in as with that direction |
|---|
| 0:07:11 | and also when you train is the but is the bound T this sound guys you to move in a |
|---|
| 0:07:16 | in a direction to for of the the bond but in a wise five |
|---|
| 0:07:21 | now to do these okay we had again three segments |
|---|
| 0:07:25 | a a background and |
|---|
| 0:07:27 | foreground and |
|---|
| 0:07:28 | and the board there |
|---|
| 0:07:30 | but now um |
|---|
| 0:07:31 | the inside the still constant but the outside is the one that was the tax from now |
|---|
| 0:07:36 | so |
|---|
| 0:07:37 | i don't know if you should be these from one side |
|---|
| 0:07:40 | i O |
|---|
| 0:07:42 | i |
|---|
| 0:07:43 | right |
|---|
| 0:07:43 | yeah |
|---|
| 0:07:44 | and and then when you are on the board the you he of the board the sound but again in |
|---|
| 0:07:49 | the direction that you're supposed to move |
|---|
| 0:07:51 | okay and and |
|---|
| 0:07:53 | we |
|---|
| 0:07:54 | to |
|---|
| 0:07:55 | in in a |
|---|
| 0:07:57 | practical suggesting you if if phones |
|---|
| 0:07:59 | and the sound comes from that a |
|---|
| 0:08:02 | right side |
|---|
| 0:08:03 | and we i wanted to a realistically simulate more late |
|---|
| 0:08:06 | the idea of the thing of if your it is over there |
|---|
| 0:08:09 | to have the sound coming from the object |
|---|
| 0:08:12 | or if you are here to be coming from the object then is you approach the object the sound becomes |
|---|
| 0:08:17 | a loud that and we wanted to simulate at least E |
|---|
| 0:08:20 | gays well |
|---|
| 0:08:21 | there is a sound source in maybe in the coordinate of the object |
|---|
| 0:08:24 | and there's your finger approaches |
|---|
| 0:08:25 | you hear a quite you would have to you were standing there and listening to be lot |
|---|
| 0:08:31 | okay case of for this |
|---|
| 0:08:32 | the mathematical formulation that you need for this is what is called the head related transfer function |
|---|
| 0:08:36 | to to of the directionality and then you play back was that okay |
|---|
| 0:08:41 | okay now again a is a said |
|---|
| 0:08:44 | uh the list and there is assumed to be the pose just on your fingers so so you out of |
|---|
| 0:08:48 | it early that is |
|---|
| 0:08:50 | on on the screen |
|---|
| 0:08:51 | okay and then you hear the sounds coming from that property direction |
|---|
| 0:08:55 | okay again if you in the background from close just point the object |
|---|
| 0:08:59 | and when you're on the board from the direction in which you need to move |
|---|
| 0:09:04 | finally we i i i mean this i i and define an object eight |
|---|
| 0:09:08 | but are also interested in an overall layout |
|---|
| 0:09:11 | and and if you if you to test this idea |
|---|
| 0:09:14 | is i mean if we have a very simple layout but you can imagine you are standing at the window |
|---|
| 0:09:19 | and you are looking at the buildings across the state |
|---|
| 0:09:22 | and and two |
|---|
| 0:09:23 | but was from the from the blind |
|---|
| 0:09:25 | but examples |
|---|
| 0:09:26 | i mean typically a blind person like have a a and and and try to explore than about "'em" in |
|---|
| 0:09:31 | front of them |
|---|
| 0:09:32 | but might think these being the features stick that you that at the objects on on on the |
|---|
| 0:09:36 | you know a across the study |
|---|
| 0:09:38 | and and and you you explore what's there |
|---|
| 0:09:40 | okay |
|---|
| 0:09:41 | so if this is a three let's say in a very simply far you make have sound |
|---|
| 0:09:46 | of wood |
|---|
| 0:09:49 | and and this may be a glass house |
|---|
| 0:09:53 | a so you may have another sound |
|---|
| 0:09:55 | a a a and this maybe |
|---|
| 0:09:58 | oh sorry this was |
|---|
| 0:10:00 | a one is met that and is defined as a |
|---|
| 0:10:03 | yeah a few times |
|---|
| 0:10:04 | okay so so so now the sound here |
|---|
| 0:10:07 | conveys is the something about the object |
|---|
| 0:10:10 | and also you get an idea of the layout so we not |
|---|
| 0:10:13 | in in that in the same here if you need to explore a shape maybe you have another remote what |
|---|
| 0:10:17 | used mean and and you are like what we discussed before |
|---|
| 0:10:20 | but this can give you a nice idea of what the layout days |
|---|
| 0:10:23 | okay and you don't have to be realistic with the sounds even though you are a scanning with your fingers |
|---|
| 0:10:30 | i mean we can |
|---|
| 0:10:31 | the the egg |
|---|
| 0:10:32 | pretend that you are tapping with with a with a steak |
|---|
| 0:10:35 | and and and you can have very testing so it's quite is if you have a different object to you |
|---|
| 0:10:39 | think it maybe you'll get |
|---|
| 0:10:40 | almost the same sound or no sound the all so we don't have to be really |
|---|
| 0:10:45 | okay |
|---|
| 0:10:46 | now a few words about about that experiments |
|---|
| 0:10:49 | we use the subjects in the five configurations |
|---|
| 0:10:52 | and i i mean i have the details if your of and to a that in the paper if you |
|---|
| 0:10:55 | want to |
|---|
| 0:10:56 | and to to look them map we use the and and i but that screen |
|---|
| 0:11:00 | and for our for out of the most a sounds and i have one with me |
|---|
| 0:11:04 | if you want to play with it after the the session |
|---|
| 0:11:07 | and and use the is that or headphones |
|---|
| 0:11:10 | and |
|---|
| 0:11:12 | now it was very important |
|---|
| 0:11:14 | how would and the experiment |
|---|
| 0:11:16 | so first to gave taxes to the subjects but we give them no information about what objects that are going |
|---|
| 0:11:21 | to be looking at |
|---|
| 0:11:22 | and that actually even during the experiment |
|---|
| 0:11:25 | if they were guessing one object to really did not make it clear that the same object my a appeal |
|---|
| 0:11:29 | again so the at you did not have any idea of what that exploring the only thing maybe be that |
|---|
| 0:11:33 | that are let at a relatively simple |
|---|
| 0:11:36 | and we use the training example to to make them |
|---|
| 0:11:38 | am |
|---|
| 0:11:39 | can for the these |
|---|
| 0:11:41 | a we did not want them to be blindfolded okay well actually you would say i have a screen that |
|---|
| 0:11:46 | "'cause" nothing when need but even if you watch your finger that's visual feedback so one to eliminate the visual |
|---|
| 0:11:51 | feedback because out assuming a case what you can not see |
|---|
| 0:11:55 | so so what do you D we put them |
|---|
| 0:11:57 | that that screen and that of box so they could not see that that screen or the thing |
|---|
| 0:12:02 | but you didn't want to i mean typical i guess in in in this type of experiment |
|---|
| 0:12:06 | you don't want to blind for people because that's not a very comfortable thing so that the most not sort |
|---|
| 0:12:10 | of thing is to to have the the |
|---|
| 0:12:12 | the ipod pod and the and that of box and then you have eye contact the experiment there his i |
|---|
| 0:12:17 | contact with with a subject |
|---|
| 0:12:19 | and then they try to explore |
|---|
| 0:12:20 | but to explain to them what's happening first they tried it by looking at the shape |
|---|
| 0:12:25 | and and and X and the fink get an exploding |
|---|
| 0:12:28 | then we turn off the the the visual |
|---|
| 0:12:31 | signal day |
|---|
| 0:12:32 | the triangle or very |
|---|
| 0:12:34 | the cost |
|---|
| 0:12:35 | this case |
|---|
| 0:12:35 | okay and and |
|---|
| 0:12:37 | they could see only the thing get and finally they went to that is situation for they could not see |
|---|
| 0:12:42 | the finger or or or or |
|---|
| 0:12:45 | okay of course we use the different objects for training then for for the testing |
|---|
| 0:12:50 | actually use the cost for the |
|---|
| 0:12:52 | for the training and and it's quite a set in the triangle for |
|---|
| 0:12:57 | for the testing |
|---|
| 0:12:59 | okay and |
|---|
| 0:13:00 | and and you know some details about the fourth configuration basically what kind of a that the you have we |
|---|
| 0:13:05 | use |
|---|
| 0:13:06 | uh and and we had |
|---|
| 0:13:09 | i mean a date that you have it's important to much to that to the specific characteristics of of the |
|---|
| 0:13:14 | user |
|---|
| 0:13:15 | and we had the very simple configuration of whether are two plus is and you could calibrate one of the |
|---|
| 0:13:20 | other but of course in in a critical situation you might have an individual it's to you have |
|---|
| 0:13:25 | for the particular user and that is expected to work but |
|---|
| 0:13:31 | so |
|---|
| 0:13:31 | now the end of the experiment we did not immediately ask them to tell us what's a these |
|---|
| 0:13:36 | but again lead from that from the literature |
|---|
| 0:13:39 | a a i and and and the work of winces is that the that of a uh in a well |
|---|
| 0:13:44 | i i actually |
|---|
| 0:13:46 | is is done it do to his work |
|---|
| 0:13:47 | and |
|---|
| 0:13:49 | it |
|---|
| 0:13:50 | we asked them to first the this shape |
|---|
| 0:13:53 | okay with with a pen on on on the on and |
|---|
| 0:13:56 | one on paper |
|---|
| 0:13:57 | and then ask them what do you think this a piece |
|---|
| 0:14:00 | and that |
|---|
| 0:14:01 | that what but there are much better for them okay because it it helps you a realise what you have |
|---|
| 0:14:07 | five |
|---|
| 0:14:08 | okay |
|---|
| 0:14:09 | so again we did not give that the subject any feedback |
|---|
| 0:14:13 | i i i do you all the experiments |
|---|
| 0:14:16 | i Q know that is out |
|---|
| 0:14:18 | okay in the first configuration the simple of fig iteration with |
|---|
| 0:14:21 | a object and background we got |
|---|
| 0:14:23 | sixty seven percent i |
|---|
| 0:14:25 | in the second configuration of we had that the ball there we got seventy seven percent so there is significant |
|---|
| 0:14:30 | improvement |
|---|
| 0:14:31 | that i a low |
|---|
| 0:14:33 | really didn't help we what that the back to |
|---|
| 0:14:36 | to low where a rate and finally detect X sound good the big impact |
|---|
| 0:14:40 | in can is to to if three press |
|---|
| 0:14:42 | the cats of course sees |
|---|
| 0:14:43 | he i it wasn't |
|---|
| 0:14:45 | if very immediate affect it took like three to four me needs |
|---|
| 0:14:49 | to be able to identify say |
|---|
| 0:14:51 | but our whole please |
|---|
| 0:14:52 | that these these subjects were not playing |
|---|
| 0:14:55 | and and is you train sound it's especially visually impaired subjects who depend critically on on doing these |
|---|
| 0:15:00 | that that will become much much five |
|---|
| 0:15:03 | okay again i well as |
|---|
| 0:15:05 | what can you a sound only |
|---|
| 0:15:06 | all obviously you can you can combine with with that |
|---|
| 0:15:10 | a that that signals when the device that are available and then you can improve more |
|---|
| 0:15:15 | but but but i think it's of very it "'em" see what that of the means of |
|---|
| 0:15:18 | i one sound is |
|---|
| 0:15:21 | finally for they seem layout |
|---|
| 0:15:24 | these these what to use this a training seen of this as the actual testing scene |
|---|
| 0:15:27 | and we have the subjects six indicate that a this was used sounds |
|---|
| 0:15:31 | this sage but the couldn't tell much about the as you "'cause" there not this |
|---|
| 0:15:35 | in know detailed is the individual ones |
|---|
| 0:15:37 | and the material that there are made |
|---|
| 0:15:40 | and that were quite well |
|---|
| 0:15:42 | they were able to identify the materials |
|---|
| 0:15:44 | quite quite uh |
|---|
| 0:15:46 | reliably there was some confusion between glass and metal bell and met the link last |
|---|
| 0:15:50 | okay |
|---|
| 0:15:51 | and and |
|---|
| 0:15:52 | and of course is as is like a first i if you do you record of the sounds more get |
|---|
| 0:15:56 | for your big different sound you can do even but |
|---|
| 0:16:00 | so |
|---|
| 0:16:01 | we present an approach for conveying graphical a pictorial information by hearing and that |
|---|
| 0:16:06 | and the subjects were able to identify |
|---|
| 0:16:09 | basic geometric shapes |
|---|
| 0:16:11 | okay |
|---|
| 0:16:12 | and and and compare to other approaches that didn't go into details of existing approaches a had to so significantly |
|---|
| 0:16:19 | better performance |
|---|
| 0:16:20 | and we also tried if its all came approach square |
|---|
| 0:16:23 | it's a basic simple steps and then subjects that it able to locate and identify all object |
|---|
| 0:16:28 | and again but that's sell applications out of my map perception and navigation |
|---|
| 0:16:33 | and of course |
|---|
| 0:16:34 | yeah at goal is |
|---|
| 0:16:36 | to combine with that's |
|---|
| 0:16:37 | and and find and and use the best of its modality |
|---|
| 0:16:40 | to to do the best and conveying the visual information |
|---|
| 0:16:43 | thank you very much |
|---|
| 0:16:51 | i from X or to start with a weird rubber and the now we can have some questions for houses |
|---|
| 0:16:58 | hmmm you are are and the where some former audience |
|---|
| 0:17:01 | stop |
|---|