Redefining Digital Accessibility with Scott Davis – Techstrong Unplugged EP14

Hey everybody and welcome back Techstrong Unplugged. This is episode 14 of our series, and I'm your host, attend Solomon on Techstrong unplugged. We dive into all things tech and get people up to speed with everything happening across the industry.

Recently, our cos Cassandra Chen went to the Great International Developer Summit, also known as gis. At GIS 2024, Cassandra sat down with Scott Davis, who is an expert in digital accessibility and cutting edge technologies. Scott dives into the evolution of technology from typewriter to modern smartphones, as well as the importance of multi-sensory architecture and computing.

Finally, he emphasizes the significance of accessibility in technology. Without further ado, let's head over to GIS 2024. Welcome back to Textron Unplugged.

I'm your host, Cassandra Chin, and today we have Scott Doula's short introduction. Oh, it's so nice to be here. My name is, uh, Scott Davis.

Um, uh, i, I do a lot of work in, uh, digital accessibility and conversational UIs and, and, uh, cutting edge technologies. Can you tell me a little bit about how you first got started into technology? Yeah, it's kind of fun.

Um, my parents, um, met at IBM, and so they were, they were, um, my dad was a software engineer, and so acorns and trees, that's, that's where I'm as well. But my mom was actually, um, working with the IBM Electric typewriters and, and these typewriters back in the 1960s and seventies were some of the earliest ways to kind of automate, uh, uh, the, the IBM electric typewriters had a memory record features, so if you were constantly signing letters the same way, sincerely, Scott Davis, you could very carefully type that in once and then have just, uh, hit replay on that. I didn't know typewriters could do that.

Isn't that something? And again, that predates, uh, uh, computers. But, um, I literally grew up with computers in the house.

The first IBM PC was released in 1979. I was nine years old at the time. And so it was my dad that taught me how to build spreadsheets and how to do programming and things.

But it was my mom who had the hardware experience, so she was fearless about cracking open the computer and adding more ram or adding new drives or things like that. And so, um, there was no hope for me to end up in anything other than computers. Given my upbringing with both Ferentz, I could definitely see that I had both the hardware and software side to see.

Absolutely. Absolutely. Um, so you played with typewriters a lot.

Ever take them apart. It was really something, even something as simple as changing the font, we would just go up to a view menu or a font menu and change that. Right now, the way you would change the fonts on the IBM Electric is there was a golf ball sized head that had all the letters of the alphabet on it.

And so if you wanted to change fonts, you would pull out that metal golf ball and drop another in. So I love that with software. Um, so much of what we do in software is build, um, metaphors for real world things.

And so having a real world typewriter in my background as a child, it, it led really easily to having a keyboard saying, okay, I'm not typing on paper anymore. I'm typing on a computer. And now that leads to smartphones where we're typing on glass and oftentimes we're not typing at all, but we're using our voice, we're using gestures, we're getting haptic feedback as our phone buzzes.

It's, it's an amazing, uh, communication device beyond like the very traditional keyboard that we kind of associate with being a computer programmer. So You really got to see each step along the way of how the typewriter evolved and eventually to a phone. Oh, Absolutely.

Absolutely. I remember when the first, uh, um, iPhone came out in 2007 and the first Android phone came out in, in 2008. And it was hard calling it a phone because I don't know about you.

I rarely speak on my phone anymore, but I constantly pull up the internet or I text or I, you know, play with apps or all of those kinds of things. So it's amazing seeing that kind of change in society where in 2006, no one had a computer in their pocket. And now, um, the market penetration for smartphones is 90%, 95%.

It's something that's just become ubiquitous, um, in my lifetime. Like I'm with you that I don't actually talk on the phone anymore. Right's, like our AirPods are kind of the new phone.

It's the truth, right? But we do end up talking to our phone quite a bit. You know, you do end up saying things like, Hey, Siri, play this album.

Or, Hey, Siri, set a timer, or, Hey, Google, do this, or, Hey Alexa, do that. And so it's interesting that we're not talking to other humans anymore. We're talking to our phones through conversational UIs.

But as a programmer, um, I know that I can't just say to my phone, Hey, Siri, do you remember that one movie that was out with that one actor and that other actor? And I really loved it. And then it, you know, she's not gonna understand these kind of very open-ended vague questions.

And so when you're in conversational UIs, you kind of realize you need to talk with a purpose. And, and so understanding even the nuances of the differences between talking to a computer and talking to a human, um, you know, just make it interesting. What Do you think of talking to ai?

I think it's fascinating. Um, speech synthesis or text to speech has been around for decades. When Steve Jobs introduced the original Macintosh on stage back in 1984, he had the Macintosh speak.

He said, oh, it's so good to be out of that bag. I'm so glad to be here. And all these things.

So speech synthesis is something that's been with us for quite some time, but it's only been in the last couple of years where speech recognition has come in. And that's just a fascinating new aspect of being a computer programmer because you can't be limited in, in what you hear from, uh, a user as they're talking to you. You need to be able to say, Hey, Siri, play this album.

Or Hey, Siri, play this song. Or Hey, Siri, play this artist or this playlist, you know? And so there's a lot more nuance, um, in a conversational UI that's trying to recognize what you're saying.

'cause the human language is so rich, there's so many different ways to express yourself. The, the computer, the uh, recognition, the speech recognition has to account for all of those nuances. So Do you think AI is kind of going to fill that gap and make speech more possible with the nuances?

I really do. I really do. Um, the, um, current way we talk to computers, we have a very limited what's called bounded grammar.

They're just very few words to that that, that you can use, you know, set a timer or play a song or turn on the lights. They're very set of Feel like programming. Yeah, yeah, very programming.

And what AI is allowing us to do with generative AI is, is we do get to be more conversational. And I could say something like, oh, I was just in India and I had my Soma salada for breakfast every morning. And I just love the cultural differences of India.

What are some of the things you can tell me about India? What are some of the things that are different, um, from India than Colorado where I'm from? And so we, we can be more open in the way we talk to computers and our expectation is we'll get more open, more artistic, more fluent answers.

So that's the next generation we're coming into. It'll be exciting. Do You think we'll hit a wall where like there's still something which AI with we can't do?

So it's funny because it's not magic to me. I know that there are humans behind the scenes building these kinds of things. And so you do understand the subtle nuances and the limitations of the technology.

I don't think that we're, um, anywhere close to true intelligence. I prefer machine learning. I think that's a much better, uh, job of that.

But if I show, uh, uh, a machine, a million images of breast cancer, um, that's machine learning. And so when I start showing it new images of, of, of X-rays and it can detect cancer, um, in there and everything, that's wonderful. And I almost would prefer that level of technology then.

Hey Siri, tell me a good joke. Actually agree with you. Like, we teach the computer and we kind of get out what we expect.

Yes, yes. But the con conversationally why the speech is just another way to interact with the computers. Um, a lot of what I talk about is multisensory architecture.

And that computing now really needs to touch all of your senses. It can't be just one dimensional. And so if you can read something on the screen, can you hear it as well?

Um, if you can type something on a keyboard, can you say it as well? And so what I love is we're moving into a new era of sophistication in computing where it's not one dimensional where you have to type commands in all upper case at a command line. You know, we, we, we can bring more nuance in and I can talk to the computer when it's, uh, convenient.

If I'm making bread and my hands are covered with dough, I wanna be able to say, Hey, Siri, set a timer for 20 minutes. Um, and we're at the era now where I can, if I've got a timer going off, I can just hold out my hand and click my fingers in thin air. And so we're just discovering a whole new set of interfaces.

And once I start doing gestures in thin air and computers can recognize it, that's what makes me feel like I'm living the Future. So maybe technology is even getting to a point where we'll spend less time staring at a screen. I would hope.

I would really hope. Um, you know, uh, uh, VR goggles are, are kind of an interesting way because it's not only just you looking at the world, but it's a computer overlay of things. So I love that.

Even Yelp on the iPhone. Um, you can certainly search for individual restaurants and read reviews and things like that. Or you could just hold up your iPhone and look down the street and Yelp reviews will pop up over each one of the restaurants and you can say, oh wow, that restaurant is so close to me and it's got a bunch of four star reviews and everything.

That's a lot different than typing in search criteria into a computer system, you know? Um, it's holding up a camera and, and looking at my surroundings and understanding the context and reacting to that, that's what I think is gonna be really exciting about this next generation of computing. Yeah.

'cause right now we kind of trained ourselves to learn how to read a map. Yeah. Hopefully We can kind of like connect the digital and real world more.

Exactly. Exactly. And like Google Maps, um, came out in 2004.

I had a, I was working on Google Maps originally, um, not with Google, but for a satellite imagery provider. And it's amazing to think that I used to drive without a GPS. I mean, it's amazing to think that, uh, as I traveled from city to city, I'd have to do research and I'd have to buy paper maps and I'd have to plan my route from the airport to the hotel and the hotel to the conference venue and things.

And now I can just kind of show up in a city unprepared and I can almost say, Hey Siri, how do I get to my hotel? Um, so excuse me. But yeah, it's um, it's just uh, interesting how computing is getting more sophisticated, um, which allows me to be less sophisticated as I interact with computers.

I Can see that. Do you wanna talk a little bit about the accessibility in your talk? Yes, yes.

So I'm giving a keynote tomorrow Digital modernization through accessibility. And so many times people look at accessibility and they even say the word accessibility, but they hear the word disability. I could definitely see that, right?

I had a keynote that I gave years ago called, it's spelled accessibility, not disability. I really tried to address that upfront. But if we think about all the ways we're interacting with computers now as I'm talking to a computer, if I had limited mobility, that might be the primary way I talk to a computer.

Or if my speech is limited or unintelligible, I might have to, to type with a computer, I might have to begin doing these kinds of gestures. So I take a very yes and approach to digital accessibility. Yes, it's for our blind and low vision users.

Yes, it's for our deaf and heart of hearing users. These, yes, and 25% of the world has some form of disability. This is not a small market segment.

If you have a disability, any disability, you're a member of the largest minority in the world. There are over a billion people worldwide that have a disability. So imagine a country the size of China, imagine a country the size of India and it gives you an idea of the number of people with disabilities in the world.

But while this technology is important for people who have limitations in any one of their senses or a combination of their senses, what I love is you and I are using these same senses to interact with our phone. So as I said, when I'm cooking, I love making sourdough. If my hands are covered with dough, I want to use my voice.

If um, I've got alarm go off, I want to tap in mid air, sometimes it really is easier just to tap on my watch and start a timer again. So what I want is the ability to use all my senses and for me to be able to choose. And I think for people with disabilities, that's all they want as well, is to have a multisensory approach to how they interact with the computers and not be limited to any one's sensor one, one input modality.

So you think like maybe more companies would build applications which use lots of senses if they sold it for convenience and it would also happen to fulfill the needs for the dis dis accessibility? Yeah. Yeah.

So I was working for a client and they were getting ready to roll out a new, um, catalog of 20,000 items that they were trying to sell online. And I said, well, we need to have alt text on each one of those images. We need to have words that describe what's in this image.

So if you're selling a baseball hat with a green tractor on it, you need to have alt text on there that says a baseball hat with a green tractor on it. Um, and so for people who have blindness or low vision, they won't be able to see the image, but they'll be able to hear the alt text and be able to make an informed purchasing decision. So this is the yes and approach to accessibility.

Yes, we need alt text for all of our blind and low vision users and we need alt text for the rest of the users as well. Because how do you think someone is going to search for a baseball hat with a green tractor on it? They're gonna go to a search engine and type in those words baseball hat with a green tractor on it.

So all of a sudden, accessibility is also related to search engine optimization. It's related to your sales goals and your marketing goals. And so what I really try to talk about with my clients is we don't need to thin slice accessibility.

We don't need to narrow our focus on that and put it to the side. What we need to do is realize that all of your customers have a variety of senses and they want to use all of their senses. And so yes, accessibility is so important for that 25% of your customer base that has a disability and it's important for the other 75% as well.

I think This is so important and I'm really happy that you're going to reach out to so many people. Oh, thank you so much. It's something that's just so near and dear to my heart.

It's just, uh, it's a joy to talk about. It's been really lovely talking to you, Scott. Thank you so much.

It was a pleasure meeting you. Thank you.