Season 8 Episode 9 Jun 9, 2022

Diablo Immortal and Video Game Accessibility, The Challenges of Creating an AR System, The Recent Wave of Tech Layoffs, and More


At least they got the accessibility right.


This week we’re talking about the recent wave of tech layoffs, and React Eurasia’s controversial decision to locate this year's conference in Dubai. Then we speak with Steven Spohn, senior director of development at The AbleGamers Charity Foundation about Diablo Immortal’s native voice chat transcription and speech-to-text accessibility feature, and where we are in terms of accessible design in the video game industry as a whole. Finally, we speak with Jeri Ellsworth, co-founder and CEO of Tilt Five about what some of the inherent challenges are with creating AR devices that might be making it difficult for both Apple and Google to launch the glasses they’ve been talking about for years.


Saron Yitbarek

Disco - Founder

Saron Yitbarek is the founder of Disco, host of the CodeNewbie podcast, and co-host of the base.cs podcast.

Josh Puetz

Forem - Principal Engineer

Josh Puetz is Principal Software Engineer at Forem.


Steven Spohn

The AbleGamers Charity - Senior Director of Development

Steven Spohn is an expert in the field of technology and disability.

Jeri Ellsworth

Tilt Five - Co-founder & CEO

Jeri Ellsworth's love for invention began with building race cars before working with hardware design, creating a complete Commodore 64 system on a chip housed within a joystick, called C64 Direct-to-TV. In 2011, she was hired by Valve Software to build and run their R&D team and was a key contributor to the technology used in the popular HTC Vive virtual reality headset. While at Valve, she began work on the AR technology that would become the basis for the technology behind Tilt Five. Her vision for the future of gaming is driving innovation for the entire Tilt Five team.

Show Notes

Audio file size





[00:00:10] SY: Welcome to DevNews, the news show for developers by developers, where we cover the latest in the world of tech. I’m Saron Yitbarek, Founder of Disco.


[00:00:19] JP: And I’m Josh Puetz, Principal Engineer at Forem.


[00:00:21] SY: This week, we’re talking about the recent wave of tech layoffs and ReactEurasia’s controversial decision to locate this year’s conference in Dubai.


[00:00:29] JP: Then we’ll speak with Steven Spohn, Senior Director of Development at The AbleGamers Charity Foundation, about Diablo Immortal’s native voice chat transcription and speech-to-text accessibility features and where we are in terms of accessible design in the video game industry as a whole.


[00:00:43] SS: So nobody questions and says, “Why should a blind person be able to read a webpage?” Well, they should because that’s the right thing to do. Right? But people go, “Well, why would a blind person want to play a video game?”


[00:00:54] SY: Then we’ll speak with Jeri Ellsworth, Co-Founder and CEO of Tilt Five, about what some of the inherent challenges are with creating AR devices that might be making it difficult for both Apple and Google to launch the glasses they’ve been talking about for years.


[00:01:08] JE: It was a lot of tradeoffs. I mean, there were times where we had to give up our dreams of the way it should work because we learned there was a better way.


[00:01:27] SY: So we’re starting this episode off with a move by ReactEurasia that has inflamed Tech Twitter this week. The organization made an announcement that they would be hosting this year’s tech conference in Dubai, which many developers and other tech folks were quick to point out might be a poor choice for location for anyone in the LGBTQ community, given the United Arab Emirates’ problematic and often horrifying history regarding LGBTQ rights. Oh, and the irony of ReactEurasia making this announcement during Pride Month, of all months, was just the cherry on top for a lot of developers. Kat Cosgrove, Staff Developer Advocate at Pulumi Corporation, tweeted, “Happy pride to everyone except the huge ReactJS conference moving to a country where queers and women are regularly imprisoned or killed just for existing.” Brandon Dail, Senior Software Engineer at Discord, wrote, “Dubai is an ongoing human rights nightmare. Attempting to whitewash that is horrendous. Many of the most amazing people in the ReactJS community are categorically excluded by this location choice at the risk of death. This conference will never represent the community.” And then we have Emily Freeman, Head of Community Engagement at AWS, who said, “If you hold a conference in a place where people attending that conference are at risk due to local laws, I will not be there. It’s just that simple.” What is your take on this? Obviously, a bad decision.


[00:02:51] JP: Happy Pride Month!


[00:02:54] SY: Happy Pride!


[00:02:55] JP: Couldn’t wait until July 1st, because I had to do it this month. Huh? Look, I get it. I think it’s been a really hard couple of years for conference organizers. In a lot of cases, I know for my friends at organized conferences, a lot of times you have to sign the contract, getting the venue years in advance and COVID really screwed that up.


[00:03:16] SY: Oh, I’m sure. Yeah.


[00:03:16] JP: And you add onto that. I know a lot of conference organizers are struggling with where in the US to have their conferences. They want to avoid certain states that have progressive or unfriendly politics, like for example, conferences that want to support women’s rights are avoiding Southern states that are passing laws against abortion. And they might have to still have one conference there because they signed the contract like four or five years ago. But on the other hand, it’s not like Dubai just started being troublesome for the LGBT community. And secondly, the LGBTQ community has a lot of representation in JavaScript. It’s a really great diverse community. And it’s a really gross oversight. I don’t know if the conference organizers will change the venue.


[00:04:07] SY: Can they at this point?


[00:04:08] JP: I would hope they would. But yeah, that’s the thing. One of these points, when you’ve signed a contract for a conference, you’re kind of locked in.


[00:04:14] SY: Yeah.


[00:04:15] JP: Let me ask you, what do you think, how could ReactJS, Eurasia make this better? What would you like to see them do?


[00:04:22] SY: Okay. So part of it is what can they afford to do, right?


[00:04:26] JP: Oh, fair. Yeah.


[00:04:27] SY: I mean, the thing that people may not appreciate about conferences, especially ones that are not funded by a FANG company, right? It’s not a Microsoft build, it’s not one of those conferences, is that they’re very expensive and you need to figure out how to pay for them. And usually, the ideal situation is that tickets cover the cost and sponsorship is where you’re able to actually make a little bit of money out of it, depending on how popular the conference is, how good your sales skills are, et cetera, et cetera. So with ReactEurasia, I don’t know where they sit financially in terms of what they’re able to afford to do, but if you can’t get out of your contract and if you have to pay for an entirely new venue, I mean, venue is usually one of the biggest expenses, period. It’s tons and tons of money. The bigger the conference, the more ridiculous the prices are for venue. So I don’t really know if they can afford to move it. I mean, I guess they could just straight cancel it, which…


[00:05:20] JP: Then that would just like bankrupt them, right? Yeah.


[00:05:21] JS: Again, exactly, it’s another financial decision. Can you afford to just sink the money essentially? I don’t know if they can afford to do that. So I feel like the best realistic option for them given the financial issues that I’m assuming is probably to make a donation. Obviously, a public apology would be a good start, but something along the lines of, “This was an oversight. We didn’t appreciate this point or didn’t think about.” I don’t know exactly what happened. That led to this, but something that says, “We’re sorry, especially to our LGBTQ React community members. You can’t really make up for it, but as a way to try and address it, we’re going to donate X percentage of our proceeds to an LGBTQ fund or nonprofit.” That’s probably their best realistic option at this point.


[00:06:16] JP: I think you’re right. Apology from the conference organizers. Donation is a fantastic suggestion and then maybe just some acknowledgement that they are going to try to do better the next time they pick a venue.


[00:06:30] SY: But it also kind of makes me wonder, how did this happen?


[00:06:34] JP: I guess this speaks to the importance of having diversity and representation on your board and in your organization. And I have to suspect that maybe there’s not a lot of diversity at ReactConf Eurasia’s board or conference committee or whatever. You think they would’ve caught this, right?


[00:06:50] SY: Oh yeah, for sure.


[00:06:52] JP: Well, this next bit of news has probably already been felt by many of you in some way or another. In the month of May, more than 15,000 tech workers lost their jobs according to There’s also been a wave of hiring freezes across the industry that include places like Meta, Twitter and Coinbase. This quick pendulum swing is a shocking contrast with the great resignation and then hiring bonanza that the industry has experienced recently. There’s a really great piece in Forbes, which we’ll put in our show notes, that deconstructs this economic moment. The piece titled “The Split-Screen Job Market: Low Unemployment, High Turnover, Plenty of Openings–And More Layoff Headlines” talks about how unintuitive the current narrative is given a lot of data about our economy and tells a tale of two job markets. The piece goes into how earlier this month we had better than expected job numbers where we continue to add jobs and unemployment was low. And the labor department even reported that hiring demand was strong. So why all the layoffs and freezes? The article points to high interest rates, the pandemic and our current geopolitical turmoil, like the war in Ukraine, as factors that have made investors slow down their funding. But you would think this would only affect venture-backed companies and not necessarily explain bigger companies like Tesla wanting to cut jobs and freeze hiring. There are also things like our economy's rising inflation that could be contributing to a feeling of uncertainty and concern about the economy, but I’m not so sure that’s the whole story. Saron, you’re a business owner in the tech industry and you’re someone that is involved in the venture capital game. What’s your take?


[00:08:23] SY: A game. It is a game. That is a discouragement.


[00:08:25] JP: It is a game. What is your take on what is happening with our economy and tech in general?


[00:08:32] SY: Okay. So here’s the thing. First of all, what I’ve learned is that the VC community is very raw, but also very small at the same time where people follow each other.


[00:08:42] JP: Ah, groupthink.


[00:08:43] SY: A hundred percent groupthink. That to me was probably the most surprising thing that I learned about just working with VCs talking to literally like over a hundred VCs in the last couple years and reading their blog posts, their Twitter, newsletter, whatever, just being in that world is just how group bank is very, very prevalent. But there’s also this, especially for the smaller venture capital firms and the individual investors, there’s also this need to prove that they are different. They're thought leaders. They want to kind of push back against the tide, but in a very marketing kind of way, if that makes sense.


[00:09:27] JP: Like it’s their brand to go against the zeitgeist?


[00:09:30] SY: Yeah. And I remember reading, I think it was like some blog post where it was basically commenting on that fact and commenting on how a lot of VCs want to use this opportunity to kind of prove that they are tough on companies and that they take business seriously, they take profitability seriously, and they’re serious investors and they care about the fundamentals. I don’t necessarily think it’s disingenuous, but I do think that this downturn is an opportunity to kind of buckle down and show how serious they are as business people. And so part of me, frankly, is trying to figure out what is real and what is not, like how do you really feel versus what are you marketing to me? I’ve had a couple investors send out private emails, some of which they ended up actually publishing publicly because people asked for it where they’re very reassuring to their individual entrepreneurs. They’ve said, “Don’t worry. Wherever you are now we’ll support you, we’ll help you kind of figure things out,” et cetera. I’ve seen other VCs send more warning emails of, “We don’t know how long this is going to last. Just assume the worst. Whatever money you’ve got, hold onto it. You don’t know when you’re going to be able to raise and vary…” and personally, I appreciate that because I’d rather plan for the worst and hope for the best than be caught screwed in the end. And so I think there’s that element of it. And then I’ve also heard one particular venture capitalists say that she doesn’t believe that the stock market trends of these public companies, huge companies really correlates with true startups, startups meaning super early stage precede, kind of literally figuring out what your product is level startup and that these two economies are not actually related. And so for her, she’s like, “Just ignore it. Just ignore what the stock market stuff is doing. Don’t get caught up in the headline.” She knows she’s talking to early stage founders, of course.


[00:11:31] JP: Right.


[00:11:32] SY: “Don’t get caught up in the headlines. Do whatever you need to do. Focus on your product. Build, build, build and don’t get distracted.” So I’ve seen kind of different takes across it. But when I hear about the layoffs headlines, to me, it feels more like we are going to be safe because we just don’t know more so than it is anything else.


[00:11:57] JP: Right.


[00:11:57] SY: We don’t know when we’re going to be able to raise next. I mean, even just forgetting the stock market, just thinking about where the pandemic is, we like to pretend it’s over, it’s not, like there are definitely surges that have happened in different cities in recent weeks. And so there’s the COVID unknown as well. And so it feels like a preemptive move and it also feels like, and this is me more speculating than fact, but it also feels like an opportunity to make tough decisions without the judgment.


[00:12:31] JP: Oh, because if everybody is saying it’s a hard economic environment or that hard times are coming, companies might have a little more cover to maybe do some layoffs that they normally… because if you think about through last three months, hiring in the tech industry was crazy. If a company wants to lay off people, it’d be like, “What are you doing?”


[00:12:49] SY: Exactly, a hundred percent. And again, this is more just my personal opinion, but I do kind of feel if you wanted to do layoffs a couple months ago, you’d probably look stupid.


[00:13:01] JP: Right.


[00:13:01] SY: You’d probably look like, “What is so wrong with your business?” You know what I mean? That everyone else is hiring.


[00:13:05] JP: And you're probably still hiring people as well. Right?


[00:13:08] SY: Yeah. And you’re getting rid of people. You’d probably look bad. Anytime there’s a layoff, there’s speculation. Right? And it’s funny because the layoff can be as small as 10 people and people are still like, “Oh my goodness! What’s going on with this company?” Is it all hype? Are they going down?” There’s so much speculation, but if you do those same layoffs, but under this bigger headline of, “Oh, it’s not just us. It’s also Twitter. It’s also Tesla.” You know what I mean? Everyone’s doing it. Then it’s like okay, yeah, you’re just responding to the market. You’re probably seen as being responsible. You are getting ahead of the problem. You know what I mean? Like it’s more forgivable. So part of me feels like, do you really need to do layoffs? Is it actually a requirement for your business or is it a good opportunity to do layoffs that maybe even thinking about, maybe would help, but it’s a good time to do layoff? So part of me is a little suspicious about the whole thing.


[00:14:04] JP: Right.


[00:14:05] SY: Especially when you look at the fact that a lot of these, not necessarily thing, but a lot of the hot startups, the billion-dollar unicorn startups that are laying people off, they have money.


[00:14:16] JP: Yeah.


[00:14:16] SY: They just closed like hundred-million-dollar rounds. So I know you got it.


[00:14:22] JP: I’ve been seeing companies tweet about how they've closed this route of funding. They closed that route of funding.


[00:14:26] SY: Literally months ago. Yeah.


[00:14:28] JP: Or like literally this week, but it’s not the same company that’s doing the layoff. Other companies are a little more quiet, but that’s a good point. It’s not like we saw a ton of announcements from say Tesla that their revenue is down.


[00:14:41] SY: Right.


[00:14:42] JP: I feel you on that. It does feel like there’s a little bit of trying to possibly maybe sort of get ahead of something that maybe sort of would happen. It’s interesting.


[00:14:51] SY: More than it is I actually need to lay off people, right? More than it is I actually cannot afford it. It feels more preemptive, right? It feels more it could be an issue in the future. I’m not really sure. Maybe this is a good opportunity to kind of clean up, tighten belts, et cetera, but it doesn’t come from a, “If we don’t do this, we’re going to be bankrupt in six months’ situation.” It feels more like, “I mean, we do have the money, we could afford it, but just to be safe, let’s fire a couple hundred people,” which sounds very cold.


[00:15:21] JP: Right.


[00:15:22] SY: But I think that’s what’s happening.


[00:15:24] JP: Well, our heart goes out to anyone that’s affected by layoffs.


[00:15:27] SY: Oh, absolutely. Yeah.


[00:15:28] JP: We can talk about macroeconomic trends all we like, but at the end of the day, if you’re the one that gets laid off…


[00:15:34] SY: These are people.


[00:15:34] JP: It super sucks. So yeah.


[00:15:37] SY: Coming up next, we talk about Diablo Immortal, an accessibility in the gaming industry after this.




[00:16:03] SY: Here with us is Steven Spohn, Senior Director of Development at The AbleGamers Charity Foundation. Thank you so much for being here.


[00:16:11] SS: Man, you do that really well. Can I hire you for all of my appearances?


[00:16:17] JP: This is only like her ten-millionth podcast.


[00:16:21] SY: Only been about a decade.


[00:16:23] SS: Go on. Just keep saying my name repeatedly for the next hour, a long time.


[00:16:28] SY: Well, that’s awesome. Okay. So tell us about The AbleGamers Charity Foundation and what it’s all about.


[00:16:34] SS: Yeah. AbleGamers is a 501(c)3 nonprofit charity that helps people with disabilities be able to play video games by either giving them the tools that they need to help defeat social isolation or combating any of the challenges that you might have associated with video games. So part of our thing that we do is having players come into our headquarters and try out different kinds of assisted technologies that might bridge the gap between their ability to play and their desires, and that involves a long peer counseling process where we have experts go through and each step along the way consult with you on what might be an option for you and what might make your game better and allow you to play with your friends and family.


[00:17:21] SY: Wow!


[00:17:21] JP: Very cool!


[00:17:22] SY: That is so cool. And how did you get into this kind of work? Why is it such a passion for you?


[00:17:27] SS: The truth is that nobody wakes up as a small child and goes, “Man, I hope I can be an advocate when I grow up.” You decide you want to do something and you realize that there’s barriers in the way. And so when I was young, I wanted to be a productive member of society. I wanted to go out and do the whole yay capitalism thing, like they all teach us to do. And I couldn’t do that because there’s a lot of laws in the way for people with disabilities, about how much we can earn, how productive we can be before the government starts staring at you. And on top of that is the whole social change portion where people don’t necessarily value you as a disabled person. So getting into this kind of work was never something I intended. It was simply I enjoyed playing video games as a way to continue to be social and hang out with my friends. My favorite way to express it to people is always talk about how, when I was younger, there’s a club in Pittsburgh, Pennsylvania that was called “The Zoo”. It was like an under 18 where you could go drink fancy root beer with your friends and that was what all my friends want to do was go to the club. And the problem for me was that they all wanted to go do that, but I couldn’t get past my mortal enemy, the single stair. And so my wheelchair couldn’t get over that staircase. And therefore, I had to figure out other things to do to stay connected with them, and video games ended up being a great way to do that. So I found along the way that helping others to be able to play video games who were in similar situations was what I found my calling to do. I found that the most selfish thing I could do for me was to help other people.


[00:19:12] JP: So making headlines this week has been the recent game by Blizzard called Diablo Immortal and it’s getting a lot of flak for being a free-to-play game and all the things that that encompasses in a mobile game in 2022. But one of the interesting parts about Diablo Immortal is that it has native voice chat transcription and speech to text, which seems like a great accessibility feature on the surface. I’m curious if you’ve had any chance to play around with it and what your thoughts might be on that.


[00:19:39] SS: The truth is that it hurts me in my soul to see the direction that this game has taken.


[00:19:48] JP: Oh yeah.


[00:19:49] SS: Yeah. The problem for me here is that I’m an accessibility champion, right? So accessibility is always what I’m going to feel for and I’m going to attach to a person. This game is just an accessibility achievement to the point where I took it back to our board to be like, “Look, when I applied all of the lessons that we have been drumming about for years, look at the cool game they came up with and how many amazing accessibility features it has. You can use only the mouse and a left click. You can use a touchpad. It can be in multiple devices. You have auto navigation. You have voice to text.” There are so many cool features, it’s just such a shame that such an accessible game is now in this tarred up area where you just are now talking about microtransactions and crypto. My focus, accessibility. I hope other games at this level of accessibility.


[00:20:45] SY: What are some other accessibility features that other games have integrated and gotten right in the past?


[00:20:51] SS: There’s an entire website that AbleGamers has put out for Accessible.Games. It’s a free developer resource that people can go and check out exactly what you’re asking, the different patterns that we have for both input and for control to teach developers how you can make your game more accessible, both as a developer and what you’re going to look out for as a player. And then there’s examples on each one of those. For example, one of my favorites is Helping Hand. And that’s anytime that the game provides a way that you can automatically receive help from the game. So if that is coming along with an NPC that can help you clear out some mobs and then you get through your quest or whether that is the game pointing out with racing track arrows, the point where you need to go so you don’t get lost. Whenever the game automatically comes along and helps you get through it in a very subtle way, that’s just another accessibility feature, something like a second channel is another one. And the language I’m using was specifically built by AbleGamers to be able to have very complicated conversations. Right? So let’s talk about what I just said there, Second Channel, right? That’s just two words, very simple, but it actually describes a complex problem like Fortnite where you have bullets coming in that are shooting you. You need to identify what’s happening very, very quickly, right? Well, what if I can’t hear the bullets being shot? What if I’m a deaf gamer? Well, I can see on the screen that the screen is lighting up in red and I can see there’s a directional indicator where the bullets are coming from. Also, I can see the spark coming out of the other person’s gun, and there’s an audio alert. Also, there’s a digital readout that says that I’m being shot and how much damage I’m taking. Second Channel is when you have more than one way, that information is coming at you at the same time. But instead of having to drum up about that for 45 seconds in the developer meeting, I can say, “Oh yeah, there’s a Second Channel problem here where we’re only seeing one way that this person’s shooting,” and then you know exactly what I just said. 


[00:23:02] JP: So maybe this is an issue of which audiences are paying attention, but it seems like talk of accessibility in games maybe hasn’t gotten as much attention in the tech industry as accessibility in say web development. And I’m curious, do you think the game industry overall is talking about accessible design in the right ways? Or do you think they could be doing better?


[00:23:25] SS: So it’s a multi-stage question. Right? So number one, when I got into this industry, there was a huge stigma against people with disabilities. In fact, we went to the Game Developers Conference when I first started back. This was probably in 2007 or 2008 and we actually still have the YouTube on The AbleGamers channel where we asked developers one question, “Have you ever thought about designing for people with disabilities?” And a couple of people said yes. A whole bunch of people said no. And one dude laughed at us and walked away.


[00:24:03] SY: Oh, wow!


[00:24:04] SS: A very nice gentleman. Hope he has a lot of salami on his sandwiches in his lifetime.


[00:24:10] SY: Salami.


[00:24:12] SS: It’s come a long way from there now. A day doesn’t go by when somebody at AbleGamers is working with a AAA or an indie studio to make a game more accessible where there’s not somebody coming to us and asking how do they do things better or somebody tweeting or emailing. And so things have come a long way in that. Unfortunately, people get really caught up in the WCAG and web3 development and whatnot and it is something that… it’s gotten a lot of press and it’s gotten a lot of attentions toward accessibility, but also they have their own internal squabbles about what is the right way to do screen readers for the blind, what is the right way to incorporate audio, what’s the right way to have navigation for if can or cannot use a mouse or a peripheral. So video games have the opposite problem where we had to not only show that this is something that should be accessible, but now how do we do it? So nobody questions and says, “Why should a blind person be able to read a webpage?” Well, they should because that’s the right thing to do. Right? But people go, “Well, why would a blind person want to play a video game?” Because it’s a visual medium, right? Sure, that’s true. But also blind people will have feelings and enjoy stories that we are all sharing, just as much as anyone else. So you can’t segment them out of the audience just because they don’t get visual input. They can still absorb the story. So it’s the silly side of the advocacy that is the longest drawn-out fight is convincing people that they should care, convincing people that they need to be involved in this. So I think to answer your question of, “Has the video game industry gotten better?” Yes, absolutely, tenfold. With companies like Microsoft and Sony and the like coming in, unfortunately, I can’t say Nintendo because one thing accessibility it doesn’t do is Nintendo don’t.


[00:26:14] JP: Yeah.


[00:26:14] SS: Everyone else, really on the same page, and it’s come a long way and honestly has. I think right now the one area that we’re squabbling with more than anything now is how to do it right. There are a lot of consultants out there that are trying to teach companies the right ways to do things. AbleGamers has a database of more than 500 gamers with disabilities who are already and willing to be Q&A testers for the industry in a program we call Player Panels. And a lot of companies have come to us and have used that, but you still got these publishers that aren’t sure that the dollars need to go there. They’re just not sure. What if we put in a million dollars in accessibility? Are we really going to get that back? And so it’s now falling on the community to prove that, yes, in fact people with disabilities also have money. It’s weird. It’s a strange concept. I know.


[00:27:08] JP: So you mentioned that there’s been a huge change since you became involved in the industry.


[00:27:14] SS: Yeah.


[00:27:14] JP: And today with how seriously the industry is taking accessibility, what do you think has encouraged the industry to change or take it more seriously?


[00:27:22] SS: I’d like to think that it’s the people who have come along and pushed advocacy. So when we started, there were petitions, if you can imagine this, where you had to sign a petition just to get remappable keys. It was a controversial item just to say that maybe people wouldn’t want the X and the Y button where they are. Maybe they’d want to swap them. This was something we had to argue as if it was some sort of United Nations area of debate. Like, “Why wouldn’t we want things to be customizable? Why wouldn’t we want things to be comfortable?” And so I’d like to think of those early days where myself and Mark Barlet, the founder of AbleGamers, and people who have come along to help push in various jobs and positions are the ones that did that. Will the historians agree? I have no idea. I’m hoping that they will put my name in there somewhere. But I just want to think that it’s because of gamers that this came along. Now the truth is it’s probably because we had conversations, like the one I had with Alex R. from Harmonix, where we went to Harmonix and they don’t take a lot of these kinds of in-person interviews, but we happen to get an opportunity to go speak to him because he was a huge supporter of the charity and what we wanted to do and we were talking about Rock Band and I asked him a very simple question that was not on the doc or the script. It was supposed to be about supporting the charity and what they could do to help. And I said, “Hey, while I got you here, why is it that I have to be able to push three buttons to play Rock Band? Why not one?” And he goes, “Well, we just didn’t think anybody would want to push one button in order to play a game. We never found anybody who would.” And I said, “Well, I would.” And the color just drained out of his face. And he’s like, “Oh, man, you actually would?” And I was like, “Yeah. Why would I not want to play a game with my friends even if it’s just clicking one button at a time to be in the same group?” And so at that point, it kind of got through to him that there would be someone out there. And if I was standing in his office in my power wheelchair on a ventilator saying, “Hey, I want to play, this is the only way I could play,” then there would be other people out there too. And I like to think that those kind of interactions happened with a lot of different advocates, a lot of different gamers reaching the people in the right places to say, “This was important and this is why you should believe in it.”


[00:29:59] SY: So you mentioned that one of the issues that you all are going through is pretty much just deciding what is the best way to address different accessibility needs. I’m wondering, what are some other areas where the gaming industry might fall short in terms of accessible design? What are some other places that you’re hoping to create some more change?


[00:30:18] SS: We need to figure out a way to incorporate different levels of technology as they approach us. So we have VR on the horizon. Absolutely no one in the industry, including AbleGamers has figured out how we’re going to take VR and make it fully accessible to everyone from various disability groups. You have people with neuromuscular disabilities where they may not be able to move their body the way that everyone else can. You have technology that is looking for very specific movements and signals to tell it where human is. A great example of that is we worked with Microsoft to help them figure that a spinal column exists differently when it’s sitting in a wheelchair than when it’s standing up, essentially the connect would not see someone sitting in a wheelchair as having a spine because it was looking for four limbs and a trunk. And so that was how it would locate a spine. And so we had to help them figure out, “Here’s how a spine would look in a wheelchair. It’s slightly curved. It’s going to be below the shoulders.” And so they did that. And when they did that, it allowed people not only to use them from wheelchairs, but also sitting down. So it wasn’t just the disability community benefited from that R&D. It was everybody. And that’s the kind of technological understanding that we need to get out to the public is anything that benefits the disability community also tangibly benefits everyone.


[00:31:49] JP: I’m curious, as someone who is working to make accessibility more of a priority, what has worked for you to get people to prioritize it? Is it anecdotes like that? Is it examples? What gets people on the side of accessibility that haven’t thought about it before?


[00:32:06] SS: You really need to figure out where the developer stands as a human being beyond just being a developer. So a little known secret that not all the gamers out there might recognize is that developers are just humans. I know. I know.


[00:32:20] JP: They’re not just lazy devs. What?


[00:32:22] SS: No, they’re not robots. They don’t have feelings. It’s really strange. I know. So we have to first do is figure out as a human being, what kinds of people have they had in their life? Have they had someone with a disability? Have they had someone with a profound disability? Have they had someone who is blind, who is deaf, who is mobility impaired? Who have they had in their life? And do they understand that there are challenges outside of the typical able-bodied human? If they have never had that touch point of dealing with someone who has a disability, they may not recognize the struggles and the barriers that exist in the world for people with disabilities. So they’ll then bring that into their development practices because they’re not trying to be jerks. It’s just that they’ve never dealt with a human that couldn’t move a mouse more than a centimeter in any one direction, like me. That is an atypical case and it’s something that one in one thousand people actually has. My disability, SMA, is something that lots of men across the country have. It’s more dominant in men because of the way the genes work than women. And so there’s a lot of men that have SMA. There are women that have SMA. There are people who are blind. One in seven people is color blind. And if you haven’t ever met one of these people, you might not know what some of those challenges are. Right?


[00:33:48] SY: Yeah.


[00:33:49] SS: Once we figure out, “Is the person being a jerk or are they just not experienced?” Then you can tackle the problem. Because if it’s just lack of experience, well, then you just need to go out and talk to some disabled people. You just need to go and find out what the challenges are and AbleGamers can provide that path. And honestly, going to Twitter, looking up disability advocates, there are plenty of great people out there who are happy to tell their stories, given the chance. And so that’s how you start with someone who is empathetic, but just doesn’t have understanding. Then you have people who just don’t like disability, and this is perhaps the hardest problem that we’ve come up against. And it’s one that we haven’t even really found a great way of tackling. And it’s one that we like to pretend, even in my corner of the internet doesn’t exist, but there are some developers I’ve met along the way who just don’t like people with disabilities. They have a bias. It’s that ableism. It’s in them so hard that they just don’t care. And for whatever reason, empathy hasn’t struck them. They haven’t themselves yet become disabled, which by the way, by the time you’re 70 is 90% likely that you yourself will have a disability. It’s not something that they want to think about. Lack of experience can be tackled. Lack of empathy is much harder.


[00:35:04] SY: What do we as developers need to do to push things in the right direction?


[00:35:10] SS: It’s a combination of doing your homework, doing your research, and knowing your audience. Right? There are more than 46 million players with disabilities potentially in America alone. More than 50 million could be a new number that we start coming out within the next couple of years as more and more people have been disabled through the pandemic. In Canada, it’s eight to nine million, and the number just continues to climb from there. And this is something that we continue to see happening all across the world. And so it’s not just designing for people in general. It’s a matter of designing for everyone when we use the hashtag so everyone can game. It’s not just fancy little social media, but we figure it out to get lots of retweets, it’s a battle cry. It’s a message to developers to say, “If you want the largest amount of people possible to play your video game, don’t exclude this particular audience because you might be inadvertently doing just that.” And their money spent is just the same as everyone else’s. So let them into your world. And what we found is most developers don’t want to exclude people with disabilities. They don’t want to necessarily keep out people who have to play differently than they might. Sure, there are those companies out there who have the get good mentality and their entire core audience is built on whether they can or cannot do the game. So the idea of putting in an easy mode is like asking a duck to fly. And it’s not something that everyone’s going to be comfortable with. Ducks are pretty and they go in the water, right? So one of the things that developers can continue to do is talk to each other and convince each other of the importance of including this community of people, especially with how large it is. Again, explore that empathy. See if they’ve had those touch points and talk about how easy it is to do. The amount of times where we at AbleGamers have run up against a developer that should have known better and just didn’t think about accessibility is mind boggling. My favorite anecdote to mention is one time I was at PAX East. This was probably 10 years ago. StarCraft was huge in that time. And I happened to be doing a panel where you’re talking about the greatness of accessibility and there were Blizzard folks there. And one of the people who was responsible for StarCraft II was there and they were talking about how we go the game, do you like it, do you play it. And I’m like, “Yeah, actually I love StarCraft. It’s one of my favorite IPs. I'm really good at being a Zerg and I'm good at murdering people. Ha-Ha! I'm so good.” And so meanwhile, I’m talking about how great I am and this guy is like, “I’m sure you’re good.” I’m talking about accessibility and whatnot and using the funny anecdotes to kind of bridge the gaps. He understands where I’m coming from. And he goes, “Is there anything that you would do differently?” And I said, “Well, off the top of my head, the one thing that I really think is mind boggling is your mini map uses green and red. One in seven people are color blind. You could easily make that different colors. You could put symbols. There are a lot of things you could do there to make the mini map better for anybody who has any kinds of color deficiency.” And he stops and he cocks his head to the side and he’s like, “You know, I probably should have thought of that because I’m color blind.”


[00:38:46] SY: Oh my goodness!


[00:38:47] JP: Oh my God!


[00:38:50] SS: Yeah.


[00:38:51] SY: Wait. How has he been using the maps then this whole time? He’s just putting up with it?


[00:38:55] SS: Because if you have a disability yourself, you have to come up with means of getting around the world that was not made for you. So he figured out ways of just knowing that a certain unit would be here and that the size of this dot meant this and he figured out ways of getting around the color disability. And so for him, it wasn’t a disability anymore. He had figured out ways because he was so ingrained, so into the weeds. He just knew the game inside and out. He never even thought about it, but everyone else who’s just joining it, yeah. Right? Way different story. One of the messages that I’m really trying hard to get out there is that we’re all just gamers and we all just have a different viewpoint on how to game. Maybe you mind if someone plays on easy mode or maybe you don’t want to develop your game as an easy mode so someone can play it and I ask you to just run the simple equation in your head. You’re sitting in a fork in the road, and if you go down Path A, you can change your game so that it is acceptable to anyone who needs it and there will be nobody left out, and therefore, everyone is able to play on their own terms. It doesn’t actually take away from anyone anything else is doing with that particular game. Or you can go down the path where you are catering to a specific niche audience and only that particular player can play in that particular style and everyone else is left out. In those scenarios, one road, A, allows everybody to be able to play. The other road, B, excludes certain people and it hurts them because they’re not able to play that kind of video game. And I ask you as a developer, what kind of person do you want to be? Do you want to be the person who includes as many people as you can? Or do you want to be the person who leaves people out on purpose? And that’s the answer to your own internal question. You got to decide whether or not you want to join us in this fight to get everybody on board.


[00:40:58] SY: Well put. Well, thank you again so much for joining us.


[00:41:02] SS: Yeah. Thanks for having me. I appreciate it.


[00:41:16] SY: Coming up next, Josh speaks with the founder of an AR device company to talk about some of the inherent challenges of creating AR devices and some of the factors that could be contributing to the extremely long wait for Google and Apple’s AR glasses after this.




[00:41:47] JP: Here with us is Jeri Ellsworth, Co-Founder and CEO of Tilt Five. Jeri, thank you for joining us.


[00:41:53] JE: Thanks. It’s my honor.


[00:41:54] JP: Can you tell us about Tilt Five? What is it and what was the impetus behind launching the company?


[00:42:01] JE: Yeah. The folks at Valve saw my YouTube channel that I had just recently started where I do kind of hardcore science in my garage and showed how anybody can do that. They hired me to run their hardware R&D department, and that’s where I got the bug for augmented reality and XR because bringing people around the table, having holograms pop up out of the table is like the best way to get the whole family to play games, whether you’re a hardcore gamer or if you’re just more casual or just want to kind of explore sandbox-type of environments. I just really felt that was the right route to do. So Valve, at the time though, was really getting excited about virtual reality. And I didn’t feel that virtual reality really fit the mission and they ended up canceling every AR project and let a bunch of us go. And I actually went back to Valve and bought a piece of technology that I had been developing there. So our goal was to make an AR system that is optimized for tabletop entertainment. So playing video games on the table. So holograms springing out of the table. You can directly reach in and interact with your whole family, your friends can come around, look into this like big volume of space and see this magical world and play board games, action games, puzzle games, sandbox-style games, sports games in the round, which has never been done before, which is really kind of a friction point for playing video games in the living room.


[00:43:37] JP: Right, especially together.


[00:43:39] JE: Yeah. Like in my living room right now, I love to play video games with friends. So I have two televisions, two Xboxes, two PlayStations, two Switches. I have two of everything, probably have $4,000 worth of equipment to have a kind of broken two-player experience. So our goal is to make it so it’s like seamless. You could just sit around the table and I can look my competitor in the eye or I can form alliances with my friends across the table and play these games.


[00:44:11] JP: So it seems to me that embarking on this journey to build, you’re not just building a game or building one piece, you’re building the whole ecosystem. You’re building the hardware, the operating system, the software, the development kits. It sounds really daunting. Where do you even start trying to build something like this from a hardware and software perspective?


[00:44:34] JE: In the very early days, we had to prove out pieces of the technology. Can we make something that looks like a hologram on the table, even in the simplest form? So let’s get that stood up and then take a look at it, like what’s the friction points for our audience? And first of all, who is our audience? And once that’s defined, what’s going to be a frustration for them and can we address that? And so let’s incrementally improve on this. Over time, next thing you know, it’s like, “Hey, this thing’s really working.” And people appreciate it. We just came from the Augmented World Expo and the Game Developer Conference and our booth was packed the entire time. We were probably the most popular booth at both these events.


[00:45:14] JP: Oh, very cool.


[00:45:15] JE: And we’re shipping them now too. And they’re at a consumer price point. It’s what you would pay for a Nintendo Switch. And it took eight years for us to get to this point. It was a lot tradeoff. I mean, there were times where we had to give up our dreams of the way it should work because we learned there was a better way and you just have to toss it out. And that’s an engineering challenge you always have to face. It’s like, “Don’t get too tied to things. If it’s not working, you have to move on and side step and find a different direction.” And it’s cool in our office, we have the whole evolution of the product, and it’s funny looking back at all the different iterations of how wrong we were as we were doing it. And we’re probably still very wrong right now, which is funny.


[00:46:05] JP: So with Tilt Five, can you explain a little bit about how the gaming ecosystem works? I think this is a topic that a lot of our developers have been interested in. Is it a closed app store? Is it an open market? Is it a peripheral that runs games on a traditional computer, I guess? Can you tell me a little bit more about the software and the game support?


[00:46:28] JE: So we wanted to break down any barriers to get developers onto our systems. So first of all, our SDK’s free. Just go to our website and get it today, even before you even have the hardware and start playing around with it. The SDK, this comes from our advisors and mentors. They just beat into our head, like, “If there’s part of the flow that’s difficult for developers, try to make some tools to make it easy.” So we’ve added a bunch of features, SDK, things like real time editing. So if you’re using Unity or Unreal, you just plug the glasses into your PC, hit the play button. You can move stuff around in the editor, and it happens in real time on the glasses. This is a real friction point if you’re developing for augmented reality or virtual reality. You have to build, deploy it to the system, look at it, realize that you did it wrong, then strip all the hardware off and go back. And so we made it. So you could do all that stuff real time. We try to make as much stuff like drag and drop as possible so you don’t have to be coding a bunch just to get your first scene rendering. It’s kind of funny when we work with developers that have existing games. You tell them like, “It’ll take longer for you to load up your game project than it is to get it rendering the first time.” They never believe us. And they’re like, “Well, load it up. Let’s do it.” They load up their project, it takes two or three, five minutes, whatever, and then we just like drag, drop. All right, hit the play button. It’ll be rendering now. Other things like all the scaling and adjusting and getting the virtual world to follow your character and stuff, we’ve made all these tools, UI, which is tricky in AR and VR. We made tools for that. And so on the developer side, we try to make it really smooth and easy. Now on the content distribution side, we also didn’t want to have the friction of having our own boutique store flow that people aren’t used to. And so you can get your content off of Steam. You can get it off Google Play. Once we get the iOS drivers going, you can get it off the Apple Store. The way the system works from a hardware perspective, we can plug into Android phones. We can plug into PCs. We can plug multiple headsets in one PC. So you don’t have to have one PC per device. And then we’re working on all the hardware dongles and stuff to do iOS. So there’s a lot of flexibility there, which system you want to run on. Like PC developers, obviously they get an advantage because they have full on graphics cards, really capable, so you can have these beautiful scenes. Another piece of the technology that we worked years and years on, and this really makes it nice, so that you can use it on all these different devices. For those of you that are familiar with virtual reality, you have to have more simplistic scenes because you have to maintain a really high frame rate or it’s a bad user experience.


[00:49:26] JP: Or a sickening user experience.


[00:49:27] JE: Yeah, exactly.


[00:49:29] JP: I definitely had VR sickness.


[00:49:31] JE: Exactly. So we wanted to solve that. And eight years ago, when we started this, we were all like, “Okay, someday it’ll be possible to do what we call “reprojection” right in the headset. So all the head tracking, all of this image stabilization, reprojection happens in the headset doesn’t require a high frame rate from the game engine. So we actually connect to your phone or PC through just generic USB. The frames of video come across from the game engine asynchronously. You don’t have to worry about a vertical sync. It doesn’t have to be a particular frame rate. You can drop a frame here and there and we upscale it to 180 frames a second in the headset, and we’re constantly repositioning and doing transforms on the image to keep it stable on the game. And so we just went to the Augmented World Expo and one of our favorite demos to show has tons and tons of geometry in it. And it frequently drops to about 40 frames per second. And no one notices because it just upscales and interpolates to 180 frames a second.


[00:50:34] JP: Oh yeah. Yeah. Because if you were doing like a VR headset and you dropped to 40 frames per second, it’s bucket time.


[00:50:40] JE: Exactly. And we have some other advantages because it’s AR. You’re locked to the real world as well. So everything is just around it.


[00:50:47] JP: Oh, right, so you lose that sense of scale and where you are and the inner ear nonsense.


[00:50:51] JE: Yeah. Yeah. So it’s really a pleasant experience for even the most sensitive people.


[00:50:57] JP: One of the things that’s really interesting to me about AR versus VR is that to a neophyte like myself, on the surface, it seems like, “Oh, okay, I’m just projecting some stuff on top of the real world.” Surely, that would be easier than trying to simulate an entire field of view. And yet, here we are so many years after Google Glass, HoloLens, I think Google’s going to take another run at AR. Apple, maybe. There’s been all these attempts at AR. I wonder if you could talk about the technical challenges, but then also the product challenges. Why don’t we have AR everywhere yet?


[00:51:40] JE: I know.


[00:51:41] JP: It’s a little, “Where’s my flying car?” But you know.


[00:51:43] JE: No, no, it’s a great observation. A lot of times people forget how much time it takes to really evolve an emerging market. Look how long it took us to get cellphones that are basically super computers for our pocket. It’s like 30, 40 years. It starts off a little clunky, early adopters use it and get a lot of value out of it. And home computers. Right? Another one, took 30 years before things were like grandma could get one. So that’s part of the challenge. I think there was a lot of delusion in the early days and I was very susceptible to that as well when I was working at Valve. We had these big dreams that we could solve these challenges faster than we could. And what’s really cool right now, and I love and it’s perfect for us to be launching, is all the cards are on the table. Magic Leap is out. No more flying whales and hype and promise. Google Glass is a great example of that over optimistic view of what could be done in a short amount of time. They showed people jumping out of the planes with it and learning how to play a mandolin and super immersive graphics. All I could do is put up text messages basically because that’s just the limitation of the hardware at the time. So some of the challenges we’re facing, why we don’t have minority report today is just the laws of physics. It’s hard to control light and get it into your eyes in a really great visual way. Like most of these systems, like in HoloLens, in a Magic Leap, when you look at the AR experiences, they’re very washed out. You’re competing with all this ambient light. And when you dissect that, like, “Well, why don’t you just do something about the light that’s coming in from the real world?” And you start thinking about it, it’s like forever you like pixel that you have in the virtual space, there’s a cone of light coming and passing through that one pixel that you’ll have to come up with some unobtainium magical thing that can switch that light from any particular angle and you need to understand the light field that’s coming towards you. And it’s like, “I want to block this light ray and not that light ray.” There’s nothing that exists that can do that. And so that’s why, in our case, we threw that all away. Let’s turn the objects inside out. Let’s use our special optical board that you put on the table and let’s stretch this problem out and have a longer distance. That’s how we can draw blanks and do a life guild and all the things the other folks can’t do. So the reason we’re seeing so much VR is it’s trivial to do VR.


[00:54:22] JP: With all this interest in VR and AR and you founding an independent AR company, this is probably a very loaded question, but where do you see the state of the AR, VR, XR environment? And are you worried about competition or do you think the space is at a point right now where we’re just looking for adoption and a rising tide is going to lift all boats?


[00:54:49] JE: I totally agree. It’s a rising tide. I think there’s a lot of opportunity. There’s a lot of opportunity around enterprise uses and even people are buying our system and doing professional uses with it, even though we didn’t optimize it for that. We’re very much at, Tilt Five, about looking at the history of evolving markets. So we have all the early home computers here, the Commodores and TRS-80s and stuff and we think about that a lot. It’s like, “What was it like for them when the personal computer market was brand new?” There was a lot of opportunity. You had huge viable companies for decades all in their own little niche, whether it was business or a Commodore that might be mostly a gaming computer and things like that. As much as we like to dilute ourselves that we’re much further along, we’re probably more like, “What was it like in 1980 in the home computer market?” Right?


[00:55:46] JP: Yeah. Well, it’s early days, but it’s very, very exciting. And thank you so much for being here today.


[00:55:52] JE: Well, thank you. It was an honor.


[00:56:03] SY: Thank you for listening to DevNews. This show is produced and mixed by Levi Sharpe. Editorial oversight is provided by Peter Frank, Ben Halpern, and Jess Lee. Our theme music is by Dan Powell. If you have any questions or comments, dial into our Google Voice at +1 (929) 500-1513 or email us at [email protected] Please rate and subscribe to this show wherever you get your podcasts.