Mobile Application Development Without Hardware – Syed Hamid, Sofy

This is Textron tv. Hey guys, thanks for the throw. We're here with Ed Hamid, who is c e o for Sophie, and we're talking about how we're making it easier to test mobile applications.

Ed, welcome the show. Yeah, Thank you for having me. You guys are building out a new facility where you're making it possible for people to come and test against all kinds of devices.

What's the thought process behind that? I know you have one in Texas now, but there are others I imagine. And are you gonna build these around the world and how do you get people there in the first place?

Yeah, so I think, uh, um, uh, yeah, I mean, good thing is that, you know, as we kind of growing it, we started in Seattle, obviously, uh, in our first data center was in Seattle. So we have like a, um, no-code test automation platform for mobile app testing. So in order to test, people have to test on physical real devices.

And today people have to buy like a iPhone 14 for $1,500 right. For each device. And if you have tens of developers, you have to spend tens and thousands of dollars in buying devices, right?

And testing it on a daily basis. So we have physical real devices in our data center in Seattle today where anybody in the world can access it via browsers within 10 seconds. So our s l a is 10 seconds, all real devices.

And with the huge demand that we are seeing it with our customers, um, almost doubling compared to last year, um, um, we needed kind of a more redundant environment. Uh, so that's why we just opened our second facility in Seattle, in Dallas, Texas this week where we'll have another few hundred devices that anybody can access anywhere from the world within 10 seconds. Mm-hmm.

And so I think this through for a bit. I don't have to go physically travel to the actual lab. I can access this stuff remotely.

So is it a virtual instance of these platforms or am I actually accessing physical devices remotely? What's the nuance there? Oh, yeah.

It's actually all physical real devices, uh, that you are accessing through browser. Um, and, and instantaneously available today, engineers have to spend 30 minutes to an hour every day configuring their environment to test it. And that's where our whole value prop is that anybody can actually access that devices within seconds real devices and start automating and testing it immediately.

We see a lot of people talking about providing access to virtual devices. So what's the difference between giving folks access to, say a virtual instance of a mobile device versus a real device? Yeah, so there are quite a few things, uh, like in emulators, specifically that in Android emulators or um, or or virtual devices.

Um, a lot of things that you can't do it, you have to mock the G p s, like, you know, a lot of people have to test, uh, one time password, right? So you have to have the sim guard validated end-to-end scenarios. And, and those things becomes, um, almost to a certain extent impossible.

It's easier to go and validate it, uh, like the basic scenarios and, and also different form factors have different versioning and different environments to do it, especially if you're doing the, like a gaming or the streaming apps. Um, and testing more, more, more, more kind of a detailed in depth things. Like for example, if you want to simulate your environment in different countries or different things.

So people have to kind of simulate and marking it you to avoid all those things. We can do it in, in real devices instead of simulators and also, and, and emulators. People have to set up their own environment, right?

All the time. So you have to write, if you're running as an Android, um, emulators, you are, it, it's very time consuming and configuring and very, um, power hogging and, you know, memory utilization on your desktop becomes significant. So it's like ease of use, right?

And we guarantee a ten second SS l a on getting your environment up and running and compared to what they have to do it on simulators. Mm-hmm. Do you think as we go along, that the tolerance level for end users, especially when it comes to mobile applications, is dropping in terms of the experience?

Some folks, I know if that thing doesn't work within two seconds, they're onto the next app. So I guess back in the day a lot of developers felt they could maybe experiment directly on end users. 'cause they were like, well, we'll fix it in the next update.

But it seems like those days have come and gone. Oh yeah, absolutely. I mean, the people demand, like, you know, I I used to tell one of the things is that, you know, uh, that, you know, people not only demand all the features available immediately, but they have tolerance for low quality is is very low.

They don't want, especially in iOS, right? Um, and we see a little, still see a little bit okayness in in Android because people assume it's an open system. So sometimes they, and there's a variation of lot of low-end Android devices as well.

So if you're running like, you know, very powerful app like with streaming on a very low-end Android device, so the quality won't be as good. But on iOS, on Apple product, people have a very high expectation of quality. And, and, and, and surprisingly we have like lot of customers prioritizing their app on iOS, um, on Apple devices with respect to higher quality expectation from the customer than the Android.

Mm-hmm. Is the whole testing process getting easier? I feel like one of the issues that we've had historically is, and we run out of time, and that happens for a thousand different reasons, and most of it is probably our own fault.

But, um, can we, you know, accelerate this process in a way where there's just no excuse for not testing? Yeah. And you know this, every time we talk to the VP of engineering, uh, it, this is one of the things that comes all the time.

Okay, look, they run out of time, they want to get things out. They're shipping the shipping the feature or shipping a very high quality feature is always kind of, uh, do it. And, um, and we are realizing that I think we are getting closer to our, our nirvana where we can actually, uh, get test, uh, tested fast.

We announced our product called Sophie Sense, which is integrated with open ai. So actually now you can just point your functional spec and, uh, our system can not only generate the test cases, but can also write the code and start testing it as well with very minimal supervision. So that the thing that used to take hours to do it, now you can almost do it and manage.

And so I think we are getting to a point where, um, where things are kind of converging for more effective testing and faster testing, especially with, uh, G P T and, and open AI integration that can enable those scenarios. That was I possible before or was very, um, manually centric, right? People have to redocument then create test case and create automation, run it.

So this whole process is becoming very streamlined and easy to use. So I'm super excited with that. With the rise of AI is the cognitive load for the developers on the testing dropping because I don't need to know how to build a test.

And I think part of the problem that a lot of developers have had is that they just didn't know what to build for the test and then they would rely on somebody else and somebody else was busy. So next thing you know, we were just throwing stuff over the wall. Yeah, yeah.

AB absolutely. Actually we, um, as I mentioned earlier, we kind of announced our product called Cent, which can generate the test case pretty much automatically. You can just point to a functional spec in your Confluence page in Jira or you know, or, or, or any, or any other test repository or functional spec repository.

And it can actually just take that and generate the test cases based on the learning. So we have done two things is that as you are testing the product, we are constantly learning. And then when a new functionalist spec comes, we can bring these things together and generate the test cases for you and be able to run it.

So we are already seeing a very significant, um, improvement in our ability to basically, you know, uh, generate the test cases, uh, automatically. And, you know, that is a very unique generative AI use case where given a functional spec generate test cases, generate, uh, automation and be able to run it at scale. What will the be the relationship between the developers and the testing team as we go forward?

'cause it seems like the developers will be able to do a lot of the tests themselves. So what will the testing team focus on? I think, um, I think there, it still, I I wouldn't call it that it will be kind of completely removed.

What, what I think the interaction would be is like a more customer centric that hey, what the feature they're building, what are the use cases and what are the, what are the, uh, things that they need to validate and input, like call it the kind of the prompt engineering for the, for the test case development. So they'll be more focused on, you know, what are the, what are the edge cases? What are the things that we need to go validate?

As an example, um, if you're releasing in a, in a worldwide, uh, line of business applications, right? Um, uh, for example, tax calculation and things like that, right? The, the current AI systems are not, um, eff efficient enough and effective enough right now.

I think the, the, uh, the, the, the, the interaction between the developers and the testers will be more focused around that the business logics and business rules rather than, you know, more end-to-end users can be taken care by ai, but the more detailed and in-depth variations of that would be. And the second scenario that will be very relevant in this case of test and the developers is that generation of test data. How do we validate complex systems, especially if you have integrated with machine learning in your own system, how do you generate the test data that can validate different scenarios?

Hmm. As we go along? What is your ultimate sense then of the, well, the quality of the applications we're building materially improve?

Will we notice that and see that? And will they be more secure? I mean, what ultimately will be the outcome of all of this?

Um, that's a pretty loaded question. I mean, when we started Sophie, we always believed that hey, um, that, that in the world where, uh, automobiles can be driven autonomously, software should be able to test itself. And we call it the intelligent automation, which is effectively constantly learning and constantly improving itself.

The test generation test execution and test reporting. So if you look at the kind of the life cycle of the testers, right? How they test it, they get environment up, then they install application, then they generate test case, then they execute and then they report.

So they do four, five diff distinct activities in a given day, right? And, and we believe that they all are coming together because now you can analyze the data effectively. As an example, um, today, if you get a failure, then the developer and tester sit together and find out, Hey, what is the use case?

What were the steps to repro repro, and how do we go fix it? Right now, this whole can be integrated itself. Now you can actually given a failure with open ai, you can actually suggest how to go fix those problems as well.

So I think I can definitely see opportunity where we can start, um, getting to our world where this whole, uh, lifecycle becomes shortened and effective based on, because if you see, a lot of developers are also using lot of open source components. So there's lot of community data that's available both for testers and developers that can significantly improve, um, uh, the, the, uh, the communication between developer testers, but also significantly decrease, uh, the testing time. Mm-hmm.

So will a lot of the steps consolidate? 'cause I think one of the issues we also see is there's, uh, regression testing and functional testing and unit testing. Is that all gonna become one integrated motion?

Yeah, I think, I think it's going to be getting more and more integrated motion. Because, you know, there's, there's like, I think those days of, I used to call it the pyramid, right? Reverse pyramid, right there, you have a lot of functional tests, right?

And lot thousands of unit tests, and you have functional tests and you have integration tests. I think those days are kind of gone because, you know, now we have so much data about the application and what are the building blocks of that integration that people, you can just give to the system and system should be able to smartly determine what changed, what do I need to test and just go run that, right? Previous model was that, hey, anytime I get the run, I have to run all these things to make it a thumbs up decision right now, because the release cycles have also decreased and, and people are becoming a little bit open, okay, look, okay, if I make a mistake, I can turn around within few hours, right?

And they do a lot of, uh, like, uh, testing and production with a very small percentage of customers right before releasing it to all those things. So I'm seeing more and more people doing one rather than classifying in different categories and then run it. All right.

So as we enter this brave new world of testing, what's your best advice to folks? 'cause we are seeing, um, a, it's becoming easier to access the devices. B is becoming simpler to create the tests, and there's ai it's helping us understand what the implications are.

So where do I start? Uh, I start for, for who? Like a fresh graduate?

What, uh, what do you mean by the start? For who, Where do I start if I am a, either a brand new developer or B, if I already have a, an extensive DevOps workflow in place, how do I kind of get from where I am today to where the promise of nirvana is tomorrow? Oh, yeah.

So I think it's really kind of boils down to the, the maturity level of the individual organization, right? Um, if you are a very mature organization, then you have lot of test data or the data available so you can actually start reporting, uh, where the failures are happening and the patterns are happening. So, so people spend a lot of time, as an example, generating custom reports, figuring out what's going wrong or not, right?

So you can you leverage that to like, as an, in this case the AI to, to understand what's happening and why it's happening and make it easy, right? But if you're starting a kind of a early stages startup and starting now, I think it's starting with a, uh, with, with a, with like a ground up approach on building the test cases as one unsom and, and generate like both, all types of tests through, uh, open AI and, and, and generative AI and be able to execute that. It's is a, is a, is a good start.

So it's like really boils down to what's the companies or their business goals are or slash their customer's profile is. And you can take multiple route, you can do a top down route, which is, I mentioned you have a lot of data already, or you can do a bottoms up route, but at the end of the day, my, my advice to people has been, Hey, look, every five years we see new things coming in tech technology, right? I mean, 10 years back we were talking about the same thing about cloud adoption, right?

Before that we were talking about IDs, right? So, so every time it comes in, so my take is that, you know, be open and try out, one thing that's great about both cloud and AI is that it'll literally take you few hours to try out what's working for you. You don't have to spend significant amount of like months and months of resources to do anything.

You can do pilot and prototype on, on generative ai like companies like us or or other use cases with, with environment analysis literally in few hours. And, and so I think trying out that I think is more important in my opinion. And the trying out cycles has also decreased significantly now, right?

You don't have to month spend hours or, or days or months to do that. So my take would be that, hey, try out now it takes really less time to try things on open source. If you are open to using open source components, you can use that because there are several variation of generative AI modules available on, on open source or, or use chat g p t itself to kind of generate that.

So I think a combination of those things would be my suggestion, because you never know for your scenarios how efficient and effective it can be. All right, folks. Well, you heard in here, I mean, the best thing you can do is go test the testing tools because it's a whole different ball game these days.

And if you're still using the same workflow as you were using five years ago, chances are you're falling fairly far behind. Zed, thanks for being on the show. Thank You.

Thanks for having me. Appreciate. All right.

And back to you guys in the studio.