CloudBees Automotive – Eric Melski

All right. Thanks for joining my session. My name is Eric Melski.

I'm the architect of CloudBees Accelerator and one of the inventors of the product. I'm here to talk to you a bit about using CloudBees accelerator in the automotive industry. It's been 10 years now since Mark Andreessen made his famous statement that "software is eating the world", and in the decades since then, I think we've really seen just how true that statement has turned out to be.

I think everybody now would agree that every business is a software business, but there was more to what Mark had to say than just that software is eating the world. " And what that means is that, now that everybody is building software all the time into all of their products, and that's no longer much of a competitive advantage for you. What remains as a competitive advantage is how quickly you can turnaround updates and enhancements and fixes, especially in an environment like we have today, where sudden unexpected catastrophes can massively disrupt your work force and your supply chain.

Your ability to respond quickly to those events is critical. And that's where CloudBees Accelerator comes in. CloudBees Accelerator is fundamentally a software build and test acceleration tool.

So it's something that's used by your developers in order to enable them to do their work faster. So the process of taking your source code and turning that into a final product that's delivered in a car or in any other embedded system or on it on a handset for your phone or whatever it might be. And the way that we do that is by combining a variety of patented technologies for paralyzation, distribution and caching in order to execute that process more quickly, but still guarantee the correctness of the result at the end.

One of the things that we like to focus on, of course, in that context is what is the impact of having a faster build to your business? So what you can see on this chart on the left, is just some rough times for build duration. So at the top, we have a 14 hour long build that maybe sounds outrageous to some of you.

Maybe that sounds fast already to some of you. Some of the companies that I've worked with have had builds that literally took several days to execute. So you started on Sunday and it finishes the following Saturday.

When you're in that kind of environment that seriously impacts your ability to be agile. There's nothing continuous about cycles that are longer than 30 minutes, as we say, at the bottom of this. What we typically see with accelerator is that if you're at the beginning of your prosthetic separation journey, then accelerator can typically drop you about two levels.

On this chart, so if you have a 14 hour build now, it should be pretty straightforward for Accelerator to get you down to two hours. If you have a six hour build, we can probably get it done for you in 30 minutes. If you are already somewhere down that road where you've already invested in making your processes faster, maybe you're already using parallel builds or you're using very capable hardware or something of that sort.

Even in those cases, you can see that accelerator will give you a benefit. The X Factor won't be as much, but it'll still be noticeable. So, of course, it's one thing for me to say that it's another thing for me to actually show you.

So what we did is we did some benchmarks of some pretty typical use cases for the automotive industry. The first one we have here is Android. And this is an example of where somebody has spent an enormous amount of time and energy already trying to optimize this build and that somebody in this case is Google.

They have vast, vast resources and entire teams who are dedicated just to making this thing go as fast as possible. That's probably far more resources than you have to contribute to your build and test acceleration initiatives. So the best that we can do with an Android open source project, this is on 32 core system.

And this is building Android "Q", which was the latest version as of last fall. So the best time that I could get without accelerator was about twenty four minutes. And with accelerator, it's just a bit more than 13 minutes.

So it's nearly twice as fast. Again, this is on an already accelerated process, making it even faster. The next example is Yocto.

That may be familiar to some of you, it's a commonly used tool chain for building an entire embedded Linux system. Again, without accelerator there, the build time was a bit more than twenty six minutes. And with accelerator about 18 and a half minutes or about 40 percent faster.

And the third example I have is running MISRA compliance checks. 9 kernel and you can see without accelerator and not using accelerator, but still an accelerated parallel process. This took about 48 minutes.

And with Accelerator, it was an astonishing 70x faster. Just thirty nine seconds long. That may seem unbelievable, but I can tell you these are not atypical results for Accelerator, I've been working with customers all around the world and some of the biggest, nastiest builds that you've ever seen, and this is not uncommon for the results that we see.

The reason that the mystery results are so compelling in this case is that happens to be what we call an embarrassingly parallel problem. And it lends itself extremely well to the caching technology that we have built into accelerator. So if you're not doing accelerator, what can you do today?

And the answer to that is not a lot, There's a lot of Band-Aid, clunky solutions to the problem. You can try and parallelize your builds and your tests by hand. But that's really difficult to do, right?

And it's kind of a waste of your developers time. They're best effort is put into things that will produce value for your customers, not just fixing structure. You can buy bigger, faster hardware or bigger VM's in the cloud, but it's hard to efficiently utilize those resources.

And, of course, is very expensive it's not and it doesn't scale well. You can't easily leverage that across a team of 100 developers, for example. You're not going to buy a 64 core machine for every one of them.

You can reduce the amount of building or testing that you do. But this is going to cause your developers to become overly cautious and your customers end up doing your QA for you, or you can just live with things the way that they are, but this will cost you in your competitive advantages. It will make it difficult for you to rapidly respond to unplanned events and crises like the one that we're living through right now.

What if you didn't have to do those sorts of things? That's what Accelerator is all about. Accelerator gives you intelligent fault-tolerant workload distribution across a variety of compute resources.

Accelerator can optimize the dependencies and guarantee the correctness of your build, even if you are not sure that your dependencies are all completely specified. We can avoid doing work that's been done in previous builds by reusing outputs from other workload's, and we can dynamically provision resources to scale your compute resources up or down as needed. Typically, we see that this results in a significant reduction in the infrastructure required to support your development team.

And finally, the accelerator provides sophisticated analytics that enable you to do capacity planning and fault analysis. So in a nutshell, that's CloudBees Accelerator. Wait less, innovate more, through acceleration, resource optimization and insight into what you're doing.

Now I'm going to walk you through some of the what we call our optional technical slides. I apologize if some of this seems a little bit too down in the weeds. As I mentioned, I'm at one of the inventors of this technology so it's very exciting to me.

I'll try to keep it at a level that's comprehensible for everybody. Again, we have several patents related to this stuff, ranging from things like schedule optimization in the context of the software built to automatically detecting and correcting paralyzation faults while the build is executing. So there are two main thrusts to Accelerator's ability to make your stuff go faster.

That's parallelism and caching. In terms of parallelism, this is not a new idea in the universe of a build and test acceleration but it's very difficult to get it right. For two reasons.

The first is that if you don't have your dependencies completely, perfectly specified, then it's possible that although your builds work correctly on a single CPU or on a few CPU's, maybe up to four or even eight, but past that point, suddenly your build starts failing in bizarre and unpredictable ways. If you're lucky, it's a very visible failure. If you're not lucky, you end up consuming the wrong output.

Maybe that maybe a build up of a previous run and something you shipped buggy code without even realizing it. In addition, Accelerator provides distribution across a cluster of computers. So you're no longer limited to the number of cores in a single host.

The second part is caching. Again, this is not a new idea, although there are very few build tools that provide the sort of comprehensive across the board generic caching that accelerator has. You may have heard of things like C cache, which is great for doing compile caching, but accelerator can do more than just compile caching.

We can compile Java Doc generation, we can compile, we can cache miswrote checks. We can cache Java compilations, all things that don't readily exist outside of accelerator. Once upon a time, maybe you might have use Clearcase with Clearmake, which had similar sort of capabilities built in.

But I think it's been about 10 years since I've seen anybody who is seriously using Clearcase for anything more. So between these two things, we're able to deliver performance improvements even as much as ninety five percent. And this result in better quality for your customers through faster feedback to both your developers and your QA teams.

One of our core patents is around, as I mentioned, dependency management. And this is specifically in regards to the execution order of the steps in your build. As I said, it's very difficult to get this right when you're talking about a large scale system.

Something like Android, for example, has hundreds of thousands of distinct compilation steps in the build. So the idea that you might be able to specifically accurately specify all of those is almost laughable. Again, a company like Google could do that because they have a team of several engineers whose job is entirely to manage that process.

That's out of reach for most companies. So what you end up with instead is something that's pretty close and works fine again at low levels of parallelism or in serial as you try to scale that up, suddenly, the build starts failing in surprising ways. Accelerators technology enables us to automatically detect and determine when things have been run in the incorrect order because of missing dependencies and in addition to that, besides just telling you that that has happened, we can fix it on the fly.

The end result is that you have a build that executes with the speed of a massively parallel system, but with the same output as you would get if you had run it serially, guaranteed. The last thing that I want to mention here is the use of resources. As I mentioned earlier, one possible solution to the build speed problem is just to get bigger, faster hardware, but that's difficult to efficiently share among your development team.

You could conceivably get one big machine for each developer then nobody would have to fight for that resource, but that's prohibitively expensive. You could instead get a smaller number of machines and parcel them out so that some smaller group of developer shares each one. But then if you have multiple developers trying to do something on that machine at the same time, they're competing for those resources and everybody suffers.

What Accelerator allows you to do is combine all of those resources into a single virtual supercomputer, which is then parceled out as needed to the tasks that are executing on it at that time. So we can take a cluster of machines of various capacities and join them together into a virtual cloud as you can see. Then, as you start to execute jobs against this or as you run builds against this cluster, the resources are divvied up according to everything that's running at that time.

So we might start out with one build that that's using about half of the cluster. A second build comes in and needs some portion of that as well. And then finally, a high priority build comes in and actually pushes some of the previously running stuff off of the resources they're using in order to get more capacity for itself.

Of course, with our fault-tolerant system, all of the bills that are running see no disruption in their execution. There's no faults. There's no failures of the builds.

Perhaps one of the builds takes just a smidgen longer than it would have otherwise, but it's all transparent to the end users. Finally, I'm just gonna walk you through sort of a real life use case with Accelerator in this example, we have a build that serial took about an hour and 10 minutes or 70 minutes to execute. The first time we run this with accelerator, we call this our initial learning build.

And during this time, Accelerator will determine where those missing dependencies are, as well as build up the cach in order to facilitate faster builds in the future. You can see even in this case, the learning build was already faster than the serial build at about 44 minutes. But you can see there were some places where the dependencies were incorrect, which cost us some time.

The next build leverage is the information that we gathered during that learning build in order to further optimize the execution. Again, no user involvement is required, this is all automatic and transparent to the end users. In this case, applying the learning in the caches that we got from the first build, we're now able to execute this exact same build in under five minutes.

And as I mentioned, we have a variety of analytics that we can use now to do capacity and planning on this system. Such as this, which is our electric simulator report, which allows us to project how long this build would take if we were able to give it more resources to execute. In this case, we can see that the bill could, in fact, run in under two minutes if we had sufficient CPU resources for it.

Well, thank you for joining my session about CloudBees accelerator for automotive uses. If you have any questions for presentation, you should be able to post them in the chat window, in this session. And I will answer them as quickly as I'm able, as I'm attending the session live right along with you.