Além do Piloto: O que realmente é necessário para colocar a IA em produção, com Beth Williams, da Dell

Welcome to Espeaks for EW week. I'm Corey Nolles. At this point, most enterprises don't need another demonstration that AI can do something interesting. The harder question is more what what happens after the demo. Moving from a successful pilot to AI that can operate reliably across a business introduces challenges around data, infrastructure, security, governance, cost, and even who is ultimately responsible when these systems make decisions or take action.

Joining me today to unpack that transition is Beth Williams from Dell Technologies. Beth, thank you for being here. >> Hi, Corey. Thanks very much for inviting me. >> Excellent. Glad to have you. We've kind of reached a point where AI pilots are everywhere, but why is moving from a promising pilot to a real production environment still something that's really difficult for so many enterprises? >> Yeah, I mean, we're seeing this a lot with our customers, a lot of stalling pilots, people just staying in that phase.

And I think there's lots of different reasons for it, but one of the main reasons we see is when people are focusing on pilots or demos, they're really just trying to get the technology to work, right? So, they've possibly got their own little environment somewhere. Maybe they've got a nice little GB10 or something. Uh, it's not a scalable environment. They've probably handpicked the data, right? They've curated the data, found it all, put it all in to get this to technically work, you know, and they probably also handpicked the users as well if they are opening up to limited users.

In fact, we're in that stage now with one of our pilots where literally we've just chosen a subset of users to go and play with this stuff. So all that's fantastic for a pilot, but the minute you move to production, it's a completely different game, right? So first of all, >> you're probably going to need something more than a laptop to run it on. So the platform you're going to run these things on is a big part of that and many of the customers that we have have not actually established a platform yet for it to run on other than maybe using public cloud and then part of that equation then becomes the data.

I've talked about the fact that a lot of pilots you kind of handpick the data and curate it and get it to work technically. But when you start to move into production quite often what you're looking for is really automated data, right? So you want data to be available when you need it and you want to trust it. And a lot of the data we're seeing with customers is just not ready, right? And so it's just a combination of lots of different things. Then if you add things like Aentic, and what we're seeing right now is lots of people are piloting Aentic workflows because they're so powerful and the autonomy is so appealing.

The minute you get into that realm, then you're into a really interesting security conversation. Like you've probably seen quite a lot of stuff in the media recently about agents going a little bit crazy, >> right? >> Going a little crazy. They have >> gone a little crazy. Yeah. get on get on little forums and talking to each other about how to break out of, you know, sandboxes and all sorts of fun stuff. So, you've really got to start thinking about that, right?

So, that can't be something you think after you push to production. You've got to build it in from the start. It's all about things like policies, permissions, security, and then also, you know, who's going to support this down the line, right? So, again, a lot of people are very focused on getting that technology to work. And then when you think about moving it to production, right, who's going to own it, right? who's going to own whether or not it's behaving right, how's it going to be monitored, how you going to be able to roll back and so on.

So all of these questions are quite difficult to solve unless you've done it before. Of course, we've done it before, which is why we can help or unless you've already got a platform you can roll it onto, which is something like the AI factory with Nvidia, for example, right? It's just a readymade platform you can roll it onto. So as I say multiffactorial, >> you know, there's been a lot of pressure on these companies as well to find more AI use cases, but maybe not every experiment needs to become a production system.

How do you think leaders should determine which pilots are actually worth scaling? >> I think a lot of people get into this mindset that, you know, if I've got a pilot that works technically, it should always go to production, right? Um, and piloting is about learning, right? Essentially, it's about trying to learn whether or not something's going to work for the users, something's going to technically work. And with AI, it becomes very interesting, especially with the fact that you're using models that might hallucinate.

So, part of what I try and help customers and internally know is that when you do a pilot, you've really got to be emotionally unattached to where that pilot goes next, right? So, you've got to say, okay, what did we learn? Can we move this to production now, or should we maybe pivot on what this is doing? Or maybe, and this is the fail fast model, maybe we just say this isn't suitable for AI, right? We we need something that's 100% trustworthy and this is not giving it to us.

So maybe we just stop and rethink it. So, you know, a lot of it is about being very unemotional about where you get to with the pilots. But also, it's a case of when you're running the pilot, what kind of KPIs are you looking at during the pilot itself? So again, one of the things that we do internally and we also help our customers do is going through what we call like a proof of value for a pilot. Let's decide what the important KPIs are. So let's prove those things during the pilot.

Not just the technology, but is it giving you the business outcome that you're looking for? Is that you know revenue or is it um cost savings or is it more performance? What what is it that you're looking for? And let's measure those during the pilot. And also just be aware that when you move to production, you want to continue to measure those as well, right? So those aren't just one off is it continuing to give you all of those things as well. One of the other things that we did within Dell, because by the way, use cases aren't the problem, right?

Use cases are definitely not the problem. There are hundreds and hundreds and hundreds of potential use cases out there. >> Same. >> So we we actually you had like 800 plus use cases a few years ago. We were starting the journey and everybody was going off doing little pilots and it's like guys, we need to really pair this down. we need to look at what is it that's going to move the needle for us, right? So, what are the really important use cases?

And we looked at what our business was, what what do we do as Dell? And we landed on three or four use cases. That was it. Three or four use cases that we know made Dell what it was. agreed all of the KPIs and then we went through a whole data readiness side of things as well because it's all very well saying these use cases will move the dial but if it it takes ages to get the data ready for those use cases again it's one of those decision points you go well maybe this is not great for those first pilots right >> so again we did that internally prioritized the use cases that we wanted to push to production made sure that we understood what the data was that we needed to get ready for that as well and we got ready for that production deployment and Then we continue to monitor those KPIs going forward. >> Well, you know, I'm glad you mentioned data.

We always hear about how AI strategy is ultimately data strategy. Once a company gets serious about production AI, what does data readiness actually mean in practical terms? >> Nobody really wants to talk about it, right? So, um, because it's not the it's not it's not the fun bit, right? So, it's not the I've just got this gentic workload flow. I'm doing this, I'm doing that. Look at this is amazing. it's kind of not fun um unless you know you work in a field of data.

So um we actually have a data readiness assessment that we do. We do as part of our own work and we do it for customers as well. And the sorts of things that we look at is first of all we say don't look at all your data, right? Because that's just going to that's just overwhelming, right? So when we talk about data readiness, talk about those prioritized use cases and you will probably see it's a limited amount of data sources that you need to pull in, right?

So just focus on the readiness around there, right? Where is that data? Can you find it? Is it discoverable? Secondly, and this is really important when you start getting into the agentic space, have you got the right permissions around it? Right? Because again, you really need to be careful with autonomous agents accessing data they shouldn't access and then, you know, can we trace it? You know, can we take it back to where it came from? Is the lineage clear of the data?

Have we created something like a data product? And we talk a lot about that, right? Data products. Treat data like products. But that's essentially it. It's it's have we got the right permissions for it? Can we create trade the lineage for it? Is it for the right use case? Is it owned? Is it governed? And really important as well, does it have life cycle management around it? Right? So when you move into data as a product world, you are really putting ownership uh onto those data product managers to make sure that not only are they providing data in a timely manner, but they also got the responsibility to make sure that data is right and correct and governed and permissioned and policied.

And until you get into that model, it becomes very difficult to scale because you get back into that curated data set where it's all manual data pipelines aren't doing it for you. So yeah, moving into that kind of data product world is a really good game changer. We did that fairly early on at Dell uh to help us with a lot of the use cases that we're rolling out now. A lot of organizations began experimenting with AI just using whatever infrastructure was easiest for them to access like what they have when usage becomes persistent and starts scaling across an entire enterprise.

How does that infrastructure conversation need to change? >> So you're right. We a lot of the pilots that people do tend to be on whatever they can get their hands on, right? Might be their own. I mean I'm guilty. I've done it on my laptop. you know, it's could be anything, right? Could be a borrowed resource, could be could be anything. So, their aim is to try and get it to work. Once you've got to that point where you go, okay, so now I need to move this to production.

The infrastructure it's running on as a pilot is very unlikely to be good enough, right? It's very unlikely to be able to scale. So, you got to start thinking about, okay, what am I doing for scalability from an infrastructure point of view? There's a few different ways you can slice and dice that, right? So we started off saying okay everybody needs to buy a big AI factory and you know but that was a very very big investment for customers right especially if they've got one use case right when you get to like multiple use cases you can absolutely see that infrastructure story and that AI factory story makes total sense at scale so what we've done at Dell is we've kind of paired it down a bit and said well actually let's take the learnings that we understand now from the industry which is people are tending to start small so all you need to maybe do is make sure you've got one good GPU new AI server for example with the right software stack on that might be good enough to start with you might just be doing some smallcale inferencing and then you can upscale you can add these modular infrastructure pieces in so you can go maybe to enterprise inferencing and then even distributed inferencing by keep adding in those layers those modules if you like so starting small is still really important the gap between a pilot infrastructure and a big distributed AI factory is large so having stepping stones to get you there for what use cases are that you're running is a really important, you know, strategy.

The other thing to talk about as well is, you know, it's not necessarily always going to be on prem for everything, right? So, you're going to be looking at a kind of hybrid approach, right? Most customers are, right? So, if you look at it from an AI workload point of view and you look at especially when you talk about agents, you're going to see different parts of those workflows potentially not needing to be on prem, they could possibly be in the cloud.

Maybe it's got different kind of scalability that it needs and there's going to be certain things that you would do want to keep on prem right because of the data potentially right and the security. So one of the things that we do is we come up with a what we call a workload placement strategy based on the use cases that we're implementing with customers and then we decide okay based on workload placement what do we need in terms of platforms right this particular use case needs to stay on prem okay fine AI factory this particular part of the use case could potentially run on a public cloud right model like that's fine okay so so it's that kind of strategy so you know we talk about hybrid models but it becomes an AI workload decision-making process as to what your infrastructure needs to look like, what you can use in public cloud, what you need to keep on prem, uh what will get you to the fastest outcome. >> That makes sense.

And and that kind of leads us really well into the elephant in the room, which is cost. AI can seem really inexpensive when you're running a small proof of concept, but at scale, when you're talking about compute, storage, inference, networking, and data movement, it adds up fast. and turns into big numbers. How should enterprises think differently about the economics of AI once it becomes an operational workload? >> Yeah, I mean I think we've all heard the phrase tokconomics.

It's probably one of the most overused phrases right now. And when people think of tokconomics, they think, okay, it's token burn or token cost, which is, you know, just the model cost if you like. And that's just when you mentioned that that's just part of the equation, right? It's part of the equation around what it's going to cost you to run this in production is if you are using some kind of tokenbased model what that's going to look like but that's just only part of it and so you know a kind of total cost of ownership exercise needs to happen right which includes that hybrid approach which will include okay maybe there is token burn cost over here for this particular workload that's running in the in the cloud but let's look at what the cost is of running stuff on prem as well and that needs to be part of the equation.

Yeah. >> So we look at things like comput, storage, networking. We look at data movement. We look at you know ingress egress sort of cost. We but we also look at you know software costs power cooling. You know every time you you run a query on an on-prem model it's still taking power right. So you that is part of the equation. So you know we need to think of it in terms of a kind of unit cost for doing a piece of work a valuable piece of work end to end.

And what does that really look like depending on where it's running? It really is a complicated calculation and it's an ongoing calculation as well because these things tend to change, right? So not every model is going to be the same co to token cost going forward. So you kind of need to have this ongoing I say broad tokconomics conversation, not just the token burn, but everything else around it. You need to have that monitoring going forward so that you can adopt and change and move things to the most optimal place that they need to be depending on whether or not they need to have you know high frequency latency depending on what they're what you're trying to use the use case for.

Uh and having that kind of automated workload placement based on those kinds of cost prompts. >> That's really sensible and I think it's the thing people forget about sometimes is that uh you know the old costs are still there too. It's not just tokens. >> Exactly. You know I mean this is the thing right. So private uh cloud, public cloud, we know we've had that conversation for many years, right? So it's basically an AI cloud, right? So yeah, very similar.

So the TCO calculations are very similar up to a certain point and then there's that added nuance on top of okay, now I'm using AI. So what else do I need to incur? What else do I need to factor in when I'm looking at that kind of conversation around what's my best economic strategy for running these AI workloads? But you're right, it's it's very there's not a lot of difference between what we used to talk about with public cloud private cloud comparisons with AI.

They're just these slight nuances. >> That's right. That's right. Well, once AI is in production, even the model itself is just one part of a much larger system. So, what needs to be monitored and managed across that life cycle to ensure that the application keeps performing the way the business wants it to? I mean like we just mentioned there's there's parts of this solution including the platform that we already have pretty good solutions for right in terms of monitoring right so there's a layer of monitoring in there logging and monitoring that customers at an enterprise scale already do right so first of all make sure you have those things in place that's the ground rules right table stakes if you like then you need to look at okay what are the new things that we need to think about when you start talking about AI especially when you start talking about agentic AI Okay.

So then we need need to start thinking about okay so um the model itself maybe [snorts] drift around the model we need to look at that and have some way of monitoring that also looking at the data. So that's people often forget that right but data drift is really important. So looking at the data, looking at also the adoption of the the use cases, solutions we put out there, right? Because again, so many people are keen to push stuff to production, but then not really see whether or not people are using it and then getting the actual cost returns or whatever it might be, revenue, whatever, whatever that KPI was in the pilot.

I mentioned before, continue to monitor that going forward, right? So it's not just about monitoring technically in terms of okay, is my stack behaving? Is my AI behaving? Is my agent behaving? You know, am I making sure that I don't have any attack vectors on my model and so on, but also am I actually achieving the goals I said I would around KPIs? That's a really big part of the monitoring equation as well that people often overlook. They'll keep things running in production and but actually maybe they're just not adding the value they need.

Maybe the ROI is not what we need it to be. So again, keeping that ongoing cost in mind and that ongoing ROI in mind as well as the technical monitoring. So production AI also changes the stakes around governance. I would say when these systems start influencing decisions and interacting with company data or taking action through other agents, where does that accountability ultimately live? >> Yeah. You can't say, well, the model told me to do it, right?

You know, no, that doesn't work. >> People try. Some people try, but but no, it's that's just not going to work. So it lands in a few different places, right? So if you're using AI to go and do some perform some action at work and you're the final human in that loop and you're going to do something with that action in reality that's your responsibility right so you know we drill that into ourselves all the time so we have lots of great AI tools internally and you know we're told all the time hey listen if you use this and you do this with it you're responsible for that last action all of the buildup to that it's your decision so one one person in that loop is the user of it right but the other person in that loop I'd say especially if it's more autonomous with agents is the business process owner the the business process that's being if you like automated and made better with AI and Aentic AI they are the ones that are kind of responsible for that business process which includes all of the AI that it's using now underneath that business process owner you've got a number of different owners as well right so we talked about data products if you're in that data products world the data product owner is going to be responsible for the data that makes part of that solution ution.

You may have a platform owner like AI factory with NVIDIA platform owner. They're going to be responsible for making sure that platform is behaving. You maybe have a model owner if you're training your own models. So there's underneath that overarching business owner for the process are all of these key players as well that make sure their component piece parts are all complying to the rules that we've set up and performing in the SLAs that we decided we needed from them as well.

So, lots of different players, but ultimately the user, if it's there, if they're the end person in that workflow, make sure it's right before you press go, but then the business process owner whose business process you're actually automating with AI, those are the two main accountability tiers. I'd say >> that's a really good approach. I think uh you know, understanding that you have these humans in these key elements here who are this part is you, this part is you, just keep it working.

Make sure it's doing the thing, make sure it's not going crazy and going off and attacking websites or something. You know, you want to That's right. You you want that kind of focus. >> How much of successfully putting AI into production winds up being a technology problem? And how much do you think is an organizational problem? And once you move beyond that isolated innovation team, then who takes over and owns it? >> We always say the technology is the easy bit, right?

It's funny because it's the bit we focus on all the time, right? We talk about it all the time. We talk about our pilots being technically capable and so on. Honestly, the hardest bit really is the organizational change. It's it's and it's always been the same any any kind of transformation project, right? It's not the technology transformation, it's the people transformation, it's the organizational shift that needs to come into play. That is the hardest constraint for you to get over in these in these um use cases and we've always seen that, but it's slightly exacerbated now with the autonomy of AI as well because things shift quite dramatically in that space, right?

So organizations are shifting much faster and much quicker than they used to before. So the sort of things that we talk about is you've got to have an owner, right? So that's the first thing, right? So if you're looking at, you know, moving things to production, organizational readiness needs to look at things like, okay, who's going to be the owner? Who's going to fund this when it's running in production? So you know, where's the money coming from?

Who's going to approve changes? Right? So if I need something to change, who do I go to? Who's approving that? you know who measures the value ongoing and who says this is valuable or not. essentially who is accountable when things start going off the rails. All the things that we you know we know for sure with software is now even more important with AI software right so operating model business sponsor product owner data platform ownership we've talked about those before security and risk need to be key players in all of that right you need proper governance in there you need to have an adoption plan you need to see things are actually going on with the adoption as well so organizational shift is not just about how who's going to keep the lights on in the solution it's also about how do I encourage people to use this solution?

That's a big one, right? Because if you think about AI and you think about some of the fear around AI, people are sometimes a little cautious about, you know, I'm going to start using this for my job and maybe it'll start you do my job for me. So part of the organizational change is to help people understand AI is there to help them be faster, better, more efficient, take all the boring stuff out, right? So organizational shifts and we've had to do that internally as well.

So, we've gone through a number of different programs to help us adopt AI internally. So, not only is there teams of people making sure the solution itself is still blinky lights and performing well, there's a whole team of people making sure that people are are using it in the right way and actually using it to help them do their jobs. So, lots of different areas, lots of bit similar to most transformational change, but some nuance again with the AI, especially around adoption.

So if we look ahead a few years, what do you think will distinguish companies that that have truly operationalized AI from companies that are still running a collection of disconnected experiments? >> Yeah. Yeah. What's it going to look like? How am I going to tell if these guys are are really there or not? You know, if we look to ourselves, I think we're we're pretty much there. We've got we've got quite a few good production ready use cases and we have a good process around it in terms of governance.

You know, one of the things that we did early on was, you know, try and make sure that we had, for example, not just a set of AI pilots, but an AI portfolio, right? So, let's take a look at this as a set of services that we can either offer ourselves internally to be more efficient, be more productive, or offer our customers. These are all services. These are all products now, right? So, treat them as they are services and products. The other signs of maturity as well as you know this is a portfolio not a set of myriad of pilots running somewhere is the reusability aspect.

Right? So you can tell somebody's operationalizing when they can take a pilot that's proven all its KPIs and go boop now it's in production because I've got all of this repeatable stuff underneath that I can reuse. I've got the repeatable data products. Right? I maybe created a data product earlier on that a different use case was using. I can reuse that data product now for this particular use case. The underlying factory itself, I mean, we call it the AI factory with Nvidia because it's a factory, right?

It's meant to be that kind of just turn it and it'll come out, right? So really easy to deploy onto an existing platform that's already secure and governed. So that's another key part that the repeatability of it. And again, the idea that you're not just measuring deployment clicks, you're measuring whether or not this thing is giving you the ROI it needs to give you going forward. Ongoing measurement about KPIs, performance characteristics, all of those things.

That's a highly functioning operationalized AI environment. >> Well, Beth, thank you so much for joining us today. Before we let you go, where can people learn more about what Dell Technologies and the work you're doing around enterprise AI? >> There's a a tremendous amount of stuff and available on on dell.com which I definitely encourage people to go and have a look at. Right. So from there you can jump off to lots of different points. You can jump off to to my area which is the services part.

You can get to see all the great services that we've got like the governance services, the AI factory services, the data services. All of those things are on there as well. So start with dell.com. Fantastic amount of information about our products, our services, our approach, how we've done things, and that will leap you off onto whatever you're interested in. >> Excellent, Beth. Thank you very much for taking the time to speak with us today. >> Thank you so much. >> And for more conversations like this, visit eweek.com.

You can also like and subscribe wherever you're watching and follow EW on LinkedIn, Facebook, and X. I'm Corey Nolles. Thanks for watching. We'll see you next time on Espeaks. >>

This transcript was generated automatically from the video's captions and may contain errors.