BMC’s CTO Ram Chakravarti on Generative AI in the Enterprise

foreign speaks we're talking about the only topic that anyone is talking about these days when it comes to Enterprise Tech which is of course artificial intelligence we'll look at what's driving AI in the Enterprise along with the challenges of this emerging Tech and how to handle them to discuss that I'm joined by a very special guest with me is Ram chakravarti CTO at BMC Software Ron very good to have you with us today James pleasure entirely mine thank you for inviting me to this August forum and for the five cents worth of my ideas and perspectives great all right so artificial intelligence I can't believe the amount of surge of interest that is getting since uh chat chat EPT came in the scene last November seems like the amount of headlines I've never seen more Buzz for for a new technology than AI of course AI really isn't that new the Enterprise uh from from where you sit what trends do you see as companies rush to adopt Ai and they they do seem particularly eager to adopt AI these days yep absolutely I mean as you rightly observed James this is the topic the preeminent topic uh and AI in general and generative AI specifically have taken the tech World quite by storm and that impact is being felt across companies and let's also acknowledge that this is not AI is not new companies have been at it for a few years now but generative AI has served as an even greater Catalyst if I may in terms of the buzz and the potential uh opportunity opportunity potential that it offers now you probably read as much or more as than I have that surveys seem to indicate that CEOs across Industries are largely enthused by the potential that gen AI offers and are really looking at it as a source of competitive Advantage uh differentiation as well as something that promises a lot of productivity gains now on the flip side in my conversations with cios and ctOS whether they're BMC customers or otherwise it is evident that they are looking at the benefits of AI but also weighing the risks from an implementation standpoint as well as from security data privacy accuracy the ethics and all of these considerations given the nasancy of generative AI in particular and then they ask the right questions about implementation and operate operational considerations and these are the absolute right questions to consider with the new text such as gen AI right I mean the promise is that it can automate processes it does summarization well captioning insights and recommendations we which is kind of what we are at the Forefront of as an Enterprise software company as well but you also need to consider that a lay person looking at an open AI chat GPT or something else to get uh responses is very different from Enterprise use cases right so that's where the AI models what the choice of the AI model and more importantly the choice of the Enterprise knowledge repository or the training data is really really considered to hyper focus and this is where internal data sets become super important and while the benefits are compelling you also want to kind of consider the downside if you have misuse flawed outputs leakage of sensitive data or inherent biases because of a poorly trained model right and for example if these models are trained over time on and feed on self-generated data that has a set of negative consequences as well so that's the way I look at it people are basically in the early stages and our customers are looking at us in terms of okay you are a trusted partner you've we've done business with you and you're talking about gen AI what is it that you can provide and it's not just providing something but how can you help us operationalize Implement and operationalize this that's where it really becomes real you know curious you talked about talking to a lot of Executives about Ai and generative AI do you get the sense that they really are very eager to invest right now or of course they've been somewhat interested in investing all along for several years but I think it's a real no eagerness or is it like well we know we need to but we're still going to think about it a bit more what's your sense on that angle no great question I think it's a pragmatic approach people are in uh intensely curious and rightly so um I think a lot of them want the onus is on the vendors that are basically inclusive of ourselves who are basically who believe that there is an opportunity and we believe that we offer something differentiated so the onus is then put on vendors such as us to prove what is it that is different about us and so I would characterize it as a pragmatic approach for the most part with our customers yeah the point being that okay come in do some hand holding Let's uh start with the use case let's do a proof of concept show me the value a quick win then let's start talking about adoption across the board or scaling it up plus there are multiple people so what do I want to choose what uh foundational model do I want to choose those are all considerations that they're grappling with you know you also you mentioned the challenges and I think of course they're numerous there's sometimes the Gen AI programs are inaccurate there's bias sometimes employees put in proprietary information and that causes all sorts of problems what's a particularly Big Challenge that the companies face when they choose to deploy AI as you see it great question I mean that that is a multitude of challenges right so I'm going to put them in three categories and then elaborate on a couple of them sure I the three categories are organizational the second is data and Technology the third is what I call operational and organizational challenges I'll highlight three or four uh lack of executive support or Waning executive support is one second is the talent and skill shortage that organizations face the third is it's inadequately funded from a budget standpoint and fourth in this macro environment there can be a change in business mandate and priorities so uh fatigue sets in and if particularly if you don't deliver value against the promise that you offered so those are some of the organizational challenges from a data and Tech standpoint data quality is by far the biggest issue and I'll elaborate a little more on that data availability becomes another important consideration because you need the right data to train the model and for the inferences that you can get and then of course you need to integrate these with your existing systems and new sources of data and that by itself presents a fair bit of challenges the spotlight particularly with generative AI is even more so on the quality of the data that flows through these analytics pipelines as an example and then from an operational consideration uh there's infrastructure performance what we talked about uh are alluded to earlier ethics and biases come into uh play the Regulatory and legal considerations also come into play so let me maybe elaborate on a couple of these right so data quality absolute Biggie and most people uh look at this as the number one challenge right the point here is that if companies apply Ai and generative AI in the wrong use cases and with and or with bad data the consequences can be severe misuse flawed outputs leakage of sensitive data that I alluded to and the ramifications from a business standpoint are disruptions to the business compromise data integrity and seriously a loss of reputational or a reputational risk with customer dissatisfaction so those are that's from a data quality standpoint from an operational consideration standpoint be really the challenges are about ensuring reasonable implementation and addressing vulnerabilities as well as IP concerns so particularly in AI generated responses issues such as hallucinations relate that are related to security as well as ensuring that sensitive data is not leaked or other additional big considerations my takeaway in response to your cons your larger question is that llms are still at a nascent State there is tremendous height a lot of it is Justified over large language models because they allow end users to interact with systems much like I'm talking to a friend or a colleague maybe it's a little bit of a hyperbole on my part but uh you get the points right it's the potential for democratization of what were seemingly complex tasks at one point in time is considerable now what I would suggest is basically organization should really look to apply AI to very specific use cases using domain-specific models in I mean when I say use cases these are Enterprise use cases using domain-specific models that can deliver immediate value or quick wins and your prior question to tie all this back together they can't expect Nirvana it's not going to wholesale transformation and uh reaping the benefits is not going to happen overnight but again I have a structured approach that can potentially help you that's the way I look at it well I think the the really interesting point there that a lot of companies are grappling with is the idea of a large language model it seems like it's it's getting to this state where do we need to build our own large language model or you know our competitors llm is is is is larger than ours and maybe ours is not as specific uh what is your sense if companies are grappling with you know do we need to create our own large language model what what what's your sense of that yeah so um large language models are popular it's about the right horse for the right course right it's really about the Enterprise knowledge repository that you have for the training of the uh uh model and it can even be a small language model but here is the fundamental uh issue that I see uh particularly when we talk to our customers they are busy running their business and doing a whole bunch of I.T and operational things that are fundamentally important to run the business they don't not many of them have the WHERE with or to go and pursue a line of Investigation or the skills in-house necessarily at scale because people are busy doing other things to dedicate the time to go and build this from the ground up and that's where we uh basically look at it in terms of to the extent that we can help fast track it with domain specific large language models that we've pre-built and then can integrate with their Enterprise knowledge repositories and train it gives them a head start and not necessarily an issue of a comparative difference between me versus my competitor in terms of my ability to build something from the ground up all right so I think I want to ask the question that would maybe take four hours to really answer fully but I'll ask it to you anyways uh so you've talked about this a bit but what advice do you give to companies as they seek to optimize artificial intelligence it's a big topic they know they need AI for competitive Advantage but what should they be thinking about as they really make the best of it yeah um so here is my take right it's not just about AI it's about orchestrating complex data pipelines and building on your existing technology Investments to streamline information delivery for each use case and these become inherently important to achieving growth and competitive advantage or whatever else your desired business outcomes right and companies have immense amounts of data that's only growing exponentially by the day and used the right way to add business value and or monetize it can this becomes the fuel that drives differentiation and increases cooperational efficiency but there is a challenge the data is complex so how can companies hardness date the data to the benefit that they desire without creating incredible strain both in terms of the technical uh strain as well as the resource trade and the second important question is also how can companies use technology Investments that they've already made and kind of extend the scope of their technology Investments I don't want to use the term modernize but it kind of is modernizing in some way and advancing what they deliver and AI is the answer to that right so the way I look at it is very simplistically Ai and data are intertwined you need good data to train AI without which the AI method is of little value conversely with the cons I mean the constantly increasing complexity of data it's difficult for humans to fully unlock the value and that's where AI is required to unlock value from data and ultimately if you really distill it down to the basics AI enables two categories of use cases one make existing things better or improve outcomes for existing use cases and number two make new use cases possible that were inconceivable a while back so my advice to customers against all this backdrop is there is really no foolproof approach given how quickly the technology is evolving and new things are coming by and it's an even more exponential storm of Technology um that's uh being foisted on them so the guidelines then are abide by good design and implementation principles to conquer complexity one use case driven approach based on value and time to realize value second not just Innovation but operationalizing Innovation number three use the right horse for the right course in other words you use the right AI method and do not over index on the technology number four don't boil the ocean as the cliche goes start small with one to three high value use cases and ensure that these are feasible the data is available the data quality is good appropriate AI methods and tools are open invest in the data quality for the at the outset for these use cases and going tying this back to the skillshot shortage go with trusted partners for at the outset and from and even over the course of time put the onus on them to help you deliver quick wins as well as scale and operationalize and this includes not just enterprise software vendors such as us but implementation Partners as well I could I mean like you observed I could probably uh go for hours on this I'll leave it with just one more it's a bonus if you have the killer resource what do I mean by the killer resource it's a resource that has the judicious mix of both the technical expertise as well as a functional domain expertise that is the resource that's going to make a difference otherwise you're going to waste Cycles on bringing in external technical help internal functional domain help and I mean that kind of flies in the face of a quick win as well as internal ownership of the asset or the use case and value delivered and from a longer term standpoint those are some of my uh advice I mean that's some of what I would advise uh customers yeah good points to be sure right I think the one that sometimes doesn't get as much attention is what you said about data is that it seems like before artificial intelligence came along and took up all the headlines the big topic was data analytics it was what everyone was talking about and really that hasn't gone away it's actually even more important in the world of AI I mean if you want to get involved with AI and make the most of AI you need as you say a solid data infrastructure that's that's really a big part of the piece yeah absolutely uh James I mean um right wrong or otherwise the spotlight on good quality data uh is going to be even greater so it behooves organizations to if they haven't already to invest in their data infrastructure ensuring that data quality and availability are I don't want to say a given but you're reasonably confident uh of those aspects before you embark on uh the new use cases that you want to drive with AI talk about BMC in particular so how is BMC addressing the AI needs of its clients it's a big job to be sure uh yes it's pretty daunting and we're up for the challenge let me maybe give you a little bit of a perspective on our view and how it drives every uh investment decision that we make right we at BMC we believe that every company continues to be on a tech transformation Journey the destination to which we Define as the autonomous digital Enterprise so our mission is to help our customers in this journey by harnessing the power of AI to unlock business value from data by bringing best-in-class capabilities in our domains of expertise these being observability orchestration and automation as well as ensuring the resilience of their hybrid technology landscape so we encapsulate the value that we deliver in a term that we've defined as connected digital Ops and this becomes the umbrella framework which comprises five areas of innovation for us and these five areas are AI Ops service Ops devops data Ops and autonomous Ops so I would basically say that we are hyper focused on harnessing the power of new technologies specifically AI to exploit intelligence as well as the Need for Speed and flexibility while balancing quality within a customer's environment and a we've embedded AI in pretty much everything that we do across these Five Focus areas and generative AI in particular so I'll outline this with two aspects of our product Innovation that is emblematic of the larger AI driven product uh Innovation or better said maybe harnessing the power of AI and data to deliver value for our customers in our domains of expertise so number one observability and AI Ops we offer two AI absolutions BMC Helix AI Ops for distributed systems and BMC any AI Ops for the Mainframe and they both are based on a common Helix platform as a foundation and we offer five areas of differentiation in our AI obsolution one is open observability and the high availability highly scalable Cloud native microscale Services based data ingestion platform right and what we do is we provide full stack observability here where it's not just your usual suspects such as metrics logs events and traces we also integrate APM data network data Mainframe data to ensure that full stack observability is provided that's differentiator one differentiator two automated service modeling which ties an overarching business service to all the technology components that enable that business service number three causal AI for Better Business outcomes uh root cause analysis with AI is one such example and it's an example of doing a traditional use case better in other words making existing things better which I talked about number four a category of AI use cases that we call predictive AI for continuous service optimization and this is an example of AI making new things possible where the AI is constantly working looking for patterns and data and identifying service insights that they may not be aware of and it actually has an extension of the organization and it could potentially tell you based on historical data analysis that a service degradation is possible next week on Thursday between 7 and 8 AM or something like that for a particular business service because of device X as well as recommending a set of corrective actions so that's our value proposition in uh observability and AI Ops and what we've achieved as a result of that is an industry leading position a recent Forester wave as of June 2023 would does firmly as the leader in process Centric AI Ops so that's bucket one the second is a huge thrust in generative Ai and this is an area of focus across service Ops devops and autonomous Ops in addition to AI Ops and data Ops so back in uh we've had I mean generative AI is not new to BMC right so we've had generative AI capabilities as part of our Helix service op solution suite for more than a year and we made the formal announcement of BMC Helix GPT which is our domain specific large language model that we rolled out and we've got a baby of use cases and we are working on a whole slew of use cases across service my Ops AI Ops Automation and a whole host of others and devops also to enhance the SRE experience with BMC Helix GPT so the point is we are have our domains of expertise are clear so we are focusing on our domains of expertise and customers Enterprise knowledge repositories so it's not World Knowledge repositories regardless of what Foundation model that that we deploy at our customers based on our customers choice or based on our choice right uh and it varies based on the customer on top of this what we do is we train the model with the Enterprise knowledge so you're basically limiting the knowledge sets to Enterprise knowledge repositories so you're already minimizing hallucinations and other things that can come from World Knowledge repositories but that's not all we've got three sets of guard rails that we've put from the outset uh with respect to training then fine-tuning the model which I talked about second is a prompt engineering and a context grounding layer and the third is an additional safety layer that basically eliminates anything that might seem erroneous based on the response to a query so we're doing all of that and the proof of the pudding is in the eating and we've had tremendous conversations with customers tremendously positive conversations with customers in terms of interest but again the oh it's early stages the onus is on us to go and help them and hold as needed deploy and operationalize and help them realize value that's how I look at it well I think you said it I appreciate the product run down and also the advice about deploying AI as you say it's early stages I think companies will need a lot of help with this they'll be looking for help uh thank you for sharing expertise today and please come back and talk with us again sometime my pleasure James it's always a pleasure to interact with you and uh the folks at EV so uh I'm up for it I would love to have a follow-up conversation and thanks again for uh inviting me

This transcript was generated automatically from the video's captions and may contain errors.