hi I'm James Maguire and on today's webcast we're separating the hype about artificial intelligence from the Practical reality we're talking about the top generative AI use cases that are providing real world value for business users and we'll discuss the future of generative Ai and how businesses can prepare for that right now to discuss that I'm joined by a leading voice in the field with me is Muhammad Ali senior vice president of IBM Consulting Muhammad thank you so much for joining us today James it's such a pleasure to be here with you so I think there's a a certain amount of confusion out in the in the business Community you know managers are very excited about AI they want to get on board or or get more on board but they're not sure how to do it I think maybe it's magic we we bring in the AI we sprinkle it over the software it makes everything's okay although maybe they they realize it doesn't quite work if you get practical right what what do you think about you the top generative AI use cases that are driving real business value for for actual real world clients yeah I mean James you're right there's so much excitement over this technology I don't know about you but you know in my career I don't think I've ever seen so much investment going into a single technology so fast never never I know what you mean exactly right tons and tons of excitement and you know I think in the consumer space I mean we're all so excited playing with it to you know write emails write poetry plan a vacation but you know for businesses it's sort of not a game you've got to get it right and U and I think that's where the challenge uh starts right like how do you pick the thing to focus on and then how do you get it right you know someone said to me um they were talking about the the strategy that some business use they say well you know we're going to go for the lwh hanging flute uh fruit first the easy stuff other and then this executive said to me no no don't do that because sure it'll be easy but it won't really Drive much value so go ahead and and be you know go towards the ambitious project right off the bat so people can point to it and go hey this really counted for us I mean do you have an opinion on that issue I do I have a very strong opinion on this issue so you know after I graduated from college which was on your side of the world there I actually started neural network company this is back in the early 90s and I remember like one of the first things that we try to do was try try to diagnose medical conditions and that is a really really hard problem but but it turned out that there were actually a lot easier problems that we could solve much quicker and much easier with neural networks and the same thing exists today you know a few weeks ago I was with the CEO of a large Energy company you know very large Energy company and he said Hey listen you know if you could help us predict the weather we can station our our equipment in in the right place when a storm happens wow and I said really that is an extraordinarily hard problem I don't think that's should start right where I think we should start is you have all these customers calling in and they have all kinds of problems and guess what generative AI is extraordinarily good at parsing through all kinds of uh documents and coming up with the answers for your for your customers and it turns out that if you look at you know we've done about a thousand of these generative AI projects uh over the last year or so and one of the things that we're noticing is that in fact the low hanging fruit has a lot of opportunity so customer service is one of the top three areas where we see our projects driving the most value the other being coding right like writing code um because well it happens to be very text based and you know you can you know generative AI is very good at text things and then and other areas of uh what we call employee productivity and you know I can go into some of that but I think those are three areas of low hanging fruit that have tremendous benefit like 20 30 40% productivity gains right there right okay so you you do believe in the in the value of the Ling and fruit it makes a lot of sense yeah I do yes you know interesting I don't know what my question is here but I think it's interesting you know AI is both a consumer and an Enterprise technology so I see friends you know searching for businesses using we we search for a bike route over over the weekend actually chat helped us and you know people are using it's the right thing so it's right it's it's remarkable it's not I think the adoption rate is really quite rapid it's not like it's just people are people are into it yeah I mean it is one of the fastest technologies that have gotten adopted right I mean if you think about it the internet took a while flout took a while the mobile phone took a while been 18 months and everybody it seems like everybody on the planet is using it you know I was going to London with my daughter and we used it to plan the trip the things that she's interested in um so yeah I mean the adoption has just been incredible but you know the consumer space um a little bit slower in the Enterprise space for the reasons I described but even in the Enterprise space I mean every CEO realized that they have to lean in otherwise they might get left behind so the adoption is quite incredible well all right talking about that lwh hanging food so I know that sub businesses have started small with AI but they want to develop their deployment what about some strategies for scaling AI across the Enterprise and driving adoption that seems like it's a big issue yes so we have seen a lot of this because like I said we've now you know our Consulting business we're about 160,000 Consultants so we see a lot of projects and like I said we've seen about a thousand of these projects some of them we've done some of them we've been brought in after the fact and and the after the fact is interesting because client will do a proof concept it works it works really great we're ready to scale it then all of a sudden they'll realize hey we forgot to secure this thing right so we didn't put the right governance around it so we have to kind of sort of take a step back go figure out those governance mechanism because well for one thing you have to make sure they're secure right there're new there're new things that we didn't we didn't have before like prompt injection so when you issue a prompt and it calls an llm how do you know something isn't intercepting It One Direction the other direction so people are doing this so you actually have to deploy security to prevent that so there's a whole set of new attack vectors on AI right so first thing you have to secure it second is you've got to figure out well is there personal information being passed around here is it creating biases right is it in ining in other people's intellectual property how much is it costing every time I do one of these uh uh prom calls if I have 10,000 call center agents you know is it going to cost me tens of millions of dollars to run this thing right so you've got to put a governance framework in place and so we often help clients sort of back up put the governance framework in place then put the the the generative AI techn you know Tools in place so they can they can consume it so yeah I mean I mean there's a there's a lot of starting small but in order to scale you You' got to put those foundational items in place and then the second piece about cost is really important so I was with a a um a customer the earlier this week and this customer is a very very very large um uh uh customer service provider let's say you James I'm sure have used their service many many times okay so I bet yeah the guy said to me look I have 10,000 agents and we built these these L you know we have this LM and this prompt calls it and it's consuming all this it's very expensive it's consuming all these tokens what they're called and we can't R it so what we've done is we've created a kind of a new interim thing called prompt caching so if you're G to ask this thing the same question it doesn't go to the llm which consumes you know oh costly it it it knows that somebody else ask the same question and a respond so all kinds of techniques now are coming into existence to manage cost so in order to scale I think there are two big things right this governance and then how do you manage costs as you as you uh as you expand usage you know it's really interesting about the prompt caching it makes perfect sense and it seems like if if 500 agents have asked that one question we don't really need to bother the llm itself for the answer that the answer is right there in front of us we don't need to incur the C I think we just hit in a new startup company it's like AC promp cash.com let's started I'll I'll uh you know I'll I'll contact andreon Horowitz we'll get funded by the end of the week no doubt about it that's right you and me well I want to go back to something you said I thought there was really interesting in that um you talk about intercepting the prompt between I guess the the this the software and the llm that is feeding into it I always thought I just for my education I thought that like sort of a pretty clear-cut API path there that probably would not be be intercepted but am I wrong about that is that is that a possible interception point there yeah this is a real thing and luckily there's now soft there's security software to actually um uh you know tra for that right so we actually in our implementation within our organization we um use a product call polar security and polar security allows us to do prompt injection checking right and so and James just to give you a sense within Consulting we are big users of uh of gener AI as as tools right as sort of digital worker tools to to a our consultant so our vision is um and we're we're sort of on this path already is for every of our consultant they have 10 agents that are helping them so 60,000 Consultants you have 1.6 million digital workers wow so we actually have to build a governance platform with security so that as they use these agents it to help them and these agents are calling llms this is a secure environment so by building it for ourselves you know we're also able to advise our clients on how they should do it you know that that is really also quite F fascinating so there's 60,000 Consultants they each have 10 agents I guess I'm curious I mean know 10 is an arbitrary number could be eight could be 11 I I'm assuming but I mean it seems like they would not need that many and that you is there not like like a versatile handful a five I you tell me right right right so this is interesting right so it's 160,000 consultants and that's why we sort of came up with this 1.6 million okay yeah all right and so you're right it doesn't have to be 10 right so we just sort of picked the number that that we thought sort of made sense because today in fact when you use chat GPT you're using one at a time right ask question it's coming back right and this is this is what we call assistance in the in sort of the the AI world but that's recently changed and it's moved to these things called agents so an assistant you ask it it responds if it doesn't know the answer it tells you it doesn't know the answer right an agent is different an agent you ask it a question and it says hey James I don't I don't know the answer would you mind if I send an email to your colleague uh an and ask an if she knows the answer sure so then the agent goes off and it's it's doing that and it could be doing multiple things why you wait for that agent to come back right that agent could be slow because it's not just compute it's waiting on another human being right okay and so you launch that agent and then you're you know then you want you're doing something else you launch another agent and you launch another agent and we believe that they will come it won't be that long where we're launching many agents and so we just picked the number we say said 10 so we wanted to build you know a a a governance layer but not just a governance layer a compute infrastructure underneath that would not be that that would be cost efficient to run even at that scale at 1.6 million digital workers because right now with a computer infrastructure that exists we will actually not be able to afford 1.6 million digital workers so by having that Vision that's where we need to end up it allows us to then architect our internal platform such that we can get there and so yes so whether it's eight or 10 or 12 it's you know it's less important but this concept that you know you as an individual will have multiple agents working for you simultaneously ly is where is is where actually the technology is today all right yeah it's really fascinating and I I know there's a startup here in the Bay area that is working or has has already achieved the idea of of a very functioning agent where say you want to buy a refrigerator the agent will go price the refrigerators they'll make the decision they'll work on getting it you know into your house they they really it's it's a f it's willing to do five or six seven eight steps while you just wait for the refrigerator to show up so I guess that's really what you're talking about in terms of the agent yeah yeah and I don't know James if you got a chance to go to dreamforce um not this year go I go most years but not this year yeah it was incredible the whole thing was about salesforce's agent force right how it's really all about the agents going forward and um and in some ways uh you know Salesforce is a very close partner to us at IBM and many of the uh uh technology that they're bringing to Market um we have a joint uh relationship with we are powering it with some of our Technologies um we are um you know helping them deploy this in the field but um but but dream force was fascinating because it was almost 100% about agents they were literally like uh uh you know costumed uh characters that look like walking around right and so um you know it's it's not just us it Salesforce is actually uh I mean I think within the next 12 months you know that the the industry will migrate to agents well I guess it's so if I see someone using chat GPT I could say hey that's an assistant that is so 2023 I'm into agents myself I'm into 2025 so I can sort of look down on them because I'm in agents not assistants yeah and uh you know sort of the the agents sort of live on top right so you know chat is an assistant application in some ways that live on top of the llm right so the llms will evolve on their own right at their own pace but the things that live on top are becoming more sophisticated and you know I'm sure there will be the agent version of Chad GPT and there'll be the agent version of you know everybody's uh AI product out there right right you know you've talked a little about this but I think it's important I want to make sure we get back to this you what what do you think about critical considerations for cyber security and AI governance in an open approach what what do managers need to know and I know you did touch on that a little bit I just want to make sure I get your full thoughts on that one yeah so look you know we talked a little bit about cyber security and how important it is right and so um you know we we've done some studies and um you know 75% of CEOs out there and cxos believe that they you know they have no choice but to move towards gener ai ai to increase their productivity of their companies but less than 40% of them believe that they have the security in place to do that which interesting because nonetheless they're going because if they don't they're competitors will right almost an existential threat so um so so core security is really important but long term uh you know you sort of mentioned this word open We Believe open is really important because when you when you have uh uh neural network models where you don't understand what's in these models and they're locked down it's it's actually hard to know how to secure them either you know kind of core types of security or um you know things like hallucinations you just don't know what's in there and so there's been a whole wave of companies including meta IBM and others that have been bringing open models to Market where you know you know what's inside you know what the weights are you even know how the data you know what data is used for training it and so visibility throughout that that um that sort of supply chain that builds the the AI technology we think is really important if if you're going to have you know a good shot at securing these things well does you said does does the open format make it more challenging or less challenging I want to make sure we think it we think it makes it more secure because there are more people really looking at it and understand where where the where the faults are right so that and there are more people building solutions to protect against those you know those points of exposure sort of you know more eyes on the problem um we think creates you know more secure environment yeah makes makes perfect sense um well I want to I want to get your sense to the Future where we're going in the future because I think that's that's really the the big big question in this topic I guess hey what what do you think about the the transformative effect on J on on the Consulting industry touch on that and then we get to the future but generative and Consulting going forward what what is your sense of that yeah I mean we we think that that it's going to have a massive impact on the Consulting industry and so uh you know as um as I I mentioned to you before the call I I rejoined IBM about a year ago and uh and before that 14 years ago I worked at IBM and I got a chance to work on many of the software products and so you know part of the reason I rejoin is because I saw this opportunity for significant change and it's exciting um I mean it's it's a little bit you know there's some trepidation in how right go through this um but but with this level of productivity I think we'll be able to do so much more than we ever could before and and I think there's going to be actually more demand for we so the type of IT projects that you might have not done before and accumulated a lot of technical debt the fact that now we can do those projects quicker faster lower costs people are going to deploy their it budgets to do those things that they weren't doing before and so but in order to do that you know consult companies are going to need to deploy these digital workers to and these agents to support their Consultants to work more efficiently and it's a big part of why you know instead of us just sort of adopting generative AI in kind of a raw sense we went out and we built a whole platform platform that provides the governance the security um and we call it IBM Consulting Advantage it's an internal platform we're on our third release of it and um and it allows us to build these thousands of Agents on top of it and then align those agents to our particular Consultants so that you know they're getting they're getting agents that matter to them so like you James as a reporter you'll get a pack of you know say 10 agents that are specific to you and what you do so we're investing heavily in it we're also he investing heavily in skills development of our team member in order to take advantage of this but yeah I I believe there's going to be a significant transformation in the Consulting industry and uh you know the companies that invest in this type of approach are going to be the ones that are successful interesting really interesting I love the idea of the agents and and each of the Consultants having multiple agents that that is fascinating if I think as a as a closing question about the future of business I I think um one of the problems with that question is that ai ai has so much potential but there's really it's hard to know exactly where it's going I think even some of the smartest Minds don't know exactly where it's going because it's going to develop exponentially and it has the ability to develop by itself to a certain extent it's one of the only Technologies maybe the only technology humankind has ever created that can you know develop by itself to a certain extent uh so I guess I'd ask you what is what what is your sense of the future of gen in business and then as part of that does something anything worry you about that what's your worry and what do you see as the future James a very good question so I think there there are multiple Paths of geni right so there's the consumer path which uh uh will have sort of its own uh trajectory and then there's the business path and I think in businesses uh businesses are going to be much much more careful about how they deploy gen because they can't afford for it to go wrong like it could be destructive to your company and so this is a big part of why we built that IBM Consulting Advantage platform which is an internal platform for ourselves because we need to ensure that how uh generative AI is used is well governed according to good policies and we are U you know sort of uh containing it and in in a way that allows it to be productive for us but um you know but but not do things we don't want it to do not sure that's going to be the case always in the consumer world right because world moves faster um in some ways it um you know it's a proliferation of potentially smaller uh uh entities that sort of do what they do in businesses we you know we can't afford to do that right because one error and you know it can affect the business dramatically and in some ways you know for us at IBM uh you know we're sort of in in some ways uniquely positioned right for for many many decades you know I ibbm has taken a very measured approach to technology you know ensure that there's trust around technology and um and you know in we are the only consulting company with a full technology company inside of this so so we have the ability to sort of think about how to balance these things and also think about it long term I'm not sure you know everyone is necessarily going to take that approach but that is the approach that we're taking and that's the approach we're taking for our clients that's interesting uh Muhammad I think you said it uh fascinating and and thank you so much for sharing your expertise today uh please come back and talk with us again sometime James it was it it was great pleasure to be here thank you so much
This transcript was generated automatically from the
video's captions and may contain errors.
