Komprise’s Krishna Subramanian on Optimizing Data for AI

hi I'm James Maguire and on today's webcast we're discussing some of the challenging data issues that face companies as they work to get the most from artificial intelligence in particular we're talking about inferencing we'll explain what that is and how to set up the right data strategy to optimize for it to discuss that I'm joined by a major industry thought leader with me is Krishna subramanian president and co-founder of comprise Krishna good to have you with us as always likewise Jam so you know there's been so much I mean it's been a fast two years since chat GPT came on the scene I guess it was actually November 2022 so we're right about at the two-year Mark uh so much Evolution so many companies scrambling to to make the most of it with with some success or limited success in some cases what do you see now in terms of businesses and how they're focusing on in terms of their AI strategy yeah you know as you said AI has caught everyone's imagination especially generative AI because we can see how powerful it can be um and uh you know a lot of companies have invested in training their own models that's where we are seeing a lot of innovation right now that's why for companies like Nvidia are doing really well um and as you have these models stabilizing what we're going to start seeing is Enterprises starting to figure out how to use those models with their own corporate data and that's what we mean by inferencing you know you know using a pre-trained model you know augmenting it with Rag and then using it let me make sure I understand the term so it's really it's using a pre-trained model and that means the company has put its own proprietary data into the model it it's not like an off-the-shelf model so to speak um and correct me if I'm wrong on that and and then and then exactly what are they doing with it to to call it inferencing yeah I'll give you some examples maybe that'll make it um you know easy to understand so let's say you have a model that knows how to recognize images so it knows if James is in an image or if it knows if the e-week logo is in an image and now you want to give it all of ewe's images to to find all the images that have James doing a podcast with e week's logo in it um so that would be an example where the model was first trained to recognize images using a different data set but now you're using the pre-trained model with your own corporate data for it to add value to what you have so when you use it that way that's what's called inferencing and use um that's one example another example is maybe you have a chatbot and it knows how to answer questions about your company's health benefits plans and now you know um Europe adds some new um benefits to their health plan but the model doesn't know that because it wasn't when it was trained that document didn't exist so how do you feed it updated information so that would be like using something like rag how do you augment your prompt and say answer benefits questions but use this data to add to what you already know so those are the two use cases you know two common use cases in inferencing um and that inferencing is nine times plus bigger than the training Market mive Market the the the the inferencing model is is far bigger than the training Model Market which itself seems rather large yeah I mean you could think about it in another way how many of us generate electricity I guess before solar very few of us did but how many of us use electricity almost everybody right and that's the same kind of ratio between AI model training and model influencing and use uhhuh and and in the middle of that you use the term rag retrieval augmented generation uh tell me a little about that if you would that might help sure so rag is just a technique in inferencing so basically you know a model is trained on some data and when you ask it a prompt like when you say answer this question you ask it a question you say answer this question using this additional data along with what you're trained on so that's the augmentation part so it's just a technique for augmenting a model some people also call it prompt grounding so when you have the prompt you ground it with some data these are just techniques to add more data or add more context after a model has been trained so you don't need to retrain the model you're just saying in addition to what you already know use this additional context all right so we see that emping is is a key Focus I mean more companies want to be doing more of it but what what's the challenging with livering why is it hard in in a data sense yeah so influencing uh people are trying to figure out how to use it you know how how companies can use AI uh with their corporate data and there are two big challenges U when you try to use AI with a company's own data the first challenge is who is doing the work um you know if you really want AI to be widespread in its use anybody in the company should be able to use it so any user could be giving corporate data to Ai and asking it to do something with it so now it's not just data scientists or AI Specialists uh you know it's anybody in the company so the chance of data leakage or improperly sharing sensitive data all of that just increases exponentially because anybody is using it right uh that's the first problem data governance and data leakage is a big problem uh the second problem is uh you know how do you help people you know find and feed the right data to Ai and how do you do it where it can still control what's being shared because you know you don't want this Shadow AI movement where willingly everybody's just sharing something and then it's kind of out of control right so that's that you know systematic management of the AI data workflow with it oversight um people are still trying to figure out how to make that happen it it seems like it's a real problem issue because I'm sure that Executives across the country if not the world are thinking oh gosh we can't be feeding all our proprietary data into this AI model and we need to have only very few people feeding it in even even they are suspect and you know once we let go of our data who knows what's going to happen with it so it's I this feels to me like it's a very new area and an area that seems very perilous to many Executives yes I think you know if you look at a lot of the surveys around AI even our own surve we do an annual survey James just to find out the top issues for unstructured data uh and you know the for the last two years you know the feedback we got is we're exploring AI but really not using it um because I think most companies are sort of in this they're kind of paralyzed by it they don't know how to use it with their data I would agree with you so it's it's kind of an irony in that they're storing all this data and there's so much value to be made from the data to be gained from the data but they're not really sure how to get the value from it yeah and I think you can start and there I mean the good news is there is some places where you can start you know so the first step is just actually indexing the data you know if you have all your corporate data and you have some way of analyzing and indexing it across all your different data stores then you know that's a start because you at least know what data you have and then you can create systematic data workflows around it and you can you know and you do need some automation you need some tooling to do that uh so that any user could use AI but the workflows are automated by it uh so there are solutions uh you know data Management Solutions um that are starting to address this problem and and who would do that particular work that that you just mentioned is that only the department or the data professionals who's actually doing the indexing of the data yeah it's a great question James so we have customers who are actually using comprise right now for some of these Ai workflows and I can tell you that uh it's usually a collaboration between the business users and it so you know what they do is they have business users um generating queries uh using a like in our product for as an example our product indexes all the data and then we provide ways where a scientist could go in or or a in a research could go in or or a you know student could go in and they could Search and say I want this data this is the data that's useful to me and they could create queries like that but moving that data to AI you know that policy it has to approve it uh and once they approve it when we move the data we also keep an audit trail of what got moved because that's the other thing for data governance you do need to know what was shared with which a process when by home you you want all that information definitely well that's that's one one idea to help anything else you'd say in terms of recommending best practices you know getting companies to be successful in terms of their their data yeah I think you know also good uh policies are important uh because I think uh you know if if users don't know even what they're supposed to do and what they're not supposed to do with AI uh then you know uh you you're kind of letting everybody just decide what the policy should be that's not good so having good corporate policies uh and then having good education for people on what the policies are and then following it with some systematic workflow management I think that's like a good three-pronged approach and this area is evolving rapidly so the other thing I would say is you know start in small and then you know it's going to you're going to learn continuously but of course starting small means your data has only so much value I mean the more data you can put into that model the more valuable it becomes you're saying well true but it's we we need we still need to keep in a narrow diet a data diet for the model well I was many it start small I me start with specific use cases not so much that you put the data on a diet but maybe you pick a use case first like if tagging images is your use case start there you know that could be a good use case maybe identifying sensitive data is your use case so do that you know maybe transcribing you know um chat sessions is your use case my suggestion is pick one or two use cases where you want to apply AI you can apply it on all your data but you systematically build that workflow you learn from what's working what's not working then you can extrapolate to other use cases yeah makes makes perfect sense anything else in terms of what company should know to to create a best practices to strategy for their data I think you know the biggest thing I would say is you know always you know uh analytics is like the first step to any of these Journeys uh because it's hard to manage something you don't know and uh the problem with unstructured data is you don't know how much unstructured data you really have you know you're in the dark so always start with the knowledge you know know first uh and you'll see that it's not as difficult as it seems H well I like that optimism of that I think companies might say yes but it's it is difficult but I think if you you know start small and and make progress It's it's a good strategy um looking to the Future because I think you know companies really want to know where we're where we're going and how can we get ready for that now what what do you think in terms of the future of infine if infine as it pertains to Ai and best data practices what do you see like in the next so you know one to two years I think a fascinating area James uh where I think we'll see more uh work uh is a human AI interaction um so you know these I think there's right now people are thinking oh AI is going to take this over uh and then people will get cut out I actually think reality is not going to be that it's going to be that there are going to be some AI helpers just like you know just like software helps us uh you know AI is going to be the next step you know it's going to assist a human and there's going to be human oversight over everything the AI does and so that interaction how do you make it easier and how do you enable iteration over it so maybe there's an AI assistant for a human then the human has validated it and after it's been validated something else happens so I think that kind of you know human enabled AI workflows uh I think that's what's exciting that's where I think there's a lot of opportunity in the Enterprise well I mean I think it's you're getting into this area I mean I hear so much of these days is it agentic AI so the you know the AI agent doesn't just give an answer like chat gbt might it's actually the agent does a series of tasks and really does become a digital worker yes yes yes yes yeah is is that I mean like guess is something you deal with in your day to best practices or that's not you don't get into agentic AI at this point no we can I mean even uh agents uh require data for their work and that's where our intersection would be you know we focus kind of on that bridge of data to Ai and and I want to make sure I give you a chance to talk about how how comprise can help customers with this journey if you kind of put that in a nutshell like how how how ISS comprise helping customers yeah so comprise you know analyzes moves indexes and manages unstructured data meaning any data that's not in a database uh and so you know um where we can help is again you know if you want to know homogen structure data you have or you're wondering how you can build data workflows for it those are all the things that compris to us great definitely a market for that um chrishna thank you so much for sharing your expertise today and please do come back and talk with us again sometime thank you James it's a pleasure as always

This transcript was generated automatically from the video's captions and may contain errors.