DataStax CEO Chet Kapoor on the Data Market and AI

foreign speaks we're talking about generative AI in the database Market we'll also look at how AI affects Enterprise workflow and we'll dive into issues around the data market and large language models it's a lot of Hefty topics to discuss that I'm joined by a major industry expert with me is Chet Kapoor chief executive officer at data stacks Chad do we really know enough to handle all those complicated topics absolutely absolutely I won't use I won't start the thing like you know fake it till you make it because that's not the thing I actually think that we've learned a lot over the last 30 years of seeing different waves and I think I and I think you you know we can learn a lot from history and try to make sure that we're prepared so very true well I want to dive in I think it might be worth getting kind of the the 11 seconds on what exactly data Stacks does um you know we'll get to what they the real product pitched later but I mean just I know a lot of people know what it is to tell those who are not as familiar what does data Stacks actually do um we help customers build developers build generative AI apps that change the trajectory of Enterprises they work for and so um we have the most scalable Vector database in the world and it is based on Apache Cassandra and we have some generative AI streaming technology that actually puts information in and out of the vector database as well as we actually bring all kinds of data sources together so that developers can actually build more than just chat Bots they can they can build autonomous agents that affect core processes for companies and so super exciting time we've we've always had what we would call a data platform or real-time data platform and what is really interesting is that the market has kind of come to us instead of us going to the market because we have the more scalable data platform in the world for real-time apps and now all real-time apps are AI apps right and so and so we've just it just works beautifully for the technology that we built over the last decade so it's if someone said oh data Stacks they're they're a database company is that is that person accurate information if they say that I would um I would say we used to be a database company okay um we are we we've we've moved to being a cloud company because we we focus a lot on database as a service but most importantly we're an AI company and I'll tell you why not because AI is hot it's because what do you use data for um artificial intelligence I mean yeah yeah AI goes nowhere without data so there's there is no AI without data and we provide the most scalable data platform in the world and hence we are an AI company and so that's how I think about it I think it's a lot more than databases but we are very happy being the most scalable data platform or database platform in the world very happy with it but I think it's absolutely if you if I think James about my conversations with developers and my conversations with cios right the two constituents and Architects it's all about building generative AI apps yes oh totally well all right that that brings me to my first extremely insightful question which is it's one of these four-hour questions you could I think we could really talk about for four hours Chad how do you see the rise of generative AI affecting Enterprise workflows and obviously that's a that's a massive topic one of a one of the topics of course is that it's it streamlines things it also might might confuse certain things a certain amount of things because management doesn't really yet know how to handle generative AI what what do you see out there when you talk with clients uh this is actually a eight hour discussion not a four-hour discussion but um Let me let me it's something this is something that um are our developers are all are are very firmly in experimentation mode right anything that they see they're playing with right and so that's a good thing right that's awesome but the cios are getting calls from the CEOs and from their business colleagues and saying the only conversation we're having on our dinner table is about chat GPT why can't we do this right and why can't we make that happen and so they are definitely thinking this through so so they ask me you know Chad what do you think right having long-term relationships and us really being part of their data strategy right and I I've gone back to what I have seen not what I've read right and my whatever whatever what I've learned over the years is that every wave and I'll talk about how generative AI is different because we chatted about it before we got started sure um is every wave goes through two kinds of use cases there's the incremental use case what I affectionately call bolt-ons right and then there is the transformational use case which I affectionately call Core yeah oh yeah bolt-ons are if you're if you remember we used to you know the first thing was just let's put a static web page on mobile but guess what that was just a bolt-on then came retail banking how often do we go to branches but those the the way they shut down their branches the way that took that took time because it affected the core processes of a bank so I think those two use cases are still what our customers will go through and the type of customer you are changes the time frame in which you will adopt those use cases so smbs are doing startups mid-sized companies are doing incremental use cases now chat Bots copilots all that's happening now large Enterprises are going to do that over nine months because there's more risk you have a brand you're regulated financial services companies cannot have a chatbot and give advice when they're when the chatbots hallucinating right because they're legally bound to the advice that they give right right so so that will take longer to do but the transformational use cases is where the real differences that's where the world changes around us and smbs will do that over the next 12 to 18 months and I think large Enterprises doesn't matter whether they're regulator or not um will take about 24 plus months to actually Implement those transformational use cases well let me let me introduce a notice skepticism just for a moment I think one of the one of the challenges is going to be is that the technology as we spoke about before we started is is Relentless in the way it's changing it's almost it's this devouring in the landscape so generative Ai and you know may of 2023 is not going to be the same as generative AI in January of 2024. so the technology is this shifting Target Plus I sometimes think that the executives and a lot of the folks in the c-suite don't fully understand the technology which doesn't mean that they're you know not bright people they are but it's it's AI is complicated it's more complicated than cloud ever was it's more unpredictable than Cloud wise it's not just like data analytics it's the first technology that actually grows without human uh human help it can grow by itself can iterate by itself so technology is changing the folks don't totally understand it I think it's going to be pretty challenging so um yeah and all true so let me give you a perspective on all three right the first thing is the technology will settle down it always does every wave you think about when we started building mobile apps when we would start doing internet the web apps everybody was doing different things there was there was you know web servers were not there app service came you know and so there was a bunch of Technology just settles in I believe the stack to build generative AI apps and I'm not talking about simplistic chatbots I'm talking about things that are what we call autonomous agents I think that stack generally settles in in the next 18 months but for the next 18 months there will be a lot of experimentation by the way the generative AI stack um is getting settled in for startups now because they're building businesses on it right we have startups who have already put things in production on our products right and are going to go to million two million three million dollars of consumption over the next 18 months so that part is already started so I think I think the experimentation will continue but I think there will be a time you know there'll be 18 months 24 months maybe nine months some Forward Thinking companies I think the stack will stabilize I think things will move around the stack but I think the core stack will get stabilized like for example are you using llm on premise are you using it in the cloud are you do you have to use a vector database great are you using some AI tool chain the tool chain may change right they may have version one version two version three but they'll start having the tool chain what are you using to integrate all your data sources right how are you doing streaming how are you doing generative AI streaming are you doing it old school so those things will all settle in I think right I'll go ahead please the the second part is the cios I think oh yeah because of what they are because of how the Relentless pace of Technology I think the cios I believe are doing the smart things smart because and I and I'll tell you I'm only saying it from the people I talk to and maybe this is the best of the bunch right but um their perspective is just twofold I'm going to let the developers experiment like crazy let them let them start projects fails start projects fail and the second thing is I'm going to continue to think about where I can get the most amount of impact right so they're doing actually the smart thing which is they are not getting in and saying I need to be as technical as a developer notice I'm not using the word architect I'm saying developer right right so the practitioner the person who's using it they're giving a lot of room for the practitioner to go off and use things but what they are focusing on is what are the use cases for the incremental use cases and what are the transformational use cases so I actually think at least the ones I'm interacting with are getting it right now just for the record they are not moving as fast as they should okay you know clearly not but at some point they have to do risk assessment on the brand getting the wrong information to different people and things like that so that part is there these things work out in such a way that you will I you will miss it by about six months or 12 months no matter what but you miss a lot if you screw it up right because it is not cloud cloud was not visible to business Executives at all Enterprises that I work with right mobile walls Cloud was not visible generative AI is visible to everybody everybody right so that's my second Point go ahead you had another you had a question well I was going to say really the the you've you've brought up subject of the data market I think it's a really it's a Hot Topic in terms of the large language models and some companies are thinking well do we need our own proprietary large language model with this Niche data that we depend on a lot it's expensive to build or should we should we you know borrow a large language model like how do we handle the the idea of large language models because underneath the headlines that's the thing that's really making your AI app go so what's your Take On The World of llms um so um I'll give three different comments one is I was uh at a uh a conference for it's called Enterprise retreat in Half Moon Bay and uh we I saw openai present which was fascinating they obviously think that they are too expensive they say that they say they say that large language models are too expensive yeah the usage models are really and they want to bring the costs down because it's cost anybody you talk to anybody you talk to will tell you it's too expensive and okay and on top of that by the way there's a shortage of gpus and what lands up happening is when there's a this is simple Supply demand there's a hell of a lot of Demand right and and because of that you can because Supply is short you can actually raise prices so I think that part is happening then everybody knows in the industry that this will straighten out and flatten out over a period of time so I think that part will happen but the more important question is when you're building an autonomous agent are you actually going to be doing it off Bard or GPT or whatever else or whatever else you do it and my take is going to be no I think you will still use very large language models that you go off and call in the cloud but a large portion of what you do will be language models that your agents use locally for the work they do right I don't need my language model to run my company to recite haikus I I don't right I don't need Haiku help right I need to actually run a process and it's a very specific thing to my company which is one of the reasons why I win and I want to make sure my agent works with the language model specific for this and I think a lot of people whether they're running I think a large portion of people will run it in the cloud there'll be many people that run it in data centers right and go and make that happen and I think so number one thing you said was cost right I think that will come down but the second thing is I don't think this is anthropic open AI Google and bedrock and all that I think there's a lot more and I actually think believe it or not I think you know llama 2 from for meta is is actually pretty cool you can run it on a laptop right the competitively priced or not necessarily it's open source or it's a resource can really set that up and use it though without technical absolutely and you can take a look at you can just you can take a look at what you know go to hugging face and you'll see so many different models that are coming out so we think the future of autonomous agents is agents that have their own models that may or may not call home and by the way if you really get deep into talking to the hyperscalers they will tell you that's the future as well wait clarify that what what is the future the future is that you know agents have their own models that they work with but they also call large very large language models in the cloud but not as often as you think in other words you're saying there's something like chat GP not chat GP itself but something that is not chat GPD but very purpose built for that specific task they think about I'm a logistics language models correct so I have a large language model but it is very specific to Logistics right because I want to schedule somebody moving X to to why that doesn't I don't need you know 100 000 gpus crunching data for the world I need a little bit of that because I want weather information and things like that I want map information I can get that from Google but that's only 20 of what I need the 80 is about where is my where is my boat where's my ship where's my you know where's my plane where's my train where are the trucks right right right and that's the part that a lot of people are not uh focused on now okay so let's go to the topic that is very close to the idea of the vector database in making more companies more competitive in the world of AI what what is what is your view in that I have a feeling you have a strong opinion on that on that topic um I think um the the way llms do things they're native llm's native language is um is embeddings and the way you search for embeddings is actually through vectors right so they vectorize embeddings you need you need to make sure that you have now every language model out there in the world large language model medium language models small language model is nothing and this is something James I talked to cios about llms are nothing but part of the system of Engagement every Enterprise has this forever you have some kind of ux forget about what the ux is it could be phone it could be web it could be a chatbot it could be voice doesn't matter but there's a system of record and a system of Engagement we've done this now enough times with the web and we've done enough with mobile llms are part of your system of Engagement they are practically useless without the system or record right so what what we do we are the system of record for everything out there right what is Chet's recommendations and relevance on Netflix what is it what what what are the things that Chet has done from a FedEx point of view everything I've done at Verizon how many times have you ordered a you know latte over at Starbucks all of that six sits in Apache Cassandra and in data Stacks right so that information is extremely relevant to an llm right because llms are fairly context read they don't know anything about chat right and so what we bring is context to llms right and that has to be done through a very scalable Vector database that that that does two things very importantly it does it with very low latency and it does it with significant scale because let's be very clear if chat GPT took two days to respond it wouldn't have been successful right we need milliseconds we need absolute milliseconds and that's what we do right we do that really well and the other thing A lot of people are forgetting in this generative airspace it is not just about vectors and embeddings right it's also about the data you have on chat so you've got to do this concurrent read write that means you have to be able to you need to be able to write very quickly vectorize it and then be able to read your vectors but also be able to read the data that chat is about right independent of the vectors and so what we bring to the table is we have we have taken our absolutely massively scalable database the more scalable database in the universe right that's what a nosql database that we have and now we've completely vectorized it we've run some I was just before this meeting I was just going through some performance numbers and absolutely beautiful fast you said very fast very highly very very fast very massive scale very low latency and the most important thing highly relevant because relevancy becomes really important right relevancy is in a sense the coin of the realm if it's not really really relevant people are not excited about it yeah and by the way that's what we talked about right people will not deploy it and things like that but I think people need to and by the way all of this stuff you need to make sure it's available worldwide it's secure it's highly available right so all those things we do very very well we've been doing it for many years and so that that's why we think we have a this is a beautiful time that the market came and asked for everything that we've been building over the last five years are so I think the the really big question the question that everybody everyone's talking about in board rooms Across America and and the world for that matter is is the future of generative AI in the Enterprise we've talked about it somewhat what if we look a little bit further out even beyond the next year or so what what are we going to be talking about if we talk about gen Ai and say oh 2025 2026 what do you think it's going to be looking like then chat it's not really an answerable question but I'll ask you anyway so I'll tell you what I have conviction on and we touched on this before we got started for all the waves I've been through and rationalizing and using my past experiences and and getting us and feeling the technology in my bones um I think this one's going to go faster I don't think it's going to take that long for it to plow through incremental use case into transformational use case it'll just go it'll be relentless on how it happens right so that's one I think speed will matter in this space right um the second thing I will tell you is two to three years from now all incremental use cases all Enterprises would have implemented right I just got something this morning from one of our board members um Luke the AI right he has sold 16 million dollars of wine and he's an AI he's just an AI agent that's all he is and 16 million dollars in the last X number of weeks this is this is a chat bar this is a chatbot right and so so this this is my point is all this will happen so quickly there's the incremental use cases will just happen I can go through what is the name of the chatbot I'm sorry to interrupt it's called Luke the AI I should actually look it up okay yeah as we speak and it's called Luke the sales guy all right it's what it's called so um it's very interesting I only listen to half of it um so so I think the incremental use cases will happen I think what people are going to very quickly realize is that it is not about machines against people and this is really important I said this at a at a world economic Forum thing as well it's not machines against people it is people against people with machines yeah that's what it's so so it's not AI versus people it is people versus people with AI and so what is going to happen is that every person out there is going to have multiple agents and those agents are going to help us be more productive and make that happen and so in 2025 we will see that happening more often it will not be saturated by 25 or 26 but I think you're going to see that happen a lot more because that is the future right if I think about our business we are implementing that right now on how we run our company yeah it makes sense that the the the conflict so to speak the friction might be you people against people with with AI that makes a lot of sense I I sometimes think one of the problems with artificial intelligence and the job market is that it's not like people moving from the farm to the factories and people could be trained certainly some people will be trained to handle AI but part of the problem is that kind of training and education needed is pretty Advanced more than going from the farm to the factory ever was and so I'm not sure where it's set up to really train people like that I don't think so I actually think that I think in this particular case um I think AI will be able to write code faster than a lot of people so I think it's going to help and I think the humans will actively interact with agents like they were the digital twins and so I don't I actually I I don't yes will we need to reprogram some people on how they do their jobs the answer is absolutely we will have to do that but robots did that for factory workers as well right there was everything done manually and people the machine started doing a lot of it but they were still involved I think they will be a displacement of jobs James no question about that happens every time but I think we will come out net positive in making this happen and I think that part I firmly believe in because everybody gets more productive because of it I like the optimism of that that's a that's a it's a positive Vision to end on uh Chad thank you uh fascinating stuff uh thank you for sharing your expertise today and please do come back and talk with us again sometime it is always a pleasure James thank you very much

This transcript was generated automatically from the video's captions and may contain errors.