Alation CEO Satyen Sangani on Trends in Data Catalogs

hi i'm james mcguire and today we're talking about all things data catalog we'll look at exactly what a data catalog does and we'll talk about some tips and best practices for optimizing a data catalog to discuss that i'm joined by someone who knows about as much about data catalogs as anyone in the world probably uh with me today is sachin sangani ceo of violation sachin thank you for joining us today hey james it's great to see you again absolutely and i think we met might have been 2016 2017 we never did figure that out but it was it's been a few might have been even before that yeah but it was a great dinner and you were the best one of the best things coming out of that dinner great so speaking of best things there's an amazing piece of news that just broke elation raised 110 million dollars in series d funding the last time i checked 110 million dollars is a pretty serious amount of money i sat down with a calculator i figured it out it took me a while but i figured out that's one tenth of a billion dollars uh so it's just a lot of money uh sachin and congratulations yeah it's it's weird to think that somebody would trust an organization led by me with that kind of thankfully there's lots of smart people who you know make me make us look good and and so yeah i know it is an incredible amount of money i mean it is right you know and and kind of inconceivable in some ways because you know when we started nine years ago i mean 110 million dollars was a com was what was a great exit right right and so and what's even crazier is that we we're likely to spend it i mean we're likely to spend it and we're gonna spend it you know we're gonna spend it aggressively because unlike nine years ago i think the um you know the market momentum around data broadly but what we do specifically is just incredible and the opportunity is just amazing you know it's it is and i i was talking about this beforehand that um i'm amazed looking at the i.t sector in that the only thing that anyone wants to talk about these days is data i mean cloud is used to be hot and it's still hot and we talk about other things edge computing etc which reaches the data story itself but every every vendor every provider their story no matter what they sell this story is data believe it or not yeah and i mean i think it's because you know the innovation in the stack used to be around business process optimization and defining the business processes and defining the engagement processes and allowing one step to move to the other but the world of computing has broadly and and b2b software has broadly graduated past that and so now every transactional experience every process experience has to be informed by what the next best decision is and what the only way you can do that is with data and so everything has to be intelligent everything has to be guided everything has to be patient based upon the needs of the consumer or the actor at any given time and so data powers all those experiences so you know all of the investment on the margin now is going to data which is just a fabulous place to be because i mean that means that you know people like us who are trying to enable lots of people with data um you know are obviously selling tools um during during you know what is way more than a gold rush because there's actually gold to be found right right yeah okay so so i i want to get back to the 110 million because i don't think we should leave that too far behind um i like to think about that number it's a lovely number uh but let's get uh a little a little 101-ish if we can because i i know you know companies have a database they have a data warehouse they might have a data lake but why do they need a data catalog and of course the term sounds self-explanatory in the face of it it must be a catalog that holds our data but but what exactly does the data catalog do if that's not too dumb a question and why do we need one yeah not not at all i mean it's a it's it's you know you would think that in a world where there's so much data everybody would just understand the data they have i mean you have lots of emails you probably broadly know where your emails are um but the reality is there's so much of it that you forget and you know if you think about those three words right those three three words actually have different meanings so database um there are different types of databases but often they fuel and you know are are the basis for transactional systems and then you've got sort of your data lake which takes all of the sort of core transactional data from those systems and puts them into a single place for storage and analysis and then to your point you've got this data warehouse and in today's world the warehouse is a place where all that information is not just stored but also structured so that you can commonly ask questions in a repeatable consistent way and process the data in a repeatable consistent way and so those are three different levels of processing almost like you've got you know uh sort of iron ore and then you've got the base metal and you've got a finished product you know that's essentially you know how to think about those three things now the reality is that in each of those things you've got you know company like pfizer or like you know goldman sachs like these are customers that have hundreds of thousands of databases and they've got scores of data links and they've got tons of data warehouses and data marts and so in a world where they've got tens of thousands of employees they don't even know what data they have what data their colleagues are working on what data is in a warehouse what data is in a database where they can get it how they can get access to it and then even when they find it inside of a database there's this question of what does that data even mean right right and and so you know we can get into like what that looks like but then there's this question of like oh well customer id number 3712512 actually that means ibm and how do i decode that and how do i figure that out because that's not in english and so you know these problems of both discovering data and understanding data are just getting more and more and more and more complicated and in a world where data is sort of the power that runs the world it is impossible to get by without something to help you navigate that landscape so the the data catalog really is is a pretty meta thing then it's it's it's it's sort of it's almost like the navigational tool that you need for all your data data entries data warehouse data lake where is it all if without the data catalog in theory i would be lost yeah i mean it's like you know there's so many analogies like you could the catalog analogy obviously is yelp or amazon or linkedin but you could even use an analogy like yahoo maps right or google maps right like i mean you know just how do i get from point a to point b or even extending that farther like ways well i want to go use this data but it's got data that's got pii i need to comply with gdpr can i use this data and so having this catalog really is the entry point for anybody who wants to understand information and analyze stuff so i'm sure that there are plenty of of data catalog users out there who are like encountering some challenges encounters some headaches i mean what what are some common challenges involved and it seems like one of the first problems would be how do we get someone or a team or a series of teams to actually catalog idea given that we have so many pools and repositories of data someone has to go through and actually create the data catalog true true or false yeah so i think this this um you know that's great so there's this idea this catalog is a noun and catalog is a verb okay right so catalog is a noun is the place that you go to find something right and then there's this question of like catalog is a verb and it used to be the case that cataloging as a verb was done by an army of accenture consultants or deloitte consultants who would come in and perhaps charge you hundreds of millions of dollars a year and the only reason you would ever do that kind of like painting the golden gate bridge is because you would need to comply uh-huh right because there just wasn't enough people to just like document all this information i think we've turned catalog as a verb into something that machines do and so we've really helped the world sort of you know kind of catalog or understand their data by taking ml based approaches to crawling data sets just like google does and then looking at the logs around how people use it using the stuff just so we can understand like hey james looked up this data set this data set therefore is probably pretty important because james really understands it and so this is a great data set for it and so being able to sort of understand that history by doing all of this machine learning allows people to document the information in a way that they would otherwise have to do manually where every individual piece by piece would have to do this work you know data set by data set which is just impossible so so data catalog is is a process it's it's both a not a noun and a process am i correct in thinking that yeah i i think that's that's exactly right and we think of it you know and it's something that i think people now understand that they need but of course you know what what's happening today is that that core that that core capability that core catalog is becoming the basis for which a whole bunch of traditional data management applications are now being delivered so once you catalog your data you can deliver better you know descriptions of whether or not that data is high quality you can understand whether or not that data has been loaded at the appropriate place and time data observability you can master your data better because you know which data you have you can migrate the right data to the right cloud storage system because again you know what data you have you can merge it with external and third-party data sets because again you know what data you have and so all these things that used to be historical point tools and point capabilities are now being sort of built on top of this catalog as a platform for this broader category that we described as data intelligence and that to me is what's so exciting because you can take all of this work that historically has been done really by like narrow audiences in it and explode it so that now data management is an enterprise capability for any business person that wants to do anything with data and that's the opportunity for elation yeah all right i i definitely see the need for that to be sure so you you mentioned ml so so ml and ai certainly play a role in in doing doing the heavy lifting here what what is the role that i mean i guess humans could not do this it's partially done by ml and ai yeah i mean just think about some of the you know so i'll just say i'll give a couple of examples just to give you know a sense of texture for what that feels like so inside of enterprises inside of databases if you think about databases you know the most you know fundamental databases are relational databases and there's just tables like you would have in excel and often what happens is that when people label those databases the very column header you know they'll say oh it's txn underscore id numbers underscore c that's like a great column name okay do we not call name for a programmer because a programmer you know basically allows you know they care about succinctness and being able to type really fast um but it's not really great for people who are consuming that information because they don't know if txn is transaction or trucks right they don't know if id is identifier or something else and so if you think about the myriad of literally billions of different columns column names inside of many companies how do you label that stuff at scale well if i know the txn is transaction i can go find that pattern as a machine and just explode that a billion times through but i couldn't do that as a human being because i'd never be able to just do the work manually well enough and so machines can now do these things that otherwise would have been so hard for people to do other uh today and so that's what's exciting you know because you can now take that and now people once they search for transaction can find that information the catalog so so what is a common problem for the for those data catalog users out there if if they have a certain classic headache what might be the classic headache or challenge of the data catalog user and and what might you recommend to say hey don't do that try this so you won't be surprised if the first and most fundamentally used case is just search and discovery if i have a customer question or a question about my customers where do i find the customer data so that's right generally the first thing that people want to know and that seems like a trivial thing but there are often many of our customers 4 000 different data sets or 10 000 different data sets or 40 000 data data sets that speaks to a particular aspect of customer relationships or customers and so you know data discovery is one piece another aspect of it is what we define as data literacy so once i've gotten this stuff how do i get enough context to know whether it's the right thing to use so within those 40 data sets that i might narrow down to which one do i use how do i know it's usable and then the last piece of it is just trust like how do i know actually how to use this and what's the way to use this so that's kind of the basic frame of what we do and then the more fundamental business use cases um productivity and sort of self-service enablement is a big deal because lots of data jobs or lots of jobs are becoming data jobs um another use case for us is data governance because people to govern the data need to understand where the data is ironically lots of tools historically that would do data governance would actually be disconnected from the underlying data state which sounds really weird but literally like you would have data governance glossaries built and workflows built where no underlying data systems were connected strange yeah yeah right right like super strange and and of course you know intractable and unmanageable and then what you would find is things like cloud data migration data privacy with gdpr people are doing a lot of work in data exchange moving data back and forth between organizations so lots of interesting use cases are emerging as people are becoming more and more sophisticated with data interesting all right so let's look to the future if i think about what is the data catalog of 2025 going to look like and i guess we that gets us back to the 110 million which i can't stop thinking about i i assume you're going to obviously spend the 110 to help build that data catalog of 2025 so how are you going to spend the money and or what what is the future of the data catalog yeah so we we have been talking a lot about this idea of the data catalog as a platform for data intelligence and i think this sort of platformization for us is just such a critical idea because to get this catalog up and running it's got to be super seamless super easy to do super easy to deploy and today that is not as true as it should be right we've got 100 so we've got 200 and you know over 250 enterprise customers most of them are global 2000s we think that numbers should be you know 10x over the next three to five years and to get to that point you just have to make the process super easy and so being in the cloud making sure that you're ubiquitously connectable making sure that you have in the same way you've got self-driving cars you've got self-driving connectors that's a big part of the investment of building that platform capability out and making it totally seamless a big application capability that we're investing is data governance we think the data governance today is just too hard too cumbersome too inscrutable too impossible to set up three years time to value you know nobody has that time most people are in and out of two jobs in that time right um and so you know in in our world we think data governance is a first-class use case and then the big part of building a platform is of course enabling an ecosystem and so you know we really believe with partnering with you know great data quality vendors great data privacy vendors great cloud vendors like snowflake and certainly salesforce in order to be able to build this great ecosystem of partners who can both feed data in but also take use cases out and so that's what we're going to be working on for the next you know few years and it's an amazingly rich road map i i can only imagine this is going to be really neat to follow it i hope we can speak again sometime uh sachin thank you very much for sharing your expertise today thanks james it's always great to see you

This transcript was generated automatically from the video's captions and may contain errors.