Hello everyone and welcome to ES Speaks, the show where we unpack the biggest ideas shaping enterprise strategy and performance. I'm your host Corey Nolles and today we're exploring the power of openness. How an open data ecosystem accelerates AI innovation. So, as enterprises race to harness the potential of AI, from model development to real-time analytics, many are discovering that openness isn't just a philosophy, it's a competitive advantage.
To help us break it all down today, we're joined by Dr. Chris Hillman, global AI lead at Terodata. Chris has spent his career helping global organizations operationalize data and AI strategies across hybrid cloud environments. At Terod Data, he leads efforts to make enterprise AI faster, more open, and more scalable through connected data ecosystems. Dr. Hillman, welcome to East Speaks. >> Hey, thanks. Nice to be here. Thanks, Corey. >> Excellent.
We're sure glad to have you. Well, Chris, I guess let's uh let's start with the big picture. Why has openness become such a crucial success factor for enterprise AI today? >> Yeah. Okay. So, so I I you know I I think as a business as a as a top priority, you you've really got to be ready for what's coming next. And and it's really hard to tell what's coming next, right? Because it it's difficult to tell what the big players in AI are working on in their R&D labs.
Uh you can check academic research and see what you think is coming down the line. But, you know, if you were reading about Transformers back in the summer of 2022, I doubt you would have foreseen chat GPT. Not many people force saw that and the impact it was going to have agreed >> and you know if you if you look at smaller scale models things like that's on things like hugging face you know there's new stuff every day uh and you know are we at the end of the line with transformer based models you know I I don't think so I think they're amazing there's lots of development um but there will be something else along the line it may be an evolution it may be something completely different but you know I'm not actually in a position can tell you what that's going to be.
And I'm pretty sure most people working AI in in business in departmental data science teams, they can't tell you either. So, you've got to be open to whatever's coming next. And that has to reflect um not just in the ecosystem. And um it's got to be in the ecosystem and the company mindset. Otherwise, you're going to miss out. You know, you've got to be able to switch, go with a new direction, new method, new type of model and and at least to test it out even if ultimately you decide to stay where you are because you know uh another thing to consider actually is that um you know data science is an inherently creative role and and if you fix everything down, if you lock it too rigidly, you've stifled a lot of that and that you're losing the superpower you employed those people for in the first place there. >> That's true.
That's very true. And you know, I love what you said about how things are changing so fast. It really is a tough time and and smaller models are dropping non-stop. I check a giant list every single day and uh I'm trying a lot of local ones quite regularly. So when you talk about open ecosystems, what does that actually mean in technical terms like architecture, tools, mindset? Yeah, I think I think ecosystems to me I mean my me and my team were we're more users rather than builders or architects.
So uh we do have a lot of in uh influence in the design and the choices but ultimately those decisions are made by by others. So for me it's all about access to the the tools I want. So, so most of the work my team and I do is in the pre-sales area and we do a lot of factf finding and investigation and I I always remember talking this was a while back talking to the head of data science large telecom company and I said to him you know what tools do you use for data science and model training and what applications you use and he basically said we use one of everything and two of some of them because anything that's out there we're using it. >> So true you know. >> Yeah.
Exactly. and and anytime somebody new joins, they bring something else with them, you know, and and I was thinking how how do you reconcile that um the possibility that anyone coming into the business is skilled in a different tool set and I know we've kind of converged on on Python. You know, we don't see R that much out in the field apart from like really specific companies where stats is uh super important like it should be. Um, but it is mostly Python, but I still want the ability to use pretty much whatever coding tools and libraries, whatever I want, basically. >> Yeah, >> that makes a lot of sense.
Yeah, you don't want to be strapped in too much. >> Yeah. >> How are organizations balancing that need for flexibility with governance and compliance? Yeah, I mean I um you know I want the ability to use pretty much any tool and it's important to the team, but I get it that creates a potential well it's an actual nightmare. It's not a potential nightmare and I think one of the ways you can mitigate that is to to fully separate uh discovery and evaluation from production.
So the idea is you've got a clear production environment where uh things are conformed. Um you can have all these different tools for working things out, trying new technologies, libraries, methods, but when you push something into production, there's a set plan, set methodology, set of tools, uh production platform that's capable of doing everything you need to do because um what it means is whoever's doing the evaluation work um you know, I really try not to say playing around because it kind of belittles it and it's really important. >> Yeah.
Yeah, >> but it is kind of exactly the people doing that um they can have the mindset that they know that they know that route to production. They've got it in their heads when I've done this, I've done that. If the business accepts it, that's where it goes. That's what I have to have in place for someone else to accept it. Um and you know, most companies have pretty strict policies and governance. You know, in Europe, we've got the EU AI Act. um that has to be complied with if you're working with anyone that's in the European Union whether you are or not yourself.
Um and you know then there's things like guard rails models um and if you have you know anyone who's uh in this area really needs to get up to speed on guard rails at the moment. It's [clears throat] that coming area. Yeah, >> for sure. And uh so from your perspective then what what are the risks of not adopting some type of open approach in today's AI landscape? >> So I think you know I think it's pretty clear to me that you can get stuck in a dead end.
Uh you can be late to market with innovation. I know there's kind of commercial issues around this like uh vendor lock in stuck in contracts and that kind of thing. But you know for me uh key consideration is I want to use new stuff when new stuff comes out. I don't want to have to rec tool or reconfigure and all that kind of thing. So so staying open like I say is as much a mindset as it is a technology play. You know everyone in in it the architects they have to accept this is the way we want to work as well. >> That makes sense.
Well let's talk a little a little about what it's like to build an open data ecosystem. Uh, and I'm curious about Terterodata's partnership with AWS. How does this joint ecosystem you have embody openness in practice? >> Yeah, I mean, we we've got a really good uh partnership with AWS. Um, there's a lot of uh interoperability between our tools. So, you know, if you think about it, um, Terodata at its heart is a is a database and it runs on SQL. >> So, we have a Python interpreter for SQL.
It's a bit like imagine like pi spark but it's for for SQL. >> Yeah. >> Um and and so you write code in Python and it gets converted to SQL under the covers but you don't have to know all the details of that. Now all you need to do is install that library in the AWS SageMaker notebook. Um, and then if you're a data scientist working with Sage Maker, you can be firing off commands, executing them in in terod data, pulling data out, writing data back, but you don't even have to know any of the details because you just kind of set up a data frame.
You think you're working with a data frame, but in in reality, it's executing on the parallel system. Um, so you know, that that that seamless experience is is pretty essential if it's going to work. Um, you know, we we did we had a conference recently and we had the guy I forgot he's Chris something as well, but he he's the guy from Shazam who was like the the real innovator driving force behind it. Yeah. >> And he >> Shazam the the music app. >> Yeah, that's >> I still use that thing every day. >> Fascinating. >> Absolutely. >> Um, basically they designed it for something that didn't even exist yet. you know, they needed a smartphone, but they didn't exist for like seven years after the company formed.
Um, but but he was talking about eliminating friction. Um, and friction is the enemy in any experience. And, uh, >> you know, if I've got to do this, do that, mess around here, export it, I'm not going to use it. Um, and and that I think is the key for our our our joint our partnership with AWS is it's kind of seamless, you know. Yeah. Um, and we really, you know, we work with customers together. We really kind of live that better together message.
Yeah, it's good. >> That makes sense. So, what are some of the, I'd say, biggest technical enablers that make true interoperability possible across multicloud environments like that? I >> I think one of the the biggest problem is multicloud and and hybrid. Um, it it's the data movement. And, you know, that's something we've always tried to minimize. Um we still have a a fair few customers that are just on premise and we've got a few that are uh either hybrid or very interesting hybrid all kinds of reasons at the moment but um you know it's data interchange that's a key consideration there.
So and open table formats and object stores are key to that. >> Um you know if you if I guess an example from a IML is the feature store. Um so if you know if you're building a data set that's going to be used to train models um you transform the data into something we call features. Um I used to call them independent variables but now it's features. That's what we say. You know >> now that the normies need to know what they are. Yes. >> Exactly.
Exactly. Independent or input variables if you prefer. Yeah. Um but but you know you if you're if you're training an a IML model then it's the input data that makes it accurate. You know [clears throat] that is the key to success in there. So so the idea is if somebody else wants to build a similar model why do all that work again? Why not reuse those features? So you know waste all that time. So it's it's kind of key concept for data science productivity.
Mhm. >> Um but if I'm using uh this platform for um uh discovery and evaluation and then this platform for uh production and they're in different places, different clouds, one's on premise, one isn't. >> Um you know, you don't you you'd have to intentionally move all that data. Um and if the model then needs retraining, which it it will regularly, you've got to recreate all that data back. So um you know and and even more complicated all that data has come from all different places you know the data warehouse the data lake operational systems. >> Yeah. >> So so the idea is if you if you build that feature store um in open table formats on object stores wherever that is then everyone can write and read to the from the same place and it doesn't matter who wrote it where it is.
You haven't got to you know you've only got to move it the once basically. >> That's right. Um, and obviously you're then relying on acceptive format, the right place, the right naming, metadata, etc. But that can be part of that, you know, that kind of production workflow, >> all of the important hygiene stuff. >> Exactly. Yeah. Yeah. >> Well, can you explain how uh something that intrigued me is how Terodata's bring your own model approach can help enterprises accelerate deployment and at the same time at excuse me, at the same time stay secure and compliant. >> Yeah.
So, so, um, BYOM, uh, as we we always call it, bring your own model, you know, it's it's, um, it's it's kind of a it's been a key thing for us because we we realized quite early on, you know, our systems parallel and any algorithm you want to run on it, you have to if you want to train a a model, you've got to rewrite it to run in parallel, which is no easy feat. So, so we realized we were never going to keep up with open source, you know, Python libraries.
And to be honest, we don't we don't really need to or want to. So, >> you know, the the thinking about how you can use any tool to train and evaluate models, how how do you get it into one place for production? And that's what BYOM is. So, you you take a trained model um export it to a common serialization format. Um you heard of PMML, predictive model markup language. >> Um and then the newer version is ONNX, open neural network exchange. Um, you know, and then there's formats coming out.
There's like GGF, the GPT generated unified format. I think that's called that. That's what I run so often on my local computers. >> Yeah. Right. Right. So, so, so the idea is you export your train model to that common format, >> then you can import it to a different platform to use it for inference. >> Yeah. >> Um, >> and it works really well for traditional ML like uh regression. Nothing wrong with a bit of regression. SVM, you know, and it works for for the bigger ones like ensemble models like XG Boost.
Um, >> and and it also works for the models you find on hugging face. So things like um you know these these smaller task specific language models >> uh like they might do vector embedding creation, topic detection like very specific things, >> very niche. >> Yeah. >> Yeah. Exactly. and and things like image classification, you know, those those models are small now because they've been sort of optimized for for mobile phones. So, >> yeah. >> Um, >> yeah, you know, what what I really like about it is when you import it into a system, you've basically broken all of the dependencies on on what was there before.
Um, so, so you don't, you know, if if the discovery system is using Python uh 3.6 6 and your production 3.7 or this library, that library, it doesn't matter because the the format kind of takes that all into consideration. You know, the the the the execution part of that ONX or or PMML that works now. So, you've you've kind of really split the two environments, which is just perfect for for production. Um, it sure is. >> And then I I I guess for us, um, the superpower, why why would you use Terodate for this? is that you know the the inference part is typically uh embarrassingly parallel.
So it's like you know data function output um and and we're set up as MPPP. So if you've got a thousand units of parallelism parallelism uh you can run a thousand inferences at the same time. So that's you know that's that's why you might look at that as a as a way to go. Yeah, [snorts] >> that makes sense. Well, I'd like to I'd like to maybe share some real world innovation stories along the way. Are you able to share an example of an enterprise that's leveraged an open terod data ecosystem to scale AI faster? >> Yeah, I mean, we've got we've got um some great I mean, it's one thing I particularly like about working at Terodata is the customers you get to work with and the massive data sets and crazy stuff.
Um yeah, but uh the a good example um that people might not have heard of is the the lottery company, lottery corporation, sorry, TLC in Australia. >> Okay. >> Um it's the Australian National Lottery, really successful. Um they've got they've been a longtime um customer of ours. Um particularly for data management because they can they handle and analyze like masses of data. Um and a lot of its customer data set has to be really secure. Um, but they've they they now use Terodata as the AI production platform that you know using the things we've been talking about.
So they they've got a great data science team there really innovative and pretty much everything they do is is Python uh running on uh AWS instances. So you know a lot of them have got their own version their own instances. So they've got their playground they can do what they like. Yeah. Um when it when it comes to production, they they manage their entire system in in Terodator. Um and and they're running things like uh next best offer, churn, uh engagement, um and early interventions are really really um important thing for them.
So they're they're checking for like uh things like harm, gambling, addiction, that kind of stuff. Real serious problems that they, you know, they they work hard to to mitigate. um and and they use that BYU method to put everything in production. So they get the scale and speed um and then the enterprise readiness and and the they still get all that freedom from being able to do what they want in in native Python. So yeah, it's a good system. >> That's awesome.
How do you think openness impacted outcomes like time to insight, data accessibility, maybe governance? >> Yeah. Yeah. I mean another uh good example is uh Rafien Bank. Now that's Austria and I keep getting as you do Austria and Australia around the wrong way. So like Sweden and Switzerland but this is Austria right? So they they they migrated to um terod data on AWS you know a while back and it was basically for handling the huge volumes of data and all the integration challenges they had um easily over a million queries a day. uh and you know all that with very high concurrency.
Um so what it's let them do is the really increase the uh the data granularity um and so they got much higher visibility into financial data compliance uh customer behavior um and and of course what that means is they're now supporting hundreds of applications in the in that open ecosystem using using AWS. Wow. Um and and it's just the you know the real insights are coming every day from what what they're doing. It's really really impressive to see. >> That's awesome.
Were there any unexpected benefits like cultural operational maybe that emerged along the way? Yeah, I think um going back to uh TLC um what what was what was surprising to them I think and to us which is which shouldn't have been surprising to us but it was was hadn't fully appreciated the efficiencies they gained from having that that production system like that. So, you know, their entire um a IML operation um for for like regular production, it's only two people that manage the whole thing now. >> Um so, so because it's just it's you know it it's basically monitoring it and making sure the retraining happens regularly and things like this.
So, >> um they've been able to take that resource, they've been a refocus on the uh innovation and development over all that day-to-day stuff. Um, and we see that a lot is like existing uh teams get bogged down in stuff. You know what it's like get away from it. So, >> yeah. Yeah. Um but you know something else that's interesting is um and it's because you know agents are the thing at the moment is that it it turns out if you've got hundreds or even thousands of agents running which is kind of predicted then >> um if they're if they're using tools to query a data a database which they will be doing a lot you've got very little control over what when and how those agents are working you know they're just doing their thing.
So, so that translates into, you know, thousands of queries firing whenever they get submitted uh 24/7 and and that potentially causes a big problem with concurrency in existing workloads. >> Um because you know there's other stuff going on that's often mission critical. So um we've actually found the the existing uh workload management system we have and the parallel nature is is ideally placed to handle that. That means that, you know, all of those tools in the open ecosystem are highly performant because they're not waiting for data and they're not getting interrupted by what these agents are doing from from external sources. >> Oh, that's nice.
Yeah. >> Well, let's uh let's talk a little bit more about the future. Um, so AI is evolving incredibly fast. Like we've talked about along the way here. Uh we regularly joke here that what's true at lunch isn't at dinner some days. And uh how do open platforms futureproof an organization's AI strategy? >> Yeah. So I think you know it's like like we were saying at the start, you got to be ready to to jump on that innovation when you see it when it comes out. >> Um and at the very least you've got to be ready to test this kind of thing, see where it's relevant to you.
Um, so you can't close it all down and and tie up in regulations. And you know, as I said before, the open mindset is is is key. So, um, you got to allow the evaluation. Um, and again, if you think about agents, because it's a subject taking up a lot of headlines and commentary. Um, >> for sure, >> if you're in a high, you know, you're a highly regulated industry like a like a bank and you want to try this out, you've got to be so careful because, um, automation is is a real minefield.
Um, yeah. and and you know um in in people's thoughts in in practice and technically um and and you've got to be even more careful if your um automation includes large language models cuz cuz we know they hallucinate and we know that things can go wrong. >> Um >> so you've got to have a way of testing this out, but you've also got to have I like the this expression, you got to bring people along for the ride. Um the business has to has to get it, you know, and trust it.
So, you know, I I like uh a lot of work we do, you think you got to control the variables. You can't have too many um variables in one thing you're testing. >> So, you know, you you want to be thinking about how can I test this out with an existing problem so that if something is wrong, I can tell it's wrong. You know, it's there's no point tying out a new process with a new tech because you don't know which bit didn't didn't work. >> Um Yeah. And then of course you know if you've got that lockdown ecosystem and mind mindset you can't even start on on that kind of journey >> because you don't have that same level of internal visibility I suppose that you do with an open system. >> Yeah.
Exactly. Exactly. >> So where do you see the the biggest opportunities for open ecosystems in the next two or three years let's say? Yeah, I mean it's um the the innovation is just increasing, you know, the pace is just increasing and um you know I I I I think we're going to see a lot more open source. You know, I think um at first um there there was a lot of doubt about whether open source could keep up with the you know crazy huge amounts of money in R&D going into commercial uh transformerbased language models.
Crazy. But um you know so far open source is giving them a run for their money. Um, >> I I think we'll see more of that and more of this kind of uh somebody in our product management called it private genai which I quite liked you know it's for you and it and it's it's it's uh you know um sandboxed and things like that. So I I think we're going to see as I said earlier a lot more uh focus on guardrails models uh governance and you know I was I was reading up there's quite a few uh startups now that are basically bringing out um applications for AI governance and and I I think it's going to be a big growth area there. >> Absolutely.
Governance a big topic right now and you know I like that you mentioned that uh open models are are keeping up close. I think I think the figure I saw last week and I've seen it floated around for a while, but it seems to continue being true is that there's about a two to three month gap before open source catches up with the new closed source frontier level. And that's insane. That is so fast. >> Yeah. Right. Consider, as you say, considering the investment into >> Yeah. when you consider these other companies are spending hundreds of billions of dollars and >> um what advice would you give to enterprises that are just now beginning to modernize their AI and data infrastructure?
Yeah. So, so having talked about uh exciting stuff like agents and um you know newer models um you know if you go back a while I always when I was presenting I always used to show this slide and it basically had word big words in the middle of the screen that said it's all about the data stupid >> right and marketing told me to stop doing that >> in the brand guidelines but it's true it's um you know you you can do whatever you like with AI, but if you haven't got the data accessible, if it's not trusted, if you can't find it, if it's not cataloged, uh you know, we we've been showing some great demos recently using uh just a simple MCP server um and a language model and query a database in natural language and and it works brilliantly.
Um it's really smart and everyone's impressed. But if you really want that to be reliable, if you want your non-technical people to have natural language access to a a complex data set, the data has got to be sorted in the first place. You know, all the metadata about those tables and columns has got to be present because otherwise that LLM is just going to going to make up what it thinks the data is about and and you'll get results whether they're right or not.
And we've even see it where it makes up the results completely. Um so so you got to have a serious look at data quality and the way it's structured and you know it's not it's always a a scary subject and there's always people looking for shortcuts and you don't have to do this the way we did 20 years ago. You don't have to go and you know um start from scratch because the AI tools >> are there to help now. So you can create an agent to do the work for you.
You know it's not it's not hard. You could do it in an afternoon, come up with a data quality agent. Um, as long as you know what metrics you're checking for, and it's not that hard to come up with an agent that will look at an existing schema, you know, the set of tables and give you >> at least 80% of what you need. Um, so, so, so you, yeah, I think that's the main advice. Get that sorted because once you've got that sorted, you can do amazing things with AI, but if the data is wrong, you'll get the wrong answer and sort of, you know, discredit yourself right at the start.
So, that's the Yeah. >> Yeah. And if there's anything we know, it's that most data is a mess. Most companies data is everywhere coming from a million different sources in a million different formats. >> And uh that's that's very much the go-to is is get your data house in order >> and then build as you wish. >> Yeah. Yeah. Yeah. And unfortunately, you say it's not the most exciting message, but it's it's the right message. >> It's the right message.
I agree. I agree. Well, that brings us to the end of today's e speaks. I want to give a huge thank you to Dr. Chris Hillman for joining us and shedding some light on how open ecosystems are really powering that next wave of AI innovation. Chris, before we go, where can our audience learn more about or even try these ideas out in action? >> Yeah, I mean obviously um terodata.com there's, you know, we've got all the information on there. Um we have you can actually try this out.
We've got, you know, an environment you can start up a twode system and try all this out. So that's just clearcape.odata.com. Uh clearscape is kind of the brand for all of our AI capabilities. So yeah, clearscape.odata.com and you can get a free account and try it all out or or contact us on on LinkedIn, you know, that's always available there. So yeah, absolutely. That's great. Thanks, Corey. That's been that's been really good. >> Thank you so much.
For anyone listening, we'll have links to Terodata's Clearcape Analytics experience and related resources in the show notes right below this video or at espeaks.com/terodata. That's T E R A D A T A. If you found value in today's conversation, don't forget to follow, like, and share this episode with a colleague who's working on building their own data strategy, their own AI and data strategy right now, excuse me, and uh, you know, may have their own issues. So, thanks for watching and we'll see you next time on East Peaks.
This transcript was generated automatically from the
video's captions and may contain errors.
