All right. Well, hello and welcome to Espeaks. I'm Corey Nolles, managing editor here over at EWeek and Technology Advice. Today, we're going to talk about the evolving role of API platforms as product ecosystems, kind of taking a step beyond the traditional plumbing approach at surface level that you might have thought of a few years back. Uh we've invited Kong onto our show this week to discuss how Kong Connect is helping teams better manage the growing spiderwebs of APIs that connect our world.
With that said, I am super thrilled today to be joined by Saju Pilai. [snorts] He's a senior vice president of engineering at Kong. Saju, welcome to Espeaks. >> Thank you very much for having me on, Corey. I'm looking forward to chatting with you. >> Nice. Well, I guess let's dive right in. So the API landscape is is pretty constantly evolving. How have APIs shifted more into products as opposed to traditional plumbing? >> Yeah, I actually I it's very interesting you you use the word plumbing many many years ago.
That's exactly how I would describe uh you know the APIs. It's about connectivity, right? So you have you you have business logic that is packaged up um you know on the server side so to speak and there are other entities that need to access that business logic and you got to solve the plumbing problem. I think I think a few years ago something fundamentally changed um in the API world which was the realization that business logic did not require the same company that owned the business logic to also own the user experience in terms of mobile client client or other experiences.
And in fact there is as I mean as as you would see for example with the uh in the ecosystem where you have these super apps for example in some countries especially Southeast Asia the chance that you can get your app on the user's phone not that high the fact that you may want your business logic to be integrable in a super app is just one simple example where the user experiences are now getting specialized by specialist companies that still need to access your business logic.
Another quick example would be let's say you are a big airline company and you are releasing airline seats to be you know so that people can purchase them the chance that they are actually using some kind of aggregator application or or or a specialized mobile app for ticketing is higher and you want your business logic to go there. So the realization that APIs are products and APIs are not just a mechanism for plumbing. That realization happened and I think that kind of changed a lot of how people think about APIs and uplifted APIs from a pure technical perspective to actually a revenue generating monetizable uh idea. >> Wow, it's changed a lot.
So, uh what would you say are the hidden costs then of poor API visibility? >> Oh, yeah. Yeah. actually look I mean many people will tell you when you ask them about hey API visibility is that some of the stuff is obvious there's there's duplication that can happen there is lost effort that can happen uh but one one way to you know maybe just in a prioritized way uh to think about this would be for example the security surface area so let's say you you are aware of certain APIs that are exposing you know let's say you're a bank and you're exposing some account creation credit card management etc APIs and you you you are making sure that they comply with your regulators standards they apply you know your security mechanisms etc but what if you also have these what we going to call shadow APIs perhaps where you don't have full visibility therefore you cannot even make them comply you don't know you don't know that there is a gap so those risks are extremely uh I would say you know in terms of things you should worry about um if you don't have visibility you can't secure you cannot make them comply maybe you're under a regulated environment or so forth force then the infrastructure like is something that you know people should think about overprovisioning how many how yeah how much extra hardware is now running the same or very close to same API that you don't have visibility over fragmented analytics users coming in with use cases that are powering through let's call let's call them your visible APIs but some of them coming through these less visible APIs and now you have this fragmentation that's actually going out into your ecosystem which I think is a is a is a major problem you want to change your business logic, how will you roll this out?
It becomes a huge problem. See, without uh, you know, there is a saying, right? I mean, if you can't see it, you can't measure it, you can't fix it. Um, it comes down to that. Uh, at the end of the day, >> it's exactly what I was just thinking. If you can't see it, you can't fix it. And that's a that's a big problem. >> Exactly. >> In that case, let's talk about Kong Connect a little here. How how is it meant to rein in some of the common struggles as teams manage you know increasing loads of APIs? >> Yeah, great question.
Um, so we we let me tie this back to some of the stuff we already talked about. We we we talked about APIs as plumbing becoming APIs as product. Um, and and there we we talked about visibility uh from a governance perspective, from a security perspective. Um, and I'll just use that framing. If you think about the things you could do with APIs, I would claim and you know um I would say something like you can you can build, you can uh run, you can govern, you can discover and you can secure like five things.
Build, run, govern, discover, and secure. Right? These are the things you would do with APIs. The life cycle of an API. Kong connect our goal is to be the onestop shop the one platform for APIs that gives you all of these capabilities. So from the pure connectivity perspective uh you know not just HTTP but there are other types of APIs out there. So we provide data planes for a lot of that class of APIs. So we solve the you know uh connectivity at scale connectivity across clouds connectivity across heterogeneous environments. uh you want to protect, authenticate, authorize, you want to rate limit, you want to protect your APIs, you want to shape your traffic, observability.
So we provide you all of that. But that's just kind of almost table stakes you would you would argue, right? You what you want is to discover all the APIs in your ecosystem. You want to discover everything that is even in planning. So let's say I'm a developer. I'm planning this API. Maybe I need a place to go where I can go look up does this API already exist or is there API that is close enough? What is the uptime on that API? How do I internally consume that API to build my API or my solution on top of? [snorts] So catalog class of problem you know in engineering we call it an enumeration problem.
Can I enumerate all my APIs and ask interesting enumeration questions on that? So that's the catalog. Kong connect offers you the catalog. You want to take these APIs you want to run analytics on them. You want to see their usage patterns. You want to see both the traffic and business use case, right? How are they been consumed? What is the sequence in which your APIs have been called? And then that's analytics. And then you discover some of this this this this these patterns and richness and you realize you you you you have grown to the API as a product thinking and you you're saying I want to take some of these APIs together stitch them in a very particular way so that they become a user journey or use case and expose them out on a portal so my partners can discover from outside and come in and now take that flow and go all the way into metering and monetization and building out the whole monetization mechanism.
So you can actually monetize. I mean not not just in theory but literally meter and then run plans and then have have people come in uh you know for for uh at at a you know like a gold tier user for a particular API product versus a silver tier user different pricing plans different uh kind of features that you're releasing. So all of this plus the ability to look at vulnerabilities the ability to look at the security risks that are going on within your API surface.
For example, the OASP top 10 vulnerabilities in your API surface area as a as an example, right? All of this integrated is the Kong connect story. Like we are trying not to just help one particular team but the enterprise, the company with executing expressing and executing your API strategy. >> Wow. And I guess that's where the the combination of of a developer portal and a service catalog really help improve that experience probably both for consumers and producers. >> 100% 100%.
I mean that is the you know it's interesting we we we divide the world into consumers and producers in this this context but if you look internally inside a company you have both internal consumers and producers like like like I was saying earlier as a as a let's say team that has to release some business functionality out and I choose to do that as an API I probably want to build on top of lower level APIs that are already out there I have an internal discoverability issue now where are my APIs and how do I call them and and you know my user you know some some rather complex problems like my user the authenticated user coming into the system how do I translate them into an identity that is acceptable to those lower level APIs and there's all that funky problems to solve but but it's it's it's you know so so so the so so the cataloging system you you would you're right you would come from that from a from a from a from a producer perspective saying here are all our things that I'm putting up into the catalog but there are I mean catalog is automatically also an internal portal where people want to come you know get richer info discover avoid duplication and so forth and of course then once you have these this catalog and you have this rich APIs you start doing monetization part and now you want to start exposing it outside and that's where the portal comes into play a nicely branded portal that looks exactly like you want it that behaves the way you want it where people can come discover you discover your documentation register sign up for monetization tiers you may have and then full life cycle activate like as in nobody in this world probably wants to you know send paperwork to you and say okay now I'm going to wait for 7 days for you to turn this account on in some data plane it's all has to happen at speed of light that's what we now fun thing so all of what I described I mean traditionally portal is meant to serve humans right you come you look at the documentation you are evaluating what about agents what about all this agentic traffic that is slowly climbing up We are already I mean as Kong uh based on interactions with our customers and from our open source footprint etc. we're already seeing um agentic traffic starting to climb up, right?
So, so our portals and our cataloges are actually also been built for agents and and I think that's very awesome. Yeah, it's very cool. >> That really is and you know and you have event gateway as well. Uh >> oh yes yes yes my my my favorite >> what gap does that fill you know in the API life sake. So I mean a a reason I said is my favorite gateway as in you know this if if you'll just permit me to do a little bit engineering geek out it is it is uh so we use the word data plane to describe stuff that's you know effectively doing the plumbing portion so this data plane is actually built on new technology from scratch that we have written it's on rust and the Tokyo scheduleuler and we're doing something really funky this this data plane can proxy today the Kafka brokers in the future all kinds of message queuing systems out there, right?
So, it's a general purpose event proxy. Uh but the the the the you know, if you look at our footprint, it's about 10 MB in size. I mean, it's amazing what the right technology choices and you know, all of that will let us do. Uh but right now, no is you know, it's just um I just enjoy talking about that data. >> Oh, yeah. Yeah. It's fascinating. But but just just uh you know just answering your question uh imagine you know without quoting names um you know one of the one of the biggest uh u uh departments of the UK government is a is is a customer of ours.
They they use us for HTTP u you know and and APIs that run on top of HTTP but they also have teams that are actually using eventing systems internally. Right. So, so now this the team that is chartered the central team that is chartered with governance on APIs they're also looking at the eventing systems out there and saying but they are also APIs at the end of the day I mean even if they're encoded in terms of this kind of protocol and and you know messages and whatnot you are still executing business logic on the other end right that's what's happening and then if you think about it a lot of the things that we already solve in the HTTP world automatically apply those problems and therefore of the solutions apply in the event world.
I want to see my messages. Where can I get this message from which is same as does anyone else have this API already? Uh what is the shape of it? How do I subscribe to it? Like what what are the rules and so forth and now one very interesting problem. So there's a catalog is what I was describing and schema evolution and so forth and analytics on top of that. But very interesting problem uh you know uh that that we are enjoying solving right now is let's say you have you are a fully venting company. you are completely bought in.
You have all these events going on and you're you know you have all your business logic firing and now you want to bring third party partners like external people to call into your system. The problem is brokers that central brokers whatever your message queuing system be usually cannot be put on the internet. It's not uh you know it's not it's not recommended that you you you open them up uh uh on the internet. So you need something in between that can take eventing information and kind of upgrade that to HTTP, ship it over the internet to the other side and then ship it back and then translate back.
Yeah. So we are we do that too at the boundary, right? So so these are some of the things our event gateway um uh is trying to solve. >> Oh wow. So that's really opening a lot of doors then. >> Yeah. Absolutely. I mean you know teams are realizing um the the central architecture teams and and and the central API teams are realizing that there are multiple types of API traffic out it's not just about HTTP I mean LLMs as as applications talk to LLMs that traffic is climbing you want governance on top of LLMs and LM governance is not the easiest because instead of structured information you're now speaking English so you have a prompt it's English or you know whatever language and and maybe maybe you're in a regulated industry maybe you're a government or or healthcare or something and you say no, I cannot send what could be potentially PHI or even biased information out um uh to the LM exiting my boundary.
How do you detect that or or even the more technical problem of inference speed up? It takes too long for the LLM to reply. Can I respond fast? Actually, Kong solves those problems using our AI gateway. uh but uh my point is that governance on all of these are going rapidly becoming concerns of the same team so that these teams have to invest in technology that will give them cover over all of these developing systems not just one class of traffic. >> Yeah.
Oh wow. Oh wow. Yeah. That is a that's a that's a can of worms. So So I guess that's a that's a good segue into talking about you know AI gateway and how how it can maybe simplify integrations and at the same time maintain security. >> Yeah, absolutely. Absolutely. And I I and if I may uh if you permit me, I'll also add compliance as an element to to the problems that AI gateways have to solve. Um but to your point about simplifying you know kind of models and integrations and so forth like one of the one of the one of the things we see um we have we have some of the largest um AI I'm going to call it AI strategy implementations people who very very large companies that are walking the AI journey are choosing to express their AI strategy and their AI governance using using our AI gateway and and you know the the the the the the the special no I I would let let me let let me let me go back a little and say the the question you asked about model integration what is a hard problem see models are coming every day there is a new model and a new awesome >> every day it's non-stop >> you can't you can't keep up right you just can't keep up um but then but then let's say a new model comes out that gives you better responses for a certain class of prompts or or better faster inference cheaper inference for example >> your application probably is modeled build and and written to work with the API of that other model and now you have to rewrite portions of your application to make it work.
How do you keep up? It's it's too hard instead impossible. Yeah. >> Exactly. Exactly. in instead imagine you had this central thing sitting between your application and the LLM that always exposes a stable API to you but you could trivially switch the models just with an argument or or or you know extra annotation somewhere. So it it gives you stability against change on that side at the same time your inference speed up semantic features. For example, you could say if my prompt has this bias or has say customer uh looks like it has customer contractual information or it is talking about um let's say citizens of another country and I can't have that because of these reasons exit out of my my boundary then understanding the semantic meaning and then applying rules on that.
Kong gateway will do that. Kong's AI gateway will do that. Kong's AI gateway will can I I mean I maybe I'll give you a trivial example. If you were it's my favorite Google. >> Yeah, I'd love to hear it. >> Yeah. If you ask what is the price of the latest iPhone and you get an answer let's say back from from the LLM and say what is the price of the newest iPhone? It is the same semantic meaning. Why would you hit the LLM? Right. You you want something but see it's not it's not cachable in a traditional system because the the cache keys again the questions are different but semantically they mean the same thing.
It's the same answer. Yeah, it's a similar problem to keywords through AI too that they're they're always going to be a little different. >> Exactly. Exactly. I mean the the the the the technical complexity that that this LLM traffic brings in uh you know it's it's in addition to everything that HTTP kind of stuff brings in is the semantic element to it. You have to have that semantic multimodal semantic analysis is required. Um I I'll finally say this one last thing. um uh you know for AI gateways uh their problem space is actually dual as in if you think about it your application is not just working with prompts it's also possibly making HTTP calls and other things are going on and at the end of the day all of these whether is streaming prompts or direct uh you know request response chat API prompts there are still all HTTP underneath so not only do you need an semanticaw aware model aware LLM aware gateway but you can't take away the HTTP basics right you still need that and we are the only gateway that can provide you that full stack experience >> oh wow I didn't know that that's really interesting so you know I guess let's let's go here suppose I'm a developer starting work on a new project next week can you walk me through kind of the flow of a project without using connect versus using connect and how that's different, >> right?
Yeah, absolutely. uh the the if if your problem the project you're working on is probably you're expressing that or intend to express that as a as a as a micros service or or a service let's say connectivity how am I exposing like there is this whole you know what what we were jokingly calling the plumbing problem the plumbing problem is now back if you don't have a robust data plane where you can express your authentication policies your authorization policies your routing policies your DNS concerns your shaping concerns and all of that class of problems.
So that's there and in the connect stack it our connect stack starts from a data plane and goes all the way up. Right? So you run into the plumbing problems first you have to solve all of those and and then you climb up to the okay before I start writing the code. I need to know what can I build on top of what I'm trying to build. Maybe is this as simple as I just compose over four APIs that already exist. Why am I writing all this code? Ship faster, right?
That's what you want. You want to ship faster. That's the mantra. Exactly. Right. Why would you rewrite everything? Right. And and so so a place to go discover all of this, right? Uh so that's a catalog, a place to understand what is already available and how do I sign up and and and whatnot. So so without that catalog, I mean you know I have done this at previous organizations back long back in my career. Guess where I mean guess you know guess where people record uh you know their their existing landscape >> spreadsheets notepad uh post-it notes >> and then confluence page perhaps even more sophisticated but yeah it's it's it's not um you know what I what I say it's not live what I mean is you need a system perhaps into which you have done github integration or gitlab integr your source control for the service you already have in production production is linked into the catalog and the catalog is actively monitoring your source control to even know see what changes are happening the you need a catalog that is actually monitoring your SLA uh the the actual actual yeah your actual uptime it's actually monitoring for example say with page of duty or equivalent uh what's your uptime what's your open tickets you know what's going on right this needs to be live catalog it's not just a static record because as you well know spreadsheets are out of date the moment you hit save.
Um yeah so you need you need something like that and then you need and and and and going from there let's say if you wanted to publicly expose this outside you need the whole portal mechanism imagine a back-end team trying to build a portal by themselves uh is is not the best use of the time what you need is yeah a technology that already solves that for you right and then there is a whole analytics side telemetry side and debugging side so this is what I meant by as a developer you want to be able to to to builds.
That's your mock testing and we have we have this program called Insomnia that lets you do that. It's integrated into into our bigger story. But u you need to be able to build you need to be able to run, you need to be able to govern. You need to be able to discover what's going on and you need to be able to secure and all those guarantees. If you are a programmer using connect, we just we let you focus on what you must do the business logic and take care of the other stuff for you >> where you can check it all in one place. your your Oh, that makes so much sense. >> Yes. and and and if I may there is we also help one more role which is the you know in the central architecture teams and let's say security team wants to ask the question how many of my APIs that are out there are suffering from say this particular OAS vulnerability as a question or or or uh the central architecture team says it is required that all of our APIs must when they return an error must have this format like standardization.
So you need a place where you can express this and then apply it as in these are non-negotiables. So these can't be done just through you know documentation and instruction but through machinery right. So that also we allow the centralized governance uh you know uh and and expression the the the the the correct word we say is actually we say federated governance in the in the mean the meaning is individual teams can move really fast and you can still have a central place to go in and put your core governance rules and we decouple uh you know these two fairly so that you get the guarantees you need but you don't slow down your teams. any chance?
Uh I know we've talked a little about clients and that you weren't comfortable sharing names, but uh are are you able to share maybe a story where connect was able to cut duplicate APIs or maybe an accelerated AI use case? >> Absolutely. Uh I mean I I actually can quot I mean quote a referenceable customer uh case studies published online. Uh so this is one of the oldest and biggest private banks in Philippines. Um uh I think they were uh incorporated in 1951 if I remember security bank from Philippines.
Um and uh they uh I because you know their technology has evolved over time. They uh actually had a full enterprise service bus ESB almost acting like an API gateway. They moved from that model to connect and across 10 core business large business units they were they actually identified and brought down redu duplicate reduction to about 25% of their API. So so some direct duplicates some so close enough that you just wanted one of the APIs to have an additional parameter.
That's it. And why why would you double your infra right? So huge TCOs. Um I can also you know if you if you'll permit me I I can't take names here but u uh you know we we have we have some I'm going to say like fortune50 customers that expressing their entire for example their AI strategy and their API strategy using Kong where we have actually run into traffic throughputs of like 7 million requests per second steady state right it is yeah I mean it is I mean well well this may this may not be a published fact but it is actually uh true that uh outside of the biggest uh e- e e- retailer black Friday events we are probably the third largest black Friday events powered through Kong.
The amount of traffic that we push through at at tremendously low latencies is is is you know is just awesome >> and the consumer has no idea. I I bet that Corey today you and and I'm I'm pretty certain most of our viewers right now have gone through a Kong proxy or Kong powered API today as if you have been on the internet you have gone through Kong. >> I would bet I have more than one tab open right now that's going through Kong >> 100% 100%. >> Oh my gosh.
Well, looking ahead, um, what does a frictionfree API ecosystem look like and and how do you think Kong will push that frontier? >> Yeah, that's that's that's a great question. Actually, the way you described this, Cory, is exactly I would say that is the mission of Kong. That's what Kong wants to do. That's that's our true focus, right? So with an example that we just quoted before, you have HTTP teams that have traditionally owned HTTP APIs and and and been responsible centrally for them now want to extend those same visibility, governance, security, etc. to eventing APIs to AI gateway and LLM kind of use cases and there will be many more like streamings and VAF and you know there'll be many more of these class of use cases where that class of traffic will have the same governance burden on it so to speak right and and and and so maybe a prediction from me uh and we can check how how true it'll turn out I'm sure it'll >> we'll come back in a few years >> so so so at the lower layers are the plumbing layers so to speak API traffic is going to become uh multifaceted or multi-ature multi-ype that will be companies will have mix and just because you are not HTTP or not this or not that doesn't mean those governance those those build govern you know uh uh discover secure or none of that those obligations are not there those obligations are absolutely there so what do you want is you want a system where new types of this class of uh uh plumbing traffic can be brought in where new teams can be trivially onboarded and the dayto-day life cycle are completely solved.
You have a catalog that solves your problems. You have an IDP integrated that solves your problem. You have a vault that manages your secrets for you and all of this integrated together. So the the incremental cost of bringing in a new class of Denisen or citizen into your API world should be really low, right? And and and that for me is a truly frictionless place where you get to and that's what Kong is building >> kind of like with with email or where some think AI is headed.
This this idea that to serve up one instance is is going to be a fraction of zero. >> Exactly. Exactly. I mean the incremental costs truly have to come down everywhere right whatever you want to do with an API whatever type of API that full life cycle the incremental cost of adding new people onto it and performing action really needs to go down it should be frictionless one single integrated platform that's what Kong is building as we as we you know grow as we become bigger that's the big large durable company that we want to build >> Sanji this has been great be before we wrap If listeners want to go learn more about Kong Connect for themselves, where do they go next? >> cloud.hq.com. >> We'll put that at the bottom, too. >> Oh, yeah.
Yep. Absolutely. At the bottom, cloud.hq.com. Right. You can go sign up, social login, and just take it for a spin. >> Well, that's all the time we have for today. Thank you so much for watching Espeaks. Uh, Saju Palai, thank you so much for joining us. It's been super interesting learning how how Kong Connect can really turn sprawl into speed. >> Thank you so much for having me on, Cody. This was absolute fun. You're a fantastic host. >> Well, I appreciate it.
Uh, if you enjoyed today's conversation, be sure to take a minute to subscribe wherever you get your podcasts and leave us a review. Really helps others find the show. I'm Corey Ols and I'll see you next time.
This transcript was generated automatically from the
video's captions and may contain errors.
