AWS Vice President Bratin Saha on the Bedrock Generative AI Tools

foreign speaks we're talking about the hottest sector in technology today which is a generative AI in the many years I've covered technology I've never seen a more massive surge in interest than we've seen about generative AI in the last few months really really stunning stuff let's talk about that I'm joined by a very special guest with me is Broad and Saha vice president of AI and machine learning at Amazon Broadway very good to have you with us today thank you James nice to be here so I want to ask you about the certainly the AIS sector would you agree it's been really obviously a an enormous surge of interest in generative AI in the last few months stunning is it not yeah and it's a really transformational technology and you can imagine it's going to really lead to new user experiences new conversation user experiences more natural interfaces um and we think it's going to help us and others reimagine how products are built Trends I think driving the sector but if you point to a trend or perhaps a couple Trends what's I mean obviously the technology itself is very powerful what is driving the generative AI sector now as you see what sort of forces are shaping it so I think the what the first one is really the amount of compute and data that helps you train bigger and bigger models to give you some stats uh three years back 2019 you know the state-of-the-art models used to have maybe 300 million parameters you can think of parameters as knobs that you can use to tune these models and train them right um today you you can have up to 500 billion parameters or more okay and that's a 1600 X increase in just three years that's like you know super exponential and what that has allowed us to do is take these really large models and train them on a lot of data and what has allowed that to happen is for these models to have these very interesting capabilities in terms of generating text generating images and so on and so forth and so as a result of that you're seeing a lot of these really Innovative use cases which is you know summarize a dog for me or you know create an email for me or write a draft of this document for me that I can edit or you know create these small videos or images that I can subsequently edit so on and so you know a lot of um what we are going to see is generative AI being used as an assistant in many creative tasks you know in many generative tasks and I think that is what this you know this a Confluence of a lot of compute lot of storage enabled primarily by the cloud because you know you really need the cloud to be able to have access to this kind of data this kind of storage with the right volume right governance and so on so the cloud enabled a lot of developers to be able to deploy this kind of compute and this kind of data to drain these large models that has led to them having these generative capabilities and that is leading to a lot of interesting use cases well actually I think there's there's many companies out there right now thinking to themselves well we need to get on board with this we need to leverage this for our own competitive advantage any advice any advice you would give with them how should companies deal with this now prepare for this now given the enormous power that this technology has so I think it's really important that every company think about how they can reimagine their products reimagine their customer experience by leveraging generative Ai and you know the question you ask is okay if I have a product or a service if this product or service were designed after generative AI was created how would it look like right um and I think the place to get started would be starting with you know you can take a model and these Foundation models and try to train it yourself and so on but that's going to be a very expensive and pretty hard and that is where I think all of the products that we announced yesterday are so relevant so we announced Amazon bedrock and Amazon bedrocks gives customers a choice of foundation models that are pre-trained and the important thing that differentiated offering with Amazon Bedrock is the choice and the flexibility so you get models from AI 21 you get models from anthropic you get them from stability.ai and you get them Amazon's own first party models called Titan and so you're getting this choice and this flexibility because we really think that you're not going to have a single model that rules them all and this is this thing that we're giving customers choice I think is really key and really important for customers and they're getting all of this choice and all of these pre-trained models with all of the security and privacy guarantees that Amazon provides and it comes integrated with AWS services and we are familiar environment so you really we are basically taking care of all of that heavy lifting in terms of taking the data curating the data you know filtering out inappropriate content training the modeling this model training can take weeks months on thousands of gpus okay so all of that we are doing and I think customers can now start using these models they can also fine tune it with their own proprietary data um and you know build generative applications on top of this so I would highly encourage customers companies of all kinds to start looking at Amazon bedrock figure out how they can leverage its choice and its flexibility integrated with all of the security and services that AWS provides and start reimagining how they're going to run their businesses well you mentioned the three companies that are on AWS that that enterprises can choose from and then of course there's Titan itself Amazon Amazon's own own model is there a way for companies to choose between those three or four different offerings and something that distinguishes I mean how would they how would they go about researching that are even investigating that you know I think ultimately it will depend down on the use cases so for example if you're looking more at kind of Imaging Visions kind of use cases then you would probably go with stability.ai um Amazon Titan you know we have a text model but we also have an embedding model these embedding models can be used in such kind of use cases so at the end you know we are effectively giving customers the choice on the flexibility through bedrock and you know they can then use the model that works best for them and we also have more models we also have a lot more models on the you know in Sage maker that customers can try out but the advantage of using Bedrock is that you get a completely managed experience you know you get the choice the flexibility integration with AWS Services the security and the privacy well I guess when a company thinks about you know using one of those Services they might say to themselves that's great we might need an in-house AI expert or even an in-house AI team to what extent is is there some hand holding going on if I go to bedrock I'd say I want to use one of these services do I need a team of AI experts in-house to be dealing with that you know the the Bedrock models uh what we have done is we've done the pre-training so you know we have done all of the work of building the model and training it and making it ready to be used in many situations you can just use the model out of the box you know what is called zero shot okay so you can just go ahead and type prompts and ask it to do stuff and it will generate output and in those cases you you need no ml expertise you need even no software or development expertise and it's built that way now there might be situations where you say you know I have some proprietary data I have some custom data and I want to tune the model to my own particular needs in those cases if you have to do some model training it's still pretty easy you know you still point a model to your data in S3 and it drains but in those cases you may or you will probably need some software or machine learning expertise when you're trying to train it but for many use cases you don't need to do any of that you know you just come in type a prompt and you get a response back um you know I think one of the big trends sweeping though the tech industry is the idea of the democratization of Technology where it's not just the upper level executive all sorts of staffers that would have access to the technology it sounds somewhat like this is is going into that like you wouldn't necessarily need a team of experts to access artificial intelligence which to me feels like sort of a turning point in the history of AI absolutely and uh you know these launchers that we made yesterday I think a really key in democratizing machine learning generative AI you know we've done the same with our machine learning and AI Services we have more than a hundred thousand customers we that is what made machine learning and AI pervasive and I think these launches do that so Amazon better off democratizes access to these generative AI models you know the use and access to these generative AI models but we also announced infrastructure Innovation so we announced the inferentia two processors and the terrainium one and processors and these significantly reduce the cost of training or serving on these models they provide up to a 40 Better Price performance than any other ec2 instances and now when you look at this generative AI models you know it takes a lot of effort and a lot of GP you know it takes a lot of processors to be able to train them and so it can be very expensive and so being able to use inferentia or inf2 or terranium 1n actually can reduce your cost by almost 2X and that's a huge deal if I have to democratize access to these um models and then finally we also launched Amazon code Whisperer in GAA and it's available for free for every developer okay and so now what we have done is we have said look we are going to be innovating on all of these vectors there is the Bedrock models you know with all its choice and security and flexibility and integration with AWS there's the infrastructure innovations that significantly reduce the cost and make this a lot more accessible and then there is code Whisperer which is a great application of generative AI for software development it generates code for you and it's available for free to every developer so that's huge in terms of making this making generative AI in general accessible to all our customers hmm well I think that it I think this indeed is a turning point of history of AI it's you in the last few months ever since open AI you know debut chat gbt uh well let's talk about the future of generate AI I mean how do you see it developing over the next 0012 months or so and you're in your view you know I think you are going to see uh these generative AI models increasingly get integrated into all kinds of applications you know some of it will be in the form of assistance for Creative tasks like a generate a draft of this for me or here is a document generate a summary of this for me or you know I I have some information coming in generate a draft of an email for me all kinds of that or you uniform let's say an analyst a marketing analyst okay generate maybe some ad copy for me or generate some small videos for me and then you go in and iterate on it and keep refining it and so on and so I think what you're going to really see or you know change the way we interact with information like the change the way we search for information change the way we look at documentation like you won't have to price documentation you just ask a question and out comes the answer so I think you know you're going to see this increasingly get integrated into all kinds of applications such as rethinking how customers and users get their information and how they act on that information because you know you are you are also going to see these you know once you have the information you can imagine that these apps will then be able to act on your behalf to get stuff done well I mean I think part of the excitement or part of the interest around General today is that there's so much potential but then mixed in there's a little bit of a you know there's a little concern like oh gosh we don't know exactly what exactly what effect it's going to have there's I don't know what the percentage is 80 optimism 20 Warrior maybe you know 64. yeah I don't know what I wouldn't guess on that but I well what would you say to those people who say oh generative AI I'm worried about it in terms of jobs or some other some other aspect what would you say to the Warriors out there you know I think we have we have made a lot of effort to make responsible AI responsible use of these Bedrock uh and code Whisperer possible like we've put in a lot of effort we already have a very robust way of doing responsible AI because you know we have been doing Ai and ml services for a long time but we you know but we took it up a notch with bedrock and with code Whisperer and so we actually went and curated a Content filter the content for inappropriate use cases we have an output filter on the Bedrock model so the Titan model that actually looks at the output and filters out inappropriate content when you look at code Whisperer what we have is a reference tracker so when it's generating code that may look similar to some other code that it has been trained on it'll actually output the right copyright notice for you so that means possible for you to use it we have Security checks in the code being generated so we check it for security best practices for the bugs that you have in the overask test Suite so you know we have put in a lot of effort to make sure that you know these tools these models these AI Services can be used in a responsible way Brian that's a lot of good stuff uh anything you'd like to add in closing about generative Ai and and or the Amazon offering I think these at exciting times you know I think there's just a lot of innovation happening and I expect that base of innovation to keep accelerating and so I think it's really important that customers and companies start thinking of what is the generative AI strategy and how they're going to embed generative AI in all facets of how they're on their companies and I think Amazon bedrock you know Amazon code Whisperer from the perspective of software development inferentia to trainium one a really key capabilities and you know customers should really check those out right I think you said it uh it's going to be a fascinating sector to watch in in the months and years ahead thanks so much for sharing your Insight and please don't come back and talk with us again sometime happy to James thanks for having me

This transcript was generated automatically from the video's captions and may contain errors.