Waking Up With AI

Katherine Forrest: I'm going to tell you about a definition that I really like. It's from a position paper called “Agent AI Towards a Holistic Intelligence.” And the first author, for those in our audience who want to look it up, and I really encourage folks to read it, is from an individual last name, Huang, H-U-A-N-G, from Microsoft Research Core. And the definition is this, and get ready for it, because you might need another swig of your coffee.

So here's the quote: “An AI agent is an intelligent agent capable of autonomously executing appropriate and contextually relevant actions based on sensory input, whether in physical, virtual, or mixed reality environments.”

Anna Gressel: That's a little bit of a mouthful. But for folks who have been our regular listeners, AI agents combine important topics from two of our prior episodes. The first is training AI, and the second is multimodal AI, which we spoke about a few episodes ago and is an important part of this whole agent dynamic. But let's explain that a bit more, Katherine.

Katherine Forrest: Yeah, well, an AI agent learns about the world from a whole variety of data, all kinds of different data, from visual data, audio data, text-based data, and the point of the agent is that it's able to learn from the environment around it and then use that information to execute tasks appropriate to the environment. And they can do so sequentially. It can do a number of tasks in order.

Anna Gressel: Yeah, I mean, one thing that agents might do is independently navigate websites, or if it's robotics, they can do one task over another and they can learn from humans actually about how to do new kinds of tasks. It's very impressive.

Katherine Forrest: It's a little bit like how children learn. So, Anna, a moment ago you mentioned a possible use case, and let's spell that out a little more. What are the use cases that people are talking about for AI agents? What are you hearing?

Anna Gressel: What I'm hearing are things like complex assistants. So one thing you can do is set up a software agent. So this is something that really only exists in your computer, kind of like an app. They can do all kinds of complex processes. They can set up an entire vacation for you, from finding and planning the itinerary to booking the travel and activities to sending you the results. It's like a personal assistant and even something like someone sending you all of the spreadsheets that you need for work or aggregating information and making sure it gets into your inbox.

Katherine Forrest: And another use case that I like to think about is robotics, because AI agent capabilities really have the potential to bring our concept of robotics and cognitive robotics to life and to place the robotics field on the cusp, really, of an enormous breakthrough. You can imagine a robot that's not limited to being a single task robot on an assembly line or vacuuming your floor, but that can actually work with you on multiple home or work tasks throughout the day. Again, what an agent, an AI agent, is doing is it's working sequentially through a number of tasks, not necessarily with a break in between them and not necessarily being directed in between them.

Anna Gressel: So we've talked about two different kinds of agents, Katherine, essentially. We have agents embodied in robots, that's one kind, and then the other kind is what we might call software-only agents.

Katherine Forrest: Right.

Anna Gressel: And what's new about these? Let's tell people, why is this different than what we've seen before?

Katherine Forrest: Well, these are built on top of advances that we've introduced, as you had mentioned earlier, Anna, in prior episodes about large language models and MLLMs. They do multiple things, and they have the flexibility to pivot also if they run into issues, because they're working within an environment that they recognize in multiple dimensions.

Anna Gressel: I think we're going to be hearing a lot more about AI agents all throughout the course of this coming year.

Katherine Forrest: Oh, for sure.

Anna Gressel: So what's your practical pointer for our audience today, Katherine?

Katherine Forrest: The practical pointer that I want to really sort of want to make sure that people understand right now is that AI teams need to look at AI agents and possible use cases, have those on their radar screen because they can add efficiencies not by stopping at one task, but by completing a whole series.

Alright. Well, that's it for our short morning show called “Waking Up With AI,” a Paul, Weiss podcast. I'm Katherine Forrest…

Anna Gressel: I'm Anna Gressel, and we hope you become a subscriber, and don't hesitate to let us know about topics you want to hear about. Have a great week.