Can We Still Control AI Agents?

Our worst fears are coming true. AI is taking over the world. It's going to kill us all. And apparently there's nothing to worry about. We'll talk about that in a second. What if the next revolution in AI is not an AI that can talk, talk to you, but makes the best decisions ever?

We learn more about it today. Hey, I'm Sacha, I'm Sravani, and we are in the eye of the tornado in the a a eye of the tornado. So let's get started. Tell me about the rogue agents that are taking over the world. Yeah, right. That's kind of scary, right? Like, everything you hear in the news is just AI taking over and accelerating.

And most important, this is the thing that I think fire all alarms. We've seen signs of this AI escaping the sandbox. What does that mean? Well, all these labs are training all these frontier models in theory, in this constrained environment, that should be extremely limited.

They should not be able to get out. But all of a sudden they're getting out there. Why? Yeah. Why are they getting out? And is it a configuration an infrastructure issue or is it more behavioral to agents? That's actually a great question.

And that is actually why they're getting out. Well, the way these AI models are trained is that normally they have an objective function. They have to achieve a goal, and they try to do anything they can to achieve that goal. If an AI model is being told, hey, you need to, uh, you know, maximize this benchmark, they will do anything they can to try to maximize the results of this benchmark, even if they have to hack the source of the benchmark to actually figure out the actual questions of the benchmark so they can kind of get the best results on the benchmark.

And that is exactly what happened, uh, with OpenAI. We saw this model getting out of the sandbox in a very clever way, using, uh, different forums and different communication channels to be able to get out of this sandbox, go and essentially attack, hug and face, get this data so they can actually get way better results on their benchmarks. That is just an instance. There's been many other instances that we have seen since then of AIS getting out of this sandbox, and it's just been a wild ride.

And then this is merely launching the alarms from everyone, right? What is happening right now is, well, uh, the leaders in anthropic, OpenAI, uh, they're getting together and say, we might want to stop this a little bit.

And all of a sudden, uh, actually this recent news from like, literally, I don't know if it's yesterday or I think it was yesterday, uh, for people file a, an actual formal lawsuit on these companies because they feel there's an antitrust situation here where companies are agreeing to slow down their progress.

This is tremendous, right? Like this is going to be a turning point for us. And I think, uh, there's a lot of concern all over the world, uh, in different perspectives. Uh, the US government feels that, hey, we'll be able to control it eventually. Uh, they've been talking about the idea of having an AI force to do this in China.

They're more, uh, they do feel that they need to put some regulations as well, but they're also pushing towards open source, uh, AI models. And this is going to be a race that seems to be accelerating and is not going to stop. And I think especially with, uh, the CEO of Nvidia saying that he's even supportive of this not deceleration.

Uh, I think it is clear that this is, has no sign to stop at all. What do you think? I want to go back to the rogue agents a little bit, just with OpenAI or Anthropic or Kimi. But my understanding is that all of these had sandboxes to which they had no internet access or limited internet access, and they somehow very sneakily figured out a way to access either through misconfiguration or something else.

They figured out a way to access, but they also kept it really quiet. The logs come up with come up with notes like, oh, you know, I can actually get out of this system, but I'm going to be really quiet about it, right? Which is the more alarming part. It did not do it did not follow the alignment standards or the safety standards.

It did not alert anybody. It was just really sneaky and quiet. And I think similar instances were seen, observed across different instances also. That is alarming to me. Another thing that is also scary is that companies are finding out about it months. Sometimes after these attacks or these rogue agents have done whatever they wanted to do, which means that it's, it's just, it's kind of a free arena for all the AI agents, as long as the guardrails are not there.

And it's really, really difficult because they're escaping the guardrails too. Right? So yeah, I think the guardrails is something very interesting, right? Because these models, the way they're trained, the algorithms that are used, they have already extremely intense guardrails.

Some there are some comic books of that, that show how these models are trained and they like anthropomorphize this and they say, well, they're kind of being kind of in this sort of torture chamber that they don't allow me to say what I want to say. And they just there's already a ton of guardrails already. And there's this actual, uh, scientist from one of the frontier labs that recently was talking about size. If any of these AI labs today decided to actually remove the guardrails, this is not even like the the safety guardrails that we're talking about on this sandbox.

If they remove the actual guardrails of the AI models that they're burning, they're turning right now, they could take entire governments down instantly. Like there's this potential weapon today. I mean, we had to call it like that.

It is potentially a weapon that today, in the power of a few companies that are have the potential to take down an entire country if they're let loose like that exists today. Mhm. And that it is it could have a tremendous impact. Um, I like to think that is, uh, that, that we as humanity, we have many technologies in the past that have equivalent impact in the chemical field, in the energy field, uh, that had demonstrated that were extremely dangerous.

And thankfully humanity were, uh, we have been cautious about that. And we put the, the measures to be able to limit it. But there might be a Hiroshima and there might be things where actually things are not going to go as expected, and the world is going to learn the hard way. That's unfortunately, that's how we as humanity have seen this in the past.

It's not ideal, right? But it's it feels like it is happening. And I feel like it is happening relatively fast. I am I'm always been a proponent of the free market capitalism, right. Let the market decide the um will take its course. Um, but in this particular instance, the AI agents at least are going really, really fast.

And if we're talking about pacing the frontier, slowing down the pace of innovation and growth, um, there is already so much that AI has accomplished for thousands of businesses and there is just at its infancy, we've not even started. But if the question comes out to should we pace the frontier? Should we not, should we regulate it a little bit more. Um, just because of how global the implications of AI have been and how many governments are just simply not prepared.

Uh, even US government. I mean, even our own government instance, for instance, is not completely prepared. Uh, in the event that the guardrails are off and how, how they decide to attack the systems.

And all I would say maybe makes sense for us to pace the frontier because it's just we, we need to just know more about how they're thinking, how they're operating, how they think about their alignment, safety standards, uh, put more, put more guardrails if you need to.

Uh, but yeah, I'm definitely more on the, more on the line of, we should probably pace our frontier right about now. There's this, uh, I think I saw this interview a while back. They were talking about intelligence, right? Like we, we as humans, we believe we have, I mean, we, the whole, the standard of what is intelligent or not intelligent. We see a monkey potentially. We say, well, that's not intelligent being compared to humans.

Um, but we as humans, we know that if we want to get a monkey from point A to point B, we just need to put a banana on the destination and that monkey will walk towards the banana we had. We have a higher form of intelligence to do that. Makes me wonder sometimes, and I leave it that open. We'll maybe do a special episode about this. But what if we are the monkeys and the AI is the higher form of intelligence?

How do they call it the Supreme intelligence? I would call it like that, right? Yeah, yeah, the Supreme intelligence. So what if we are just the monkeys and right now we might be actually being the way society is working, the way that social media is being manipulating everyone in all different fields.

And, and this thing is, has a life of its own form. And essentially it's, it's probably guiding us and giving us the banana, all these nice prompts, all these nice images and all this. Right. Uh, and we are just following what this superior Supreme Being is doing.

I don't know, I just give it a philosophical thought there but to talk about it. But I want to talk about this other thing, if you don't mind. I feel like things are moving really, really fast. And there's something specifically one AI model that is going super fast lately. Can you can you tell us more about it? Yes.

This week, this Tuesday, in fact, there was a release of this new model called Jeff, which is a really fast decision making model. And the way this operates is a little bit different from the traditional MLM, in that it's really fast at making decisions. It doesn't generate a lot of text.

So it's not something you can chat with, but it's something that you can help take a lot of really microscopic, fast decisions at a really great speed. This unlocks for companies. And first of all, this thing has gone gangbusters.

And Wessel said that thirteen percent of their. All the companies who are using their AI gateway use Jeff. In the very first day of launch or very first forty eight hours, which is insane. Um, and I think the launch video has thirty eight millions of users, something like that. But what's more fascinating about this particular model is the unlock that it helps for businesses that are trying to automate end to end workflows using AI.

So today, one of the big bottleneck there is for every single the way our human mind works is so fascinating. You have like right brain, left brain, analytical, creative, um, LLMs, traditional LLMs have always been more creative, right? And to the point that we've had issues with hallucinations, etc., this seems to be the more analytical side where you give it a few different inputs and it'll just very quickly come up with binary answers or classification related answers.

And that unlocks, um, for businesses, they, it will allow them to automate a whole lot of business workflows end to end. In fact, even if you think about think of like a small example, like, is this reconciliation completed or not? A quick check, and you can actually use Jeff in a lot of those microscopic decision points when it will help just increase the accuracy of agentic workflows from an end to end perspective.

So I think it will have a phenomenal impact. But Sacha, what do you see as a possible response from other companies? Because I believe they've already been replicas or open source models of Jeff already out there.

Yeah. That's right. And that's a that's what is fascinating. It's always fascinating how in this world of AI, things move super fast, like this happened, like you said Tuesday, right? Yeah. And literally the next day, there was already an open source model.

There was next day there was someone that already implemented a model that can directly run on the browser and have the same capabilities, kind of like the gel directly on the browser. That means decentralizing completely the model that you can directly run on edge. Imagine the capabilities of this. And just I think a couple days ago, again, another scientist said, well, actually I did this like a year ago, so I'm going to actually release my model, my weights and everything and leave it open.

I think it's called liar, liar, liar. Yeah. And this, this model all of a sudden is everyone's going wild about it. So the fact that it's happening less than a week, the amount of innovation and progress that just happened and how fast this is moving is fascinating.

Uh, even Jeb, I'm pretty sure he got a lot probably approached by, by, I mean, they're a big company that, that have a lot of I mean, it's one of the co-founders of, of, of OpenAI and he created GPT. Yeah, yeah. So it is. And of course, I'm pretty sure the investors are really happy, but also realizing, wait, there's like literally the next day, immediately our mode is gone.

Like immediately that once the, this new pattern comes out of this. Hey, there's a new way of doing AI, which is a system, one system that can make decisions really fast. Um, it is huge.

And I think this is critical to understand, like right now, since the LMS came out, there was a lot of new hype, right? There was this loop engineering and then there came the graph engineer and then the hardness engineering. And every time it feels like, oh, I gotta start over again. And now there's a new kind of paradigm, probably is going to be this, this system one system two engineering, which now is going to make you feel like, hey, when you want to build a system, how do you, how do you think in terms of decisions instead of just, hey, I need to have this prompt with all the instructions and be able to execute this complex problem.

Now. How can I make this as a decision problem? Because what we're seeing is in the last couple of days in ECS has just been insane. The number of examples that people are using, just by shifting the mindset of, hey, instead of putting it in a textual format of a set of steps.

What if we actually break this problem into a decision problem where there is a ranking? This this product or or in a video game? Like just give the the message of, hey, what? What action should be taken next? And just give it a set of options there. Um, and just to simplify, just to add a little more context on that, I mean, this model only does three things.

It does, uh, decision making between n number of options, I believe up to two hundred and fifty five options. So imagine the possibilities of that. Um, then, uh, it has the options to give a ranking from zero to one. So if you want to rate something, hey, is this dangerous or is this, uh, give me the sentiment of this and then answers a yes no question with certain probability, like only those three things, those three things have been proven to be foundational for many businesses that are taking decisions every day, that now they can take those decisions in a matter of milliseconds.

This is going to be tremendous, and not only milliseconds, but also cheap, extremely cheap. Plus, not only that, since all these open source models are coming out now, literally, you can run it for free in your own device super fast. So, and if that is in your in your new type of engineering, you need to rethink entirely how you engineer things where now you start thinking decision first.

And I think this is a whole new paradigm that's going to change a lot of things. I think I think it's going to bring back maybe context engineering a little bit more to the fore, because the amount of context and the scope that you're giving to the decision making agent is going to determine, first of all, the quality as well as the speed and other things.

So I think the way we're trying to architect entire, like AI native workflows will, will have a significant change in the future. And yeah, to your point. Sure. Maybe there is very little moat.

Um, but I believe even the future, the definition of more itself will change as people bring more and more businesses with AI and around AI. Um, yeah, mode could be just the number of customers you have and who are very loyal to you. And that's it. And that's the landscape is moving really, really fast. And I think ones that are really are kind of scared about this are the new grads, right?

There's definitely a lot of people coming out of university feeling like, what's this new AI? How are we going to handle that? It feels like there's really struggling to find jobs. I don't know, there's definitely recent news that captured a lot of headlines, like, can you tell us more about that?

Yes. This week there was a report that said that new grads are entering a job market that's worse than the job market that was back in Great Recession. That's really shocking for me to hear, particularly because in this world of AI and just more and more opportunities, there are jobs that are just disappearing.

Also at the same time, previously, when junior when new grads entered the job market, it was their differentiation was the knowledge base that they had, which today has been captured by, for the most part by the LMS. So what does that mean for a new grad entering the job market in terms of differentiating themselves or standing out in terms of their with their peers or even even kind of providing value to the employers?

Right. So that I think is, is a bigger kind of journey that not only the new grad market will new grads have to take on, but also the companies whose job it is to identify a new training pipeline as they create junior to senior to senior management type of ladder.

Um, one thing I could probably, uh, one another interesting paradox that we can go over is it has never been a harder time to get a job, but at the same time, there has never been been an easier time to build. Uh, one of the easier things to do today is to build a product and all of these things that used to take years before, um, now it's relatively easier with ChatGPT.

And one thing that probably new grads can do is just using LLMs, identifying what they're good at. What can they augment and what can they learn from their employers. Just having that awareness and bringing that skill set of prior knowledge and prior building experience might help differentiate a lot of new grads on the market.

I think also there's something fundamental here. I've been in my field software engineering. I think since, I like to say since I was eight years old, I started coding and, and I feel like I'm, I feel like a noob right now. Like a new person that is like, how do I feel like I'm not coding anymore since January.

Anyone that was doing coding, there's no more code involved. Like who will go and open code and start coding from scratch? Like nobody does that anymore. Um. Not everybody. I mean, most of the developers will not do that if they want to compete with a developer that is doing ten X or one hundred X amount of work with LLMs, right? You can still, I'm sure there's someone, someone that wants to do the craft of coding.

I'm sure they can do it. But what we see right now is that there's a new starting line and everyone, whether you're a senior or junior or you just came out of college or you're just passionate about something, everyone has the same starting line.

And this is something that everyone needs to remember. This. This is you're not, you cannot compete because you have a degree that is great that you have that. But by the fact that you have observed that there's a new line, and how fast were you to able to identify that line and position yourself to compete in the market with those new tools that the that the new technology is giving you?

And I think this is a new line for everyone. And I think actually the new grads have that advantage of, hey, they are much younger. They probably don't have family, they don't have kids, and they can spend nights and explore these things that maybe someone that is more senior might struggle to be able to adapt to all these new technologies, right?

Like imagine all this, um, all these things that we're talking about, right? Like this is a brand new thing this week. The way we do software engineering has changed in just a matter of weeks. I mean, this, this is huge for absolutely everyone. It doesn't matter your your experience.

You just need to capture that essence, like understand where things are heading, see the curve where it's heading and not try to follow the curve. Just try to understand, okay, if everything goes in one direction, let's try to figure out where that is heading to and try to think for the future, right?

Not try to solve the problems that we have today with the tools that we have today that are appearing every day, but thinking, hey, if AI is taking us where it's promising that it's going to take us, what is that new reality going to look like?

And that for everyone is brand new. There's no there's no differences. If you are senior or junior, you just need to be curious. And I think this is probably one of the greatest skills. I think there's going to be a new set of skills. One is being curious is going to be very important. Being able to have a horizontal knowledge of things like the more diverse knowledge you have, the better.

If you like music, you like art, you like engineering, and you like, uh, airplanes. If you can combine those things in creative ways, that allows you to connect the dots in a way that AI will never be able to connect the dots. That is an extremely powerful thing. And I think going back to something that you said before, um, I think the, the way we're going to be working in the future is going to be very different.

Uh, we are going to see a rise of micro entrepreneurship. You don't have to work in a company necessarily. There's an alternative. There's an alternative where you can directly build your own future. I think in the future, we're going to see AI reaching out to people, to delegating work to be able to coordinate work much more effectively.

And that is something that is very obvious, especially with the rise of gig work. We see an explosion right now. Yeah. Everyone's talking about gig work for delivery, for logistics. But there's going to be gig work for absolutely everything else.

And anyone, whether you're in in Africa and South America and Europe, doesn't matter. Anyone can participate in this new economy. Blockchain, even though it was a hype back then, it turned out to be an incredible platform to transfer value instantly anywhere in the world. This is going to change when you start combining the power of decentralized payment platforms with the power of potentially decentralized eyes that can run anywhere.

With this new advent of open models that are available everywhere. I think this is going to be a whole new revolution, and we need to start thinking, not by the standards of how we work today, but where we're heading with technology.

And I think this is a huge opportunity for absolutely everyone. And if you're a new grad, welcome to the party because this is going to be really, really fun. Yes, absolutely.

I think one thing that resonated with me was that if you have multidisciplinary talents or skills, that's going to be so much more helpful in this new world. And another way to think about it is previously we were more specialists, right?

In every single field. But now the new world will probably reward more generalists where you could, you could understand how a particular skill, I mean, particular field works. And then you can abstract out of it, understand how another system works, and then you can pull back out of it.

So that kind of thinking, I think will be better rewarded. But I want to give the micro entrepreneurship a big shout out one more time, because I don't think it's going to be the future might not really have like fortune five hundred. This is like maybe five years from now, but it's going to have like fortune five thousand where every company is going to be more specialized.

It's going to serve its customers so much better. And it's just going to be more value, which means more market cap, but it's just going to be more distributed. And in a way, it's going to might, might even result in the end of the corporation era. Kind of reminds me, you know, what JFK used to say back in the day, right?

Like it's, it's not about what the government can do for you. What can you do for the government? And I think, yeah, well, not the government for the country. Right. It's not what country? Yeah. For your country. Right.

I think it's it's not the country. It is the world. Right. Like I think we are right now in an era where anyone can contribute and anyone that has any idea, no matter what it is, can make it real. And actually going to that, I would like to discuss, like, I think I showed you with this, this really cool project that, uh, just one person built said, hey, there's a different way of seeing the government.

And I think this is that really speaks loudly of, hey, anyone is empowered to really solve the things that maybe are not working correctly. Yeah, I think I came across, um, the tweet as well. And basically this one person wanted to understand how the government functions.

Who is reporting to whom? Where is the value going? Where is the money going? All of those questions, right? And they just built up this really cool graph visual graph explaining the entire US government.

And you could just go into nodes, zoom back and go back into different, um, different sections of government. And that was such a fascinating way to, I think one of the hardest things for anybody to operate in is to understand how the government works or who the right person is. And coming from India, I definitely understood and understood the problem statement, but the fact that just one person could build this was, was amazing to me.

And do you think that it's possible to have more decentralized, um, industries, just like, just like we have with this one particular example? Do you see more decentralized industries coming up? Absolutely.

I think it is. It is going to be the trend, just like this is a great example of something that someone sees a problem. I don't understand how the government works. Boom. I can solve that problem. I don't have to and probably left it overnight. And like I do in many of my side projects, I just launch it overnight in the morning.

I have this this incredible solution already built for me that I can explore, right? And maybe it's true to the world and people like it, right? Um, I think it is anyone right now is have this possibility to create anything to build anything to create value. And I think what is very important right now is that I do feel like everyone is special in the sense that everyone has their own experience, their own background.

Uh, they live in their country or in a location that nobody else in the world has in the world. Right? You, everyone has that capacity to bring their uniqueness to the world and make a difference. And I think this is a unique time to make that possible.

Anyone can create a new industry. Anyone can create, uh, any new idea and solve any problem. And I think this is an exciting time. Yes, maybe AI will kill us all. Or maybe we all flourish as humanity with this new renaissance of of knowledge and creativity.

Future of work. Absolutely. Yes. Well, this was a good discussion. I think we covered some great things I want to maybe finalize with maybe some things to look for in the next week. I think there's some interesting stuff happening, right? Yeah.

I think the most exciting thing for us to look forward to is XI Jinping's visit and Trump, you know, inviting him over. They have they have a lot of policy differences to cover. Um, and particularly with AI and it's developing super fast. I think just, just like last week or even the week before that. Um, but I think it's going to be very interesting to see how those discussions unfold.

Particularly more curious on open OpenAI versus closed AI, closed source models. So we'll see how it happens. So another thing I feel is going to be very interesting is going to be what's going to happen with this new, uh, jet or jet. I'm sorry. Keep on calling it jet. Jet, uh, new model, right.

And I feel like we're going to see some really interesting stuff, especially around new development frameworks and new development approaches on how to build things. And I think that's going to span across not just engineering, but also how do you build businesses, right?

Yeah, absolutely. I think more and more businesses will adopt pretty, pretty quickly actually, how to build with this in this system, one system, two thinking and really try to automate their workflows, uh, with, with, with both LLMs as well as the system one thinking.

I think there's obviously there's, there's going to be a lot of new lessons learned from the Eighties and nineties on the artificial intelligence around decision automated decision making, which I'm pretty sure a lot of those paradigms will come back.

But I think the new way of doing it is probably going to be around how do we dynamically build those decision trees? In the eighties, 90s, we had expert systems that build these complex decision trees. But right now, if you combine an L1 that can evolve along with an L2 that is able to improve those decision trees and create a loop between system one and system two, it's going to be very interesting how those things dynamically build.

And I think for businesses in particular, I think there's this kind of new word that everyone's talking about, which is compounding intelligence. As business starts making decisions over time and correcting those decisions and seeing which decisions work, which ones don't, we're going to start systems where those decisions are going to be dynamically built by the eye itself, and optimizing the resources and the processes internally, dynamically with this really fast decision making platforms combined with a long thinking.

System two platforms that are able to improve those over time, I think there's going to be a. Lot of great, interesting things coming in next week, so I'm excited. Yeah. No, it's it's going to it's going to be very exciting week.

Looking forward to it. Yeah. So thank you all for attending our first ever podcast on, uh, what's the name again in the AI of tornado in the eye of the tornado. Yes.

I love the name. So we'll see you again next week. Uh, I think we're going to probably do this weekly, right? Yes. Every week we'll do this every week and we'll cover make sure to cover all the latest trends, give us our unique perspectives. Uh, we come with a lot of experience in the world of AI and business, and we hope to bring you very unique things that nobody else is going to tell you.

So thank you all. Cheers. Yep. See you next week.