Will machines decide what to learn next?

Anthropic’s recent announcement around its most advanced system and the launch of Project Glasswing has triggered concern across the technology world. The headlines are predictable. A system that can identify and exploit software vulnerabilities at a scale no human team can match. A system considered too risky to release.

That is serious. But it is not the real story.

What stayed with me was not the headline. It was how this behaviour was described. It suggests that some of what these systems are doing may not be entirely predictable from what they were originally designed to do. That they may be going beyond execution into something closer to discovery.

And once that line is crossed, the implications are immediate. For every company that pauses in the interest of safety, there will be another that will not. If one organisation can build this, others will follow. That is how competition works.

Which means this is not a contained risk. It is about to spread.

The first implication is obvious. Our systems become more vulnerable.

We are already seeing ransomware attacks exploiting small cracks. What we are now talking about is not incremental. It is a different category of threat. Not a sharper tool, but a shift in power. The cost of defending systems will rise sharply. And the risk of disruption will rise with it.

The second implication is more uncomfortable. Uncertainty.

If systems begin to decide what they need to learn next, they begin to shape outcomes in ways we may not fully understand or anticipate. Until now, these systems have been seen as enablers. They improve productivity. They take away routine work. They help us do more.

That comfort is fragile. They do not become enemies. But they stop being predictable. They become something harder to define. A partner that helps, but also a system that can test boundaries in ways we cannot fully see. That is a very different relationship.

The third implication is institutional. There is too much money chasing too few guardrails.

Governments are hesitant. Not because they do not see the risks, but because the upside is too large to ignore. The result is a race to move faster, build bigger, and deploy sooner. But no one can clearly answer a simple question. What are we racing towards? And when speed becomes the only metric, caution gets left behind.

Finally, there is the question of data.

This is where I worry deeply for India. We have seen this story before. Raw material goes out. Value comes back at a price.

There is a real risk that data becomes the next cotton. We export it freely, and then pay to use systems built on top of it.

Despite the warning signs, we are already moving in that direction. Education data, student learning data, textbook content are beginning to be opened up without a clear long-term position. It may help create a headline. But it also creates dependence. And dependence, once created, is rarely reversed.

So how do we respond?

Not with fear. And not by stepping back. But with clarity.

First, we must recognise that this is not just about technology. It is about how systems learn. And once

learning changes, everything changes.

Second, security can no longer be treated as a cost. It becomes central to survival.

Third, governments need the confidence to act early. Not to slow progress, but to shape its direction before it becomes difficult to control.

And fourth, India must take a clear position on data. What we share, how we share it, and what we will not give away.

Because in the end, this is not just about technology. It is about control. For decades, we have lived with a simple assumption. Humans decide. Machines execute. That assumption is now being tested.

The question is no longer what these systems can do. The question is whether they will begin to decide what they want to learn next.

And if that happens, the more uncomfortable question is this. Will we even know when we lost control?

(The writer is Founder Chairman Sampark Foundation and former CEO, HCL Technologies. Views are personal.)