Just Because AI Can Make the Decision, Should We Let It?

 Written by Dan Straw 

I spend a lot of my time talking to HR leaders about AI.

It's difficult to have a conversation about the future of HR without AI coming up. And understandably so. The opportunities are enormous, whether that's reducing administration, improving employee experience, identifying skills gaps, or giving managers better information to support their decisions.

I'm genuinely excited about that potential. But as these conversations have developed, I've found myself thinking more about a question that isn't quite so straightforward.

Just because AI can do something, does that mean we should let it?

We've already seen examples of what happens when AI gets things wrong. Amazon's experimental recruitment tool is perhaps one of the best known. Trained on historical recruitment data, it developed patterns that disadvantaged women. The project was eventually abandoned.

The technology hadn't independently decided to discriminate. It had learnt from the world we'd given it.

Examples like this have understandably shaped the debate around responsible AI. We worry about bias, unreliable information, and decisions that may disadvantage particular groups of people.

But I've increasingly found myself wrestling with a different question.

What happens when AI gets the decision right?

Imagine two people being considered for a promotion. Both are capable, with strong track records. An AI system analyses their performance, experience, and skills, considers the available evidence, and recommends one of them.

It doesn't simply produce a name. It provides an explanation, identifies the information it considered, and highlights the factors that influenced its recommendation.

The manager reviews the recommendation and agrees.

Twelve months later, the successful candidate is thriving. By every reasonable measure, the AI appears to have made the right call.

So, what's the problem?

Perhaps there isn't one. But I'd want to understand a little more before reaching that conclusion.

Were the criteria appropriate? Was the information reliable? Did the system overlook something important? And did the manager genuinely assess the evidence or simply accept a convincing recommendation?

Even if we can explain how AI arrived at its recommendation, that doesn't necessarily tell us whether we should have followed it.

And that raises another question.

Who actually made the decision?

When Is a Human Decision No Longer a Human Decision?

The obvious answer might be that AI should never make the final decision. Keep a human involved and the problem goes away.

I'm not sure it's quite that simple.

Imagine our manager receives an AI recommendation but retains complete authority to accept or reject it. On paper, the decision is still theirs. But what happens when the system develops a track record of being right?

If it makes good recommendations 90% of the time, how often will that manager genuinely challenge it? What about 95%? Or 99%?

And what happens when those recommendations come with detailed explanations and supporting evidence that would take the manager hours to independently assess?

We should welcome improvements in AI explainability. But there's a difference between understanding the explanation a system provides and independently validating the reasoning behind its recommendation.

The UK's Information Commissioner's Office (ICO) has long highlighted the risk of automation bias, where people place too much reliance on automated recommendations rather than exercising their own judgement. Its guidance on human review emphasises the importance of reviewers having the knowledge, experience, and authority to challenge a system's output.

That guidance is being updated following changes introduced by the Data (Use and Access) Act 2025, which permits significant automated decisions in a wider range of circumstances, subject to appropriate safeguards.

The regulatory framework may be evolving, but the practical question remains.

Are we genuinely reviewing AI recommendations—or simply approving them?

There's an interesting tension here. We often talk about keeping humans involved, as though human judgement is the safeguard against flawed technology. But humans aren't objective either. We have biases, make inconsistent decisions, overlook information, and can be influenced by relationships and experiences.

So, what happens if the evidence eventually shows that an AI system makes more consistent and demonstrably better decisions than the humans overseeing it?

Would it then be irresponsible not to follow its recommendations?

That's where the ethics become much less comfortable.

This isn't an argument for handing decisions about people's careers to machines. But nor do I think it's enough to say that a human must always have the final say and consider the problem solved.

The real question is whether that human has genuinely assessed the recommendation, understands its limitations, and remains accountable for the decision they ultimately make.

Because putting a person at the end of an automated process doesn't necessarily make it a human one.

We've Been Automating Decisions for Years

Of course, none of this started with generative AI.

Organisations have been using technology to inform, and sometimes automate, important decisions for decades. Banks use credit-scoring models. Insurers use algorithms to assess risk and price policies. Fraud detection systems identify transactions that warrant further investigation.

Those industries have already had to wrestle with some familiar challenges. How do you explain a decision made using a complex model? How does somebody challenge it if they believe it's wrong? And who remains accountable when technology has played a significant part in reaching the outcome?

That experience gives us something useful to build on.

Today's AI systems can process enormous quantities of information, identify relationships that aren't immediately obvious to us, and increasingly make recommendations in areas we've historically regarded as requiring human judgement.

The more sophisticated these systems become, the more valuable their recommendations may be. But potentially, the harder those recommendations become to independently evaluate.

That matters when we're deciding whether somebody qualifies for a loan. It matters when we're deciding whether somebody gets a job, receives a promotion, or is selected for redundancy.

The lesson from previous generations of automated decision-making isn't that we should stop automating decisions.

It's that guardrails shouldn't be an afterthought.

We need to decide what requires explanation, where people should be able to challenge an outcome, what meaningful human oversight looks like, and who ultimately remains accountable.

Ideally, before something goes wrong.

The Bigger Conversation

These questions aren't confined to HR.

Step outside our organisations and a much bigger conversation is taking place about whether the pace of AI development is beginning to outstrip our ability to understand, govern, and regulate it.

Some of those warnings aren't coming from people who oppose AI. They're coming from the people developing it.

The 2023 Bletchley Declaration, agreed by countries including the UK, US, and China, alongside the European Union, acknowledges both the enormous potential benefits of AI and the possibility of serious, even catastrophic, harm from the most capable systems. It also emphasises the importance of international cooperation and appropriate governance.

I don't raise that because I think we should be frightened of AI. I don't.

I raise it because there's an important principle underneath the debate.

At what point do you put the guardrails in?

We could wait until we have perfect evidence of every possible risk. But that means some of those risks may have materialised before we act.

Or we can recognise that innovation and governance don't have to be opposing forces. We can continue exploring what AI makes possible while deliberately deciding where we're comfortable drawing boundaries.

Organisations face exactly the same choice.

We don't need to wait for legislation to tell us whether we're comfortable allowing AI to influence a promotion decision, how much transparency an employee should expect, or who should be accountable when something goes wrong.

Regulation will continue to evolve. But organisations can start answering those questions now.

The guardrails shouldn't arrive because something went wrong. They should exist so we're better prepared when something does.

What About the Cost of Using AI?

But the decisions AI makes aren't the only consequences we need to think about.

There's also an environmental question that is becoming increasingly difficult to ignore.

According to the International Energy Agency, data centres accounted for approximately 1.5% of global electricity consumption in 2024, with demand projected to more than double by 2030. AI is a major driver of that growth.

And electricity isn't the only consideration. There's water consumption, infrastructure, and the resources needed to manufacture and operate the technology.

For some, that's become an argument against AI altogether.

I'm not convinced it's that straightforward.

AI can itself support environmental goals, help organisations use resources more efficiently, and enable people to complete activities in less time. But improved efficiency doesn't automatically mean lower overall consumption, particularly if it encourages us to use AI more.

Perhaps the more useful question is one of value.

If we're using AI to analyse thousands of data points to identify something that materially improves an outcome, we might judge the resources involved to be worthwhile.

But what if we're generating fifty versions of an image because we can't decide which one we like? Or asking AI to perform tasks where it adds little or no meaningful value?

I don't pretend to know exactly where that line should be drawn. And it will undoubtedly move as the technology evolves.

But if we're serious about responsible AI, we have to consider the question.

We can't talk about our organisation's sustainability commitments in one conversation and treat the environmental impact of our technology choices as somebody else's problem in another.

Responsible use isn't only about what AI does.

It's also about how, why, and how often we choose to use it.

HR Needs to Be in the Room

For HR leaders, I think there are two conversations happening here.

The first is about how we use AI within HR itself.

Recruitment is perhaps the most obvious example, but it goes much further. Performance, talent, learning, workforce planning, and reward are all areas where AI can increasingly influence the information we see and the decisions we make.

We need to understand where AI is being used, what role it's playing, and where accountability ultimately sits.

But I think there's a second responsibility that's potentially much bigger.

HR needs to help the organisation work out what responsible AI looks like for its people.

It's tempting to think of AI governance as an IT problem. Or perhaps something for legal, information security, or compliance.

Those functions absolutely need to be involved. But consider the questions AI is creating inside organisations.

What happens to a role when a significant proportion of its activities can be automated? What new skills does that person need? Should employees be expected to use AI? How do we measure performance when two people doing the same job may be using technology very differently?

What happens when AI allows us to monitor productivity in ways we couldn't before? Just because we can measure something, does that mean we should?

And if AI ultimately means an organisation needs fewer people to perform a particular activity, when should employees become part of that conversation?

These aren't simply technology questions.

They're people questions.

That's why I think HR has to be in the room early, not brought in afterwards to communicate a decision that's already been made.

Organisations will inevitably establish policies around AI. They'll decide which tools employees can use, what data can be shared, and where approval is required.

But responsible AI won't be created by a policy document alone.

It will depend on culture. Whether people feel able to question an AI-generated answer. Whether leaders understand the limitations of the technology they're using. Whether employees feel safe admitting they've used AI. Whether somebody is prepared to say, "I know the system recommends this, but I don't think we should do it."

Creating that kind of environment is something HR understands very well.

Perhaps our role isn't to have all the answers.

But we should be helping organisations ask the right questions.

Would We Be Comfortable Telling the Employee?

Perhaps there's a relatively simple test we can apply to all of this.

Would we be comfortable telling the person affected exactly how AI was involved?

Imagine telling an unsuccessful candidate that AI contributed to the decision not to shortlist them. Or explaining to an employee that an algorithm helped identify them as having lower potential than their peers.

What about telling somebody that AI analysis played a part in selecting their role for redundancy?

Our immediate reaction might be that those conversations would be uncomfortable.

But perhaps that's precisely why we need to have them.

If we're comfortable using AI to influence a decision but uncomfortable being transparent about that with the person affected, we should ask ourselves why.

And providing an explanation isn't necessarily enough. Could we explain why we considered the recommendation appropriate? Could we defend the criteria and evidence used? And could we demonstrate that somebody had genuinely considered the consequences?

Transparency won't solve every problem. An employee knowing that AI was involved doesn't automatically make the decision fair, explainable, or correct.

But it does force us to confront something important.

Behind every data point is a person.

And while an organisation might experience an AI recommendation as an efficiency, an insight, or a percentage on a dashboard, the person on the receiving end may experience it as something that changes their career or livelihood.

That's why accountability matters.

Not because AI is inherently less trustworthy than humans, but because consequential decisions deserve to be understood, challenged, and owned, regardless of whether they were made by a person, a machine, or some combination of the two.

So, Where Do We Draw the Line?

I started this by saying I'm excited about AI—and I am.

I use it. I experiment with it. I see enormous potential in what it can do for HR, for organisations, and for individuals.

But perhaps that's exactly why these conversations matter now.

The debate around responsible AI can sometimes feel like a choice between embracing the technology and being afraid of it.

I don't think it is.

We can be optimistic about what AI makes possible while still asking difficult questions about where and how we use it.

We can automate without giving away accountability.

We can benefit from better predictions without assuming that accuracy is the only measure that matters.

And we can innovate while considering the people, resources, and consequences involved.

HR won't answer all these questions alone, nor should it. Technology, legal, risk, sustainability, and business leaders all have a part to play.

But when the questions involve people's careers, livelihoods, opportunities, privacy, and experience at work, HR has both a reason and a responsibility to be part of the conversation.

Because perhaps the hardest ethical question isn't what we do when AI gets it wrong.

It's what we do when it appears to get it right, but we haven't yet agreed where the boundaries lie.

And I'd rather we had that conversation before we need the answer.

About the author: Dan Straw has extensive experience helping organisations transform HR through technology, digital innovation, and AI. He writes about the intersection of people, technology, and business change, sharing practical insights that help organisations improve employee experience, simplify HR operations, and make more confident decisions.