When Algorithms Learn Our Limits: The Hidden Asymmetry of AI Pricing

As artificial intelligence reshapes commerce, a fundamental imbalance is emerging: businesses are learning precisely what individuals will pay and accept, while workers and consumers remain largely in the dark about the systems governing their transactions. This asymmetry, now under scrutiny by US regulators, raises profound questions about bargaining power, privacy, and the distribution of economic gains in the digital age.

What is personalised pricing and why does it matter?

The Federal Trade Commission is currently consulting on an enforcement policy for personalised pricing, a practice that uses personal data to tailor prices and discounts to individual customers. The concern is that increasingly sophisticated algorithms could allow businesses to identify each consumer's maximum willingness to pay with unprecedented accuracy.

In New Zealand, Consumer NZ has issued similar warnings about the vast amounts of data collected through supermarket loyalty programmes. While there is no evidence that New Zealand supermarkets are individually pricing products, the organisation argues that loyalty data could provide retailers with a detailed picture of shopping habits, including clues about how much individual customers are prepared to pay.

How does AI change the balance between firms, workers, and consumers?

At the University of Auckland Business School, we teach students how businesses create value, compete, and become more efficient. But consider the same person in two markets. As a worker, their employer benefits from knowing the lowest amount they will accept; as a customer, a seller benefits from knowing the highest amount they will pay.

Traditionally, neither side knows those numbers precisely. A worker might accept $24 but receive $30 because that is the going rate; a customer might pay $20 but buy for $14 because that is the advertised price. Algorithms are increasingly reducing that uncertainty, and they are doing so much faster for firms than for the workers and consumers they deal with.

Digital platforms can observe thousands of individual decisions. A ride-hailing platform can see which jobs a driver accepts, when they work, and which incentives bring them online. A retailer can see purchases, abandoned carts, and responses to discounts.

Is there evidence that companies already use this power?

There is no strong evidence that major companies already know everyone's precise financial breaking point. However, algorithmically mediated pay, personalised worker incentives, discounts, and consumer offers are already a reality.

Lyft has documented systems that determine which drivers receive incentives, with some earnings challenges explicitly personalised. Recent research on 1.5 million Uber trips in the UK found that dynamic pricing was associated with lower real hourly earnings and greater inequality, although this does not prove that Uber calculates the minimum each driver will accept.

A recent Federal Trade Commission investigation found that pricing intermediaries had access to information including location, demographics, browsing histories, shopping-cart activity, and even mouse movements in systems capable of influencing prices, discounts, and promotions.

Why is the information asymmetry a problem?

It should be noted that markets have never been perfectly transparent. Employers know more about wage structures than workers, and sellers know more about margins than buyers. Yet there has traditionally been uncertainty on both sides.

Algorithmic systems now risk reducing that uncertainty in only one direction: firms can increasingly learn an individual's limits, while their own remain hidden. A worker cannot easily know whether rejecting $24 would have produced $28. Nor can a customer know whether walking away from a purchase today would have triggered a discount tomorrow.

At its extreme, this risks becoming a kind of digital feudalism: platforms can increasingly see the people they deal with, while those people can barely see the systems governing the exchange.

Can AI personalisation also create genuine benefits?

There can, of course, also be genuine benefits to AI-driven personalisation. Targeted incentives can improve matching, personalised discounts can help price-sensitive customers, and better forecasting can reduce waste.

The issue, however, is not whether these systems can create efficiencies, but how the gains are distributed. They could translate into higher wages, lower prices, better products, greater investment, or higher profits. That depends partly on information. Personal data has economic value because it can help predict the terms people are willing to accept, making privacy a question of bargaining power too.

What should transparency and accountability look like?

Transparency is equally important. Workers and consumers are increasingly visible to businesses, while the systems making decisions about them remain largely opaque. They might reasonably expect to know when an offer has been personalised, what information influenced it, and whether others are receiving materially different treatment. That does not require companies to publish their algorithms, but visibility should not flow only one way.

Business schools also have a responsibility. Alongside teaching pricing strategy, segmentation, cost reduction, and AI-driven decision-making, students should be encouraged to ask: effective for whom?

What is the most troubling possible outcome?

There is a difference between using technology to create new value and becoming better at capturing value from the other side of a transaction. The most troubling outcome does not require malicious AI. Companies can rationally reduce costs and improve margins while becoming better at predicting what workers will accept and customers will pay.

The question cannot simply be whether something can be optimised. We should also ask who benefits, whether it is fair, and what happens if every business does the same thing.

Frequently asked questions

What is personalised pricing?

Personalised pricing is the practice of using consumer data and algorithms to set individual prices or discounts based on a person's predicted willingness to pay. The Federal Trade Commission is currently consulting on enforcement policy for this practice.

How does AI affect workers' wages?

AI systems can observe worker behaviour and personalise incentives, potentially allowing employers to identify the lowest wage a worker will accept. Research on ride-hailing platforms has found associations between dynamic pricing and lower real hourly earnings.

What can be done to address the information asymmetry?

Policymakers could require transparency when offers are personalised, and businesses should be encouraged to consider how the gains from AI efficiency are distributed. Privacy protections also matter because personal data is a form of bargaining power.