Hacker News .hnnew | past | comments | ask | show | jobs | submitlogin
'I was misidentified as shoplifter by facial recognition tech' (bbc.com)
144 points by gnabgib on May 26, 2024 | hide | past | favorite | 248 comments


Seems like an opportunity for the lawyers to me.

Being evicted from a store is necessarily a public act. Telling someone wrongly that they are a thief, in public, seems like libel. Since most people go to stores in their neighbourhood, this could easily happen in front of your friends, neighbours, parents of other kids at your school, fellow members of social clubs or worship clubs, etc.

If this happened to me, I'd be going and seeing if I could get any lawyers to take this on a no-win no fee basis.

It seems like businesses are increasingly failing to think through what their process will look like to real human beings.


When I worked on analysis software for police & intelligence agencies, it quickly became clear that no matter how much you present something with clear "this is a heuristic, it can give you leads", a scary number of people will take it as "this will tell you whodunit" and just assume it's infallible.

If people are going to treat these things like they are forensic evidence and not lead generation, then they shouldn't be legal. Some precedent that people need to check these things before acting on them would be very good.


From The flaws of policies requiring human oversight of government algorithms https://www.sciencedirect.com/science/article/pii/S026736492... :

    human oversight policies provide a false sense of security in
    adopting algorithms and enable vendors and agencies to shirk
    accountability for algorithmic harms.
And the better The Algorithm is, the less people check, and the more of its faults pass through.

Having worked on systems that required human oversight, I've seen first-hand, and analyzed data on, how people click the "LGTM" button without reflecting. We even noticed how when we improved our predictor with expert input, so that it performed better according to experts who worked with it, it would perform worse when we benchmarked on the previously "manually checked" data set. Analysis showed that the manual checks had let through so many of the algorithm's faults, that when the algorithm improved, it got a worse score.

See also https://predictive-optimization.cs.princeton.edu/ :

    We present seven flaws of predictive optimization. Our aim is to outline a set of
    objections inherent to predictive optimization that cannot be easily fixed using a
    design or engineering change. Taken together, these critical flaws undermine the
    legitimacy of applications of predictive optimization.

    1. Good predictions may not lead to good decisions
    2. It's hard to measure what we truly care about
    3. The training data rarely matches the deployment setting
    4. Social outcomes aren’t accurately predictable, with or without machine learning
    5. Disparate performance between groups can’t be fixed by algorithmic interventions
    6. Providing adequate contestability undercuts putative efficiency benefits
    7. Predictive optimization doesn't account for strategic behavior


True, "check" was poor phrasing, because these things don't produce evidence, they produce alerts to guide investigation.

It should be a given that these systems don't produce anything actionable, they are a guide for a user to look more closely, of course, that requires someone in the chain stops and says "this isn't usable evidence".


Very interesting - thanks for giving us the benefit of your experience.

In the case of store workers, it's not obvious what other checking they could do. The only reasonable use I can see is for them to keep an eye on the person that has been flagged. But I expect that the entire purpose of this tech is to reduce the number of store guards they need to pay, so they are likely not to have the manpower. If the policy is 'don't let them in the store', I don't see how a store worker can avoid making false accusations.


There's something fundamentally wrong with these probabilistic analysis tools being sold currently as "artificial intelligence". The nuance in "probabilistic" is quickly lost and the output is taken as deterministic.

Any piece of software that puts out data that is acted upon should have clear disclaimers attached. Calling it "intelligent" is irresponsible.


"Calling it "intelligent" is irresponsible." How so ?

If a human of animal behaving the same were to be describe as intelligent, than so are those systems.


How so? People take it at face value and harm other people based on its outputs, as per the article linked.

> If a human of animal behaving the same were to be describe as intelligent, than so are those systems.

I cannot parse this sentence. Behaving the same as what? What bearing do claims of human or animal intelligence have on me stating that generating liability by trusting probabilistic analysis, and actively harming people, by calling software "intelligent", is not responsible behavior?


The bearing is on the claim that calling software "intelligent" have anything to do with trusting or liability. Dumb or not, no one is supposed to blindly trust it.


we can call it frediness, but the problem isn't what it's called, the problem is that the profit motive only works when there's competition. if there were several shops nearby and one of them started rejecting 80% of customers based on frediness, they'd go out of business and the others who prefer having a human guard would prosper. if it's the only shop within a 10 mile radius, and it only flags out 1% of innocent people as shoplifters, then the profit motive doesn't provide the balancing force. that's what regulation is for.


I can think of one possible regulation that I think would produce the proper behavior incentives.

Require that the system intentionally produce false positives (selected at random) at approximately the same rate as the true positives. This means that the users of the system will be aware that any indication has only a 50% chance of being "real", and therefore can be used as an indicator to increase caution but NOT on it's own as a signal of guilt.


Why not just flip a coin then? It would be much cheaper?


I imagine the solution space is much bigger.

I think it could be an actually pretty good solution for this. A bit like Chaos Monkey but for human systems. Introducing error often enough that operators don't assume it never happens.


If the expected rate of known shoplifters is 1 in 10,000 then the human security team can keep a close eye on 10,000 people (one of whom is probably a shoplifter) or on just 2 people (one of whom is probably a shoplifter). The latter is much more efficient.

Without the intentional false positives they could simply throw out 1 individual with no need to keep an eye on anyone at all. That's enen MORE efficient, but it utterly ruins the lives of anyone falsely identified.


they'll just throw both out.


Similar to story points and task estimates suddenly becoming deadlines


>If people are going to treat these things like they are forensic evidence

Are most kinds of forensic evidence actually any better?


Well, plenty of bunk stuff (bite mark analysis, hair analysis, etc...) has been used in the past, but in theory anything used today should be backed up by significant evidence and rigorous methodology, and obviously it'll vary based on jurisdiction, but in theory courts should require that background.

Certainly, more transparency than "we bought some facial recognition software, no, you can't see the code, that's a trade secret".


> no matter how much you present something with clear "this is a heuristic, it can give you leads", a scary number of people will take it as "this will tell you whodunit" and just assume it's infallible.

That's precisely my anecdotal experience.

Subtlety and nuance are easily ignored where their absence appears to provide easy answers and the avoidance of responsibility for their outcomes.

Similarly to "It Is Difficult to Get a Man to Understand Something When His Salary Depends Upon His Not Understanding It".


I don't trust a sales person or marketer to not hype such a system as being flawless whodonit machine. I certainly don't trust that the people purchasing these machines won't communicate to their staff that it's a flawless whodonit machine.

That's because the step from "potential leads" to "flawless machine" is a relatively small leap easily hidden by throwing out bad stats and case studies about people caught red handed. Even if you say the machine is 80% accurate, a lot of people will happily accept harming the 20% on the chance they might be in the 80%. (Though, I'm willing to bet the false identification percentages are both not public and not kept up to date, if they are even known).


> Telling someone wrongly that they are a thief, in public, seems like libel

Slander.

Libel is published, Slander is spoken. If they put a picture of her on a sheet that said "shoplifter" and posted in on the wall (or the net), it'd be libel.

But, I don't think you'd win a case based on Slander either:

* you'd still need to show it was published to a third party (though as you say, maybe you had friends present) * They can show that they reasonably believed it to be true (the system told them so). Their wording then becomes incredibly important - did they say "you're a shoplifter" or "you've been identified by our shoplifter recognition system"

They might also try to argue that stopping shoplifting is in the public interest and that this was a statement of opinion related tot hat.

I agree though, if this happened to me, I'd definitely be sending a letter or two to dissuade them from making a similar mistake in future (I think I'd also be trying my luck with Facewatch given that their system disseminated that "information")


Yes, I'd forgotten that libel is published information.

Although, given that this is third party tech, maybe Facewatch counts as the publisher.


Wouldn't there be a better outcome for those misidentified and publicly slandered and inconvenienced if they sued for stress? Some unlucky sod could have a heart attack from this.


Libel and slander must have malicious intent.

I think the better legal tactic would be defamation as it doesn’t require intent.


It seems that the proponents of this technology believe the benefits far outweigh the harms. Why not have a standard generous payout to anyone misidentified by the system. The payout would be funded by all parties involved, store, police, tech company. If the benefits really do outweigh the harms and the compensation is enough, I think most people would begin welcoming the program.

If the system got gamed (people with a history of false positives seeking to be misidentified), fine, it would encourage the technology to improve and eliminate false positives.


The payout would have to be at least enough to both fund and cover the inconvenience of moving to a new neighborhood closer to stores you aren't yet banned from. Of course, that becomes impossible if the tech is widespread enough.


It doesn’t really work like that in the UK. I doubt you’d get anyone to take it on, let alone for feee. Proving the material impact would be hard. She’s not been named publicly, no lost earnings etc. I would like to think this sort of thing means we get better regulation, but who am I kidding.


The UK has been trying for decades to become the surveillance society, hasn't it?

I remember being taken aback by the amount of street CCTV cameras back in the early '00s. Coming from Canada for a visit that feeling of being observed nearly constantly was really uncomfortable and bizarre. For Brits it looked like a norm (or they didn't show it). Also the parallels with 1984 were uncanny, especially given that Orwell was English.


Don't forget the incredible posters that used to be up in London!

http://soerenkern.com/web/wp-content/uploads/2014/06/uk-met1...


That's gotta be satire right? Looks straight out of V for Vendetta.

I'm really glad the EU just completely banned predictive policing.


It's not satire! It really was an official poster campaign (some 20 years ago).


It was an official campaign but it’s worth noting that the satirical element of it is there, deliberately, because we are way, way funnier as a nation.

Almost nobody worries about these transport safety cameras because we don’t fear or tolerate overpolicing in the same way.

Of course there should be safety cameras on public transport!

And if a US tourist was on a night bus in London they too would be happy there were cameras. A nation of drivers has a very different calibration, I suspect.


I gotta say that I didn't feel very safe on night buses in London, despite the cameras. Besides Brussels, London is the only other city in which I've occasionally felt like I was in danger. It's still a fun city, to be clear, and it probably helps that it isn't a 24/7 city so that you don't end up in weird situations at night too much.

In the end the CCTV doesn't actually prevent anything from happening ...


> In the end the CCTV doesn't actually prevent anything from happening ...

The question of whether it prevents things from happening is a question of statistics.

I suspect the data doesn't support your claim, and that certain categories of crime have indeed been deterred. But neither of us can know without that statistical analysis.


It's got a red bus on it, must be one of Boris's...


This is a propaganda piece by the state-funded BBC. Notice that the focus of the article is whether the technology is accurate enough, not whether they should be doing it at all in the first place. It's an Overton nudge, a softball position that's easy to take-- because of course the tech will get better.


Spot on. It's amazing how much the BBC is riding on its reputation and how many people are ignorant or just turn a blind eye to the fact it's Tory crony central at the top, and has been for years


The better it gets, the worse the experience for the people who are mistakenly identified.


You do not understand the BBC or its level of independence or funding model.

Yes, the tory government have been trying to stack the governance of the BBC with their own people, but it doesn’t always go the way they expect.


John Varley in the science fiction novel The Golden Globe envisioned a future where there are two separate AIs: one that is solely designed to help people, that knows everything, and a separate AI designed for law enforcement that is constrained to a limited set of inputs. I don't know if that's practical, or palatable, but it was an interesting thought experiment, although he doesn't dig into it too deeply. https://en.wikipedia.org/wiki/The_Golden_Globe


>The UK has been trying for decades to become the surveillance society, hasn't it?

Australia wants that title!

Bunnings here also uses (used?) facial recognition technology.

Also concert venues and sporting events.

https://www.sbs.com.au/news/article/bunnings-denies-reintrod...

I'd be willing to bet that supermarkets are next, if they aren't already.


Orwell understood the authoritarian streak in the national character.


This idea that the UK is a surveillance society that is meaningfully different to other western societies is somewhat misplaced, based on motivated misreadings of a US article about surveillance in King’s Lynn in the 90s.

The vast, vast majority of CCTV systems in the UK, that are part of this “culture”, are just store and private CCTV systems that police can get access to with a warrant, by going door to door. The same as in almost all western countries.

https://en.wikipedia.org/wiki/Mass_surveillance_in_the_Unite...

There’s almost no police CCTV network, and there’s no reality to the fantasy element of police being able to see you wherever you go in the UK in real time (with the exception of a few crowded high crime areas where tourists get robbed, ironically, and a few high violence and terrorism risk areas, public transport facilities and live road traffic cameras).

There are some council-owned networks in high crime areas which are pretty contentious, and then there are these police facial recognition vans which are clearly offensive bullshit overreach.

In particular a lot of visitors misunderstand the prevalence of cameras pointed at our roads, which are not live feed cameras at all but speed-triggered safety cameras.

Basically it’s important to understand that the vast majority of cameras in the UK are people and businesses legally protecting their own shit, just like in the USA.

One key cultural difference is ANPR. In America the general perception is that privacy includes where you go in your car, because you treat it as a portable living room. There’s never been that presumption here. ANPR systems probably can track cars in metropolitan areas and on some motorways, but live traffic cameras are unusual in most of the country.

There is definitely a slightly less self-centred lean to the balance of individual rights compared to community safety compared to North America. But this is also part of why we don’t have to put up with anywhere near the same risk of violence or crime, even in areas that are significantly more densely populated.

We also have pretty good laws governing this stuff.

In short, Hot Fuzz ain’t real, and I think North Americans like to project this idea of a surveillance culture onto the UK without considering that they are likely under the exact same sorts of surveillance in the same sorts of places, without as many of the same legal protections.

(And with plainly discriminatory surveillance systems like Shotspotter to boot)

Do you think retailers in the USA and Canada are not using the systems described in the article? Because you would be wrong if you do.

https://www.dailymail.co.uk/sciencetech/article-12592563/wal...


I understand that my viewpoint differs from that of many here. I don’t see the problem with the live face detection. What I see as a problem is the process followed after the detection. Simply saying with any system like this there will be mistakes. The reaction should be designed such that we account for these mistakes. It sounds like the system as designed has at least three unhandled exceptions:

- your system thinks I am someone who stole something but I am not that someone. (Mis-identification)

- your system thinks I am someone who stole something, and I am that someone but I did not stole anything ever. (Mistake in the database)

- your system thinks I am some who stole something, and I did, but I have already been properly punished for it. (Rehabilitation)

If you just bar people from the shop automatically on a positive identification you are going to make the above mistakes on a massive scale. And because of the nature of these system if they made a mistake once with someone it is possible they will make the mistake again and again, so you better handle the exceptions in a gracefull and sensible maner. And shops will absolutely not do the right thing on their own. Because it is just so much cheaper to hassle the few who has fallen off from the happy path than build in the proper safeguards, so shops on their own will just decide to “eat” the losses caused by persecuting innocents. And that is the part which is not right. That is where laws should come in and mandate proper procedural safeguards.

What are the proper safeguards? I don’t know. I have ideas on how that could go, but this comment is already getting very long. What I am saying is that I think the correct solution is not “no cctv with facial detection allowed” as many here seems to propose, but it is also not letting shops persecute innocent victims of the above exceptions. There is a third way with regulated and safeguarded procedures where I think we should be looking.


I don't believe these systems are ready for mass, non-expert use. I spent 7 years as lead developer of one of the globally leading FR systems, and I've left the industry. It is fundamentally unsafe in the hands of ordinary people, ordinary people who are not scientifically and rationally discriminate in their day to day activities. Ordinary people treat FR like an AI Gawd, and the police are hopeless with it.

The straw that broke my camel's back was learning of the wide spread practice of police asking crime victims "what celebrity does the perpetrator look the most alike?" And then the police would put that celebrity's photo into their FR system and start harassing people in their city that look like the celebrity. So, mind numbingly stupid, I got angry at clients doing this, got barred from talking to clients, and soon there after quit the industry. The industry believes it is "not their problem".


Ain't no one gonna use the tech if they have to spend 30 minutes doing a background check after it flags someone.

I also guarantee this is not how sales and marketing are pitching it.


Strictly to your point about rehabilitation, that doesn't mean the store necessarily needs to let you shop there again. So unless there is a statutory requirement to the contrary stores would still get that information and presumably there would be a chain-specific SOP about whether or not that person was allowed on the property.


That is perhaps reasonable if it's one store, or even a chain, but the article spoke about this applying across multiple chains, whichever used this software. If we end up banning people for life from shopping in physical sto ed because they shoplifted once when they were young, we are creating a deeply disturbing society.


I think the danger in systems like this is in the article. They said the shopper was banned from all establishments running that system. So, even if the accused person is not accused of doing anything in that particular establishment, they are now capable of being globally banned. That should 100% be prohibited.


> unless there is a statutory requirement to the contrary

I’m aware of that. What I’m proposing is that we should make that a statutory requirement. If you are operating a camera system with automatic face detection it must be operated in a certain manner.


I'm guessing unless you go to the press these issues will never get resolved for you and you'll just be banned from random stores. So a 1% error rate will over time creep up to a 5% surprisingly quickly.

How many grocery stores will tell you what's security system they use when they already assume your a thief?


Maybe a solution is the cost of lawsuits from false positives vs the cost of theft from false negatives means shopkeepers or their insurance companies have to make a reasonable trade-off in how they use that information that keeps most people happy. The odd incorrectly identified person can still get cleared after a bit of inconvenient legal procedure and be paid for their trouble.


I’m not so sure about those lawsuits. With the current system (as the laws are today) there is very little standing against operators who wrongly ban you. You might get lucky and sue them for defamation if they do something outrageous like call you a thief in front of people. But they probably won’t. It is just so much easier to call you asside and trespass you from the store quietly without giving you a reason, or only giving a very general one. And then you don’t have any standing to sue them.

And if we are already proposing new laws to provide standing to sue why would we not also mandate some procedure so people can work out these edge cases among themselves?


Seems like there should be some sort of legal protection against being banned from nearly every shop. If it's just one shop owner who doesn't like you for whatever reason, you can go somewhere else, but if they're all sharing information and excluding someone, that's obviously bad. Perhaps this hasn't needed a law before because it was too hard for them all to coordinate on identifying everyone on their blacklist, but with facial recognition provided by a single vendor, it becomes a real risk.

This has been an issue with tenancy in my country. A tenant goes to the tenancy tribunal for any reason - even if it's the landlord's fault, and then all property managers, who collectively control most of the rental market, blacklist that person and suddenly they can't live anywhere.

With suing, the parties can already work it out among themselves if they both want to avoid going to court. I don't think it needs a mandated procedure. No-win-no-fee lawyers exist to serve poor people who have a clearly winnable case.


When I was 18 years old, a security guard kicked me and a friend out of a grocery store, much to our confusion.

After talking to them and requesting the store manager, it turns out they had a photo of a shoplifter that looked very similar to me. Eventually we were let back in the store and it was all OK.

So, it’s not an entirely new problem? Although with facial recognition I guess there is less recourse; a manager is not as likely to believe they got it wrong if their computer tells them otherwise.


There isn't much recourse anyway, you were entirely lucky that they were willing to be reasonable. At least if you live in the US, not sure what laws are like around this in Europe

I don't think this status quo should be empowered further by error prone tech


Even without the facial recognition system, security guys can ask customers out, since those are private property. And you are lucky that store manager helped you with the issue. Most cases I heard around me are that the security guy and the manager just gives no explanation.


I would like to put our officials under cameras 24/7 with the footage publicly available complete with the means to put them on trial and lock them up. When we've accomplished that im fine with monitoring everything always forever. But no sooner.


I would rather like to put people prone to stealing from all (tax fraudsters) under cameras 24/7 with the footage publicly available complete with the means to put them on trial and lock them up.

we can limit it to those where the amount of individual stealing is relevant. not the long tail but the fat creamers.


Why should public service be even more inhumane ?


When you talk about a post office clerk or military draftee, that's when you can use this public service argument.

He is talking about the ruling class with power over all of us, so they should have more oversight than any normal person.


We are making it impossible for anyone but sociopaths to run for office by believing that abusing them is the key to getting them to behave well. This attitude is as responsible for our state of affairs as everything else.


It is not abuse, it is protection. You can pressure, bribe or blackmail people and force them to do things for you. If it is not possible to do what is asked there is also no need to ask. There is no need to take the job if there is not enough room to enrich yourself.

Like there is no need to rob a store if the clerk cant open the vault.


What's good for the goose is good for the gander, no? Besides, stealing a random item from a super market shelf is much less damage than what a corrupt politician could do. Perhaps we should prioritize them instead..


People who want power are the ones who can be trusted the least to have it.


Maybe the people who run for office are so bad because only someone who can accept constant abuse from the public can stand it. Maybe this attitude is the problem.


No, the problem is that those who don't play ball with their political machine's interests are shut out from opportunities to advance. Tammany Hall is a famous historical example that goes back much farther than the public has had as strong a voice as it does today, and though the faces have changed this is still how things get done.

The game is rigged at the level of who gets to stand for election, particularly in safe-seat districts that get decided in the primary. Very few people are able to get elected without help from their local political machine getting out the vote - enough to provide plausible deniability, but not enough to create an effective opposition.

Remember the monopsony of Sheldon Silver? He used his position to control which politicians would receive contributions for their campaigns, creating a network of people dependent on him for political survival. It's that kind of power broking that's the problem. The clear and present danger of entrenched power is why politicians must not be allowed a shred of privacy.


Because they are being paid by public money.


Ahh so you believe in wage slavery.


When you can give half your waking life to a billion dollar corporation and not even have the privilege of being able to pay for a roof over your head; yes.


There seems to be a misunderstanding of the place of technology in the criminal justice system. There is nothing wrong with using technology to sift through a mass of data points to narrow down suspects or point the investigation in a particular direction. However, this does not obviate the need for investigation to actually verify that the initial identification is correct. There should never be a case where people are arrested or prosecuted solely on the word of a database query.


Agree. A better response to '80% chance that this is a known shoplifter' is for security to keep an eye on them. It should very quickly become apparent that it's a false positive (or not).


The problem is that this technology is not sold that way, it's sold as a way to detect shoplifters. It's also extremely important how the result is phrased and presented: even "80% chance that is is a known shoplifter" simply means "this is a known shoplifter" to a layperson. But even a 99.9% or 100% confidence might be wrong so this isn't even an 80% chance - at best it's 80% times the statistical likelihood of this not being a false positive at 100% confidence, and that can never be a 100% chance.

The (psychologically) correct way to think of this is as a colleague making a claim and telling you how certain they are. But this tech is not sold as a colleague, it's sold as a machine that is better than humans. It's not a perfect super cop but that's how it is marketed and why people buy it.


I am afraid the Appeal to Authority fallacy is one we as a society are about to be exposed to on a massive level and recognize as a side effect of integrating AI into our daily lives.


For me the solution is clear: require written authorization from everyone you want to capture with facial recognition software.

It prevents privacy issues and doesn't stop research.

Exceptions could be made in gov buildings perhaps due to national security but the idea of unlimited private companies taking my photo, or using video for gait analysis, facial analysis etc is abhorrent.

I'm not against tech and love what AI can do but private companies using my images for facial recognition is a step too far.


>It prevents privacy issues

Next step: every big company/store requires a written authorization to enter. Back to square one.


That was what I was going to say. All the stores need to do is warn potential customers that by entering the stores premises the potential customer agrees to x technology being used for the purpose of y. It's the potential customers choice then, leave or enter.


Is "We don't use automated identification technology" enough of a market advantage to win over customers?


Presumably only if the average consumer cares about this enough.


I could see it playing out like organic foods, honestly.


Even just that would be an improvement.


Not if you outlaw that practice.

It's easily countered by creating a law that prevents discriminating on the basis of opting into tracking, AI and whatnot.

Problem solved.


Maybe that would encourage people to stop shopping at big companies and stores and go back to supporting small businesses.

(I do think the idea isn't a good one and am not defending it.)


Personally I'd go the other way - you can do it, but you must make it public. It also has to be reciprocal. You can look up stuff on me, but your stuff should also be available for me to look up. E.g. tracking private jets/yachts and publicizing their cost to the environment should be legal.


Then what's the course of action when every store in the city requires your written authorization to shop from there?


As I mentioned above: outlaw the practice of preventing people from shopping if they haven't opted into your dystopian facial recognition software.

It will essentially kill it immediately but still allow research.


In my experience, when face recognition gives you a false positive, it's because there is a high likeness in the first place, and a human would likely have made the same mistake.

How different is it from a police officer stopping someone in the street based on some photo on his most wanted cheat sheet? Or a shopkeeper denying entry to a customer based on the likeness to a list of mugshots held under the counter. Those are faillible processes too.

Where face recognition is really useful is where it can match faces that would have otherwise fooled the human eye, just because it focuses on specific traits and ignores others like hairstyle.


The difference is scale. "You look a bit like a shoplifter from across the country" is not a productive use of a police officer's time, so will basically never happen.

Whereas the people in this article are going to get challenged, if not outright refused or arrested, in an increasing number of stores as this tech spreads.


> How different is it from a police officer stopping someone in the street based on some photo on his most wanted cheat sheet?

There's a big difference. The cop knows that the resemblance could be coincidental and will (usually) do further investigation before actually harming the suspect. If an automated system fingers someone, the assumption is that the system is correct and innocent people caught up will suffer harm.


The problem is the dumb ass users of FR think an FR Match has authority, AND the industry does not discourage that attitude, not at all.


And there's a lack of, for want of a better word, punishment for false positives for all of: the law enforcement officers, the software publishers, the policy makers.


How is this not slander? Falsely telling someone that you're a thief with real worldly consequences like getting kicked out of a store should have real legal consequences.


Slander requires a third person. The person from the store telling you you're a thief might be wrong, but it's not slander.

I'm not sure about the computer program though. It telling a third person (i.e. the person from the store) would be libel, but since it is the result of a calculation, I'm not sure if it's considered libel.


Facewatch is telling the store.

Technically, it’s probably saying “there’s an 80% chance this person matches a thief” and the store is interpreting that as “ban them.”

I don’t know UK slander laws, but in the US, it has to be incorrect to be slander. So this probably wouldn’t pass the test.


If its said in public the general public is the audience that hears the slander. If the store had even the slightest amount of brains they would have simply asked the person to leave without saying anything about the cause.


As long as the accusation was made in public with other customers around it can be slander.

In the UK it would be for the defendant (I.e. security guard/store) to prove that the claim was actually true. So in this case they would be stuffed.

The person should definitely contact a solicitor.


Only if you have the money to sue. Things being the way they are, rather than fixing the issue, they would more likely simply increase the confidence threshold based on whether you look wealthy enough to sue


This is the uk, you probably don’t have much rights to stop intrusive public private rights violations


Remember that doco on Netflix where black people were complaining and trying to solve for not being correctly detected by facial recognition tech? They had a genetic advantage against FRT.


mistaken arrests aren’t slander


In the UK, only police ('sworn constables') have legal protection when conducting an arrest. Ordinary people conducting a 'citizens arrest' are committing a crime if they make a mistake (false imprisonment). Store workers are not usually sworn constables.

In any case, the article is not about an arrest. Making someone leave is not an arrest, it's the opposite (an arrest is detaining someone).


From the story.

"Within less than a minute, I'm approached by a store worker who comes up to me and says, 'You're a thief, you need to leave the store'."

Knowing that one in 40 people so flagged aren't guilty of a crime means they are deliberately imposing upon those so flagged. The choice to use a faulty tech isn't itself a mistake its a choice. EG if you exposed a million people to this tech daily you would falsely arrest 11,000 people.


she should get a free candybar.


The jakes have recently started parking these facial recognition vans up at protests and introduced a brand new imaginary law making face coverings illegal if they so deem it. We really aren't ok over the here.


What about the Old Bill and the Rozzers then?

Same result https://www.youtube.com/watch?v=R5Cw043lPCE ?


With physical access to the camera, disabling them can be delightfully low tech.


N95 masks for everybody?


NC (USA) just made masks illegal in public.


It's easy enough to avoid going to North Carolina. Let's hope that madness will not spread much.


“it is also clear there are plenty among the public who are willing to put up with having their faces scanned - if it means safer streets.”

Unfortunately it doesn’t mean anything of the sort. This tech doesn’t prevent typical street crime like muggings or assault. And the chance of getting hurt or killed by a police officer whose dispatcher just identified you as a criminal is much higher than what the random folks who skipped bail that are in this system would do to you (ie nothing).

Then there’s the “low” false positive rates. They say only one of 33,000 people scanned is falsely identified. Okay, that’s pretty high. If they scan one percent of the UK population with these devices each day, that’d be 20 false positives a day, and 20 more potentially violent encounters with police a day. Compared to what’s likely a much smaller number of correct identifications of actual wanted individuals, most of whom could probably be found by much cheaper, less problematic methods.


Haven't UK learnt anything after unjustly accusing and imprisoning all those people that the Postmaster thoughts had stolen money when it turned out eventually to have been a software bug?


All the people who need to learn from that seem to be learning, is new and interesting ways to pass the buck.


I don't think it was a "bug", but it is probably described as such to minimise what happened. My understanding is that people with admin access were manipulating Postmaster transactions which resulted in creating artificial losses and Post Office was denying it was possible, despite having knowledge it was happening.


I’m worried that opponents of facial recognition are shooting themselves in the foot with the inaccuracy argument. Sure, maybe face rec ends up like Siri or self-driving cars. Even if it ends up like DallE or ChatGPT they probably still have a point. But I’m a little bit afraid that it might have an AlphaGo moment and just become extraordinarily superhuman at facial recognition, becoming much more dangerous while simultaneously dismantling the case they’d been building against it.


This is in UK. The "but what about privacy?" argument has been lost there a long time ago, so they make do with what they have.


The trespassing laws for commercial establishments are very problematic. I'm in favor of laws that prohibit refusing service when selling openly offered goods or publicly offered services, with clear exceptions such as aggressive behavior, outrageously bad hygiene, requirements for proper attire etc. Rude behavior, shoplifting and other annoyances fall under separate laws.

This should apply even to past criminals.

Such laws protect against racism and other forms of discrimination, and they help integrate convicts back into society, at the cost of "commercial freedom".

In jurisdictions with these laws (I know at least one), it's impossible to create, for example, bars only for men or only for women or refuse service to people we don't like (also, because they belong to some group, incl. morally reprehensible groups). There are some hacks around this and exceptions, but generally, it works and solves more problems, than it creates. Private gatherings or clubs are likely ok (nondiscrimination laws should still apply).

UK and US are outliers here with their "can refuse service to anyone" approach.


Can you name the “at least one” jurisdiction? I’m curious where there’s a law saying you can’t reject service to, say, Nazis.

I’ve only known examples where you’re free to reject anyone for any or no reason as long as it’s not because they’re one of a very small list of protected groups.


Poland is one:

138 misdemeanor law: Whoever, professionally engaged in the provision of services, demands and collects a payment higher than the applicable fee for the service, or intentionally, without a justified reason, refuses to provide the service to which he is obliged, shall be subject to a fine.

Some forms of propagation of fascism, communism and nazizm is prohibited in Poland, but I guess it probably would be illegal to refuse service to someone who otherwise declares support for some reprehensible ideas, and does that privately, so it doesn't fall under hate or other laws.


Thanks for the example and name, so I can look it up and read further.


From the same article, meanwhile, out on the street in public space:

    On a humid day in Bethnal Green, in east London, we joined the police as they positioned a modified white van on the high street.

    Cameras attached to its roof captured thousands of images of people's faces.

    If they matched people on a police watchlist, officers would speak to them and potentially arrest them.


That seems perfectly reasonable?


It places a burden on someone to somehow prove their innocence, if they can. What if a person refused to speak to the police?

Or what about this? A man covered his face when walking past one of these vans in a public street. He protested and was then fined £90 for swearing (swearing is not illegal but The Pubic Order Act is another law that is often abused by police but that's tangential)

https://www.independent.co.uk/news/uk/crime/facial-recogniti...


This is the same burden a person would be under if a police officer simply noticed someone on the street who looked like a wanted suspect (without computer assistance). I agree that the automated system makes this situation more likely to occur, but it has always been the case that you can potentially be arrested because you look a bit like someone else. And in that situation you would have to do your best to convince the police officer that you were in fact someone else.


That's generally why you're only required to provide ID to police in the US (at least in a car. Don't think you can force anyone off the street to show ID). The right to remain silent, or to deny entry into your home cannot put you on a Arrest list.

Of course, all this assume US police have consequences for breaking the rules. So this is unfortunately moot.


No difference at all from the US here. If a police officer in the US recognizes you as being someone for whom there is an outstanding arrest warrant then they can arrest you. And of course, they could be mistaken.


You act less arrest warrants these days stop the police from arresting someone. That's part of the problem.


It's perfectly reasonable to harvest thousands of photos of unknowing people and then accost them based on faulty software that can produce an erroneous match? How so?


Nothing in OPs post said anything about accosting people or using faulty software to drive that interaction.

Simply going out looking for people of interest in east London is perfectly fine, we call it "police work" and have done it for bloody ages.

Maybe instead we could issue officers with a deck of cards with 52 faces on them and they can use that as a comparison point? Seems pretty inefficient to me but im only a tax paying pleb what would i know.


> Seems pretty inefficient to me

Police surveillance should be inefficient. People do not exist to be arbitrarily inspected at scale; it should be an expensive, manual process and you should be forced to choose carefully whom you target, by necessity keeping that list small and limited to those for whom you can articulate cause.


I'm sorry but i strongly disagree.

If you're taking money from the public and spending it you should be legally bound to use it in the most efficient and effective way possible.


The most efficient way to survey is to not survey. Nothing more efficient than 0 runtime overhead.

Snark aside, surveys are quite literally the least efficient means of gathering a sample. Scanning random people on the street when a known criminal is likely to lay low and stay inside. I can't imagine expensive surveillance cameras is cheaper than paying an investigator to track down a suspect.


[flagged]


This one officer.


It sounds like they had weren't looking for any specific person (just anyone on "the watchlist") and they were simply mass processing faces to find potential matches.

Let's ignore the privacy implications as not all countries have an expectation of privacy outside windowless closed private spaces. How is this meaningfully different from a general stop and frisk policy? Sure, this adds a non-physical filter before the physical police interaction but it's still a widespread warrantless search. How does this work with the presumption of innocence?


The counterpoint would be to ask how this is meaningfully different from a group of police officers standing on the street comparing the faces they see to a set of photos of known suspects. The efficiency of the automated system obviously makes it feel very different. However, fundamentally, being arrested for looking like someone else is not a new phenomenon. I do not like what is going on here, but I’m not sure that your particular line of argument against it is very strong.


Same deal as LLMs. Police don't get to automatically input facial data into a database when on the streets.

If there was some guarantee all that data wasn't going into some government sanctioned back end, we can start to compare it to police out on the street.

>However, fundamentally, being arrested for looking like someone else is not a new phenomenon.

Was always wrong and still is wrong.

But if you want a modern difference: companies these days (even minimum wage) are really strict about arrests. Even a false arrest can cause complications in a background check, so there is more potential (albeit indirect) slanderous effects from mistaking people nowadays.


> Was always wrong and still is wrong.

I don’t understand this comment. The police sometimes have to arrest people based on descriptions and other fallible data points. It’s unfortunate when the wrong person is arrested (and false arrests should obviously be minimized), but it’s not reasonable to expect police to get this right absolutely 100% of the time. If someone is a fugitive then they are very unlikely to reveal any information that identifies them with certainty.

Also, police in the US use the same technology: https://www.nyc.gov/assets/nypd/downloads/pdf/public_informa...


It's not reasonable to be 100% correct, no. But given current police conduct in the US I trust them less than ever to try and resolve wrongdoings when it comes up. It's not even "oops" level anymore.

It's always been wrong and still is wrong, but the reasons changed over the years. Americans have traditionally been adverse to mass surveillance and I don't think that changes much today. The government just does it in sneaker ways nowadays.


Dragnet surveillance meets fishing operation. I do not find it so reasonable.


Sure, it may be reasonable, it's still worth pointing out that the article is not just about people entering a specific chain of shops, it's also mentioning police vans sitting in the middle of pedestrian concourses outside such shops doing the same face scanning.


Not this case specifically, but the harm done by misidentification by a computer is seemingly no different for the harm done by a human misidentification - if the doorman / security guard / croupier thinks you are a miscreant then the same harm applies

What might matter is the actions taken - but would they be any different if a human said “oh look that’s the well known shoplifter, keep her out”


Humans apply much more judgement to their own choices, and actions resultant from them, than instructions from the infallible machine.

Humans may also be disciplined for not attending to an instruction from the infallible machine, whereas there are no stats or electronic records of human judgement of "is that the lady who shoplifted the other day? Hmmm, maybe... Maybe's not really good enough for how little I'm paid".


The difference is if this shitty doorman worked at every shop.


A few questions here...

How did they know she shoplifted in the first place?

Does the tech also identify shop lifting, or do they just search a database of people convicted of shoplifting (in which case they could have just checked this person's ID and realized it was mistaken identity).

Is being caught shoplifting a credible reason to deny someone access to basic goods indefinitely? What if all supermarkets in a town use the same facial rec service?


It uses facial recognition ergo a computer flagged her face as similar to the face of someone known to be banned from the store. This tech is known not to be good enough for this use case. If you match a large population of users you are going to get an unacceptable level of false positives.

Even putting a human being in the loop is trouble prone because people aren't on average good enough at this.

It would be more useful to require a free membership to shop there and validate allowed users via bluetooth and ban users devices and phone numbers.


> How did they know she shoplifted in the first place?

She didn't.


UK does not have ID cards and you do not have to carry ID. Also they may not have ID for the shoplifters often they are just pictures from cctv.


Seems like they're trying to match against known people and it has false positives.


> they could have just checked this person's ID

If random store employees start walking up to people and asking them to show ID, is that an acceptable state for society?


It seems slightly more acceptable than kicking someone out of the store due to mistaken identity.


It is at Costco /s

But note that this is the UK. Different issues from the States (where this wouldn't ever pass. We have weak social security specifically becsuse citizens rejected a national ID).


There isn’t any national ID in the UK either.


We have many studies that show vulnerabilities of these image classification models to noise (adversarial or otherwise). The confidence levels are just not enough to fit into our "innocent until proven guilty (beyond reasonable doubt)" justice system. The danger is that big tech/AI can pump enough money into politics that justice system parameters will be rewritten.


Let's say this technology was basically infallible: only known shoplifters were flagged. What then?

Should shoplifting once mean that a person is banned from all stores that use this service for life?

Also: I assume that this service wouldn't just impact shoplifters, but anyone that a store saw fit to ban for any reason. What about people who experience mental health crises? I had a friend who had psychotic breaks while her issues were still undiagnosed and untreated

I know that this ultimately depends on the store's own policies, the facial recognition tech just provides information. but frankly I don't want to live in a world where making such a minor mistake in your youth could basically ruin your life


> Should shoplifting once mean that a person is banned from all stores that use this service for life?

I mean, it depends, right? Barred from buying any groceries for stealing a candy bar is obviously wrong… but if you walk out the door with a five-figure watch you didn’t pay for, I’m happy for you to be banned from Rolex shops for life, yeah.


If a decision needs to be made, I think "access to public businesses" should be something that is decided in a court, and not by shady AI surveillance corporations.


Unless you already went to jail for it? People do change over the timelines of an entire life.


The best silver-lining of COVID was that, for a while, we'd normalized covering our faces in the presence of CCTV.

Unfortunately it didn't stick. If you wear a mask these days, you only stand out more.


The Chinese solved this by adding more cameras. If you can lookup where the potential suspect came from and pull their full details from the fuzzy query false positives become unlikely. More data is the answer! lol


Not sure why this comment's gray; it's a good point (and a sobering one). In the very near feature, Western countries likely won't be relying on individual surveillance cameras; we'll have end-to-end chains of cameras that will confirm, with confidence, which house a person walked out of 2 hours ago, right down to their interior flat entrance inside an apartment block. (As the PRC is currently trialing). We'll have near-perfect panopticon identity systems, associating facial recognition with place of residence, and dozens of other factors.

It's very important to ponder what the PRC is pioneering today because we're about three years behind doing exactly the same things, and for the same reasons. It's only going to be a brief time before things we're horrified about, and vitriolically condemn totalitarian states for, are things we're going to fully normalize within our own culture.


This is why predator drones orbiting for days over protests is so scary, this is exactly the functionality their payload powers.

https://en.wikipedia.org/wiki/ARGUS-IS

https://yewtu.be/watch?v=CpLdL8ONEm4


Poe's law. Given a few comments here, some may parade the idea. Fully giving up privacy for security. I'd hope we give a better fight than that.


It's essentislly impossible to eliminate false positives. With billions of people on the planet, the odds of someone looking extremely similar is high. Plenty of actual twins around as well.


I'm sure there are other things that could be captured and analyzed aside from images. Gait, perhaps? It's similar to web tracking - each individual method might be very unreliable, but the intersection of them can be extremely specific.

Or just implant RFID into every citizen at birth. It's for your safety, after all - if you get lost or abducted, that's how they'll find you.


I think the technology is fine, but as per usual it's misuse and abuse of the tool that's the problem; why is their policy "throw the person out"? Surely a sane policy would be "if it alerts keep an eye on the individual in the store while still being respectful" ie acknowledging that the system does not have 100% accuracy.

And for anyone that complains about tech being used this way; you're too late, it's already happened, when you apply for a bank account, mortgage, any kind of insurance...the list goes on and they've been profiling people for decades and decades.

It is impossible to prevent facial recognition being used...so laws need to be updated to ensure that the public have fair rights around this tech. But since when did laws or public opinion/understanding ever keep pace with technology?


“ The Metropolitan Police say that around one in every 33,000 people who walk by its cameras is misidentified. “

Given the UK population of 67 million, that’s about 2000 false matches.


Per time they venture near one of these cameras…


It's an absolute bullshit statistic. There is no way they have established ground truth to evaluate accuracy at this scale.

$100 says they've taken the numerator of 'people we detained where it turned out we dun goofed' over the denominator of total people scanned.


I'm wondering if this woman is still barred from stores that use Facewatch. It would be a terrible inconvenience to only be allowed in a select few shops. Where is the line defining certain private businesses as essential resources for day to day life?

I think this sentence sums it up perfectly...

> "It felt intrusive… I was treated guilty until proven innocent," he says.

I don't want to live in this kind of society.


In EU there is a ban on this kind of tech (face/gait recognition based on cctv, by private companies, plus separate laws forbidding acquisition of personal data without consent)


I guess government can still use it though?


Is there a better way to avoid shoplifting or catch shoplifters using tech?


Private shops largely have a right to bar whoever they want from being a customer, except in protected categories ("shoplifter" isn't one of those). The problems here are (a) no one understands type I / type II errors, (b) everyone trusts computers must be right all the time, and (c) there's no easy way for this particular customer to prove that she's not the shoplifter. (c) could be fixed by a process. (a) and (b) are very hard nuts to tackle.


She can contact a solicitor and go after the store/security guard for slander if they made the claim in public.

Under UK slander law it would be for the store to prove that the claim was true, not for her to prove that it was not.


The damages for defamation would be severely limited. Considering that most people in the store would barely notice, and would probably not even remember the following day, you'd be entitled to zero damages, and probably not even costs, for defamation. You might be entitled to a bus fare (not a taxi) to a different supermarket, but that's about it.


You can't get legal aid for that, so the astronomical cost wouldn't be worth it.


There are steps. A good solicitor letter may prompt a financial settlement and would serve its purpose, not least as the story is making headlines...

In any case, contacting a solicitor does not hurt.


Humans watching CCTV cameras, maybe?


In the United States local drug store Rite Aid got in some serious trouble for using AI facial recognition. Looking at some of the Federal Trade Commission's complaints against them can give you some creepy insight into how the technology supposedly works and how few checks for accuracy or oversight there are in place.

https://www.ftc.gov/news-events/news/press-releases/2023/12/...


Just today when a Kroger self-checkout machine failed to detect an item on the scale, instead of just calling the attendant over, it implied that it had caught me shoplifting and replayed a surveillance video of me. The video only showed me standing there miffed by the machine’s nonsensical instruction, but even that felt humiliating.


I think another part to this is the police not attending shoplifting incidident (amongst many others) due to the reduction in the police numbers. It's definitely out of order, what the tracking company is doing, of course. Perhaps if we had a functioning police service and reduced the need for people to shoplift (yes, a lot of this is run by gangs now, but still some steal to feed).


You’ve listened to a lot of misinformation about this topic. There aren’t huge gangs of shoplifters, and police force numbers have been growing in recent years. If anything is to blame for whatever increase there has been in shoplifting, it’s the stores who have tried their best to eliminate every bit of human staff.


Show me the data to back that up as everything being reported is completely the opposite of what you suggest.

https://www.bbc.co.uk/news/uk-england-london-67742042


The only data point in your article is that London police don’t follow up on shoplifting calls very often. That doesn’t indicate anything other than how seriously the police take the crime.


So that's a no then.


It's better than being identified as somebody who has a high probability of becoming a thief in the future, I suppose.

Do we do that? Do we want to do that?


I don't think we have that tech yet (outside of good ol' racism, of course).

Given current controversies around Her, I'm sure some techie will see Minority Report and view it as a dream to pursue rather than a cautionary tale.


Tony Blair wanted to do exactly that with psychological profiling of kindergarten children.


> The BBC understands the mistake might have been due to a family resemblance. The Metropolitan Police declined to comment.

The BBC should understand that much facial recognition software struggles with non-white faces, like Mr. Thompson's. "Family resemblance" my ass.


The Hikvision facial recognition camera in my previous condo sometimes recognized me as an Indian man, and sometimes it scrolled through tons of faces and unlocked the door. I am from northern Europe btw. Sometimes it also had difficulty recognizing me when my hair grew longer.


Can happen without technology too, of course. I was nearly thrown out of a pub before I opened my mouth and the staff realised I wasn't the person who caused trouble there the week before.


Do they think it’d be better if the police did not use the facial identification tech?

If they’re standing there with a van and immediately stopping you the only thing you have to do is pull out your ID?


Easy answer: yes it'd be better.

TFA says that it has a failure rate of 1:33,000. That's a "do not ship" rating for almost anything else.

> immediately stopping you the only thing you have to do is pull out your ID?

Someone I know was detained on the side of the M25 for an hour sorting things out after being pulled over.

He presented the police with his ID and they decided it was a fake. His name was almost identical to someone who was wanted - his middle name and date-of-birth were different.

The Police said that was common on fake IDs - just change a few small bits of information - and that they'd have to take him to the station where he could sort it.

The only reason he didn't get taken in in the end is because the description of the wanted person noted that he had tattoos on his chest. On the side of the M25, at night, the only thing that stopped my mate being hauled into a London police station was taking his top off.

Anecdotes don't make data, but the idea that a copper will simply accept ID despite a system saying "this is your guy" is incredibly naive and suggests you've not had to interact with them much.


> suggests you've not had to interact with them much.

Or the opposite is true and this has been enough to solve all my police problems? I’ll admit it is not in the UK, but I doubt they have a significantly less professional police force.

> TFA says that it has a failure rate of 1:33,000. That's a "do not ship" rating for almost anything else.

For anything where failure means death, sure. For situations where failure means a minor inconvenience, maybe not so much.

To me, the only relevant point of comparison here is the rate of misidentification by officers while _not_ relying on the face id tech. Because that’s the alternative, not having no arrests or searches at all.


> I’ll admit it is not in the UK, but I doubt they have a significantly less professional police force.

Ah, that explains a point that I'd didn't bother to pull you up on.

Carrying ID is not a routine thing here in the way that it is in some other countries (there have previously been attempts to introduce a national ID but they were staunchly opposed). So, it's not a given that you'll have ID on you to show them.

Drivers probably have their driving license in their wallet, but even that's not guaranteed (because you don't have to have it on you when driving).

The "quality" of police varies by force (and, of course, by officer).

The Met, though, have had some pretty serious issues with misconduct (including sexual assault and murder) and are still working through the processes of identifying personnel who shouldn't be in uniform at all (the Met themselves found there were hundreds of officers who should have been sacked previously).

They're working to fix things (or claim to be), but you probably don't want a force that's been described as "institutionally racist, sexist and homophobic" to be entrusted with something like this.

> For anything where failure means death, sure. For situations where failure means a minor inconvenience, maybe not so much.

I would still say the failure rate is too high given that the outcome of interactions with the Police varies quite significantly (par.

> To me, the only relevant point of comparison here is the rate of misidentification by officers while _not_ relying on the face id tech.

I'd also be interested to know this. But, I don't think it'll go quite the way you expect.

I'd expect there'd be _fewer_ overall stops: coppers simply won't (mis)recognise as broad a range of people. If they're only stopping people they recognise (or based on stuff that's been radioed through), their success rate is probably better


> If they're only stopping people they recognise

> institutionally racist, sexist and homophobic

I’m not sure those things go together very well. If they’re bored they’ll just pull ‘random’ people off the street. They might not be at direct risk of arrest, but it’d still waste your time.

That’s half of the reason I need to show my ID all the time :/

Edit: I’m starting to think you might be right, and there’s no direct analogue because police don’t just sit in the middle of the street hoping they’ll see some suspect if they cannot instantly compare with 40k wanted faces. No officer would be able to remember them all.

Question becomes more if it’s worth to have some people misidentified to catch the bad guys.


> TFA says that it has a failure rate of 1:33,000. That's a "do not ship" rating for almost anything else.

I find statements like this interesting, from a risk analysis point of view. In the UK there are breast cancer screening programmes and the risk of a radiation-induced cancer for a woman attending full field digital mammographic screening is between one in 50,000 to 100,000. That puts the rate of inducing cancer vs finding cancer at 1 in 400 to 1 in 800 (because the majority of scan results find no cancer). Should we "not ship" breast cancer screening?


"The Metropolitan Police say that around one in every 33,000 people who walk by its cameras is misidentified.

But the error count is much higher once an someone is actually flagged. One in 40 alerts so far this year has been a false positive."

I think it's a good idea to make it clear what you mean by "failure rate".


One in forty false positives still isn't bad for a system to automatically sift out wanted criminals. That's going to be orders of magnitude better than police stopping people who 'fit a description'.


That claimed false positive rate sounds unbelievably low to me. Is there any proof publicly available and verified openly by a third party?

Why I find it hard to believe is that I am imagining having access to a face photo of everyone in the world, and trying to pick a specific close friend from those photos. Given infinite time to find the photo of that one person, I am convinced that I would find tons of false positives that might be that person, and that I would never even be sure I found the correct photo.

I am guessing that there is some serious sampling bias leading to such a low false positive rate (assuming the number is not complete fiction).

According to government (referring to "live facial recognition"): "All deployments are targeted, intelligence-led, time-bound, and geographically limited." https://homeofficemedia.blog.gov.uk/2023/10/29/police-use-of...

which makes that 1 in 40 number completely irrelevant when discussing false positives from mass deployment in grocery stores or other non-targeted spying.


There is zero reason for a police van in the middle of a city to sift through the whole internet. Everyone identified as living in that particular area is fine, and cuts down on a huge number of matches. The government also has a lovely national database of facial ID’s with names, so they don’t even have to rely on shitty internet photos.


> One in forty false positives still isn't bad for a system to automatically sift out wanted criminals.

It's an absolutely disastrous false positive rate when the consequences to those who are falsely identified are more than trivial.


That one in fourth will be banned from using nultiple stores and establishments without due process and without a clear way to clear themselves up.


OK, can you show the numbers you are using?


Yeah, like in the USSR when you wanted to enter the metro and a cop asked to see your ID. Very nice, nothing tyrannical about demanding that people submit to the authorities whenever the authorities want them to.


Eventually, criminals learn how to use makeup to circumvent this technology and it will be just the poor people who get wronged, just like ban on E2EE.


In ancient times, one was innocent until proven guilty.

But, hey: Progress.


Up until the last couple of centuries most countries didn’t have any sort of pro-active policing, beyond posting a bounty or a wanted notice. Unless you were caught red handed it was up to private citizens - thief takers, bounty hunters, or the aggrieved victims and family - to attempt to apprehend wanted criminals.


That was not long ago. In ancient times you wouldn't be on trial if you were innocent.


The Code of Hammurabi says otherwise. Quoting from http://www.general-intelligence.com/library/hr.pdf, linked-to from https://en.wikipedia.org/wiki/Code_of_Hammurabi:

1. If any one ensnare another, putting a ban upon him, but he can not prove it, then he that ensnared him shall be put to death.

2. If any one bring an accusation against a man, and the accused go to the river and leap into the river, if he sink in the river his accuser shall take possession of his house. But if the river prove that the accused is not guilty, and he escape unhurt, then he who had brought the accusation shall be put to death, while he who leaped into the river shall take possession of the house that had belonged to his accuser.

3. If any one bring an accusation of any crime before the elders, and does not prove what he has charged, he shall, if it be a capital offense charged, be put to death.


Salem might want a word...


That is the sort of thing Mx Throwaway was getting at: the fact that you were on trial was often seen as a sign of guilt, because why would an innocent person be on trial? Almost everybody else on trial was found guilty, so why should this trial be any different?


i think you might have an overly romanticized view of justice in ancient times


"In ancient times" was intended as sarcasm, but labeling this as "Progress" made that point too subtly, I realize.


> The Metropolitan Police say that around one in every 33,000 people who walk by its cameras is misidentified. But the error count is much higher once an someone is actually flagged. One in 40 alerts so far this year has been a false positive.

A 97.5% accuracy rate seems… really really high!

Obviously facial recognition shouldn’t be the only bit of evidence. But that’s remarkably accurate.


It should also get worse as more people are added: In a large enough population almost every person has some look alike who happens to be a thief.

Another way to look at it is for the worst case person what is their false positive rate? It's going to be 100%: Joe Blow, who did nothing wrong to no one, will go from place to place getting kicked out for the rest of his (/the system's) life. The failures aren't random (or at least not entirely)-- some people will get a heap of error in their lap.

So while getting punted 1:10000 times you go somewhere unfairly isn't too bad, getting always punted is awful. The unequal distribution of errors creates bad incentives: the system is almost risk-less for those of us who aren't the victims of false hits, so why should we demand improving or banning it?


It lacks context, it could misidentify half of the people scanned and still have the police report that number.

eg: 33,000 people walk past cameras but only two are flagged as matching a person of interest on the watch list.

One of those is a mistake, the other was correctly identified.

From the Police PoV that's one mistake in 33,000 .. but whose to say the remaining 32,998 people were correctly identified?

Perhaps they were all criminals that weren't identified, perhaps they were all not criminals but not correctly matched.

Perhaps 2,000 were criminals and on the watch list but not identified.


So high in fact that either the numbers are cooked or the threshold required for a positive match is set really really high, so they get a lot of false negatives, which opens the possibility that the threshold can be lowered in the future.


I don’t know. Whenever I see those amber alerts to be on the lookout for a grey Nissan with license plate number something, I think isn’t that a great opportunity to use at least OCR or something. Or be on the look out for some kidnap victim kid who looks like other kids, asking people to do what technology can already do much better sounds dumb.

Machines are going to come up with false positives, but so will human beings who are asked to be on the lookout for some kid of some redacted race in a hoody.


The amber alert stuff seems useless. Maybe I’m a bad person but I just have that disabled. The emergency weather warnings are also pretty high false positive with respect to things like flash flooding but at least worth being aware of.


Oh it is quite useless where I am. Before I disabled it, I would routinely get ear-shriekingly loud alerts at 4:30 in the morning for events happening hundreds of miles away. Worst waste of resources that are mandatory on phones, ever, except for the presidential alert.


The difference here is that people are asked to be on the lookout at some specific moments when a crime is known to be occurring, while cameras can - and are, per TFA - being used for straight up fishing expeditions.

That said, yes, it would be nice if there was a way for my car to automatically scan car plates around, linked to my phone so that when an amber alert comes in and I explicitly choose to enable the scanner at that point, it would report matches for that particular number automatically.

But we all know that's never going to happen. Well, the automatic plate scanners in every car will probably happen eventually - it's too easy technologically for someone to not come up with a bright idea to mandate them "for safety". But they will be enabled and effectively controlled by law enforcement 24/7.


Live CCTV offenders here on Airstrip One

John Lewis,

Co-op,

Tesco,

Sainsbury's,

Waitrose,

Next,

Marks & Spencer,

Boots

Primark


Probably easier to list who doesn't have CCTV


Live CCTV offenders here on Airstrip One

John Lewis,

Co-op,

Tesco,

Sainsbury's,

Waitrose,

Next,

Marks & Spencer,

Boots

Primark


Waitrose self-checkouts point a camera at your face when pay so that you can see yourself on screen.

I dislike shoplifting and petty crime but I also dislike this form of cheap, petty intimidation.

I don’t want to live in this dystopia.


Whoever designed the Waitrose self-checkout was a master in the art of annoying one's customers. Not only there's a camera pointed at you, the machine will urge you to bag an item "or press finish and pay" as soon as you take a few extra seconds e.g. to solve a Backpack Problem with a can of sardines and a packet of pasta etc. And they're supposed to be the up-market chain?


I was stopped several times to get my bag searched in the Dutch upmarket chain. I wouldn’t say it means anything.

(Though people told me that was exceptional and I was just unlucky)


Is that Albert Heijn? They have nice freshly squeezed orange juice and pindakaas. I remember automated checkouts when I was in Amsterdam but I also seem to remember they were out of order. Small blessings, I guess?


The large Asda near me does this, too. Not happy about it but not sure what can be done to stop it. Probably needs either Scottish Parliament or Westminster to enact a privacy law and I can’t see that happening.

Perhaps it’s already covered under the European convention on human rights, but I don’t know if that still applies.


That's the one the Tories tried to tear down as a "villain's charter" isn't it? And Labour are very keen to show they're "tough on crime". So I guess, not.


I think Asda’s is so bad and intrusive that I’ve long since stopped shopping there. Also the worst of supermarkets on the quality front. Even Lidl is better.


That's the second reason I avoid self-checkouts whenever possible, honestly. The odds of me being hassled because of some sort of surveillance messup or because I did something wrong at the checkout are substantially lower if an employee is the one ringing up my purchases.


Just make sure you pocket stuff before reaching the checkout then.


HN yesterday: EU is so silly trying to regulate AI usage, "The Banned and High-Risk lists read like a call for startups (outside the EU)." https://hackernews.hn/item?id=40473533

HN today...


Gov exempts themselves and the powerful from regulations.


As always, it's the details of the regulation that matter. Or perhaps more important, how it gets enforced.


I firmly believe that the harm done to misidentified people is not worth the capture of people who are sought after.

In the story I also noticed that people flagged are not treated as innocent until their identity has been truly established.

What kind of society do we want? Was the money spent on facial recognition systems better spent on “prevention”? Probably not as this money would not have gone to the police so we can’t have that! /s


I have deliberately not clicked on the link yet. My thought process just based on the title:

1. Okay well maybe the technology was wrong.

2. Or maybe the shoplifter is lying?

3. Surely "my word against yours" in the case of "human vs. computer" is solved by believing the human?

4. Isn't the whole point of this technology to catch humans who lie?

5. Then surely, if it's a shoplifter, there is video footage and that will clear up the issue.

6. But what if there isn't footage. What if it's just an AI that's detected someone as a shoplifter based on their face?

7. The technology for this is really mature, now I'm inclined to believe the computer.

8. But what about the guy? (Yay, gender assumption based on the word "shoplifter")

9. Well, in order to believe him I would want to understand why the AI was wrong.

10. Oh, the whole thing with AI in many contexts is that YOU CAN'T KNOW THIS.

11. This is why people care about understanding how AI makes decisions and have serious reactions to technology that just "magics" a decision out of thin air, even when it's the right one this time.

12. But what about the guy?

Well, I've just newly understood some things about the social impacts of pervasive AI. Time to read the article.


If China surveils people, it's because they are evil and authoritarian.

If we surveil people, it's because we're just keeping the public safe.


Nobody here is cheerleading this, I think the overwhelming sentiment is that it is invasive (and inaccurate but that’s beside the point)


> Nobody here is cheerleading this

By "here" do you mean in the UK?

Somehow the UK is consistently the first non-China nation where this authoritarian surveillance crap takes hold.

Other countries make pushes towards it, but UK appears to be consistently leading.


just this particular post on hn. I'm not from the UK so I can't fully comment, but based on the stories I read here, I'd agree with your general sentiment.


As the article points out, given the rampant levels of shoplifting in the UK (and the associated abuse of staff), all of the major retailers and a lot of their staff are going to be on board with this.

For the government and police, it's a cheap option to deal with the problem.


The staff don't really care about lifting: they really aren't paid enough for it to be their problem, or cared for enough by the employers such that they care if the employer looses a bit through shoplifting. The few that do care see the heavy end of the abuse you mention, and those that don't see abuse anyway, and if it isn't on CCTV the employers don't seem to care (it is far too much hassle to prosecute) despite the higher-ups pretending otherwise: that is why the staff support CCTV and related monitoring, not the lifting.

The abuse is a very real problem, I know a few who work in shops, all of whom have been verbally abused and physically threatened at various times. One of them was not long ago punched in the face in front of witnesses. Tesco, ever caring, had her working the next day, just a few hours later (incident happened on late shift, next shift was early morning, "are you sure you can't come in?, it'll be hard to find someone to cover") and seemed to put as more effort into investigating what she might have done wrong as they put into helping the police gather evidence to investigate the abusive individual, so she was glad of the CCTV. She got out off Tesco as quick as she could. She has had notification from the police that the individual involved is being prosecuted, but for a number of other offences also so it is unlikely she'll need to testify or otherwise be involved further.

The staff want the CCTV because they don't trust the managers and higher to look after their interests otherwise.


As the article points out, most people they've asked around those cameras were provisionally supportive of them (so long as they "help with crime").


The BBC is, for one.


The Chinese government is evil and authoritarian. As a result, they do like surveillance. That does not exclude you can use the same technology or same measures for other purposes.


This tech can really help San Francisco recover.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: