There's an underappreciated comment in the other thread about SynthID and OpenAI [0] that captures what (IMO) the hacker ethos on this should be. We care about privacy, we should not accept tools that barcode our every digital move. (note that the counter of "well, they don't do that yet" is not particularly convincing)
Building a tool that tries (and probably fails) to remove the watermark (due to the arms race that large corporate machines will win) is tacitly accepting the barcode. The hacker ethos should be, first and foremost, to run open source models locally without relying on a corporation.
>due to the arms race that large corporate machines will win
Much like how the entirety of Hollywood, book publishers, academic publishers, and game developers have won against piracy despite being some of the largest corps on earth and dedicating untold billions to the issue over the past 30 years?
Yes. Winning against piracy doesn't mean you completely eliminate piracy. It means you scare enough people into not doing it and make it a bit harder to do for others.
Losing to piracy would see companies like Netflix and Spotify not thriving.
> It means you scare enough people into not doing it and make it a bit harder to do for others.
By which definition they utterly failed.
> Losing to piracy would see companies like Netflix and Spotify not thriving.
Not at all. Netflix and Spotify do well because they are a good value proposition for the average customer. Piracy is free at point of "purchase" but is (and always has been) expensive in terms of various sorts of overhead.
As long as enough people keep the pirate bays open, it will be there as an alternative when the services start their inevitable enshittification.
I for one do not enjoy the “Which service has the classic film I wanted to watch this week?” Nor having to switch services every time I want to see a new TV series.
We need (and have!) similar “free” alternatives to the watermarked generative services. Just like I hate the yellow dots on my printed images, I am not happy to have my creative assets (I do nothing nefarious) stained with SynthID.
> Winning against piracy doesn't mean you completely eliminate piracy.
But this is moving the goalposts. You can win against piracy either by making piracy less attractive or by making the paid offering more attractive. The first has utterly failed, piracy remains easy as a rule, and to the extent that they've succeeded it's not only disproportionately by doing the second thing, the DRM itself is a net negative because it has such a small effect on the ease of piracy while making the paid offering worse.
And now that they're trying to push up the margins and the streaming ecosystem is fragmenting making everything into a series of bundles again, piracy is on the rise again.
They didn’t win because of DRM. They won because of the regulations that grant a monopoly for a specific term in the form of a copyright. Society has recognized that incentivizing creative acts requires a temporary grant of monopoly to ensure the necessary scarcity to make money and recover the costs of creation. The real problem is Disney keeps expanding that time period so things never enter the public domain
This is again conflating at least two things and this is so prevalent in this context. Let us not conflate how annoying DRM:s are to us users that buy the things, with pirates thinking they somehow have a right to use any software without paying fairly for it. I would even go as far as to say that you pirates are the reason I have to have a DRM in the shit I bought and paid for.
Piracy is as easy now as it was pre-DRM. DRM is the digital equivalent to security screws on electronics, in that they’re a mechanism for lawyers to argue their client made an attempt despite being easily bypassed with a trivial amount of effort.
> I would even go as far as to say that you pirates are the reason I have to have a DRM
I think this is largely an incorrect take. DRM is anti consumer, not anti piracy. In fact, it has done very little to deter actual piracy (and remember it only takes ONE person to break the DRM), while affecting some casual pirates and all legitimate users. In the process, they got rid of reselling stuff you own.
It's anticonsumer, not antipiracy, never forget that. It means something like this would have happened regardless of pirates.
They succesfully did away with 2nd hand markets and the concept of "owning" anything. So yes, I would imagine DRM would continue to exist without piracy.
I think so, because their main goal is to prevent unwanted use of the digital product -- to the detriment of end users -- in more ways than just piracy. In fact, they don't solve the piracy issue.
I am not sure how I am conflating two things, it would be helpful if you could expand or connect to my argument. Perhaps I am misunderstanding.
My argument is that the grant of monopoly is a regulatory decision and the real cause of "winning". No amount of DRM would confer the same benefit because the ability to bypass it through piracy would be totally legal with no economic or other consequences and so a robust cracking and distribution ecosystem would emerge. Thats a drastically different story than when napster gets shut down, and limewire gets shut down and pirate bay gets shut down every time it relaunches. Imagine a world where there is are 1000 pirate bays
By the time you're building (or buying) the necessary highly esoteric and expensive ultracentrifuge setup I think you would be well outside the realm of "hobbyist" unless someone insists on the most unreasonably pedantic definition for the term.
Unless we're only considering final assembly. Just gotta get that weapons grade fissile material supplier lined up. That might or might not qualify as rich hobbyist territory depending on how high a price tag is permissible.
In theory, there's also direct laser-based isotope separation. It's a technology that is being actively suppressed, and that's one case where I very much in favor of that.
This subthread starts off with the argument that the big corps will never beat the little determined hackers, one of the founding myths of the early internet. And then every now and then a strong little branch of the argument runs up against an example and it becomes well sure, the little hobbyist hackers don't have anything there but that is because the big corps/gov/billionaires/whatever put so much into beating them.
I mean reading it all certainly sounds like the people on the little guy's side are overestimating the value of pluck, an observation Hollywood generally makes just before the heroes with pluck win for ever!
> And then every now and then a strong little branch of the argument runs up against an example and it becomes well sure, the little hobbyist hackers don't have anything there but that is because the big corps/gov/billionaires/whatever put so much into beating them.
It's almost never about the level of resources the organization puts in. The usual reason is that there isn't enough incentive to do it. What is a hobbyist going to do with a nuclear weapon? Why spend your time creating one if you, like the overwhelming majority of people, have no desire to blow up a city?
Preventing something that hardly anybody would be trying to do even if it wasn't being suppressed is a lot more practical than preventing something millions of people would do given the chance.
They did win for a while because they stamped out 99% of piracy. In the early days of streaming it was legitimately difficult to argue for piracy. Streaming was just too convenient and too cheap.
But, they are greedy above all else. And so, we are once again seeing a resurgence of piracy. Large corporations seem to snatch defeat from the jaws of victory.
It doesn't make any sense at all. That's like saying browsing the internet with an ad blocker and other privacy tools is a tacit acceptance of tracking and ads, and that you should only visit websites that doesn't track or have ads.
It's already possible to lie with text. Pixels are pixels. If we can't blindly believe pixels to show the truth, we will be simply back to the pre-photography era which managed to have a concept of truth regardless.
You and I did whenever we assumed the pixels were an upload of a digitized photo taken with an actual camera of, say, a historical event, or something in nature (e.g. some rare bird), a birthday party, etc.
The photo could be staged (e.g. Cottingley Fairies), it could be altered physically (like cutting or painting over, e.g. Stalin), it could be cropped intentionally to tell a different story (plenty of examples), and more recently it could be photoshopped, etc. All of these were possible, though harder than it is now, but let's not pretend we didn't "trust pixels". We all did, just as we trusted newspaper photographs. Now that era must come to a firm end, and I believe it's a tragedy.
For the umpteenth time, scale and ease of access and propagation matters.
A knife and a handgun aren’t comparable to a machine gun and a bomb. When you have equal access to all of those, the damage you can enact is exponentiated.
You could lie with text before, but it took effort and time and skill to do it convincingly. You could also lie with images but they took even more time and effort and skill, greatly limiting the pool of people who could do it and the possible damage.
When anyone anywhere can convincingly lie and have it do two laps around the world in a matter of minutes, the whole game changes.
It’s becoming very hard to believe that people making arguments like yours are doing so in good faith. Maybe you’re not even a person but a shill bot. That’s a very real and trivial possibility today, which is the whole point and illustrates the problem.
Fair enough. While I would kind of wish AI could be reliably detected, deep down I know this is impossible and it would be pretty bad if we had, say, a prosecution that succeeded because "this 'provably-non-AI' photo places you at the scene of the crime" because only a few underground people know how to remove a watermark.
I also wonder if being able to prove that an image or video isn’t AI generated would lend credence to it, while in reality there are other methods to produce falsified video.
Well, you just have to convince the jury. The defense attorney will try to throw up all the reasons it could be falsified, but the prosecutor will say "All of that is unlikely - the defense attorney would like you to believe that farfetched story, but this is still a compelling piece of evidence." This is how it always is with any kind of testimony and evidence.
You raise an interesting point about the artificial generation of evidence used in court. In 1992, Michael Crichton wrote the book Rising Sun, which centers around the editing of security camera video footage to coverup a murder.
What stops someone from adding a watermark to an actually photographed (carefully framed?) picture to discredit it? There is no certainty either way, just suggestions from someone else about what the truth might be.
Stalin had all the resources imaginables at his disposal.
Now Nancy, a tech-phobic waitress who has a grudge against her coworker can make up an entire scenario with one prompt and her colleagues might blindly believe her.
Let's not pretend they're the same thing.
Gen AI is inevitable. Watermarking is likely futile. But in my opinion it is still very important to discuss how, as a society, we're going to live in a post-truth world now that anybody can, IN SECONDS, not only fabricate a story but also spread it to thousands of people through their social media.
When that idea was originated, the advice was more like:
"Don't trust what you see on the Internet. Trust instead what you read in a reputable daily newspaper, or Peter Jennings or Tom Brokaw on the nightly news, or BBC World News."
Today, the Internet, especially the part which is not trustable, has nearly finished killing most of the "trustworthy" news sources, by outcompeting them for ad dollars - by being way better at targeting ads (e.g. Meta) and by scientifically perfecting addiction (e.g. TikTok). What remains is mostly controlled by governments and has far from a perfect record of being fact-based and impartial.[1] There are a ton of independent people out there in good faith posting facts on the Internet, but we just agreed that we shouldn't trust what we see on the Internet.
So doesn't this become "Don't trust anything"? And doesn't that, in practice, get implemented as "Don't trust anything that challenges what you believe to be true"? This feels like a really, really bad change to our society - and I'd argue it's already completely happened.
This isn't just ads, trust in the mainstream media, itself, is very low [1], deservedly so in my opinion. The continuous lies by omission, the outright incorrect headlines/articles that they edit after a day, the lock-step messaging, alignment, and avoidance of topics, pushed by their respective political parties/billionaire owners (6 companies own 90% of media [2]), made me switch to more independent journalists.
I have no objection to this -- I follow a few that I would say meet that definition well and which I trust. But boy do I worry that for 90% of the population, this translates to picking a bunch of enthusiastic propagandists whose bias is far worse than MSNBC, Fox News, or CNN ever were. I assume our craven and corrupt political parties will increasingly focus on propping up "independent journalists" who repeat their talking points for them.
Adequately implementing solves one problem (the making up a story because of a grudge), but creates a whole new set of likely much worse problems: how does one maintain a democracy / civil society? It's not just the trust of "social media" that you've eroded, you've almost certainly killed trust in traditional news reporting as well, especially considering just how much of traditional media is discovered via social media.
Effective democracy requires an informed voter base. Society requires its constituents to be invested in its continuity. Neither of those is achievable when we completely discard trust.
Yes, it's happened. Except a lot of people do have an exception - they'll trust the slop that reinforces their existing biases, or even if they know in their hearts it's not true, viewing their side's lies regularly still has an effect on the way they think.
Good point. Sometimes I wonder if social media, just almost every aspect of it, is the real cancer. Allowing just about anyone (globally) to anonymously deploy information warfare via the social media vector just seems bound to have horrible outcomes. It's just as bad with text as with images or video. Because of social media, we've trained at least 3 separate generations to self-sort into camps with customized ideological info sources that have incredibly-low standards for fact-checking and every incentive to tell their audience (1) exactly what they want and (2) whatever will enrage them most.
AI kind of makes this worse, but also only barely. Because most people really ought to know by now that almost any content could be AI, a video of, say, Trump kicking a baby or violating a goat wouldn't convince anyone that those acts happened (unless they already believed they happened).
Thing is, we're so flooded in biased BS, and no one has any incentive to produce non-sensational, factual news. I absolutely see 'post-truth' as the inevitability. You can't "weed a garden" when it is 100% weeds. The term "news" will cease to mean facts, and just become a branch of entertainment. Kind of the way "Reality TV" went from being supposedly a documentary (e.g. COPS) to just being a flavor of entertainment, where nothing needs to be real.
Why are humans powerless to do anything about this? Aren't we making the technology? It's kind of a big problem for the future of the justice system and politics.
Stalin controlled the state. The state controls companies. Companies control watermarking.
This sort of solution to the fake image problem, makes it easier for stalin not harder. If everyone can make fake images that is one thing. If only the dictator can, well that is much worse.
The concept of truth? A bit overblown don't you think? Because some guy can make a realistic looking fake videos that destroys the "concept" of truth? How?
AI watermarks only give the illusion of maintaining the concept of truth. The government and corporations will still have access to un-watermarked models to destroy the truth with.
The watermarking should be on those things we want to verify as something that was not generated or manipulated. Something you'd add to, for instance, cameras. Putting them on the generated/manipulated is backwards as you can never get every model to watermark.
That model is equally bad though. Given that you're writing this in a discussion about gen AI watermarks, how in the world did you come up with the idea that Gen AI wouldn't be able to add a watermark?
Perhaps not watermarks, but cameras could sign their pictures and put it in the extra data, that's not something that would be easy to add to fake pictures, or at least not the correct signature.
Not that they "wouldn't be able".. that they wouldn't do it. For Gen AI watermarks to be useful all Gen AI systems need to add them and the incentives aren't there for that to happen. On the other hand the incentives are there for the non-generated sources to add it so they can differentiate themselves from the Gen AI media.
Maybe we do care about truth, freedom and privacy but the majority of rest of society will happily accept any T&Cs just to get access to whatever the next digital sliced pan is and as for truth and accountability, if they were two sides of the same coin on the ground people wouldn't bend down to pick it up as possesing it looks too much like responsibility and inconvenience.
I'm pretty sure watermarking is (or soon will be) a requirement for AI generated images in software used in the EU, as part of their regulations for AI transparency.
If i had a dollar for every time an American cried about literally any non-US jurisdiction having an iota of effect on them I could quit my job and leave this terrible website forever.
If I had a dollar for every regulation to come out of Brussels serving no purpose other than to extract money or exert control over American companies because they have no relevant competitors to worry about impacting in the EU I could do the same.
The human ethos should be to never be misleading about the origin and truth of any content you create, forward, or pass on. If we care about honesty we should jail anyone who does so.
Its what happens when people in power are paranoid dark-triad types and want to be able to catch anyone who threatens their power and stick it to them..
It already happened with Trump claiming any unfavorable content of his administration is "AI-generated" as a defense to dismiss real, unedited media. He literally said, “If something happens that’s really bad, maybe I’ll have to just blame AI.”
ie the video of garbage being thrown out the windows that his team already confirmed was real:
but I also live in a society that requires trust to function. making a tool the obliterates that trust(genAI imagery pipelines) then creating a tool that makes it trivial for normal people to remove any hint of controls over said trust eroding system is, toxic.
I get the argument about not putting in fingerprints that identify users, Good I agree. But this also removes the things that identify this as an AI image.
Now, what are the legitimate uses of that?
No really, why would I _need_ to remove a watermark for _legitimate_ purposes? Assuming that watermark is generic, rather than a fingerprint of a specific person
> No really, why would I _need_ to remove a watermark for _legitimate_ purposes?
When removing the watermark is easy, a very legitimate purpose of making the code to do it publicly available is to make a public demonstration that it's easy to do.
As for content use cases, suppose someone is using AI to modify their appearance because they're being unjustly targeted by an oppressive government. That government naturally bans doing that because they want to be able to identify and arrest their critics, so now if you make videos with your real face you get arrested but if you use a generated avatar then the watermark enables automated censorship because the government orders anything with the watermark to have its reach automatically restricted.
> suppose someone is using AI to modify their appearance because they're being unjustly targeted by an oppressive government
Then use a mask like everyone else. digital mask, one that obscures.
which is my main point, no, there isn't a legitimate need.
realtime avatars don't generally have invisible watermarks, also they are running from your machine, otherwise you've got a (normally credit card) trail to your front door. plus a video stream
also if you are generating stuff from a public provider, then tracing people isn't that hard to do.
As someone else pointed out: if watermarks are required, then everybody will assume an image without a watermark is the honest truth, which is obviously not true. Someone will end up in prison because of some image. This is bad.
but right now, we are eroding trust at an industrial scale.
There are no reliable tools for the end user, normal person, to work out if an image is AI or not. This erodes trust and lets bad actors get away with "oh thats AI generated" or use AI to defraud users.
I disagree. Its mainly about having technical control and freedom. Reverse engineering how things work feels like peak hacker ethos. You don't have control of something if you can't remove it.
I think ethical considerations were always a bit secondary to technical power when it came to so called "hacker ethos".
After all, instructions on how to remove watermarks definitely feels like the sort of thing that would have been in phrack back in the day.
> Hackers believe that essential lessons can be learned about the systems—about the world—from taking things apart, seeing how they work, and using this knowledge to create new and more interesting things.
> Access to computers—and anything which might teach you something about the way the world works—should be unlimited and total.
> Mistrust authority—promote decentralization
> All information should be free
I phrased it a bit differently, and perhaps a little less sympathetically, but i think i was more or less saying the same thing.
In any case a tool like the article that strips watermarks seems exactly the sort of thing that would fit into what i quoted above. Its mistrusting authority - there is nothing more central authority then having a literal central authority adding hard to remove digital signatures to images. It promotes freedom of information - it supports explaining how watermarks work and what they are. Its fundamentally taking apart a system, which teaches us how the system works.
Even if you remove a watermark, the companies still have a record of which images they have generated and for whom. Even if you remove the obvious watermarks, all major image generators are using steganography to embed hidden information that you can't be sure were removed. This is a type of one-sided arms race where one player gets to be invisible if they want to.
Yes usually, since an important aspect of steganography is error correction. For example we know that SynthID is robust enough to survive resizing and small blurs.
That's not what the previous comment is referring to. They're referring to false positives, i.e "Gemini did not generate this (or process it) yet it says SynthID confirmed"
So imagine the damage you could do to an artist by having Gemini flag it then saying “see here’s the proof they’re using AI art!” That person would get eaten alive.
You can count on them doing it in a way that's economical for them. It's how email spam filters and ad blockers work. Sure somebody will always find a way to bypass it, and that's the arms race. A filter with zero false positives that removes 80% of slop is pretty darn good though.
Are markers being removed here the same or similar to ones tools might add if you use an AI tool just to edit a photo? like a more complicated object removal in a photo editor?
I feel like it is even worse if we have a marker that is ALMOST always present on AI images. It would make us more likely to be fooled by an AI image that had them removed, because we would trust the marker.
This is a bit misleading as for Gemini it only properly removes the visible watermark. To remove SynthID it has to regenerate the image at low noise with SDXL, which will likely destroy a lot of small details, plus won't work for higher res properly (NB2 and GPT Image 2 support up to 4K image outputs)
Nano Banana 2 only supports 1K resolution (1024x1024) natively. Anything above that is upscaling. So this matches SDXL. GPT Image 2 does support 4k natively (but experimentally).
Where did you get that info from? According to Google's own docs as well as my own image generation tests via the API, it supports up to 4K natively for gemini-3.1-flash-image-preview (aka NB2).
It just defaults to 1K. But I didn't see anything in the docs stating that it's just a simple upscale for larger resolutions.
Yeah - if that's true then it's even worse because the output price says
$0.067 per 1K image*, $0.101 per 2K image*, and $0.151 per 4K image*.
But if all the "compute time" is spent on a 1K image and they're just passing it to a ESRGAN or other upscaling technique, then there’s literally zero reason to generate anything above 1K. Just save the money and upscale it yourself.
With the number of fine-tuned LoRAs and checkpoints - from a realism standpoint, yes SDXL is still very viable. From a prompt adherency perspective, absolutely not.
So I use a combination of Neo Forge and ComfyUI. Forge has an easier learning curve but ComfyUI gets all the new "hotness" almost immediately since there's so much custom nodes for it.
If you're on a Mac, I've heard that Draw Things is supposed to be pretty "batteries included" simple for image gen along the same lines as LM Studio.
Watermarking images generated from trained data on stolen copyrighted material, I get why so they can try to tell if something is real or not but something seems wrong
For better or worse, I think the future has to be showing warnings on images with no C2PA-like watermarking like we do for insecure web sites. Media can be signed by authors. If the Associated Press sign an image, I can feel more confidence in its authenticity than the next decade's InfoWars. Power users can still choose to ignore it, but standardizing UX around trust sources like we do TLS will save the less savvy among us.
I don't think we've even seen the full brutal force of disinformation that we are capable of yet.
> Use cases where the threat model fits: You are preserving art or historical record against false-positive "AI-generated" labels.
Sorry, how does using AI to generate images have anything to do with this? Image generators cannot insert watermarks into things they did not generate, and it seems highly unlikely that you will get a false-positive watermark on human-generated art, especially if, as the readme says, these watermarks have high enough fidelity to trace to a specific session id. Plus the modifications to the image needed to erase watermarks would necessarily change the thing being "preserved."
[edit]: the more I read the more I'm convinced, the claimed use cases in the README are bullshit and the real reason is to provide a tool that helps people bypass "AI-generated" labels on social media for AI slop.
I mostly agree about the justification in the repo being wrong, but wanted to engage about this point:
> Image generators cannot insert watermarks into things they did not generate
It's actually very easy to take a real image, ask Gemini/ChatGPT to modify some tiny part of it (could be something as silly as lighting/shadow/etc), and often the resulting image will be detected by their watermarking tools. This way you can easily present any real image as AI-generated.
Ignoring that a watermark removal tool does not help with this threat model, the claim is still true: the original image can not be changed, and instead a copy is created.
So what? I can also open an image in Photoshop and make sure it saves out some Photoshop specific EXIF data and try to claim the image was doctored. What I can't do is go and put my deceptive altered file up in place of the original in all the places on the Internet it exists.
I had to think about it, how about if the claim were:
If you take a photograph that is misidentified as AI generated, you can “preserve the historical record“ by using this tool before publishing the image.
(Anyone know the false positive rate with watermark IDs, would’ve hoped it’s like zero)
Trusting your adversary to reliably identify themselves is folly.
We need to get to a place where we're checking digital signatures to see which humans were involved in the creation of something, and ignoring everything else.
How you use the tool matters, yes. But the freedom to demonstrate and understand this weakness with watermarks is much more important to exercise than saying "shhhhh don't tell people about this".
The takeaway should be that watermarks aren't very reliable and you should keep your guard up anyway.
I think AI watermarks are kind of a lost cause anyway.
I'd be more interested in some kind of trusted Non-AI watermark.
This is something that could get integrated into cameras for example. However, considering how much AI-processing we already have in "normal" photos, it will be difficult to decide where to draw the line.
Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term.
How is a hidden watermark that only police and providers can read better than a hidden watermark that the people can also read if they go out of their way to setup some project like this? Also your argument makes zero sense because bad guys will do this anyway. At least there's also an open source project for normal people, and it's interesting from a stenography standpoint.
Watermarks exists to create a chain of custody or attribution that can be used to establish culpability. Just like the government requires printer companies to add a dot matrix watermark to printed pages to prevent currency forgery.
Making untraceable assets means parties can’t be held liable for harms.
Just because a bad actor might poison a toddler doesn’t mean you should sell them the arsenic.
Just because the effects of arsenic on the neurons of a brain are interesting means you should feed them to a toddler and watch the effects unfold.
I support the legal freedom to pursue any idea, but we should also mentor our colleagues about the consequences of our projects and avoid unnecessary harm.
Is an ikea knife infinitely replicable, distributable and capable of undermining the entire social fabric of a community? No. so these are not the same category of threat potential.
Further, an ikea knife used in a crime can be traced back to the criminal through other evidence like finger prints, surveillance cameras, purchase receipts etc.
Arguments that have a emotional component, especially related to our evolutionary instinct to take care of kids has a way of avoiding the rationalizations people adopt to justify otherwise indefensible behavior.
You also dont do yourself any favors in a bad argument that lumps me in with some unknown group of people rather tan responding to my argument.
Information is default low-trust unless you have reason to extend trust to the source and that's been the case for thousands of years, if not the entirety of human existence.
We now have the tools to increase trust in specific information, for example: by signing images that need high trust for things like news reporting using camera hardware root of trust with time and geo stamping. If signatures are removed, that's back to a default low-trust state.
It is best not to extend blanket trust to a specific source at all.
That is how we ended up with the situation where "reputed" media organizations peddle daily lies or selective truths that are useful to their benefactors.
IMHO you cannot base your trust on something that is so easy to work around.
Trust should be between people, not a person and some data.
So showing how easy to generate "false" data, this makes it more obvious for people focus on other people. Trusting people makes life much easier in my experience, while focusing on data, again in my experience, is a game of cat and mouse.
The Kentucky primary had $1.7 million spent on deepfake political ads which were seen 49 million times. Don't know how much it effected the result, but it's not a good sign of where things are headed.
> Trusting people makes life much easier in my experience
Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information?
AI watermarks are no panacea, but at least they are a clear signal of what not to trust.
AI watermarks empower elite / those with resources and disempowers the common person.
Only people with resources will be allowed to make content that is AI generated passed off as real.
Pandora's box is open. Instead of making a multi-tiered privileged society, we need to fundamentally restructure society to adapt.
Before that restructuring occurs it is critical to keep the playing field level. These are not tools that should be controlled by a minority authority, they are far too dangerous.
> Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information?
Isn't that the scenario the watermarks are useless against? Adversarial governments or anyone with enough money will be the ones who can generate images without watermarks even if you force them on the proles.
> AI watermarks are no panacea, but at least they are a clear signal of what not to trust.
Which seems like it only makes the actual problem worse? If most of them have watermarks, that only encourages people to put more trust in the ones that don't, even though those are the ones "powerful people" can still forge to manipulate everyone. What good is something that increases the credibility of adversarial government forgeries?
You can’t, it’s an inherent contradiction. Human social structures have sophisticated and robust evolved mechanisms for establishing and maintaining trust. These dynamics are not one option among many, they are the optimum. By their definition they don’t scale to strangers around the planet. This is an immutable factor in why we have spam, bank fraud, etc. We want the benefits of trust without the cost of local constraints but wishing doesn’t make it so.
The distortion is shockingly visible in images. Especially with any amount of generational iteration https://streamable.com/9x3s4r
That said, this tool is incredibly lossy, garbling text, completely changing shapes. It also fails to remove the new gemini spark / placement that moved since yesterday.
Would you prefer if someone did the same thing and kept it to themselves (or sold it to the highest bidder)? I think knowing it exists is better than not knowing it exists.
I'd like to frame this differently: watermarking is a (weaker) form of DRM, and DRM has never worked in favor of the users.
I know of at least one music GenAI service whose ToS forbid me from using their productions in ways that are incompatible with my local rights. If Google decides, as they did with YouTube, that they'll enforce this company's watermark even though I have a right to use their results then I've passively accepted that a (foreign) company decides which of my legal rights I can exercise.
For a more pessimistic outlook: for the GenAI companies that work for the military, are those pictures also watermarked? Because if watermarks only apply to one group then the "implicit trust" argument doesn't work.
I'm in favor of watermarking the same way I am in favor of paying artists for their work. But just as with youtube-dl and DVD decryption libraries, tools like these are necessary to level the playing field.
>Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term.
would you say the same thing about a ROT13 breaker, or would you recognize how laughably naive that sounds?
I have no idea what a Rot13 breaker is and dont care that you think advocating for an ethical posture in what the tech industry builds is naive. Its naive to think bullies wont be in a kids playground. Its also naive to think we just need to accept that rather than creating norms around the type of play that is acceptable so that the playground functions.
AI generated images have never existed before, they will break our ability to use the digital tools we have built society on if we let them. Ensuring they can be identified and have attribution tracing data embedded is a reasonable step to prevent abuse.
Didn't work at all for me. Hive Moderation still shows "gemini3: 99.9%". Tried their online version and it went from gemini3: 99.9% to midjourney: 64.7% + stablediffusionxl: 16.1%.
I find the casual malevolence of this kind of thing breathtaking. Every time someone raises potential issues with AI misuse, they're dismissed as fear-mongering while a host of people immediately rush to demonstrate that not only is the fear justified, it's gleefully anticipated under the vaguest colour of lofty ideals.
People want to dismiss the potential harms of deepfakes while also excitedly releasing the deepfake-hider-3000 and saying they just really, really care about privacy (for people who make deepfakes).
the genie is out of the bottle. the sooner you let go of the notion that it can be kvetched out of existence the less frustrated you'll feel.
for example, 35 years ago PGP was a "casually malevolent" thing, enabling terrorists and pedophiles to email each other with impunity. the effort to make math illegal had (very predictably) failed and now we have encryption everywhere. did the world end? how do you feel about Chat Control and numerous other initiatives to roll it all back?
That argument is a lot less convincing (and sincere) when we're talking about a specific tool with a very specific purpose, rather than maths itself. In much the same way, you can make pro-gun arguments based on sport shooting, but not when you're holding a landmine.
This is a very specific tool with the purpose of making it easier to lie, in an area where the kinds of lies people tell are directly injurious to society. "It's just math" doesn't fly here, and all the primary uses for such a tool are malevolent.
AI itself has non-malevolent uses, but this doesn't. It's designed for negative uses.
ROT13 decryption does not have the same issues, and is not a good parallel here. Again, we're talking about a specific tool with specific use cases; a general algorithm being able to be used for negative ends is not the same as a tool being designed to be used for negative ends.
I also haven't mentioned banning anything; that's all on you. It really feels like you've jumped in with pre-prepared talking points rather than engaging with my comments.
To restate: "AI will benefit humanity" is a tough sell when so many people who are in to AI deliberately make tools that support the negative uses. This is an example of a project that superficially presents as high-minded, but is designed for exploitation.
what I've been trying to convey to you here with ROT13 analogy is that we're better off knowing that something is broken. ROT13 is a piece of shit, and so is SynthID. whether the author's intentions were noble or malicious is utterly irrelevant.
> we're better off knowing that something is broken
This is not a new insight, and it doesn't add anything to the discussion. It's effectively a platitude.
This project is absolutely not motivated by a wish to inform, and it's disingenuous to prevaricate around that; there are countless ways to inform that don't endorse.
there is no discussion. your opinion on the matter is irrelevant, same as mine. genai will be misused in every way imaginable, and there's nothing anyone can do about this. no regulations, no laws, nothing will stop it now.
for fuck's sake, Photoshop existed for how many years now? you would be a fool to trust any random photo to be genuine at any point in time. yes, yes, yes, back then only the people who had spent a few days learning it could do it, and now everyone can. that makes no difference whatsoever.
None of that is relevant to the conversation; it is genuinely like you are pasting from a random discussion a decade ago.
We're not talking about Photoshop, because it's not a parallel here. We're not talking about trust in authenticity. These are points you have plucked from the ether. Please reread my earlier comments.
it is very much relevant to the conversation because your assertion about this tool being harmful and unethical is rooted in your misguided belief that generative fakes that can fool a grandma on facebook are somehow worse than man-made fakes that can fool anyone.
Regardless of one's opinion about this particular project, it seems obvious to me that the path forward is proving authenticity of non-AI resources rather than attempting to watermark all the AI-generated ones.
Pretty hard problem to tackle when you can point an "authenticated" camera at a really nice screen and snap a 'definitely real' photo of anything a screen can display :(
There's probably a technical solution, such as the camera manufacturer cryptographically signing a GPS location and timestamp together with the pixels. Like all DRM it will probably be broken though, and more importantly, would anyone (even e.g. a newspaper editor) care enough to verify the signature?
GPS? No problem, they can take their hi-DPI monitor and their secure camera to the sidewalk in front of the White House lawn and play the AI video they made of soldiers shooting protestors.
Isn’t the goal only to prove that a photograph was taken with a particular camera? I don’t think you could ever prove that the subject was legitimate, as there are countless ways to misrepresent things. But in a world of AI slop, knowing a photo was taken on a real camera and wasn’t synthesized artificially is still a useful data point in determining trust.
watermarking only really works when the scheme is secret.
putting cyphertext in high frequency noise is old news. in generative land would be far more interesting to use the generative flexibility to encode in macrostructure.
I just saw the announcement about OpenAI or so going to use SynthID and all I thought was; what can d be read(located) can be removed. Seems the tool already exists, proving my point.
You're assigning emotions to people based on what you'd like them to feel, not on reality. For example, most americans probably don't feel shame about being american. But it's still a good decision not to go around showing off a bunch of american flags abroad, unless you want people to look at you in a certain way.
Sometimes it is not a "good thing" because it will cause people to react negatively. Whether the public's dislike of AI valid or not depends entirely on your perspective and the situation at hand.
Should people be scared of their food containing Red Dye 3? Yes. Aspartame? Maybe. MSG? Probably not. Dihydrogen Monoxide? Nope.
Should you feel "shame" for using AI to generate fake news? Yes. Art? Not really. See the numerous examples of people disliking famous human-made art just because it was presented as AI.
I don’t follow because all I see in other AI threads is how this is the future and people best adjust to the AI reality.
The hyper macho’ism on show by a large majority of pro-AI posters on other threads is frankly embarrassing and if I was trying to promote a positive spin on AI I’d be telling the bro’s to STFU.
No, there's no law requiring disclosure of AI use yet, as far as I know.
> pretending you're not American when asked
Many people do so for various reasons, for example saying you're Canadian when on vacation. Do you count that as evidence that they're "ashamed" of being american? Or that being american is automatically a bad thing, because sometimes people hide it?
There's an underappreciated comment in the other thread about SynthID and OpenAI [0] that captures what (IMO) the hacker ethos on this should be. We care about privacy, we should not accept tools that barcode our every digital move. (note that the counter of "well, they don't do that yet" is not particularly convincing)
[0]: https://news.ycombinator.com/item?id=48200060
Building a tool that tries (and probably fails) to remove the watermark (due to the arms race that large corporate machines will win) is tacitly accepting the barcode. The hacker ethos should be, first and foremost, to run open source models locally without relying on a corporation.
> [fighting against the system] is tacitly accepting the barcode.
I don't really see it. I think it's important to win on both fronts.
Especially as the open weight models are really generated by corporates, and they could stop releasing them at any time.
But we'd still have them. It's not like we're gaining much with new training anymore anyway
I appreciate my coding agent being increasingly aware of the walrus operator :)
They also have built in dystopian government authority enforcement in them unless you go to pains to sever those neurons.
Fighting within the system is accepting the system.
>due to the arms race that large corporate machines will win
Much like how the entirety of Hollywood, book publishers, academic publishers, and game developers have won against piracy despite being some of the largest corps on earth and dedicating untold billions to the issue over the past 30 years?
Yes. Winning against piracy doesn't mean you completely eliminate piracy. It means you scare enough people into not doing it and make it a bit harder to do for others.
Losing to piracy would see companies like Netflix and Spotify not thriving.
> It means you scare enough people into not doing it and make it a bit harder to do for others.
By which definition they utterly failed.
> Losing to piracy would see companies like Netflix and Spotify not thriving.
Not at all. Netflix and Spotify do well because they are a good value proposition for the average customer. Piracy is free at point of "purchase" but is (and always has been) expensive in terms of various sorts of overhead.
As long as enough people keep the pirate bays open, it will be there as an alternative when the services start their inevitable enshittification.
I for one do not enjoy the “Which service has the classic film I wanted to watch this week?” Nor having to switch services every time I want to see a new TV series.
We need (and have!) similar “free” alternatives to the watermarked generative services. Just like I hate the yellow dots on my printed images, I am not happy to have my creative assets (I do nothing nefarious) stained with SynthID.
> Winning against piracy doesn't mean you completely eliminate piracy.
But this is moving the goalposts. You can win against piracy either by making piracy less attractive or by making the paid offering more attractive. The first has utterly failed, piracy remains easy as a rule, and to the extent that they've succeeded it's not only disproportionately by doing the second thing, the DRM itself is a net negative because it has such a small effect on the ease of piracy while making the paid offering worse.
They won the long game. Everything is rented and DRM now. Very little of what most people buy digitally is truly owned.
they didn't win by attacking piracy head-on though, they made capitulation easy & nice enough for us to happily go along.
all's fair in love and war
They did a bit of everything: attacking head-on, lobbying, providing alternatives. And eventually, it worked.
Think people are getting annoyed again at having to pay even more than we did for cable to get all the streaming packages.
And now that they're trying to push up the margins and the streaming ecosystem is fragmenting making everything into a series of bundles again, piracy is on the rise again.
They didn’t win because of DRM. They won because of the regulations that grant a monopoly for a specific term in the form of a copyright. Society has recognized that incentivizing creative acts requires a temporary grant of monopoly to ensure the necessary scarcity to make money and recover the costs of creation. The real problem is Disney keeps expanding that time period so things never enter the public domain
This is again conflating at least two things and this is so prevalent in this context. Let us not conflate how annoying DRM:s are to us users that buy the things, with pirates thinking they somehow have a right to use any software without paying fairly for it. I would even go as far as to say that you pirates are the reason I have to have a DRM in the shit I bought and paid for.
Piracy is as easy now as it was pre-DRM. DRM is the digital equivalent to security screws on electronics, in that they’re a mechanism for lawyers to argue their client made an attempt despite being easily bypassed with a trivial amount of effort.
> I would even go as far as to say that you pirates are the reason I have to have a DRM
I think this is largely an incorrect take. DRM is anti consumer, not anti piracy. In fact, it has done very little to deter actual piracy (and remember it only takes ONE person to break the DRM), while affecting some casual pirates and all legitimate users. In the process, they got rid of reselling stuff you own.
It's anticonsumer, not antipiracy, never forget that. It means something like this would have happened regardless of pirates.
Would the DRM exist without piracy?
They succesfully did away with 2nd hand markets and the concept of "owning" anything. So yes, I would imagine DRM would continue to exist without piracy.
> Would the DRM exist without piracy?
I think so, because their main goal is to prevent unwanted use of the digital product -- to the detriment of end users -- in more ways than just piracy. In fact, they don't solve the piracy issue.
Almost all Pirates do no encounter DRM in any way.
I am not sure how I am conflating two things, it would be helpful if you could expand or connect to my argument. Perhaps I am misunderstanding.
My argument is that the grant of monopoly is a regulatory decision and the real cause of "winning". No amount of DRM would confer the same benefit because the ability to bypass it through piracy would be totally legal with no economic or other consequences and so a robust cracking and distribution ecosystem would emerge. Thats a drastically different story than when napster gets shut down, and limewire gets shut down and pirate bay gets shut down every time it relaunches. Imagine a world where there is are 1000 pirate bays
did they? Is piracy now impossible?
Not according to /r/CrackWatch
They've got some sort of hypervisor bypass for basically all Denuvo games.
Except the only way to watch some shows is now piracy, they've been erased from streaming and never available on disc
No? DRM gets cracked and the pirate sites still have loads of the latest shows, games, etc.
What? Some nerds on private trackers and kids on 123movies or whatever is not piracy winning by any material stretch.
They have a finite # of employees, a finite budget, and a finite amount of time.
Hobbyists do not. ROI is not a factor.
As yes, the hobbyist built nuclear weapons program.....
Legalize recreational plutonium.
To be fair the state works pretty hard to crush "hobbyist" nuclear weapons programs so you don't really know how far it could get.
By the time you're building (or buying) the necessary highly esoteric and expensive ultracentrifuge setup I think you would be well outside the realm of "hobbyist" unless someone insists on the most unreasonably pedantic definition for the term.
Unless we're only considering final assembly. Just gotta get that weapons grade fissile material supplier lined up. That might or might not qualify as rich hobbyist territory depending on how high a price tag is permissible.
You don't have to use the ultracentrifuge, though I don't suppose the power plant you would need for a diffusion plant would be much more attainable.
In theory, there's also direct laser-based isotope separation. It's a technology that is being actively suppressed, and that's one case where I very much in favor of that.
so which one is it here?
This subthread starts off with the argument that the big corps will never beat the little determined hackers, one of the founding myths of the early internet. And then every now and then a strong little branch of the argument runs up against an example and it becomes well sure, the little hobbyist hackers don't have anything there but that is because the big corps/gov/billionaires/whatever put so much into beating them.
I mean reading it all certainly sounds like the people on the little guy's side are overestimating the value of pluck, an observation Hollywood generally makes just before the heroes with pluck win for ever!
> And then every now and then a strong little branch of the argument runs up against an example and it becomes well sure, the little hobbyist hackers don't have anything there but that is because the big corps/gov/billionaires/whatever put so much into beating them.
It's almost never about the level of resources the organization puts in. The usual reason is that there isn't enough incentive to do it. What is a hobbyist going to do with a nuclear weapon? Why spend your time creating one if you, like the overwhelming majority of people, have no desire to blow up a city?
Preventing something that hardly anybody would be trying to do even if it wasn't being suppressed is a lot more practical than preventing something millions of people would do given the chance.
You don’t happen to know a certain Doc Brown?
They did win for a while because they stamped out 99% of piracy. In the early days of streaming it was legitimately difficult to argue for piracy. Streaming was just too convenient and too cheap.
But, they are greedy above all else. And so, we are once again seeing a resurgence of piracy. Large corporations seem to snatch defeat from the jaws of victory.
> No use messing with Google's watermark, fellas. Go do something else that's 100x harder instead.
> works for Google
Gee, I wonder why...
This is the “instead of using seatbelts, we should invest in trains” argument.
It is the "instead of arguing about seatbelts, we should stop driving cars" argument.
It doesn't make any sense at all. That's like saying browsing the internet with an ad blocker and other privacy tools is a tacit acceptance of tracking and ads, and that you should only visit websites that doesn't track or have ads.
Chrome is a great demonstration of my point here.
Or you can just take a picture of your monitor with your phone.
multiple things can be in line with the hacker ethos
Accepting blindly destroying the concept of thruth should not be the hacker ethos either.
Nobody said that?
Saying that watermarking fake things is bad kinda strongly implies it
It's already possible to lie with text. Pixels are pixels. If we can't blindly believe pixels to show the truth, we will be simply back to the pre-photography era which managed to have a concept of truth regardless.
When could you ever trust pixels?
You and I did whenever we assumed the pixels were an upload of a digitized photo taken with an actual camera of, say, a historical event, or something in nature (e.g. some rare bird), a birthday party, etc.
The photo could be staged (e.g. Cottingley Fairies), it could be altered physically (like cutting or painting over, e.g. Stalin), it could be cropped intentionally to tell a different story (plenty of examples), and more recently it could be photoshopped, etc. All of these were possible, though harder than it is now, but let's not pretend we didn't "trust pixels". We all did, just as we trusted newspaper photographs. Now that era must come to a firm end, and I believe it's a tragedy.
Photoshop is older then the web...
Where did I mention the web?
For the umpteenth time, scale and ease of access and propagation matters.
A knife and a handgun aren’t comparable to a machine gun and a bomb. When you have equal access to all of those, the damage you can enact is exponentiated.
You could lie with text before, but it took effort and time and skill to do it convincingly. You could also lie with images but they took even more time and effort and skill, greatly limiting the pool of people who could do it and the possible damage.
When anyone anywhere can convincingly lie and have it do two laps around the world in a matter of minutes, the whole game changes.
It’s becoming very hard to believe that people making arguments like yours are doing so in good faith. Maybe you’re not even a person but a shill bot. That’s a very real and trivial possibility today, which is the whole point and illustrates the problem.
It either works reliably or it doesn't; if it doesn't, it's better that everybody be clear about that.
Fair enough. While I would kind of wish AI could be reliably detected, deep down I know this is impossible and it would be pretty bad if we had, say, a prosecution that succeeded because "this 'provably-non-AI' photo places you at the scene of the crime" because only a few underground people know how to remove a watermark.
I also wonder if being able to prove that an image or video isn’t AI generated would lend credence to it, while in reality there are other methods to produce falsified video.
Well, you just have to convince the jury. The defense attorney will try to throw up all the reasons it could be falsified, but the prosecutor will say "All of that is unlikely - the defense attorney would like you to believe that farfetched story, but this is still a compelling piece of evidence." This is how it always is with any kind of testimony and evidence.
You raise an interesting point about the artificial generation of evidence used in court. In 1992, Michael Crichton wrote the book Rising Sun, which centers around the editing of security camera video footage to coverup a murder.
Not necessarily. Knowing an image for sure is fake has value, even if you can’t guarantee the reverse is true.
What stops someone from adding a watermark to an actually photographed (carefully framed?) picture to discredit it? There is no certainty either way, just suggestions from someone else about what the truth might be.
No need to theorize, this is already happening.
If you want to discredit an imagine, upload a slightly ai-edited copy of it.
For C2PA and exif, these aren't watermarks, just metadata. You could already remove them with exiftool.
Stalin had no issues photoshopping images almost 100 years ago.
Stalin had all the resources imaginables at his disposal.
Now Nancy, a tech-phobic waitress who has a grudge against her coworker can make up an entire scenario with one prompt and her colleagues might blindly believe her.
Let's not pretend they're the same thing.
Gen AI is inevitable. Watermarking is likely futile. But in my opinion it is still very important to discuss how, as a society, we're going to live in a post-truth world now that anybody can, IN SECONDS, not only fabricate a story but also spread it to thousands of people through their social media.
Simple, don't trust what you see on the internet, which has been a constant since the mid 90's when it was invented.
When that idea was originated, the advice was more like:
"Don't trust what you see on the Internet. Trust instead what you read in a reputable daily newspaper, or Peter Jennings or Tom Brokaw on the nightly news, or BBC World News."
Today, the Internet, especially the part which is not trustable, has nearly finished killing most of the "trustworthy" news sources, by outcompeting them for ad dollars - by being way better at targeting ads (e.g. Meta) and by scientifically perfecting addiction (e.g. TikTok). What remains is mostly controlled by governments and has far from a perfect record of being fact-based and impartial.[1] There are a ton of independent people out there in good faith posting facts on the Internet, but we just agreed that we shouldn't trust what we see on the Internet.
So doesn't this become "Don't trust anything"? And doesn't that, in practice, get implemented as "Don't trust anything that challenges what you believe to be true"? This feels like a really, really bad change to our society - and I'd argue it's already completely happened.
[1] https://apnews.com/article/bbc-gaza-documentary-hamas-sancti...
This isn't just ads, trust in the mainstream media, itself, is very low [1], deservedly so in my opinion. The continuous lies by omission, the outright incorrect headlines/articles that they edit after a day, the lock-step messaging, alignment, and avoidance of topics, pushed by their respective political parties/billionaire owners (6 companies own 90% of media [2]), made me switch to more independent journalists.
[1] https://news.gallup.com/poll/695762/trust-media-new-low.aspx
[2] https://www.businessinsider.com/these-6-corporations-control...
As bad as it is, is still miles better than Internet posts by randos.
Which independent journalists do you like?
> switch to more independent journalists.
I have no objection to this -- I follow a few that I would say meet that definition well and which I trust. But boy do I worry that for 90% of the population, this translates to picking a bunch of enthusiastic propagandists whose bias is far worse than MSNBC, Fox News, or CNN ever were. I assume our craven and corrupt political parties will increasingly focus on propping up "independent journalists" who repeat their talking points for them.
Growing up I was told the newspaper is only good for reading the time.
Adequately implementing solves one problem (the making up a story because of a grudge), but creates a whole new set of likely much worse problems: how does one maintain a democracy / civil society? It's not just the trust of "social media" that you've eroded, you've almost certainly killed trust in traditional news reporting as well, especially considering just how much of traditional media is discovered via social media.
Effective democracy requires an informed voter base. Society requires its constituents to be invested in its continuity. Neither of those is achievable when we completely discard trust.
Not simple because plenty of people do. It's not what you do per say, it's how it effects society.
People will just become numb to images and video and trust nothing: this is already happening.
Yes, it's happened. Except a lot of people do have an exception - they'll trust the slop that reinforces their existing biases, or even if they know in their hearts it's not true, viewing their side's lies regularly still has an effect on the way they think.
Good point. Sometimes I wonder if social media, just almost every aspect of it, is the real cancer. Allowing just about anyone (globally) to anonymously deploy information warfare via the social media vector just seems bound to have horrible outcomes. It's just as bad with text as with images or video. Because of social media, we've trained at least 3 separate generations to self-sort into camps with customized ideological info sources that have incredibly-low standards for fact-checking and every incentive to tell their audience (1) exactly what they want and (2) whatever will enrage them most.
AI kind of makes this worse, but also only barely. Because most people really ought to know by now that almost any content could be AI, a video of, say, Trump kicking a baby or violating a goat wouldn't convince anyone that those acts happened (unless they already believed they happened).
Thing is, we're so flooded in biased BS, and no one has any incentive to produce non-sensational, factual news. I absolutely see 'post-truth' as the inevitability. You can't "weed a garden" when it is 100% weeds. The term "news" will cease to mean facts, and just become a branch of entertainment. Kind of the way "Reality TV" went from being supposedly a documentary (e.g. COPS) to just being a flavor of entertainment, where nothing needs to be real.
Generating realistic video of arbitrary things and people at scale is quite a bit of a different game than retouching photos
A good example why fake images are bad.
Do you want to make it easier for the next Stalin?
The genie has been out of the bottle for 100 years, it's delusional to think that some voluntary watermark is going to stop that.
In reality, all images will cease to be trustworthy and there's nothing that can be done about this.
Why are humans powerless to do anything about this? Aren't we making the technology? It's kind of a big problem for the future of the justice system and politics.
Because local models exist and you can't take them away.
Drugs are banned, they still exist. Many torrents flourish (that violate copyright laws), humans can't seem to stop those.
Generative AI has too much commercial utility to ever be "snatched back" at this point through legislative means.
> It's kind of a big problem for the future of the justice system and politics.
People will adapt, but this "big problem" is going nowhere.
Before the invention of photography, all we had were paintings and drawings. You wouldn't trust a painting to faithfully represent the truth.
We already have the problem of people blindly trusting shit they read on the Internet.
Stalin controlled the state. The state controls companies. Companies control watermarking.
This sort of solution to the fake image problem, makes it easier for stalin not harder. If everyone can make fake images that is one thing. If only the dictator can, well that is much worse.
The concept of truth? A bit overblown don't you think? Because some guy can make a realistic looking fake videos that destroys the "concept" of truth? How?
It's best for privacy not to do this in the first place because:
- Watermarks are optional by AI provider so bad actors will circumvent by using another provider
- GH project proves watermarks can be removed
Given these, trying to ensure "truth" is a futile effort unfortunately, and watermarking only gives companies advantage to violate privacy
AI watermarks only give the illusion of maintaining the concept of truth. The government and corporations will still have access to un-watermarked models to destroy the truth with.
Do we care about truth?
Without truth freedom and privacy are endangered too.
The other comment talks about laws that can already handle that. How if images, video and audio aren’t reliable proof anymore?
The watermarking should be on those things we want to verify as something that was not generated or manipulated. Something you'd add to, for instance, cameras. Putting them on the generated/manipulated is backwards as you can never get every model to watermark.
That model is equally bad though. Given that you're writing this in a discussion about gen AI watermarks, how in the world did you come up with the idea that Gen AI wouldn't be able to add a watermark?
Perhaps not watermarks, but cameras could sign their pictures and put it in the extra data, that's not something that would be easy to add to fake pictures, or at least not the correct signature.
Not that they "wouldn't be able".. that they wouldn't do it. For Gen AI watermarks to be useful all Gen AI systems need to add them and the incentives aren't there for that to happen. On the other hand the incentives are there for the non-generated sources to add it so they can differentiate themselves from the Gen AI media.
I think you'll have to clarify the cause and effect of that a bit.
Also note that people have been falling for obviously watermarked videos already.
And even if they weren't, wouldn't that just make them more gullible towards non-watermarked models?
Maybe we do care about truth, freedom and privacy but the majority of rest of society will happily accept any T&Cs just to get access to whatever the next digital sliced pan is and as for truth and accountability, if they were two sides of the same coin on the ground people wouldn't bend down to pick it up as possesing it looks too much like responsibility and inconvenience.
I'm pretty sure watermarking is (or soon will be) a requirement for AI generated images in software used in the EU, as part of their regulations for AI transparency.
Of course. Regulations are the EUs primary output these days! Anywhere else they’re just sparkling suggestions.
If i had a dollar for every time an American cried about literally any non-US jurisdiction having an iota of effect on them I could quit my job and leave this terrible website forever.
If I had a dollar for every regulation to come out of Brussels serving no purpose other than to extract money or exert control over American companies because they have no relevant competitors to worry about impacting in the EU I could do the same.
I’m also Canadian.
It's not "every digital move" it's the photos you ask them to create. If you care about privacy use a local model
The human ethos should be to never be misleading about the origin and truth of any content you create, forward, or pass on. If we care about honesty we should jail anyone who does so.
Its what happens when people in power are paranoid dark-triad types and want to be able to catch anyone who threatens their power and stick it to them..
It already happened with Trump claiming any unfavorable content of his administration is "AI-generated" as a defense to dismiss real, unedited media. He literally said, “If something happens that’s really bad, maybe I’ll have to just blame AI.”
ie the video of garbage being thrown out the windows that his team already confirmed was real:
https://www.kptv.com/2025/09/03/trump-says-video-showing-ite...
Also the Lincoln Project video footage him him stumbling while walking and over his words: https://www.forbes.com/sites/mattnovak/2023/12/04/donald-tru...
When.
"we care about privacy" Yes, yes I do.
but I also live in a society that requires trust to function. making a tool the obliterates that trust(genAI imagery pipelines) then creating a tool that makes it trivial for normal people to remove any hint of controls over said trust eroding system is, toxic.
I get the argument about not putting in fingerprints that identify users, Good I agree. But this also removes the things that identify this as an AI image.
Now, what are the legitimate uses of that?
No really, why would I _need_ to remove a watermark for _legitimate_ purposes? Assuming that watermark is generic, rather than a fingerprint of a specific person
> No really, why would I _need_ to remove a watermark for _legitimate_ purposes?
When removing the watermark is easy, a very legitimate purpose of making the code to do it publicly available is to make a public demonstration that it's easy to do.
As for content use cases, suppose someone is using AI to modify their appearance because they're being unjustly targeted by an oppressive government. That government naturally bans doing that because they want to be able to identify and arrest their critics, so now if you make videos with your real face you get arrested but if you use a generated avatar then the watermark enables automated censorship because the government orders anything with the watermark to have its reach automatically restricted.
> suppose someone is using AI to modify their appearance because they're being unjustly targeted by an oppressive government
Then use a mask like everyone else. digital mask, one that obscures.
which is my main point, no, there isn't a legitimate need.
realtime avatars don't generally have invisible watermarks, also they are running from your machine, otherwise you've got a (normally credit card) trail to your front door. plus a video stream
also if you are generating stuff from a public provider, then tracing people isn't that hard to do.
As someone else pointed out: if watermarks are required, then everybody will assume an image without a watermark is the honest truth, which is obviously not true. Someone will end up in prison because of some image. This is bad.
but right now, we are eroding trust at an industrial scale.
There are no reliable tools for the end user, normal person, to work out if an image is AI or not. This erodes trust and lets bad actors get away with "oh thats AI generated" or use AI to defraud users.
The Hacker ethos is mainly about how things work: sharing, openness.
I'm not entirely sure how hiding that something is GenAI fits in here. It surely doesn't have anything to do with Privacy though.
I disagree. Its mainly about having technical control and freedom. Reverse engineering how things work feels like peak hacker ethos. You don't have control of something if you can't remove it.
I think ethical considerations were always a bit secondary to technical power when it came to so called "hacker ethos".
After all, instructions on how to remove watermarks definitely feels like the sort of thing that would have been in phrack back in the day.
There's a thing called Hacker ethic which used to be referenced quite frequently in the past: https://en.wikipedia.org/wiki/Hacker_ethic
Probably it's worth reminding of also considering we're on HN here... ;)
Well, to quote from the article
> Hackers believe that essential lessons can be learned about the systems—about the world—from taking things apart, seeing how they work, and using this knowledge to create new and more interesting things.
> Access to computers—and anything which might teach you something about the way the world works—should be unlimited and total.
> Mistrust authority—promote decentralization
> All information should be free
I phrased it a bit differently, and perhaps a little less sympathetically, but i think i was more or less saying the same thing.
In any case a tool like the article that strips watermarks seems exactly the sort of thing that would fit into what i quoted above. Its mistrusting authority - there is nothing more central authority then having a literal central authority adding hard to remove digital signatures to images. It promotes freedom of information - it supports explaining how watermarks work and what they are. Its fundamentally taking apart a system, which teaches us how the system works.
Even if you remove a watermark, the companies still have a record of which images they have generated and for whom. Even if you remove the obvious watermarks, all major image generators are using steganography to embed hidden information that you can't be sure were removed. This is a type of one-sided arms race where one player gets to be invisible if they want to.
Do they still work if you apply something like a filter or additional layers on top? Or add a subtle blur, etc.
Yes usually, since an important aspect of steganography is error correction. For example we know that SynthID is robust enough to survive resizing and small blurs.
I don't know I really like the definitive indicator that something is AI so I can completely ignore anything else that comes from them.
I think the issue is it was never definitive. This is a great way to show people that.
I have not read anyone claim that SynthID had a false alarm issue, so if it returned positive I would believe it is synthetic.
it does have a false negative issue
An issue this tool exacerbates. Labeling AI output is a good thing.
Without the tool trust into Ai generated misinformation is higher.
Precisely the opposite. This tool weakens the reliability of SynthID.
You can trivially false-flag any image by uploading it to gemini and asking it to return it as-is
That's not what the previous comment is referring to. They're referring to false positives, i.e "Gemini did not generate this (or process it) yet it says SynthID confirmed"
So imagine the damage you could do to an artist by having Gemini flag it then saying “see here’s the proof they’re using AI art!” That person would get eaten alive.
...except they can produce the same image without the fake SynthID.
They must have used the tool from TFA!
If someone's doing something you don't like, you can't really count on them doing it the way you prefer.
You can count on them doing it in a way that's economical for them. It's how email spam filters and ad blockers work. Sure somebody will always find a way to bypass it, and that's the arms race. A filter with zero false positives that removes 80% of slop is pretty darn good though.
Are markers being removed here the same or similar to ones tools might add if you use an AI tool just to edit a photo? like a more complicated object removal in a photo editor?
I feel like it is even worse if we have a marker that is ALMOST always present on AI images. It would make us more likely to be fooled by an AI image that had them removed, because we would trust the marker.
This is a bit misleading as for Gemini it only properly removes the visible watermark. To remove SynthID it has to regenerate the image at low noise with SDXL, which will likely destroy a lot of small details, plus won't work for higher res properly (NB2 and GPT Image 2 support up to 4K image outputs)
Nano Banana 2 only supports 1K resolution (1024x1024) natively. Anything above that is upscaling. So this matches SDXL. GPT Image 2 does support 4k natively (but experimentally).
Where did you get that info from? According to Google's own docs as well as my own image generation tests via the API, it supports up to 4K natively for gemini-3.1-flash-image-preview (aka NB2).
It just defaults to 1K. But I didn't see anything in the docs stating that it's just a simple upscale for larger resolutions.
https://ai.google.dev/gemini-api/docs/image-generation#gener...
From: https://aistudio.google.com/models/gemini-3-pro-image
> Produce production-ready assets with native 1K output and built-in upscaling to 2K and 4K resolutions
The API doc you linked is misleading.
Yeah - if that's true then it's even worse because the output price says
But if all the "compute time" is spent on a 1K image and they're just passing it to a ESRGAN or other upscaling technique, then there’s literally zero reason to generate anything above 1K. Just save the money and upscale it yourself.
It's not upscaling for NB2, 4K outputs are very different from 1K, and output tokens count is also different.
Is SDXL still the best local image model all these years later? Damn, that’s sad…
With the number of fine-tuned LoRAs and checkpoints - from a realism standpoint, yes SDXL is still very viable. From a prompt adherency perspective, absolutely not.
Qwen-Image-2512 / Z-Image / Flux.2 absolutely crush SDXL if you're actually generating moderately complex scenes.
Do you still need a wacky backend to run them locally or does LM Studio make it easy nowadays? Last I use a local diffusion model was late 2022.
So I use a combination of Neo Forge and ComfyUI. Forge has an easier learning curve but ComfyUI gets all the new "hotness" almost immediately since there's so much custom nodes for it.
If you're on a Mac, I've heard that Draw Things is supposed to be pretty "batteries included" simple for image gen along the same lines as LM Studio.
https://github.com/Haoming02/sd-webui-forge-classic/tree/neo
https://github.com/Comfy-Org/ComfyUI
https://drawthings.ai
It claims to remove SynthID also.
Watermarking images generated from trained data on stolen copyrighted material, I get why so they can try to tell if something is real or not but something seems wrong
For better or worse, I think the future has to be showing warnings on images with no C2PA-like watermarking like we do for insecure web sites. Media can be signed by authors. If the Associated Press sign an image, I can feel more confidence in its authenticity than the next decade's InfoWars. Power users can still choose to ignore it, but standardizing UX around trust sources like we do TLS will save the less savvy among us.
I don't think we've even seen the full brutal force of disinformation that we are capable of yet.
> Use cases where the threat model fits: You are preserving art or historical record against false-positive "AI-generated" labels.
Sorry, how does using AI to generate images have anything to do with this? Image generators cannot insert watermarks into things they did not generate, and it seems highly unlikely that you will get a false-positive watermark on human-generated art, especially if, as the readme says, these watermarks have high enough fidelity to trace to a specific session id. Plus the modifications to the image needed to erase watermarks would necessarily change the thing being "preserved."
[edit]: the more I read the more I'm convinced, the claimed use cases in the README are bullshit and the real reason is to provide a tool that helps people bypass "AI-generated" labels on social media for AI slop.
I mostly agree about the justification in the repo being wrong, but wanted to engage about this point:
> Image generators cannot insert watermarks into things they did not generate
It's actually very easy to take a real image, ask Gemini/ChatGPT to modify some tiny part of it (could be something as silly as lighting/shadow/etc), and often the resulting image will be detected by their watermarking tools. This way you can easily present any real image as AI-generated.
Ignoring that a watermark removal tool does not help with this threat model, the claim is still true: the original image can not be changed, and instead a copy is created.
So what? I can also open an image in Photoshop and make sure it saves out some Photoshop specific EXIF data and try to claim the image was doctored. What I can't do is go and put my deceptive altered file up in place of the original in all the places on the Internet it exists.
I had to think about it, how about if the claim were:
If you take a photograph that is misidentified as AI generated, you can “preserve the historical record“ by using this tool before publishing the image.
(Anyone know the false positive rate with watermark IDs, would’ve hoped it’s like zero)
Trusting your adversary to reliably identify themselves is folly.
We need to get to a place where we're checking digital signatures to see which humans were involved in the creation of something, and ignoring everything else.
Isn't this wrong? like removing AI watermarks, people like make morphed images using AI, and then use something like this, and claim it to be real
How you use the tool matters, yes. But the freedom to demonstrate and understand this weakness with watermarks is much more important to exercise than saying "shhhhh don't tell people about this".
The takeaway should be that watermarks aren't very reliable and you should keep your guard up anyway.
I think AI watermarks are kind of a lost cause anyway.
I'd be more interested in some kind of trusted Non-AI watermark.
This is something that could get integrated into cameras for example. However, considering how much AI-processing we already have in "normal" photos, it will be difficult to decide where to draw the line.
That would be great, but I always wonder: how do you prevent people from photographing an AI generating image through the 'legitimizing' camera?
Maybe LIDAR or IR focus could help with that.
But then again, the photo itself would be authentic in this case strictly speaking.
It's not an easy problem. If I had a good solution, I would have tried to monetize it already ;)
Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term.
How is a hidden watermark that only police and providers can read better than a hidden watermark that the people can also read if they go out of their way to setup some project like this? Also your argument makes zero sense because bad guys will do this anyway. At least there's also an open source project for normal people, and it's interesting from a stenography standpoint.
Watermarks exists to create a chain of custody or attribution that can be used to establish culpability. Just like the government requires printer companies to add a dot matrix watermark to printed pages to prevent currency forgery.
Making untraceable assets means parties can’t be held liable for harms.
Just because a bad actor might poison a toddler doesn’t mean you should sell them the arsenic.
Just because the effects of arsenic on the neurons of a brain are interesting means you should feed them to a toddler and watch the effects unfold.
I support the legal freedom to pursue any idea, but we should also mentor our colleagues about the consequences of our projects and avoid unnecessary harm.
What is it with people that take your position that they always need to bring children to the discussion? Where's the watermark in an IKEA knife?
Is an ikea knife infinitely replicable, distributable and capable of undermining the entire social fabric of a community? No. so these are not the same category of threat potential.
Further, an ikea knife used in a crime can be traced back to the criminal through other evidence like finger prints, surveillance cameras, purchase receipts etc.
Arguments that have a emotional component, especially related to our evolutionary instinct to take care of kids has a way of avoiding the rationalizations people adopt to justify otherwise indefensible behavior.
You also dont do yourself any favors in a bad argument that lumps me in with some unknown group of people rather tan responding to my argument.
No.
All watermarks are cures that are worse than the diseases they are trying to prevent (and never achieved to even cure anything).
Watermark is a blatant violation of legitimate privacy expectations, while not preventing anything because they can be easily removed.
So all the honest usages are punished, while illegitimate ones are not caught.
Information is default low-trust unless you have reason to extend trust to the source and that's been the case for thousands of years, if not the entirety of human existence.
We now have the tools to increase trust in specific information, for example: by signing images that need high trust for things like news reporting using camera hardware root of trust with time and geo stamping. If signatures are removed, that's back to a default low-trust state.
It is best not to extend blanket trust to a specific source at all.
That is how we ended up with the situation where "reputed" media organizations peddle daily lies or selective truths that are useful to their benefactors.
IMHO you cannot base your trust on something that is so easy to work around.
Trust should be between people, not a person and some data.
So showing how easy to generate "false" data, this makes it more obvious for people focus on other people. Trusting people makes life much easier in my experience, while focusing on data, again in my experience, is a game of cat and mouse.
The Kentucky primary had $1.7 million spent on deepfake political ads which were seen 49 million times. Don't know how much it effected the result, but it's not a good sign of where things are headed.
> Trusting people makes life much easier in my experience
Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information?
AI watermarks are no panacea, but at least they are a clear signal of what not to trust.
AI watermarks empower elite / those with resources and disempowers the common person.
Only people with resources will be allowed to make content that is AI generated passed off as real.
Pandora's box is open. Instead of making a multi-tiered privileged society, we need to fundamentally restructure society to adapt.
Before that restructuring occurs it is critical to keep the playing field level. These are not tools that should be controlled by a minority authority, they are far too dangerous.
> Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information?
Isn't that the scenario the watermarks are useless against? Adversarial governments or anyone with enough money will be the ones who can generate images without watermarks even if you force them on the proles.
> AI watermarks are no panacea, but at least they are a clear signal of what not to trust.
Which seems like it only makes the actual problem worse? If most of them have watermarks, that only encourages people to put more trust in the ones that don't, even though those are the ones "powerful people" can still forge to manipulate everyone. What good is something that increases the credibility of adversarial government forgeries?
> how do you apply that to a society at large
You can’t, it’s an inherent contradiction. Human social structures have sophisticated and robust evolved mechanisms for establishing and maintaining trust. These dynamics are not one option among many, they are the optimum. By their definition they don’t scale to strangers around the planet. This is an immutable factor in why we have spam, bank fraud, etc. We want the benefits of trust without the cost of local constraints but wishing doesn’t make it so.
The distortion is shockingly visible in images. Especially with any amount of generational iteration https://streamable.com/9x3s4r
That said, this tool is incredibly lossy, garbling text, completely changing shapes. It also fails to remove the new gemini spark / placement that moved since yesterday.
> You will erode the foundation that enables any success you might have in the short term.
It's too late already. We live in a post-trust society now.
Would you prefer if someone did the same thing and kept it to themselves (or sold it to the highest bidder)? I think knowing it exists is better than not knowing it exists.
I'd like to frame this differently: watermarking is a (weaker) form of DRM, and DRM has never worked in favor of the users.
I know of at least one music GenAI service whose ToS forbid me from using their productions in ways that are incompatible with my local rights. If Google decides, as they did with YouTube, that they'll enforce this company's watermark even though I have a right to use their results then I've passively accepted that a (foreign) company decides which of my legal rights I can exercise.
For a more pessimistic outlook: for the GenAI companies that work for the military, are those pictures also watermarked? Because if watermarks only apply to one group then the "implicit trust" argument doesn't work.
I'm in favor of watermarking the same way I am in favor of paying artists for their work. But just as with youtube-dl and DVD decryption libraries, tools like these are necessary to level the playing field.
lol
>Why? Don’t do this. Society is built on an implicit assumption of trust. You will erode the foundation that enables any success you might have in the short term.
would you say the same thing about a ROT13 breaker, or would you recognize how laughably naive that sounds?
I have no idea what a Rot13 breaker is and dont care that you think advocating for an ethical posture in what the tech industry builds is naive. Its naive to think bullies wont be in a kids playground. Its also naive to think we just need to accept that rather than creating norms around the type of play that is acceptable so that the playground functions.
AI generated images have never existed before, they will break our ability to use the digital tools we have built society on if we let them. Ensuring they can be identified and have attribution tracing data embedded is a reasonable step to prevent abuse.
Society was build on that assumption, it didn't last. Now we have to build a new one where we put in the work of explicitly trusting each other.
Preserving the illusion that that assumption is still useful only helps the people who are exploiting it.
Didn't work at all for me. Hive Moderation still shows "gemini3: 99.9%". Tried their online version and it went from gemini3: 99.9% to midjourney: 64.7% + stablediffusionxl: 16.1%.
I find the casual malevolence of this kind of thing breathtaking. Every time someone raises potential issues with AI misuse, they're dismissed as fear-mongering while a host of people immediately rush to demonstrate that not only is the fear justified, it's gleefully anticipated under the vaguest colour of lofty ideals.
People want to dismiss the potential harms of deepfakes while also excitedly releasing the deepfake-hider-3000 and saying they just really, really care about privacy (for people who make deepfakes).
the genie is out of the bottle. the sooner you let go of the notion that it can be kvetched out of existence the less frustrated you'll feel.
for example, 35 years ago PGP was a "casually malevolent" thing, enabling terrorists and pedophiles to email each other with impunity. the effort to make math illegal had (very predictably) failed and now we have encryption everywhere. did the world end? how do you feel about Chat Control and numerous other initiatives to roll it all back?
That argument is a lot less convincing (and sincere) when we're talking about a specific tool with a very specific purpose, rather than maths itself. In much the same way, you can make pro-gun arguments based on sport shooting, but not when you're holding a landmine.
This is a very specific tool with the purpose of making it easier to lie, in an area where the kinds of lies people tell are directly injurious to society. "It's just math" doesn't fly here, and all the primary uses for such a tool are malevolent.
the genie I was talking about was AI itself, not this particular tool, to address your point about "AI misuse". I should've made that clearer, sorry.
as for the rest of it: would banning a tool that breaks ROT13 make ROT13-encrypted communications safe?
AI itself has non-malevolent uses, but this doesn't. It's designed for negative uses.
ROT13 decryption does not have the same issues, and is not a good parallel here. Again, we're talking about a specific tool with specific use cases; a general algorithm being able to be used for negative ends is not the same as a tool being designed to be used for negative ends.
I also haven't mentioned banning anything; that's all on you. It really feels like you've jumped in with pre-prepared talking points rather than engaging with my comments.
To restate: "AI will benefit humanity" is a tough sell when so many people who are in to AI deliberately make tools that support the negative uses. This is an example of a project that superficially presents as high-minded, but is designed for exploitation.
what I've been trying to convey to you here with ROT13 analogy is that we're better off knowing that something is broken. ROT13 is a piece of shit, and so is SynthID. whether the author's intentions were noble or malicious is utterly irrelevant.
> we're better off knowing that something is broken
This is not a new insight, and it doesn't add anything to the discussion. It's effectively a platitude.
This project is absolutely not motivated by a wish to inform, and it's disingenuous to prevaricate around that; there are countless ways to inform that don't endorse.
there is no discussion. your opinion on the matter is irrelevant, same as mine. genai will be misused in every way imaginable, and there's nothing anyone can do about this. no regulations, no laws, nothing will stop it now.
for fuck's sake, Photoshop existed for how many years now? you would be a fool to trust any random photo to be genuine at any point in time. yes, yes, yes, back then only the people who had spent a few days learning it could do it, and now everyone can. that makes no difference whatsoever.
None of that is relevant to the conversation; it is genuinely like you are pasting from a random discussion a decade ago.
We're not talking about Photoshop, because it's not a parallel here. We're not talking about trust in authenticity. These are points you have plucked from the ether. Please reread my earlier comments.
it is very much relevant to the conversation because your assertion about this tool being harmful and unethical is rooted in your misguided belief that generative fakes that can fool a grandma on facebook are somehow worse than man-made fakes that can fool anyone.
Good. Better than only known bad actors doing this with the public wrongly believing watermarks are reliable.
I love that this is exactly one position above OpenAI Adopts SynthID Watermarks
As a photographer I say "down with this sort of thing"
Since it seems like you would know, could it also work with Non AI static watermarks like repeating oblique lines?
Regardless of one's opinion about this particular project, it seems obvious to me that the path forward is proving authenticity of non-AI resources rather than attempting to watermark all the AI-generated ones.
Pretty hard problem to tackle when you can point an "authenticated" camera at a really nice screen and snap a 'definitely real' photo of anything a screen can display :(
There's probably a technical solution, such as the camera manufacturer cryptographically signing a GPS location and timestamp together with the pixels. Like all DRM it will probably be broken though, and more importantly, would anyone (even e.g. a newspaper editor) care enough to verify the signature?
Spoofing GPS timing signals isn't as hard as it used to be. If you know what you're looking for on AliExpress you can get all the equipment you need
How about a quantum digital signature?
GPS? No problem, they can take their hi-DPI monitor and their secure camera to the sidewalk in front of the White House lawn and play the AI video they made of soldiers shooting protestors.
Isn’t the goal only to prove that a photograph was taken with a particular camera? I don’t think you could ever prove that the subject was legitimate, as there are countless ways to misrepresent things. But in a world of AI slop, knowing a photo was taken on a real camera and wasn’t synthesized artificially is still a useful data point in determining trust.
> proving authenticity of non-AI resources
You’re trying to prove a negative.
Not really, I'm trying to prove "this is an actual photo from some specific certified hardware."
watermarking only really works when the scheme is secret.
putting cyphertext in high frequency noise is old news. in generative land would be far more interesting to use the generative flexibility to encode in macrostructure.
Are there tools to apply SynthID to existing images? e.g., make AI watermarks unreliable for those relying on it
Yeah, human minds need an security patch for firehosing
To remove Gemini watermark, open dev tools and block http request to watermark. It is overlaying logo in client.
There's quite a bit of difference in the before and after. I hope they can find a way that better preserves details.
people like the idea of removing watermarks. it doesn't have to remove a watermark. do you get it? this whole product is meaningless vibes.
Can't we instead just use open source models?
What happens to watermarks when you turn images to videos on other platforms? Does every frame carry the original mark?
This is brilliant pace. What I expected to see
Yin and yang.
I just saw the announcement about OpenAI or so going to use SynthID and all I thought was; what can d be read(located) can be removed. Seems the tool already exists, proving my point.
Yes, I came from that thread and figured this kind of tool was worth mentioning.
doing the devil's work here.
What’s wrong with showing off AI bro? Why the shame?
People don’t realize how hard it can be to throw an election or impugn an adversary with manipulated imagery
Then they ask us to do it by hand?!
You're assigning emotions to people based on what you'd like them to feel, not on reality. For example, most americans probably don't feel shame about being american. But it's still a good decision not to go around showing off a bunch of american flags abroad, unless you want people to look at you in a certain way.
So letting people know you’ve used AI is not a good thing? Best used in covert is what you’re saying?
Some people are just biased to the point that telling them something is AI when it's actually not will cause them to convince themselves such:
https://futurism.com/artificial-intelligence/real-monet-ai-c...
Interesting. And now this not-AI-generated-actually-real-cropped-Monet-AI-joke has been sold to an AI art collector for 40k. https://x.com/SHL0MS/status/2055281312697647223
The most amusing part of that experiment was that even Grok got convinced that it was AI-generated: https://x.com/grok/status/2054432219460977142
Sometimes it is not a "good thing" because it will cause people to react negatively. Whether the public's dislike of AI valid or not depends entirely on your perspective and the situation at hand.
Should people be scared of their food containing Red Dye 3? Yes. Aspartame? Maybe. MSG? Probably not. Dihydrogen Monoxide? Nope.
Should you feel "shame" for using AI to generate fake news? Yes. Art? Not really. See the numerous examples of people disliking famous human-made art just because it was presented as AI.
I don’t follow because all I see in other AI threads is how this is the future and people best adjust to the AI reality.
The hyper macho’ism on show by a large majority of pro-AI posters on other threads is frankly embarrassing and if I was trying to promote a positive spin on AI I’d be telling the bro’s to STFU.
This is more akin to having a fake passport and pretending you're not American when asked.
> having a fake passport
No, there's no law requiring disclosure of AI use yet, as far as I know.
> pretending you're not American when asked
Many people do so for various reasons, for example saying you're Canadian when on vacation. Do you count that as evidence that they're "ashamed" of being american? Or that being american is automatically a bad thing, because sometimes people hide it?
Amaze amaze amaze
- Rocky