Today, we’re going to talk about reality, and whether we can label photos and videos to protect our shared understanding of the world around us. No really, we’re gonna go there. It’s a deep one.
To do this, I’m going to bring on Verge reporter Jess Weatherbed, who covers creative tools for us — a space that’s been totally upended by generative AI in a huge variety of ways with an equally huge number of responses from artists, creatives, and the huge number of people who consume that art and creative output out in the world.
If you’ve been listening to this show or my other show The Vergecast, or even just been reading The Verge these past several years, you know we’ve been talking about how the photos and videos taken by our phones are getting more and more processed and AI-generated for years now. Here in 2026, we’re in the middle of a full-on reality crisis, as fake and manipulated ultra-believable images and videos flood social platforms at scale and without regard for responsibility, norms, or even basic decency. The White House is sharing AI-manipulated images of people getting arrested and defiantly saying it simply won’t stop when asked about it. We have gone totally off the deep end now.
Verge subscribers, don’t forget you get exclusive access to ad-free Decoder wherever you get your podcasts. Head here. Not a subscriber? You can sign up here.
Whenever we cover this, we get the same question from a lot of different parts of our audience: why isn’t there a system to help people tell the real photos and videos apart from fake ones? Some people even propose systems to us, and in fact, Jess has spent a lot of time covering a few of these systems that exist in the real world. The most promising is something called C2PA, and her view is that so far, it’s been almost entirely failures.
Is this episode, we’re going to focus on C2PA, because it’s the one with the most momentum. C2PA is a labeling initiative spearheaded by Adobe with buy-in from some of the biggest players in the industry, including Meta, Microsoft, and OpenAI. But C2PA, also sometimes referred to as Content Credentials, has some pretty serious flaws.
First, it was designed as more of a photography metadata tool, not an AI detection system. And second, it’s really only been only half-heartedly adopted by a handful, but not nearly all, of the players you would need to make it work across the internet. We’re at the point now where Instagram chief Adam Mosseri is publicly posting that the default should shift and you should not trust images or videos the way you maybe could before.
Think about that for one second. That’s a huge, pivotal shift in how society evaluates photos and videos and an idea I’m sure we’ll be coming back to a lot this year. But we have to start with the idea that we can solve this problem with metadata and labels — that we can label our way into a shared reality. And why that idea might simply never work.
Okay, Verge reporter Jess Weatherbed on C2PA and the effort to label our way into reality. Here we go.
This interview has been lightly edited for length and clarity.
Jess Weatherbed, welcome to Decoder. I want to just set this stage. Several years ago, I said to Jess, “Boy, these creator tools are criminally under-covered. Adobe as a company is criminally under-covered. Go figure out what’s going on with Photoshop and Premiere and the creator economy because there’s something there that’s interesting.”
And fast-forward, here you are on Decoder today and we’re going to talk about whether you can label your way into consensus reality. I just think it’s important to say that’s a weird turn of events.
Yeah. I keep likening the situation to the Jurassic Park meme, where people thought so long about whether they could, they didn’t actually stop to think about whether they should be doing this. Now we’re in the mess that we’re in.
The problem, broadly, is that there’s an enormous amount of AI-generated content on the internet. Much of it just depicts things that are flatly not real. An important subset of that is a lot of content that depicts modifications to things that actually happened. So our sense that we can just look at a video or a picture and sort of implicitly trust that it’s true is fraying, if not completely gone. And we will come to that, because that’s an important turn here, but that’s the state of play.
In the background, the tech industry has been working on a handful of solutions to this problem, most of which involve labeling things at the point of creation. At the moment you take a photo or the moment you generate an image, you’re going to label it somehow. The most important one of those is called C2PA. So can you just quickly explain what that stands for, what it is, and where it comes from?
So this is effectively a metadata standard that was kickstarted by Adobe. Interestingly enough, Twitter as well, back in the day. You can see where the logic lies. It was supposed to be that everywhere a little bit of content goes online, this embedded metadata would follow.
What C2PA does is this: at the point that you take a picture on a camera, you upload that image into Photoshop, all of these instances would be recorded in the metadata of that file to say exactly when it was taken, what has happened to it, what tools were used to manipulate it. And then as a two-part process, all of that information could then hypothetically be read by online platforms where you would see that information.
As consumers, as internet users, we wouldn’t have to do anything. We would be able to, in this imaginary reality, go on Instagram or X and look at a photo and there would be a lovely little button there that just says, “This is AI-generated,” or, “This is real,” or some sort of authentication. That has obviously proven a lot more difficult in reality than on paper.
Tell me about the actual label. You said it’s metadata. I think a lot of people have a lot of experience with metadata. We are all children of the MP3 revolution. Metadata can be stripped, it can be altered. What protects the C2PA metadata from just being changed?
They argue that it’s quite tamper-proof, but it’s a little bit of an “actions speak louder than words” situation, unfortunately. Because while they say it’s tamper-proof, this thing is supposed to be able to resist being screenshot, for example, but then OpenAI, who is actually one of the steering community members behind this standard, openly says it’s incredibly easy to strip to the point that online platforms might actually do that accidentally. So the theory is there’s plenty behind it to make it robust, to make it hard to remove, but in practice, that just isn’t the case. It can be removed, maliciously or not.
Are there competitors to C2PA?
It’s a little bit of a confusing landscape, because I think it’s one of the few tech areas that I would say there shouldn’t actively be competition. And from what I’ve seen, from what I’ve spoken to with all these different providers, there isn’t competition between them as much as they’re all working towards the same goal.
Google SynthID is similar. It’s technically a watermarking system more so than a metadata system, but they work on a similar premise that stuff will be embedded into something you take that you’ll then be able to assess later to see how genuine it is. The technicalities behind that are difficult to explain in a shortened context, but they do operate on different levels, which means technically they could work together. A lot of these systems can work together.
You’ve got inference-based systems as well, which is where they will look at an image or a video or a piece of music and they will pick up telltale signs that apparently it may have been manipulated by AI and they will give you a rating. They can never really say yes or no, but they’ll give you a likelihood rating.
None of it will stand on its own to be a one true solution. They’re not necessarily competing to be the one that everyone uses, and that’s the mess that C2PA is now in. It’s been lauded and it’s been grandstanded. They say, “This will save us,” whereas it was never designed to do that, and it certainly isn’t equipped to.
Who runs it? Is it just a group of people? Is it a bunch of engineers? Is it simply Adobe? Who’s in charge?
It’s a coalition. The most prominent name you’ll see is Adobe because they’re the ones that shout about it the most. They’re one of the founding members of the Content Authenticity Initiative, which has helped to develop the standard. But you’ve got big names that are part of the steering committee behind it, which are supposed to be the groups involved with helping other people to adopt it, which is the important thing, because otherwise it doesn’t work. And part of this process, if you’re not using it, C2PA falls over. And OpenAI is part of that. Microsoft, Qualcomm, Google, all of these huge names are all involved with that and are supposedly helping to … They’re very careful not to say “develop it,” but to promote its adoption and to encourage other people, in regards to who’s actually working on it.
Why are they careful not to say they’re developing it?
There isn’t any confirmation that I can find where it’s got something like, I don’t know, Sam Altman saying, “We’ve found this flaw in C2PA, and therefore we’re helping to address any kind of falls and pitfalls it may have.” It’s always just anytime I see it mentioned, it’s whenever a new AI feature has been rolled out and there’s a convenient little disclaimer slapped on the bottom, kind of a, “Yay, we did it. Look, it’s fine, a new AI thing, but we have this totally cool system that we use that’s supposed to make everything better.” They don’t actively say what they’re doing to improve the situation, just that they’re using it and they’re encouraging everyone else to be using it too.
One of the most important pieces of the puzzle here is labeling the content at capture. We’ve all seen cell phone videos of protests and government actions and horrific government actions. And I think Google has C2PA in the Pixel line of phones. So video that comes off a Pixel phone or photos that come off a Pixel phone have some embedded metadata that says it’s real.
Apple notably doesn’t. Have they made any mention of C2PA or any of these other standards that would authenticate the photos or videos coming off an iPhone? That seems like an important player in this entire ecosystem.
They haven’t officially or on record. I have sources that say apparently they were involved in conversations to at least join, but nothing public-facing at the minute. There has been no confirmation that they are actually joining the initiative or even adopting Google SynthID technology. They’re very carefully skirting on the sidelines for some reason.
It’s a little bit unclear as to whether they’re letting their caution about AI generally stem into this at this point. Because as far as I’m concerned, there is not going to be one true solution, so I don’t really know what Apple is waiting for, and they could be making a difference, but no, they haven’t been making any kind of declarations about what we should be using to label AI.
That’s so interesting to me. I mean, I love a standards war, and we’ve covered many standards wars and the politics of tech standards are usually ferocious. And they’re usually ferocious because whoever controls the standard generally stands to make the most money, or whoever can drive the standard and an extended standard can make a lot of money.
Apple has played that game maybe better than anybody. It’s driven a lot of the USB standard. It was behind USB-C. It drove a lot of Bluetooth standard, which it extended for AirPods. I can’t see how you make money with C2PA, and it seems like Apple is just letting everyone else figure it out and then they will turn it on, and yet it feels like the responsibility to be the most important camera maker in the world is to drive the standard so people trust the images and videos that come off the cameras.
Does that dynamic come out anywhere in your reporting or your conversations with people about this standard — that it’s not really there to make money, it’s there to protect reality?
The moneymaking side of things never really comes into the conversation. It’s always that people are very quick to assure me that things are progressing. There’s never any kind of a conversation about incentive to motivate other people to do so. Apple doesn’t stand to really gain anything financially from this other than maybe the reassurance that people know that if they’re taking a picture with their iPhone, it could help to contribute to some sense of establishing what is still real and what isn’t. But then that’s a whole other can of worms because if iPhone is doing it, then all the platforms that we see those pictures on also have to be doing it. Otherwise, I’m just kind of verifying that this is real to my own eyes as me, the person that uses my iPhone.
Apple may be aware that all the solutions that we currently have available are inherently flawed, so throwing your lot in as one of the biggest names in this industry and one that could arguably do the most difference, you’re almost exacerbating the situation that Google and OpenAI are now in, which is that they keep lauding this as the solution and it doesn’t fucking work. I think Apple needs to be able to stand on its laurels about something, and nothing is going to offer them that at the minute.
I want to come back to how specifically it doesn’t work in one second. Let me just stay focused on the rest of the players on the content creation side of the ecosystem. There’s Apple, and there’s Google, which uses it in the Pixel phones. It’s not in Android proper, right? So if you have a Samsung phone, you don’t get C2PA when you take a picture with a Samsung phone. What about the other camera makers? Are Nikon and Sony and Fuji all using the system?
A lot of them have joined. They’ve released new camera models that have got the system embedded. The problem that they’re having now is in order for this to work, you don’t just have to do it on your new cameras, because every photographer in the world worth their salt isn’t going to go out every year and buy a brand new camera because of this technology. It would be inherently useful, but that’s just not going to happen. So backdating existing cameras is where the problem is going to be.
We’ve spoken to a lot of different companies. As you said, Sony has been involved with this, Leica, Nikon, all of them. The only company willing to speak to us about it was Leica, and even they were very vague on how internally this is progressing. They just keep saying that it’s part of the solution, it’s part of the step that they’re going to be taking. But these cameras aren’t being backdated at the minute. If you have an established model, it’s 50/50 whether it’s even possible to update it with the ability to log these metadata credentials in from that point.
There are other sources of trust in the photography ecosystem. The big photo agencies require the photographers who work there to sign contracts that say they won’t alter images, they won’t edit images in ways that fiddle with reality. Those photographers could use the cameras that don’t have the system, upload their photos to, I don’t know, Getty or AFP or Shutterstock, and then those companies could embed the metadata, and so “You can trust us.” Are any of them participating in that way?
We know that Shutterstock is a member. At the minute, the system that you’re describing would probably be the best approach that we have to making this beneficial, at least for us as people that see things online and want to be able to trust whether protest images or horrific things that we’re seeing online are actually real. To have a trusted middleman, as it were. But that system itself hasn’t been established. We do know that Shutterstock is involved. They are part of the C2PA committee, or they have general membership.
So they are on board with using the standard, but they’re not actively part of the process behind how it’s going to be adopted at a further stage. Unless we can also get the other big players involved for stock imagery, then who knows whether this is going to go, but Shutterstock actually implementing it as a middleman system would be probably the most beneficial way to go.
I’m just thinking about this in terms of the stuff that is made, the stuff that is distributed and the stuff that is consumed. It seems like at least at the moment of creation, there is some adoption, right? Adobe is saying, “Okay, in Photoshop, we’re going to let you edit photos and we’re going to write the metadata to the images and pass them along.” A handful of phonemakers, Google, or at least in its phones, are saying, “We’re going to write the metadata. We’re going to have SynthID.” OpenAI is putting the system into Sora 2 videos, which you wrote about.
On the creation side, there’s some amount of, “Okay, we’re going to label this stuff. We’re going to add the metadata.” The distribution side seems to be where the mess is, right? Nobody’s respecting the stuff as it travels across the internet. Talk about that. You wrote about Sora 2 videos and how they exploded across the internet. This is when it should have not been controversial to put labels everywhere saying, “This is AI-generated content,” and yet it didn’t happen. Why didn’t that happen anywhere?
It generally exposes the biggest flaw that this system has, and every system like it, to its credit. I don’t want to defend C2PA because it’s doing a bad job. It wasn’t ever designed to do it on this scale. It wasn’t designed to apply to everything. So in this example, yes, platforms need to be adopting it to actually read that metadata, providing they’re not the ones ripping it out during the process of actually supposedly scanning for it, but unless this is absolutely everywhere, it’s just not going to go.
Part of the problem that we’re seeing is, as much as they can credit saying, “It’s going to be really robust, it’s going to be really efficient, you can embed this at any other stage,” there are still flaws with how it’s being interpreted, even if it is scanned. So that’s a big thing. It’s not necessarily that platforms aren’t picking up the metadata or stripping it out. It’s that they have no idea what to do with it when they actually have it. And at the point of uploading any images, there are social media platforms. LinkedIn, Instagram, Threads are all supposed to be using this standard, and there is a chance that when you upload any kind of image or video to the platform, any metadata that was involved in that is just going to be stripped out regardless.
Unless they can all come to an agreement, every platform, literally every platform that we access and use online, can come to an agreement that they are going to be scanning for very, very specific details, they’re going to be adjusting their upload processes, they’re going to be adjusting how they communicate to their users, there needs to be that uniform, total uniform conformity for a system like this to actually make a difference, not even just to work. And we’re clearly not even going to see that.
One of the conversations I had, actually, was when I was grilling Andy Parsons, who is head of content credentials at Adobe—that’s their word for implementing C2PA data—I commented on the Grok mess that we’ve had recently. Twitter was a founding member of this, and then when Elon purchased the platform, it disappeared. And by the sounds of it, they’ve been trying to entice X to get back involved, but that’s just not going anywhere. And X, however we see its user base at the minute, has millions of people using it, and that is a portion of the internet that is never going to benefit from this system because it has no interest in adopting it. So you’re never going to be able to address that.
I’m going to read you this quote from Adam Mosseri, who runs Instagram. On New Year’s Eve, he just dropped a bomb and he put out a blog post in the form of a 20-carousel Instagram slideshow, which has its own PhD thesis of ideas about how information travels on the internet embedded within it, but he put out a 20 slideshow on Instagram. In it, he said, “For most of my life, I could safely assume photographs or videos were largely accurate captures of moments that happened. This is clearly no longer the case and it’s going to take us years to adapt. We’re going to move from assuming what we see as real by default to starting with skepticism.”
This is the end point, right? This is “you can’t trust your eyes,” which means you can no longer trust a photo, you can’t trust a video of any event is actually real, and reality will start to crumble. And you can just look at events in the United States over the past month. The reaction to ICE killing Alex Pretti was, “Well, we all saw it,” and it’s because there was lots of video of that event from multiple angles and everyone said, “Well, we can all see it.”
The foundation of that is we can trust that video. And I’m looking at Adam Mosseri saying, “We’re going to start with skepticism. We can no longer assume photos or videos are accurate captures of moments that happened.” This is the turn. This is the point of the standard. Do you see Mosseri saying this out loud about Instagram as the end point of this? Is this war just lost?
I would say so. I think we’ve been waiting for tech to basically admit it. I see them using stuff like C2PA as a meritless badge at this point because they’re not endeavoring to push it to its utmost potential really. Even if it was never going to be the ultimate solution, it could have been at least some kind of benefit.
We know that they’re not doing this because in the same message, Mosseri is describing this like, “Oh, it would be easier if we could just tag real content. That’s going to be so much more doable, and that would be good, and we’ll circle those people.” It’s like, “My guy, that’s what you’re doing.” C2PA is that. It’s not specifically an AI tagging system. It’s supposed to say, “Where has this been and who took this? Who made this? What has happened to it?”
So if we’re going for authenticity, Mosseri is just openly saying, “We’re using this thing and it doesn’t work, but imagine if it did. Wouldn’t that be great?” That’s deeply unhelpful. It’s his way of deeply unhelpfully musing into some system that will be able to, I don’t know, regain some kind of trust, I guess, while also acknowledging that we’re already there.
I’m going to make you keep arguing with Adam Mosseri. We’ve invited Adam on the show. We’ll have him on and maybe we can add this debate with him in person, but for now you’re going to keep arguing with his blog post. He says, “Platforms like Instagram will do good work identifying AI content, but it’ll get worse over time as AI gets better. It’ll be more practical to fingerprint real media than fake media. Labeling is only part of the solution,” he says. “We need to surface much more context about the accounts sharing content so people can make informed decisions.”
So he’s saying, “Look, we’ll start to sign all the images and everything, but actually, you need to trust individual creators. And if you trust the creator, then that will solve the problem.” And it seems like you’re really skipping over the part where creators are fooled by AI-generated content all the time. And I don’t mean that to say creators as a class of people. I mean, literally just everyone is fooled by AI content all the time. If you’re trusting people to understand it and then share what they think is real, and then you’re trusting the consumers to trust the people, that also seems like a whirlwind of chaos.
On top of that, and you’ve written about this as well, there’s the notion that these labels make you mad at people, right? If you label a piece of content as AI-generated, the creator gets furious because it makes their work seem less important or less valuable. The audiences yell at the creators. There’s been a real push to get rid of these labels entirely because they seem to make everyone mad.
How does that dynamic work here? Does any of this have a way through?
I mean, it doesn’t. And the other amusing thing is Instagram knows this the hard way. Mosseri should remember, one of the very first platform implementations they did of reading C2PA was done by Facebook and Instagram a couple of years ago where they were just slapping “made with AI” labels onto everything because that’s what the metadata told them.
The big problem here that we have isn’t just communication, which is the biggest part of it. How do you communicate a complex bucket of information to every person that’s going to be on your platform and get them only the information that they need? If I’m a creator, it shouldn’t have to matter if I was using AI or not, but if I’m a person trying to see if, again, a photo is real, I would greatly benefit from just an easy button or label that verifies authenticity.
Finding the balance for that has proven next to impossible because, as you said, people just get upset about it. But then how do you define how much AI in something is too much AI? Photoshop and all of Adobe’s tools, they do embed these content credentials in all of this metadata, it will say when AI has been used, but AI is in so many tools, and not necessarily in the generative way that we assume it’s going to be like, “I’m going to click on this. It’s going to add something new to an image that was never there before and that’s fine.”
There are very basic editing features that video editors and photographers now use that will have some kind of information embedded into them to say that AI was involved in that process. And now when you’ve got creators on the other side of that, they might not know that what they are using is AI. We’re at the point where, unless you can go through every platform, every editing suite with a fine tooth comb and designate what we count as AI, this is a non-starter. He’s already hit the point that we can’t communicate this to people effectively.
Let’s pause here for a second, because I want to lay out some important context before we keep digging in.
If you’ve been a Verge reader, you know that we’ve been asking a very simple question for over five years now: What is a photo? After all, when you push the shutter button on a modern smartphone, you’re not actually capturing a single moment in time. The phone takes several photos at different exposures and merges them into a single frame.
This process has gotten so aggressive over the years that people complain about phone photos looking “fake” — but it also means that the picture you get isn’t actually the moment you were taking a photo of, but rather a composite. Some manufacturers take this even farther. Samsung got in trouble a few years ago for simply generating photos of the moon on its phones, and of course Google Pixel phones have all kinds of Gemini-powered AI tools in them, to the point where Google now says the point of the camera is to capture “memories,” not reality. This is a lot, and like I said, we’ve been talking about it for years here at The Verge.
Now, generative AI is pushing the “what is a photo” debate to its limit. If we can’t even agree on how much AI editing makes something an AI-edited photo, or even whether these features should be considered AI in the first place, then how can we possibly reach consensus on what’s real? Camera makers have basically thrown their hands up here, and now we’re seeing the major online distribution platforms do the same thing.
I bring this up partially because it’s an obsession of mine, but also because I don’t think any of these answers are easy, which brings us back to Adam Mosseri, Instagram, and the AI labeling debate.
I will give some credit to Instagram and Adam Mosseri here in that they are at least trying and thinking about it and publicly thinking about it in a way that none of the other social networks seem to have given any shred of consideration to. TikTok, for example, is nowhere to be found here. They are just going to distribute whatever they distribute without any of these labels, and it doesn’t seem like they’re part of the standard. I think X is absolutely just fully down the rabbit hole of distributing pure AI misinformation. YouTube seems like the outlier, right? Google runs SynthID, they’re in C2PA, they’re embedding the information literally at the point of capture in Pixel phones. What is YouTube doing?
A very similar approach to TikTok actually, because weirdly enough, TikTok is involved with this. It uses the standard. It’s not necessarily a steering member, but it is involved. And it has a similar approach, where you’ll get an AI information label somewhere towards, depending on what format you’re viewing on, mobile or your TV, your computer, you’ll get a little AI information label that you have to click in and ascertain the information that you need from that.
So their problem is making sure it’s robust enough, because this doesn’t appear consistently. There are AI videos all over YouTube that don’t carry this and there’s never a good explanation. Every time I’ve asked them, it’s always just, “We’re working on it. It’s going to get there eventually,” whatever, or they ask for very specific examples and then run in and fix those while I’m like, “Okay, but if this is falling through the net, how can you stand by this as a standard and your own SynthID stuff? And you’re clearly using it to soothe concerns that people have despite its ineffectiveness.”
They don’t seem to be progressing any further than just presenting those labels probably because of what happened to Instagram, and now we’ve just got this situation where Meta does seem to be standing on the sidelines going, “Well, we tried, so let’s just see what someone else can do and maybe we’ll adopt it from there.” But YouTube doesn’t really want to address the slop problem because so much of YouTube content that’s shown to new people is now slop and it’s proven to be quite profitable for them.
Google just had one of its best quarters ever. Neal Mohan, the CEO of YouTube, has been on the show in the past, and we will have him on the show again in the future. He announced at the top of the year that the future of YouTube is AI and they have features that they’ve announced like that creators can have AI versions of themselves do the sponsored content, so that the creators can do whatever that the creators actually want to do.
There’s a part of me that completely understands that. Yes, my digital avatar should go make the ads so I can make the content that the audience is actually here for. And there’s a part of me that says, “Oh, they’re never going to label anything,” because the second they start labeling that as AI-generated, which clearly will be, they will devalue it. And there’s something about that in the creative community with the audience that seems important.
I know you’ve thought about this deeply. You’ve done some reporting here. What is it about the AI-generated label that makes everything devalued, that makes everybody so angry?
I think it’s people trying to put a value on creativity itself. If I was looking at luxury handbags and I see that they’ve not paid a creative team—This is a creative company that makes wonderful products, it’s supposed to stand on the quality of all of the stuff that it sells you. If I find that you’re not involving creative personnel in making an ad for me to want to buy your handbag, why would I want to buy it in the first place?
Not everyone will have that perspective, but as someone that worked in the creative industry for a long time, you see the work that goes into something, even if it’s something as laughable as a commercial. I love TV commercials because as annoying as they are and as much as they’re trying to get me to buy something, you can see the work that went into it, that someone had to write that story, had to get behind the film cameras, had to make the effects and all that kind of stuff.
So it feels like if you’re taking a shortcut to remove all of that, then you’re already cheapening the process yourself. I feel, from the conversations I’ve had with the other creatives, that the initial response of thinking AI looks cheap is because it’s meant to be cheap. That’s why it exists. It exists for efficiency and affordability. If you’re coming across with trying to sell me something on that, it’s probably not going to make the best first impression unless you make it utterly undetectable. And if you have a big “made with AI” or “assisted with AI” label on that, it’s no longer undetectable because even if I can’t see it, you’ve now just admitted that it’s there.
That’s a lot of mixed incentives for these platforms. And it occurs to me as we’ve been having this conversation, we’ve been kind of presuming a world in which everyone is a good-faith actor and trying to make good experiences for people. And I think a lot of the executives of these companies would love to presume that that is the world in which they operate, and whether or not the label makes people mad and you want to turn it off or whether or not you can trust the videos of significant government overreach and cause a protest, that’s still operating in a world of good faith.
Right next to that is reality, the actual reality in which we live, where lots of people are bad-faith actors who are very much incentivized to create misinformation, to create disinformation, and some of those bad-faith actors at this moment in time are the United States government. The White House publishes AI photos all the time. Department of Homeland Security, AI-generated imagery, up, down, left, right, and center. You can just see AI manipulated photos of real people modified to look like they’re crying as they’re being arrested instead of what they actually looked like.
This is a big deal, right? This is a war on reality from literally the most powerful government in the history of the world. Are the platforms ready for that at all? Because they’re being faced with the problem, right? This is the stuff you should label. No one should be mad at you for labeling this, and they seem to be doing nothing. Why do you think that is?
I think it’s because it’s the same process, right? What we’re talking about is a two-way street. You’ve got the people who want to identify AI slop, or maybe they don’t, but people want to be able to see what is and what isn’t AI, but then you’ve got the more insidious situation of, “We actually want to be able to tell what is real, but it unfortunately benefits too many people to make that confusing now.” The solution is for both. AI companies and platforms are profiting off of all of the stuff that they’re showing us and making it much more efficient for content creators to slap stuff in front of you.
We’re in a position now where there’s more online than we’ve ever seen because everything is being funneled out. Why would they want to harm that profit stream, effectively, by having to slam on the brakes of development until they can figure out how they are going to effectively be able to call out when deepfakes are proving to be a problem. The methods of being put in front of it, rather than setting up some kind of middle system like the Shutterstock model we discussed earlier, where all press images now have to come from one authority that has to verify the identity of everyone taking them. Maybe that’s a possibility, but we are so far from that point and, to my knowledge, no one’s instigated setting something like that up. So they’re just kind of relying on everyone talking about this in good faith.
Again, every conversation I’ve had with this is, “We’re working on it. It’s a slow process. We’re going to get there eventually. Oh, it was never designed to do all of this stuff anyway.” So it’s very blase and low effort really—”We’ve joined an initiative, what more do you want?” It’s incredibly frustrating, but that seems to be the reason that everything is not developing, because in order to develop any further, in order to actually help us, they would have to pause. They would have to stop and think about it, and they’re too busy running out every other tool and feature that they can think of doing because they have to. They have to keep their shareholders happy. They have to keep us as consumers happy while also saying, “Ignore everything else that’s going on in the background.”
When I say there’s mixed incentives here, one of the things that really gets me is that the biggest companies investing in AI are also the biggest distributors of information. They’re the people who run the social platforms. So Google obviously has massive investments in AI. They run YouTube. Meta has massive investments in AI, to what end unclear, but massive investments in AI. They run Instagram and Facebook and WhatsApp and the rest.
Just down the line, you can see, “Okay, Elon Musk is going to spend tons of money in xAI and he runs Twitter.” And this is a big problem, right? If your business, your money and your free cash flow is generated by the time people are spending on your platforms and then you’re plowing those profits back into AI, you can’t undercut the thing you’re spending the R&D money on by saying, “We’re going to label it and make it seem bad.”
Are there any platforms that are doing it, that are saying, “Hey, we’re going to promise you that everything you see here is real?” Because it seems like a competitive opportunity.
Very small. There’s an artist platform called Cara, which says that they’re so for supporting artists that they’re not going to allow any AI-generated artwork on the site, but they haven’t really clearly communicated how they are going to do that, because saying it is one thing and doing it is another thing entirely.
There are a million reasons why we don’t have a reliable detection method at the minute. So if I, in complete good faith, pretend to be an artist that’s just feeding AI-generated images onto that platform, there’s very little they can really do about it. Anyone that’s making those statements saying, “Yeah, we’re going to stand on merit and we’re going to keep AI off of the platform,” well how? They can’t. The systems for doing so at the minute are being developed by AI providers, as we’ve said, or at least AI providers are deeply involved with a lot of these systems and there is no guarantee for any of it.
So we’re still relying on how humans intercept this information to be able to tell people how much of what they can see is trustworthy. That’s still kind of putting the onus on us as people. It’s, “Well, we can give you a mishmash of information and then you decide whether it’s reliable or not.” And we haven’t operated in that way as a society for years. People didn’t read the newspapers to make their own mind up about stuff. They wanted information and facts, and now they can’t get that.
Is there user demand for this? This does seem like the incentive that will work. If enough people say, “Hey, I don’t know if I can trust what I see. You have to help me out here, make this better,” would that push the platforms into labeling?
Because it seems like the breakdown is at the platform level, right? The platforms are not doing enough to showcase even the data they have, let alone demand more. But it also seems like the users could simply say, “Hey, the comment section of every photo in the world now is just an argument about whether or not this is AI. Can you help us out?” Would that push them into improvement?
I would like to think it would push them into at least being more vocal about their involvement at the minute. We’ve got, again, a two-sided thing. At the minute, you can’t tell if a photo is real, but also, a less nefarious thing is that Pinterest is now unusable. As a creative, if I want to use the platform Pinterest, I cannot tell what is and what isn’t AI. I mean I can, but a lot of people won’t be able to. And there is so much demand for a filter for that website just to be able to go, “I don’t want any of this, please don’t show me anything that’s generated by AI.” That hasn’t happened yet. They’ve done a lot of other stuff on it, but they’re involved with the process behind developing these systems.
It’s kind of more the problem that they’ve set themselves an impossible task. In order to use any of the systems that we’ve established so far, you need to be best friends with every AI provider on the planet, which isn’t going to happen because we’ve got nefarious third-party things that focus entirely on stuff like nudifying people or a deepfake generation entirely. This isn’t OpenAI or the big name models, but they exist and they’re usually what’s used to do this kind of underground activity. They’re not going to be on board with it. So you can’t make bold promises about resolving the problem universally when there is no solution at hand at the minute.
When you talk to the industry, when I hear from the industry, it is the drumbeat that you’ve mentioned several times. “Look, it’s going to get better. It’s going to be slow. Every standard is slow. You have to give it time.” It sounds like you don’t necessarily believe that. You think that this has already failed. Explain that. Do you think this has already failed?
Yeah, I would say this has failed. I think this has failed for what has been presented to us because what C2PA was for and what companies have been using it for are two different things to me. C2PA came about as a … I will give Adobe its credit because Adobe’s done a lot of work from this. And the stuff it was meant to do was, if you are a creative person, this system will help you prove that you made a thing and how you made a thing. And that has benefit. I see that being used in that context every day. But then a lot of other companies got involved with that and said, “Cool, we’re going to use this as our AI safeguard basically. We’re using this system and it’ll tell you, when you post it somewhere else, whether it’s got AI involved with it, which means that we’re the good guys because we’re doing something.”
And that’s what I have a problem with. Because C2PA has never stood up and said, “We are going to fix this for you.” A lot of companies came on board and went, “Well, we’re using this and this is going to fix it for you when it works.” And that’s an impossible task. It’s just not going to happen. If we’re thinking about adopting this platform, just this platform, even in conjunction with stuff like SynthID or inference methods, it’s never going to be an ultimate solution, so I would say resting the pressure on “We have to have AI detection and labeling,” it’s failed. It’s dead in the water. It’s never going to get to a universal solution.
That doesn’t mean it’s not going to help. If they can figure out a way to effectively communicate all of this metadata and robustly keep it in check, make sure it’s not being removed at every instance of being uploaded, then yeah, there’ll be some platforms where we’ll be able to see if something was maybe generated by the eye or maybe it was a verified creator badge, something, whatever Mosseri is talking about where we’re going to have to start verifying photographers through metadata and all of this other information, but there is not going to be a point in the next three, five years where we sign on and go, “I can now tell what’s real and what’s not because of C2PA.” That’s never going to happen.
It does seem like these platforms, maybe modernity as we experience it today, have been built on, “You can trust the things that come off these phones.” You can just see it over and over and over again. Social movements rise and fall based on whether or not you can trust the things that phones generate. And if you destabilize that, you’re going to have to build all kinds of other systems. I’m not sure if C2PA is it. I’m sure we will hear from the C2PA folks. I’m sure we will hear from Adam and from Neal and the other platform owners on Decoder. Again, we’ve invited everybody on.
What do you think the next turn here is? Because the pressure is not going to relent. What’s the next thing that could happen?
From this turn of events, there’s probably going to be some kind of regulatory effort. There’s going to be some kind of legal involvement, because up until this point, there have been murmurs of how we’re going to regulate stuff, like with the Online Safety Act in the UK. Everything is now pointing toward, “Hey, AI is making a lot of deepfakes of people that we don’t like and we should probably talk about having rules in place for that.”
But up until that point, these companies have basically been enacting systems that are supposed to help us out of the goodness of their heart: “Oh, we’ve spotted that this is actually a concern and we’re going to be doing this.” But they haven’t been putting any real effort into doing so. Otherwise, again, we would have some kind of solution by now where we would see some sort of widespread results at the very least. It would involve working together, having widespread communications, and that’s supposed to be happening with the CAI, with the initiative that everyone else is currently involved with. There are no results. We are not seeing them.
Instagram made a bold effort over a year ago to stick labels on and then immediately ran back with its tail between its legs. So unless regulatory efforts actually come in clamping down on these companies and saying, “Okay, we actually now have to dictate what your models are allowed to do and what we are going to have repercussions for you if we find out what your models are doing and not supposed to be doing,” that is the next stage. We have to have this as a conjunction. I think that will be beneficial in terms of having that with labeling, with metadata tagging and stuff. But alone, there is never going to be a perfect solution to this.
Well, sadly, Jess, I always cut off Decoder episodes when they veer into explaining the regulatory process to the European Union. That’s just a hard rule on the show, but it does seem like that’s going to happen and it seems like the platforms themselves are going to have to react to how their users are behaving.
You’re going to keep covering this stuff. I find it fascinating how deep into this world you’ve gotten starting from, “Hey, we should pay more attention to these tools,” and now here we are at “Can you label reality into existence?” Jess, thank you so much for being on Decoder.
Thank you.