Ben Whitelaw 0:00

So Nicholas, for England fans this week, it has not been a fun week. Not to make it about the World Cup again, but, England's World Cup ended in tragedy once more. So yeah, I, I've had to find a lot of solace in the internet this week, to kind of perk myself up. and everyone we ask on the podcast, I wanted to, start with you and see what made you smile as you were browsing the web this week? what's brought a smile to your face?

Niklas Eder 0:23

Yeah, thanks, Ben. as an immigrant in the UK, I was also, cheering for England, of course, as much as I can and was devastated as well. then the preparation for this podcast led me to a few things that are about a serious topic, but then, in terms of language, I think still quite funny because von der Leyen was presenting the new measures from the European Commission that they are considering regarding child safety online. and then sometimes, you know, they tend to use this really grand rhetoric to speak about these things, to have this topic that everyone teams up, where everyone can come together, and then that can end in phrases such as: "When it comes to our kids' safety online, Europe believes in parents and not profits." Or there's another one: "Social media is not a toy. We need age-appropriate restrictions to platforms because childhood won't wait, and once it's gone, we can never give it back."

Ben Whitelaw 1:17

It's good stuff. It's good, it's good. It's cheesy, but it's... You gotta give credit. yeah, I, I'm not sure that prophets or, parents is the, binary choice that she thinks it is, but,

Niklas Eder 1:28

I know.

Ben Whitelaw 1:29

gonna get more into that.

Niklas Eder 1:30

Yeah. But both start with a P, which is also good

Ben Whitelaw 1:34

We love a little bit of alliteration. my kind of bit of fun, the way that I've been calming myself down after last night's game is actually by, going onto The Pudding, which is this amazing visual journalism website that some people might have heard, and, utilizing their new grass mowing game or lawn mowing game if you're from the US. it's as simple as it sounds. It's, it's right out of Mike Masnick's playbook. He, he's always doing these kind of fun games, and it is essentially you have to kind of mow the lawn on this, very small patch of grass in as fewer moves as possible. It's about efficient lawn mowing. And, uh, you know, as a kind of keen gardener myself and somebody who finds being outside very, uh, very calming, I thought this is the perfect way to come down from a big World Cup defeat. So, the Germans didn't do so well either, Niklas, so maybe we can, mow the lawn together.

Niklas Eder 2:23

That sounds brilliant

Ben Whitelaw 2:32

Hello, and welcome to Ctrl Alt Speech, the podcast where we make sense of the major debates shaping online speech, platform power, content moderation, and the future of the internet. It's July the 2026, and this week's episode is brought to you by PwC, whose upcoming 2026 Trust and Safety Outlook Report explores the forces reshaping how organizations are approaching online safety and integrity. In today's episode, we're talking about the UK failing to get the social media ban right, a federal court backing trust and safety workers at the expense of Donald Trump, and for our Patreon subscribers, Europe's move to enforcing against some of the big platforms. Ctrl Alt Speech is the internet safety and regulation equivalent of Match of the Day, Which is a very British reference for those who, who maybe follow along. With me, Ben Whitelaw, the founder and editor of Everything in Moderation, and a special guest host today, Niklas Zeder from User Rights. Welcome, Niklas. How are you?

Niklas Eder 3:24

Thank you very much. I'm great. great to be here

Ben Whitelaw 3:27

Excellent. I've been meaning to get you on the podcast for a while, Nicholas. we met a couple of years ago at Trust Con, and you gave a brilliant presentation about some of the work that you've gone on to do since. you are the co-founder and co-CEO of User Rights, which is one of the kind of fascinating organizations that have sprung out of the Digital Services Act, as one of the out of court dispute settlement bodies, users can come to you to have their content moderation, decisions validated and, and checked essentially. So, it's a fascinating, part of the DSA and you're very well, placed to speak about some of the stuff be talking about today. just give our listeners a kind of sense of, of that work you're doing on a day-to-day basis, this influx of cases you get from users and what you've seen in, in the couple of years that you've been running User Rights.

Niklas Eder 4:14

Yeah, I think in the last two years since we've started, a lot happened. I think on a higher level, one of the things that was really interesting for me to see was how a law that has been passed by the European Union ultimately becomes a reality. So my background is in legal philosophy and constitutional law. That's where I wrote my PhD. And then at some point, you know, I decided to set up this out-of-court dispute settlement body straight after the law came into effect that really is the basis for these out-of-court dispute settlement bodies. So basically, how it works is that there's a provision in the DSA, Article 21, that says that there can be organizations that are certified by national, digital services coordinators in order to take on a specific function in the content moderation ecosystem, and that function is to provide independent review of content moderation decisions. And we had to prove our expertise, our independence, our impartiality, the ability to work efficiently, and so forth. But then ultimately, we have that mandate, and once we have that mandate, platforms have to work with us. And they don't have a choice, but they have to work with us. They are compelled to do it under the law. and then I think for me, you know, as someone who was in academia for a long time, it's just been really fun to see what happens after a law is passed and how to put it into a reality. I think And you look at it more from like legal philosophy or constitutional law. So you think about it as legal interpretation. You ask yourself, "What happens if a law is done?" And then how do you interpret what it really means? And then I think, you know, there's a bit of a theoretical debate almost around it on how to do that. And then there's a very practical process. And now, you know, coming from that, a bit of the theoretical discussions around it, but then going into the practical process, that is really fun and it's really, exciting. And ultimately all of that is mixed with a bit of an entrepreneurial spirit, I would say, because kind of what this law does, what Article 21 does, is to create what, you know, you could call a regulatory market, which is basically that the legislator says, you know, "There is this thing that we wanna achieve, which is to provide everyone for free with an efficient remedy against decisions from platforms to counter the power balance a bit, but we are not quite sure how to do it. And you know, we don't wanna do it ourselves, and our courts can't do it. So how about we create some sort of market around it where the legal provisions set the framework for this market, and then we just see what happens, and we let people, you know, go out and about and try different things." And then we are one of the organizations that, that's trying out things. And, yeah, that experience has been super interesting. So we grew in like two years from two people to over 35 people, and now we are reviewing thousands of disputes. We built a nice legal tech product that allows us, to do that. And, we see that it can really work and help users when platforms also, cooperate, you know, in an efficient manner. And there we have seen discrepancies, differences in the first, year or so. In 2025, we have published a big transparency report, and there you can see that with some platforms, with TikTok, for example, it worked a lot better and a lot more effectively than with Meta during that time. and then we try to be as transparent as possible about all of this so that people understand what is this strange thing with this terrible name of out-of-court dispute settlement bodies, which is ultimately quite a fun thing that hopefully does, something useful. Yeah, so, so far it's been a, it's been a good experience.

Ben Whitelaw 7:32

Yeah. Fascinating. Thanks for, for sharing. I mean, I, w-when kind of Article 21 emerged and people started to kind of understand a bit better, I, almost likened it to the kind of advent of the report button. there was a time before on social media where there was no report button, and then the report button emerged as a way of, for users to, ask for a platform to review a case. Article 21's, I think in a similar very different way, but has the potential to be that significant, if it's adopted widely. You mentioned about the transparency report, Nicholas, and how it was difficult to get some platforms to kind of engage with the process, and how that was potentially kind of hampering, how many people were using Article 21 and, and the, the services that you and these other groups provided. can you go a bit more into detail as to what that's been like? what's the actual process of getting somebody at a platform to work with you, to give you the data to be able to do your work?

Niklas Eder 8:23

Yeah. So there's, a lot of different kinds of interactions we need to have bu- with platforms to work effectively on a case, and the first one is to have data exchange with platforms. And, you know, that requires then for us to provide them with an information package, and then they share something with us. And then already in that process, there's a lot of obstacles you need to overcome to make that work. one of them is legal. the question is, what is even the legal basis for data sharing in that regard? That as you can imagine, platforms, you know, are very careful when it comes to data sharing, and, they, can also use that step to say, "You know, we wanna be careful around data sharing. We need a clear legal basis," just to then suggest, you know, perhaps we should enter into this or that agreement around it. And then all of a sudden you find yourself, with an agreement on the table that is a vendor partner service agreement, for a platform, while you're like, "I'm actually not your service provider, and I'm not your vendor partner, and I don't wanna sign this." and then you have to start negotiating, around that. And then if the platform says, "Well, you know, it's pretty wild that you just show up here, and now we are supposed to share data with you while we don't even have, like, a contract or something like that." and then, you know, we can acknowledge that and say, "Well, it is something that is pretty new and that is also pretty unique," but, that's what the law requires. And so we go back to the regulator, and tell them, "You know, we have this issue, with this or that platform, and we need to go past it, and we feel like this is going to shape what out-of-court dispute settlement is in the future. Are we, are we service providers ultimately that are at, at the mercy of platforms or not?" and so in that process, for example, we managed, to get our approach through, that we're like, "We're not gonna sign it. we rely on the law to work with you." And that was a big milestone, for example. And then I think, you know, some others are more practical. It's like it is relatively difficult to make sure we get the right information from the complainants, then, then we share that information, and platforms can identify the relevant piece of content and so forth, and so that is a bit of the nitty-gritty work. but there is also a bit of a danger. If that takes too long, it means that too long nothing is happening, and too many users that come to us are ultimately disappointed. And we also have a bit of, you know, a period of maybe, you know, two, three, four years where we can prove that this works and that this makes sense and that this should be something that should be supported and kept. And when there is further delay and delay and delay on many things, it just undermines kind of like what we do. and so, I think with, you know, in the cooperation with platforms, sometimes you see that they're quite cooperative. They seem to wanna make it work, and others, you know, are just like these big beasts and these giants, and until they move and do something, it takes forever. And you never know if it is a stalling technique or if it just takes forever for them to do anything, because they're almost like big bureaucracies to a certain extent. but anyway, need- things need to happen. but I think- ultimately what I'm also trying to learn is to be a little bit patient on some issues. Like it is something that is very new. It's something that is quite remarkable that platforms, you know, have to work with us. They have to pay the fees, so it's like for free for everyone who comes to us, which is also quite remarkable. And I also do understand to a certain extent that these platforms are also like, "Well, we also kind of wanna see that you're doing, you know, a good job, that you're serious people, and that it ultimately also for us makes sense to somehow work with you." and I think that's what we're trying to have as a selling point at User Rights also, that we're saying, "You know, we wanna do something that complements your own processes," right? You have automated content moderation at the first level, then you have appeals where maybe you have, humans who look at it briefly, and then what then? and then maybe you can go to an out-of-court dispute settlement body that does a more thorough review, which can inform your systems, which complements them. Maybe which lets you, you know, use more AI in the first instance. I think a topic we're gonna speak later today about, So it is a, it is a process. It's a long process. I think we are slowly, yeah, making, a bit of, progress.

Ben Whitelaw 12:04

Yeah. You're very much at the kind of coal face of some of these, uh, you know, regulatory challenges, and we're gonna talk a bit today about new regulation that's coming in, in which, folks like you will have to, to figure out, a- as you've done. unfortunately, you're not gonna be at Trust Con next week, which is a, a crying shame, but Mike and I will be there. We're doing our live recording of Ctrl Alt Speech on Wednesday afternoon, so if you're a listener of Ctrl Alt Speech and you want to come along, please do. be great to see people in person. We've also got a special gift for any Ctrl Alt Speech listeners who come and find Mike and I in the chaos of Trust Con. It's, there's thousands and thousands of attendees, so if you, if you manage to do that, you will have, very much earned your special mystery gift, which will no doubt disappoint at least some of you. we're gonna jump into stories, but before we do, just to flag that, before we reach the end of today's free episode, we'll be talking to PwC's Dan Hayes about the big themes at Trust Con. he'll be giving a few tidbits from PwC's upcoming Trust and Safety Outlook Report, which is out next week, so stick around for that. we've also got a, a longer version that we'll be recording next week, going deeper into the report So l- let's dive in then, Nicholas. We've, teed up the kind of, you know, emerging regulatory ecosystem, what's happening. You can't have escaped, the news recently about various social media bans happening across the world, most notably, as we've talked about, in the UK and increasingly in the EU. And we've had a couple of announcements this week that I would say, add a bit of meat to the very bare bones that was the initial announcements, from Keir Starmer earlier this year, and actually some teeing up Leyen did, couple of weeks ago. So we're s- we've started, in the course of this last week, started to see more information about what these bans will look like. so in the UK, we'll start there. The UK, there were some very wishy-washy statements about what a ban could look like, and there was a, plan in July to expand a bit more as to what that actually would look like and how the ban would work. And we've actually started to get a bit more of that. So this week it was, shared that 16 and 17-year-olds will be unable to access social media platforms that fall under the ban after midnight and before 6:00 AM. But crucially, this is a setting that would be able to be turned off, presumably within the platform settings themselves, which we'll come onto, was not very well received by a bunch of people. the government have also expressed a desire to disable certain features, that kind of, exacerbate time spent on the platforms such as autoplay, such as infinite scroll. Those are two features that we will, we'll talk a lot about in the second half of today's podcast as well. those are things that are going to happen. what the government has said won't happen is that there won't be a VPN ban. This is something that had been, mooted because of concerns about teens circumnavigating, circumventing, the kind of measures to ban them from platforms. there's been a, a small but very vocal group of people off the back of the Australia social media ban who've, called for VPN bans and it's, something in, kind of political sphere that has a lot of traction behind it, I'd say. But the government has said they're not going down that route, and they also haven't said which platforms are in and out of scope. So despite saying that there would be kind of more detail shared in July, we're still yet to hear which platforms, will definitely be in and out. we know that the big ones, Meta, um, TikTok, YouTube will be in there. We don't know beyond that whether other gaming sites or other kind of large forums will be in there as well. It's, it's really still up for grabs. all of those measures, it's fair to say, Nicholas, have been criticized by lots of different groups, which we can talk a bit about. what was your reaction to, what the UK have unveiled this week?

Niklas Eder 15:52

Yeah, I think it's really interesting how we're now seeing more proposals, that, deal with the topic of age restrictions that perhaps are trying to be a bit more nuanced of what we have seen in other countries, such as, Australia pretty much leading, the way with a straight-out ban, for children, and I think youth until, the age of 16 or so, which was a very drastic and, you know, for the first kind of country implementing like, implementing like that perhaps went really far. and I think, these, efforts are on the one hand driven by governments also wanting to, take action on something that hopefully everybody, you know, can agree on and, like, protecting our children is probably something that everyone should easily be able to agree on, which is something that we know from the trust and safety space, quite well. for example, when you're traveling to Trust Con, you have to be more or less very careful as soon as you approach the American border it feels like to speak about trust and safety and what the responsibilities are of social media platforms and so forth, and you're gonna try to frame everything that you do and talk about under like child safety or so because hopefully that is something that everyone can actually get on board with and that's where we see, you know, a bit, more bipartisan support also in the US. So I think in general it is a topic where there was a lot of this support but then on the other hand these very drastic measures of straight-out bans I think have also received a lot of criticism and I found it, quite interesting to see who this criticism also comes from and it's a lot of NGOs, it's a lot of, organizations that are also, you know, like experts and care about children's, wellbeing and that these organizations often are also rather skeptical about it. And so, uh, looking at the topic, in a manner that there is a bit of progress perhaps like globally on how more nuanced ban can come into effect, that's how I would contextualize perhaps also this what we see in the UK. you, you know, you're saying there's a bit more meat to it but then there's also still quite a lot that is uncertain, and so maybe that is, part of the, process.

Ben Whitelaw 17:46

Yeah, I think that's fair. I mean, it's, i- many ways the social media ban was kind of designed to placate the, part of the public that had children. I think that's fair to say. I think a large amount of the push for a social media ban came from parents, and particularly the kind of smartphone-free childhood campaign. And what's interesting about these announcements this week is that they're not particularly happy about, of any of them. Um, you know, the i- the idea that a, that there's a midnight curfew is, I think, probably supported, but if, if it's a setting that can be turned off, then that doesn't do the job that the parents are looking for and doesn't do the job that the ban kind of set out to do. know, I think that the fact that they're not going down a VPN route is probably, welcome from internet experts, but again, not something that your average parent who doesn't necessarily know about of VPN technology or the, or the downsides of, removing access to VPNs, that's not gonna land very well with parents as well. So I think it's a pleasing move in some senses that the government is being a bit more careful about how it approaches this ban, but it's not gonna go down well in some quarters. And, again, like you say, experts have said this is a kind of piecemeal approach to, a social media ban that i- isn't necessarily gonna end up, yielding good results. By contrast, Nicholas, you know, look at the EU's approach now, and we teed up on the podcast a couple weeks ago how, the European president, Ursula von der Leyen, was planning to announce a harmonized approach to social media bans in September, but was waiting on a new report from what's called the Special Panel on Child Safety Online. That dropped this week, and off the back of that, she's unveiled what's seems to be the path forward for, the kind of wider bloc. top line is that there's gonna be a strict ban for under 13s, which again, is similar to, what a German commission, a few weeks ago had recommended and wh- a few other countries are going down that under 13s route with the potential for there to be kind of national measures for 13 to 18-year-olds. So if a country wants to particularly target that, teen, 18 group, then they can do. it's a much more graduated approach. It, you know, it's designed to ban social media for the youngest children, but allow access, as you get older, which is something that the experts have called for. you're more obviously based, a lot in, in the EU, very familiar with kind of European law, Nicholas. So what's your take on, the announcement this week and how it compares to what the UK have moved forward with?

Niklas Eder 20:18

Yeah, I think reading the announcement, a lot of it is still really unclear. Even when it comes to the rules for social media users under 13, I don't think it's entirely clear, what is meant. Basically, we have now this report that has been published, and we have, von der Leyen, you know, issuing a statement, in relation to this report. But in this, statement, she's also saying that she's looking forward to reading the report, so it's not entirely clear if she, you know, endorses now the report or not. I think for her, clearly this report is going to inform what they're going to do, but how exactly is still pretty uncertain. I think even when it comes to the category of social media users under 13, it's still on the table that they might be able to use, social media, if they are under parental or caregiver, supervision. So it seems like they are not, you know, necessarily endorsing a hard ban. They also didn't use that language, But more something like a coordinated minimum starting age, what I am also not exactly sure what that is, supposed to be or supposed to mean. I think So far it looks like they wanna make this attempt where on the one hand they don't go for the straight out ban and, in that manner react to all the criticism that has, existed, because of that. And then on the other hand, they are trying to look for solutions, but then what can these solutions be? And that can be, well, you know, if there's parental control and caregiver supervision perhaps, you know, then that is fine. At the same time, it's really interesting to read this mechanism, also in the context of the investigations, and the enforcement actions that the European Commission has recently taken against Meta, which was a lot about addictive design and which has a lot, to do with that because, uh, and we're gonna look at that a little bit later in more detail. But basically what the European Commission says there is that one of the mitigation measures which Meta has in place in order to protect, also children from, you know, addictive designs and so forth, one of these measures that they have in place are parental controls. But then it says in its own press release, Parental controls are only effective if parents and guardians possess adequate technical expertise as well as devote effort and time to understand them effectively." So it seems like they are criticizing these sort of mitigation measures that are in place to a certain extent in order to mitigate some harm. At the same time, they maybe wanna lean on them a bit when it comes to, you know, thinking about how, children under 13 even might be able to use, social media. So, I think, you know, that, that's where you still see the challenge. Like you, you're not gonna have the perfect solution for it and then saying, "Well, you know, we also, want to solve the whole thing not by straight out banning it, but putting responsibility on the parents." But then you say, well, in the other report you say, "Well, the parents maybe they're not always so responsible and have the relevant expertise." So, I think it's, also hard to solve it, in that way. And so to me, the European proposal also more looks like, a next step in engaging with the topic, but it is still pretty far away from anything really, concrete.

Ben Whitelaw 23:20

Yeah, it's a really good point that there is this kind of, tension, I guess, between how heavily EU is leaning on parental controls. And I think by, highlighting that y- need to have time and technical skills and an understanding of how to use the parental controls, you're the EU admitting that it's not just about what happens on platforms. You know, th- those ideas of, how much time you have depends on a lot of other things that exist outside of the platform. The platform can't give parents extra time. And so this is, where, that nuance is, really important. and where I think, yeah, as you pointed out, they get kind of slightly caught up in the narrative that they're trying to, to spin there. The other thing I'd point out is just kind of how heavily, the EU's approach relies on age assurance, which I think is kind of very, very clear in each of the, throughout the, the report and some of what, von der Leyen says. You know, they have this age assurance app that's, coming in. It was r- released a few months ago. It was touted as being kind of ready to roll. It was a matter of hours before some security researchers were able to, hack it and, it was deemed not to be very secure. So there is this also tension in, how much they're relying on this age assurance app working, plus, all of the, caveats that come with age assurance around privacy and kind of surveillance, and there has been criticism, that the EU is relying so heavily on this app to, keep children safe and, and keep users from harm in the way that it hopes to. So that's the other, the other tension, the other kind of potential pitfall I think it might find itself in, is that, users don't like that approach, that blanket surveillance approach, which obviously has its downsides.

Niklas Eder 24:57

Yeah. and I think there's also another, aspect of this which is quite interesting. Thinking about the age verification app, you know, you mentioned all the challenges that come with it and the flaws that it already has. I think, this shows also something that here clearly the European regulator needed a solution for age verification, and then they said, "Okay, we're gonna build this app." But it's not always clear whether then the regulator actually doing the things that they need in order then to enforce potential laws themselves is the best possible idea. And I think, you know, in other areas or in other markets, you rely more on platforms than providing solutions to that, which can of course also be criticized, or on vendor partners or so forth. I think what that allows is that then there is ultimately a bit more, you know, experimentation, and then perhaps there's a few that work better than others, and then over time you have see, you know, what would be ma- maybe a good industry standard. I think that approach was not, taken in, in that regard to the commission's age verification app. I think in other areas you try to see that, that the European Commission is then trying to shift work, on the platform. For example, when it says, you know, from the age of 13 there should be like a gradual phased access, to social media. it's conditional on platforms providing their services are age-appropriate and safe for teenagers. So then they wanna say, "Well, well, you have to prove something to us, and we don't really have to prove, to you that it's, that it's harmful in a certain way," which, I think then tries to play the ball into their court.

Ben Whitelaw 26:24

Yeah. And we've seen this, as an approach taken in, in some of the kind of Canada approach to social media bans as well. And I think increasingly we'll see that in the UK as well, where there's almost like a proof of safety, and that certain features are deemed safe for certain segments of the user base, and that will, open up the platforms to particular user groups, part- particular aspects of the market, that they want to be in. So yeah, I would agree with that shift. I think to kind of summarize or characterize the two different types of, what we've learnt this week, I think UK has been, uh, fairly piecemeal approach that has not gone down particularly well. and in the EU's slightly kind of higher level approach has been much more welcomed in some corners and some areas, but still doesn't have the level of detail that I think, people are looking for at this stage. And, and again, these processes are happening incredibly fast. we know that the UK social media ban is coming in by spring next year. We're expecting von der Leyen to, give a more formal, update and announcement in September about Europe's direction of travel in terms of a ban. So these things are happening at speed and, I empathize with the policy folks in both Westminster and Brussels who are having to, to work on this right now 'cause there must be a, a large inbox that they're working through. brilliant. So, so, we've gotta move on now, Nicholas, to talk about a few other stories that, we found eye-catching this week. These are stories that we feel like are worth knowing about, that we wanna put on our listeners' agenda. And the first is a topic pretty close to your heart. it's one of a, an overzealous AI moderation leading users to being blocked. Talk to us about story and why you found it interesting.

Niklas Eder 27:58

Yeah. It's a, it's a really interesting story. Like it's, uh, at the first look, I think it's a relatively ordinary story in the sense that, you know, platforms do content moderation, and they often get it wrong when they do content moderation. I think the first thing that is interesting about this story is that Discord comes out and just like publicly also like speaks about what really has gone wrong, how many accounts have been impacted by a particular error. I think the nature of the error is also interesting. So basically, what Discord has said is that about 8,000 accounts over the past two months have been banned for posting harmless images, such as video game textures that could also, for example, be images that show checkboard patterns or something like that. And then first thought is like, "Why in the world would an AI system, you know, ban these sort of, accounts?" And account ban is a very drastic measure. It's probably the most severe measure you can take as a platform, and you should have good reasons for that. and then the reason for that is probably that, images such as checkboard patterns, do bear resemblance to tactics that are used by, bad actors to conceal content such as, you know, CSAM content or so forth, and that's probably where the error, comes from. still I think, you know, that story then raises the question of what is the role of humans, in moderation systems? And then the first reaction in these sort of debates is always like, "Oh, you know, AI can make mistakes, and so perhaps we need the human in the loop, and then they can look at it, and then they will see directly, that these, check bots are not CSAM content." And I'm not sure if that is the best answer, to be honest. Like, first of all, if you then here also have a human in the loop, and then, in cases where they look at check bot content where that then really conceals, uh, CSAM content, that is also quite absolutely terrible because they, you know, just the idea that humans also have to review the CSAM content is also not great. And of course, you want the best possible AI systems in order to detect CSAM content, and, you wanna err on the side of removing content when it comes to CSAM. So it's also not as, easy as that, and I think putting the human in the loop in these scaled content moderation processes is not necessarily always the most, in the end, the best solution. I think what's interesting, is to think about more, human and the loop. Sometimes, you know, people are also speaking about, and it's a bit more than about a different concept where you say, Let's just admit that we need AI-based content moderation. We need scaled content moderation. It has to happen efficiently. It has to happen, fast, and it's going to make mistakes. So what are we going to do about the mistakes?" And another approach, rather than to saying in every individual moderation decision also you need to have then a human in the loop, is to say, Why don't we provide people who are really upset by a content moderation decision with effective mechanisms to then get, you know, remedies against these decisions?" because if you care about a content moderation decision such as, you know, your account being removed, then probably also you can do something about it. but then you need, places to go. And I think, yeah, that is of course also where platforms are supposed to have internal appeal mechanisms that users can go to. What we see is that Quite often these mechanisms are not a lot more effective than the initial review, and they might make similar mistakes. And then, you know, you end up without a remedy. And I think that's, for example, where the European Union's idea of out-of-court dispute settlement came into place because it's that very specific situation that, uh, someone could not really, you know, resolve their content moderation problem by engaging with the platform. They need an independent place to go, and then platforms have to work with that independent place so you do have access to a remedy. And that's also how we try to frame user rights to a certain extent when it comes to providing, the value add, for platforms that we say we understand that you need to do automated content moderation, and we know that you're gonna get it wrong. But those people who are really upset by the wrong decisions, they should have a place to go, and then that is what we do, you know. But then also working with us and working with us well and ensuring that these errors, especially where people are upset, really upset by these errors, are corrected, that is a good mechanism. And it, to a certain extent also then, you know, legitimizes you using AI in the system in the first place, because ultimately where there are significant errors, there should be correction mechanisms.

Ben Whitelaw 32:10

Yeah, I, I agree that this, the way that kind of Discord have, have opened themselves up to talking about this is, really interesting and, you know, I'm not sure how many of these 8,000 people are in Europe and therefore eligible to come to User Rights, but that's if you are listening and you've been affected by this, you know, you know where Nicolas is. Um, o- o- other ODS bodies are available, I should say. We, we um, but there is this kind of sense of, maybe platforms ending up doing a little bit of what you do internally that I've been thinking about. I wonder if working with ODS bodies over time will make platforms realize, okay, we find the data sharing aspect hard. the legal aspect of, giving you data is some of the most tricky parts of it. We don't like the ongoing engagement. Why don't we just do this internally better? is one thing I've been thinking about, or, or at least to some degree improving how they do appeals processes, because we hear all the time, you know, a news organization or a user asks for a post to be taken down on appeal. It's not done so. It's only done when, the report is published on the news website. So I just wonder how much of, what you do might end up being something that they say, "Well, we can do ourselves."

Niklas Eder 33:18

Yeah, great question. And I also absolutely do not understand why platforms don't have better mechanisms in, in place, particularly when it comes to account bans, because ultimately they are banning and kicking off their own clients from the platform pretty much. Like, why do they do that? And why do they not triple check that this really was justified? so I am not really sure why that, you know, is such a persistent, issue. I think one of the effects also of having to work with out-of-court, dispute settlement bodies with these external organizations, where users then can also go is, as you say, that this is rather painful for platforms to do. They have to pay for it. They have to pay the fees for it. They have to do the data exchange. It's a painful process. So perhaps the fact that this process is painful is also hopefully play some role in platforms trying to improve their own systems, because ultimately they can also look at it like, "What is more costly for me? Do I have a lot of people running to these organizations, and I have to work with them? I have to set up work streams in order to work with these organizations." And then the people from these organizations, they are gonna run around if I don't work effectively with them, saying, "These platforms don't comply with the law, with Article 21, because they don't engage with us in an effective manner that increase additional compliance risks." so ultimately, maybe there are, there are more incentives, along the lines of what you mention. And it's also something that we would welcome because ultimately it is about just getting, you know, i- the end, better content moderation systems. so yeah, something that we also hope, is going to happen. We also allow, for example, platforms when we contact them with a dispute or with a complaint to look at it themselves first. and then often, TikTok, for example, often looks at it carefully, perhaps more carefully as they did initially, and say, "Oops, we got it wrong."

Ben Whitelaw 34:56

Mm-hmm.

Niklas Eder 34:57

for us is a good outcome because then we say, "Then it's resolved." You know, we can close the case. No one is claiming anymore that there was a violation or that there was no violation. There's no disagreement, and the user, they got what they want, so that's good already. But, of course, the question why do, does there have to be this additional loop rather than it just being more, um, effective internally is a good question.

Ben Whitelaw 35:17

Yeah. Yeah. Not to do you out of a job, but, um, that, that would be that would be an interesting development to watch. fascinating story. Thanks for talking us through that. my eye-catching story, Niklas, is, one that I was very glad to see happen, and this is the news that a federal judge in the US has blocked the Trump administration from impor-enforcing a frankly mad immigration policy against non-US researchers, trust and safety workers, and fact-checkers. people might remember that last year, the US State Department announced this policy in which they would deny visas and kind of citizenship applications to people working in the trust and safety space, in fact-checking and mis- and disinformation, people doing important research for the wider field, for what they called censorship of protected expression. it was widely thought to be a bit of a scare tactic, in this kind of broader cultural, narrative around, platforms doing content moderation. Came nine or 10 months after Mark Zuckerberg did his famous speech with his big watch and his, gold chain, in which he said he was gonna row back on doing content moderation and, encourage more speech, whatever that meant. And so, that was the, kind of state of things at the end of last year. the Coalition for Independent Tech Research, CITR, sort of challenged that, and they also moved for a preliminary injunction to ask the judge to halt any enforcement of that policy until the main lawsuit was concluded. And it's that preliminary injunction that we heard about this week, and, pleasingly for people who work in this space, no researchers, fact-checkers, or trust and safety workers will be deported, will be, you know, have their rights revoked until that lawsuit has concluded, which is, I think, really pleasing. and the US judge who, was presiding over it said that, based upon CITR's evidence was likely that the policy violated their First Amendment to do this important work and was likely to censor, researchers based upon their viewpoint, which is, is obviously, goes against the First Amendment. So this is actually something, Niklas, that kind of caught up a bunch of people it caught up some Germans within this, case. It caught up some people who are, Brits who also do work in the US. the, founders of Hate Aid, and leading figures for Global Disinformation Index and the, the Center for Countering, Digital Hate. it feels very personal against Europeans, so I'm particularly glad that this has not been allowed to, proceed.

Niklas Eder 37:43

Yeah, I couldn't agree more. and you know, initially I was always thinking as a European at least I'm privileged in a way that I am not in the US and I'm not an American, and if the worst thing that can happen to me is that they don't let me travel, well, at least I'm not in the same mess as the, I don't know, some of my American friends are in to a certain extent. But I think, you know, these laws and rules, they have been incredibly intimidating. and I didn't fully understand on a personal level also what it means for people to be impacted by it and, you know, how, yeah, intimidating it can be. I was-- But I spoke to Josefine Ballon, for example, from HateAid, and then to see, you know, how that impacts her and then how such a big geopolitical, question all of a sudden becomes a personal issue, not by accident, I think on New Year's Eve or something like that was when these bans were ultimately announced. So that is clearly, a way of trying to inti-intimidate, people. And I have been thinking a lot about, you know, when I would travel to the US if I wanna do it, if I would feel like it's going to work out, how to make plans and so forth. We wanted to go in, September for like a nice, uh, a nice holiday, but then ultimately we thought perhaps like risk is too high, that it's just not gonna work out, so maybe let's do something else. And that is, you know, is, uh, frustrating. I think it's important, you know, to, to keep the relationships with everyone alive as much as possible, and of course we will go back to the US. but I think in this particular lawsuit, what I really enjoyed was to see that maybe the word censorship is finally used again in a manner that makes sense which is about this, is really about censoring, you know, um, researchers, or not censoring, but you know, putting restrictions on them based on their work, based on the positions that they take. and I think in that context it's important to basically re, uh, I don't know, reuse the, the vocabulary that is abused so much, to describe, what it actually means.

Ben Whitelaw 39:31

Yeah. And, and so did, the Hate Aid founders, did they say that it had really affected what they did, where they traveled, what they'd been up to?

Niklas Eder 39:39

Yeah. I mean, I don't wanna speak too much for them, uh, also in terms of, you know, how, how they feel about it or so personally. It was just like, just speaking to someone who's directly impacted by it, does put a different light on it, especially because you also never know how far it's going to go, right? Like, if you have restrictions to enter the country, that's one thing. If you're on a sanction list, that's a different thing because then basically American, companies are not, allowed, I think, to work with you anymore, so probably not gonna have a credit card anymore. You can't use Mastercard and these things. And then that way, like it would really creep into your life beyond not being able to travel to to the US. and that is pretty intense.

Ben Whitelaw 40:12

Yeah. No, makes sense. And I think, you know, it's, ironic in some ways that this comes just before so many trust and safety workers and researchers are gonna head to the US to San Francisco for Trust Con. you know, there's, there's something kind of quite poetic about that in a way, I'd say so. But yeah, a very welcome, update on that long-running story. so that brings us to the end of our free episode. Um, we're gonna talk more as Nicholas teed up about the European Commission's preliminary enforcement against Meta, which was announced at the back end of last week. before we go to that special section as part of our Ctrl-Alt-Speech Plus, I had a chat with Dan Hayes this week, who's the principal partner of Strategy&, which is part of PwC's network, about where the future of trust and safety is heading, what he expects to hear from practitioners next week at Trust Con, and why their new Trust and Safety Outlook report is a must-read to understanding how everything we talk about here on Ctrl-Alt-Speech fits together. Do have a listen Dan, it's fantastic to have you on Ctrl Alt Speech to discuss the exciting upcoming launch of PwC's annual outlook report on the trust and safety industry. Before we dive into that, for people who may not know PwC and its, works, the specifics of its practice, just give us a bit of a sense of what you're focused on right now.

Dan Hays 41:36

Yeah, thanks Ben. Appreciate you having me here. so PwC works very extensively in the trust and safety space. In fact, our corporate values and mission are focused around building trust in society and solving tough problems. So trust and safety is just a natural fit for us. So across the trust and safety world, we really work end to end across the life cycle. So that includes everything from working with companies up front to help shape trust and safety policy, whether it's public policy or company policy. we do a lot of work around trust and safety strategy. what is your operating model? How are you going to integrate trust and safety into products and services? and then a lot of work downstream in terms of both optimizing trust and safety operations, making sure they're working efficiently and effectively, including using new technologies like AI, and, you know, helping on the very back end of compliance controls, making sure that everything we're doing is in line with what our policies and practices say that they should be. so lots of different work across many of the major online platforms, AI platforms, and others across the trust and safety universe.

Ben Whitelaw 42:52

It, it makes sense then with, that kind of full spectrum of, services and your focus to be at Trust Con. It's, it's the fifth year, of Trust Con this year. What do you expect will be the biggest kind of trust and safety talking points at the conference?

Dan Hays 43:05

Well, I mean, what a year for trust and safety, and, you know, it's hard to believe that we're already into the fifth year of Trust Con. this year I think there's a few themes that we're really on the lookout for. one of those themes is, of course, the inevitable AI. But for trust and safety, AI really takes a couple of different forms and fashion. You know, first of all, is obviously a, trust and a safety challenge, because AI and the mere existence of it makes all of our jobs, all of us who work in trust and safety, all the more difficult. but AI is also an important tool set for us in delivering on the promise of trust and safety and executing on the policies that so many platforms and products have in place. So we think that's going to be very much front and center, way more even than it was last year. I'd say a second one that we're very much on the outlook for is this whole issue of what I would just deem, safety for protected communities, and in particular this year, children and minors in general have been so, so much in the focus, right? We've seen laws being passed all over the world that are getting into, how we protect children online, and in fact, whether minors or young adults can even engage in many online platforms. and there's a lot of mixed opinions on that. I know you've had many of them on the podcast here. A-and, and so I think we need to really be, extending that debate, and I expect there's gonna be a lot of conversation about what is right and what is wrong, maybe to be very, uh, binary about it, in this whole world of, online child safety. So those are just a couple of the big themes, but certainly there are many more and lots of hot topics in the trust and safety world.

Ben Whitelaw 44:55

Yeah, for sure. One particularly looking forward to some of those, thorny child safety debates which, you say are not, clear cut always. I mentioned, Dan, that you've got this trust and safety outlook report that you're launching at the conference. it's not the first time you've been producing a big report like this. you've done it a number of times before. I don't want you to kind of give anything away, uh, we're a few weeks out of, Trust Con, but what kind of trends or signals are you seeing, already? And what, what is the report going to kind of unpack for people that they might not already know?

Dan Hays 45:26

Yeah, so th- year's PwC Trust and Safety Outlook, you know, is really, focused on taking a forward view of what some of the major trends are expected to be in the year to come. And, we like to look at this and look at it across the very broad spectrum of platforms that we work with. And, there's a few things in there, in addition to some of the themes you just mentioned, that really stand out. one that, that I'm really excited about is the notion of how we engage communities in helping to set the rules for how platforms and how, even AI LLMs operate. what is the way that we engage the community in order to inform platforms and inform the rules and the policies that get made? the reality is none of us really want any big corporation or even small corporation, making the rules of the road for how we engage, and it's not a one-size-fits-all type of approach that will work for everyone. You really need to engage the community of users, the community of customers, of advertisers, of other stakeholders in order to help shape policies and practices for online platforms. So that's one big thing that we're gonna be spending a little bit of time talking about and understanding. another one about taking AI and looking at its impact on trust and safety, and really looking at, for example, how you can reinvent your trust and safety operations using things like agentic AI. We've all heard lots these days about, you know, massive layoffs of content moderators and such, but it's pretty clear that that's not the answer. The answer is to have smarter, more well-informed agents who will be able to handle the ever-growing volume of content moderation decisions that are needed, but to have it with an appropriate level of training, of human-in-the-loop behavior, and of continued support from human moderators who can deal with the, context-rich and sometimes ambiguous type of decisions that need to take place. So, those are some things that we're really focusing on, and then starting to look even further out into the horizon about how this is reshaping the level of risk in the content moderation and trust and safety world, and what are some of the actions that, platforms and product teams can be taking around this?

Ben Whitelaw 48:01

Yeah, really interesting. I mean, one thing that kind of often comes up, which I'm interested in your thoughts on, Dan, is this idea of trust and safety as a, cost center, and the kind of return on investment of trust and safety. It's, been a perpetual conversation at all the Trust Cons I've been. Just wanted to get your, thoughts on that because I imagine it's something that a lot of platforms and, companies come to you and ask

Dan Hays 48:22

Yeah, I'll, I'll tell you, this is probably the number one request that we get, and I've been really excited over the last year or two to see that there have been some vibrant discussions about this at Trust Con. we've also, done some really advanced work and thinking around this, including working with trust and safety leaders to help them develop the models that they use to express how trust and safety creates value within their companies. Because let's face it, all of us as leaders within large organizations have to justify our budgets and our resourcing and the investments that companies are making in our space, in our products. And so this notion of return on trust and safety investment is a really important one. So when we look at, this whole topic, we think that it's not just a, uh, a dollars and cents or, or pounds and cents type of, uh, you know, model. This is a real issue for so many leaders in this space and it's something that you need to apply some business discipline to the world of trust and safety. We can't expect to have, unlimited investment. None of us can. That's just not the world that we live in. So we need to make sure that we're, clear in how we express the impact and the benefit that trust and safety yields. there are some great techniques for that, a-and that's definitely something that I expect to be spending a lot of time on as we get into these dialogues at Trust Con.

Ben Whitelaw 49:51

Yeah, brilliant. I'm looking forward to, to listening in on those. so the report sounds very, very interesting. The outlook is, I think, gonna be very well received by a lot of the Trust Con attendees. for Ctrl-Alt-Speech listeners who perhaps will be at Trust Con this year, where can people find you to talk more about the report or any of the things we've discussed today?

Dan Hays 50:10

Yeah. Well, look, uh, first of all, I always, appreciate people who reach out in advance of Trust Con. I'm always happy to set up time to connect one-on-one with anyone who will be there. we've got a big team from PwC that will be attending, and we really, uh, all appreciate the opportunity to meet with our many clients and contacts from across the industry. so setting up in advance is always a great way. I'd say, uh, you know, bumping into people in the hallways is always fun, and there's plenty of opportunities to do that at Trust Con. But beyond that, we're also really excited to be hosting a birds of a feather event, as the Trust Con organizers call them, that's, taking place on Tuesday evening. anyone who's registered for Trust Con can sign up to attend that. think the emails have just gone out starting to encourage people to register. So we look forward to having people join us there and, and meet not just me, but also the rest of the PwC team. And we really, uh, are looking forward to Trust Con and being such a, another year of really rich conversations and learning for everyone across the trust and safety community.

Ben Whitelaw 51:14

Yeah, brilliant. Well, I particularly will look forward to, to meeting some of the team there and meeting many of our listeners there as well, at your Birds of a Feather event and elsewhere. Dan, thanks very much for joining us today. Thanks for taking the time to stop by at Ctrl Alt Speech, and I'll look forward to seeing you at Trust Con

Dan Hays 51:29

Yeah, my pleasure. See you there

Announcer 51:32

That's the end of this week's free edition of Ctrl Alt Speech. To hear the extended episode with additional coverage, become a Patreon supporter at patreon.com/ctrlaltspeech. That's C-T-R-L Alt Speech. If your organization is interested in sponsoring the podcast, contact us via our website at ctrlaltspeech.com. Thanks for listening