Mike Masnick: 0:00

Hello and welcome to Control Alt Speech. As we mentioned in last week's episode, we do not have a regular episode for you this week because it is Thanksgiving here in the us, but we do have something special for you instead. It is one of our sponsored Spotlight episodes in which we get to go deep with an expert in the trust and safety field and have a really. Interesting thought provoking conversation this week. The spotlight is brought to you by Resolver, talking about how they've just reached their 20th anniversary, supplying trust and safety services, which really is kind of incredible given just how drastically the internet world has changed over the last 20 years. In this episode, Ben got to sit down with George Blato, who's the head of Resolvers Trust and Safety Division to talk about their unique approach to scaling the detection of child sexual abuse material, better known as CSAM. They reminisce on how much the online world has changed in these past 20 years, as well as talk about how Resolver latest innovation called Athena is an incredibly powerful tool in the fight against online harms. Please enjoy this spotlight episode with. Ben and George Valeto from Resolver.

Ben Whitelaw: 1:18

George, welcome to Control Al Speech. Thanks so much for joining me today, and thanks again to Resolver for sponsoring the podcast and making Mike and I's Weekly JAB Rings possible. we're gonna dive straight in and talk a bit about Resolver Trust and Safety, which is a company that's been around a long time. It's been around 20 years. this week, in fact. For people who are new to the company who maybe weren't paying attention back in 2005, tell us a bit about Resolver, what it does in the space.

George Vlasto (Resolver): 1:46

Thanks Ben, and thank you for having me on the podcast. It's uh, it's a pleasure to be with you today. as you say, we are celebrating our 20th anniversary this week, which is a huge milestone for us. 20 years, is, almost an eternity in technology. and I was talking to our, our founder, Adam Hildrith the other day, and we were reflecting on the earliest days of Resolver, the sort of early days of the social internet and, really. The journey we've been on from being at the, cutting edge of discovering these kind of emerging harms, which at the time were really not well understood or, or well thought through. and the sort of earliest bad actors operating in places where vulnerable users congregated, particularly children. And how we, started, really in the early days, quite a sort of scrappy, Largely sort of keyword based effort to identify problematic behavior and how that's evolved over 20 years. And now we have data scientists and risk detection engineers working with our human specialists to be able to detect not just, you know, problematic content, but behavior. Trade craft as we called it, the sort of, you know, methodology that bad actors use. all the way up to our kind of, you know, latest, product launch, which is our unknown CSA detection service. And it's, it's an extraordinary number of people who've played a part in that whole range of disciplines, from some people with academic backgrounds through to deep data scientists, engineers, and everyone in between. And it's, it's pretty remarkable actually to reflect on that 20 year journey.

Ben Whitelaw: 3:05

Indeed, it's very few companies have been around for that length of time. You know, many have, have come and gone. I think it's over interesting for a number of reasons. You, you are not your kind of typical product driven company. address a range of harms, as you've alluded to there rather than just one. you have lots of different expertise kind of underneath the resolver, umbrella. Can you talk a bit about how you've, as a company developed. To cover all of these harms and with this broad range of expertise.'cause I think it is fairly unique.

George Vlasto (Resolver): 3:34

It is we, we have the great advantage, I think, in this space that unlike a lot of our clients and partners, the only thing we think about is trust and safety. So for even the most sophisticated of our clients, trust and safety is a component, is an important component, but it's a component, part of what their overall organization thinks about. For us, it's everything which has allowed us to be, although we do have, products and product management function, we are actually problem focused and I think that's the kind of our ethos ever since our earliest days in 2005. Problems evolve. Problems emerge both through human behavior and technology innovation and being nimble enough to respond to those in a kind of dynamic way has been pretty key to our success thus far. And also, I think is a reflection or is reflected in the fact that we have this very diverse group of skills and people, globally now in, four or five different countries operating kind of 24 7. And it's about being problem focused rather than having a product that we're trying to market for its own sake. It's about constantly adapting our approach to, fix the problem or address the problem that our clients and partners are are struggling with at that given moment in time.

Ben Whitelaw: 4:37

Great. That's really helpful. you, you mentioned when you explained Resolver about its focus on behaviors and kind of identifying harmful behaviors as much as kinda harmful content, and there is an emphasis on intelligence and intelligence led approach. What does that mean in practice and, and how does, Resolver think about the balance of not only kind of human intelligence, but also increasingly AI intelligence too.

George Vlasto (Resolver): 5:02

that's a really good question. And think the advent of ai, at huge scale with remarkable capabilities obviously has changed our, the way that we address this problem, actually. So the way we think about it is there are, emerging threats, emerging behaviors, sometimes. In the context of new regulations or new regulatory requirements that our clients and, partners are required to think about. And being able to understand the risk on your platform or your service and how that compares to others, and also, interacts with the sort of regulatory framework in which you're operating is really important. And that is. Still, and I can't see a world in which it's not, in the future driven by human expertise, deep subject matter expertise that enables to act in a, you know, almost a consultative way with our partners to help them understand the risks that they might be facing and how those are evolving. AI is obviously a really important part, and almost all of our, partners use their own internal AI or AI that they, bring in for the job to detect harm at scale as sort of classifiers. And a lot of what we are doing with our human expertise is informing those classifiers and helping them operate at the kind of highest level of efficiency possible to drive real value to the companies, that we work with. In one case in particular, it's very hard for our partners to train their own models, and that's in CS a detection, which is why we brought to market what we call Athena, which is our sort of best in class, new generation csam detection service. And that's kind of our, unique. AI offering a machine learning model that can solve that problem for you in a way that your own classifiers can't, because you just can't train them on that kind of material. So as humans informing policy, training classifiers and then, in the hardest and most egregious harm in the case of CSA, are actually providing a solution directly to our partners, rather than leaving'em to sort of fumble in the dark around a very, very challenging, legally, ethically, and, you know, psychologically challenging kind of issue.

Ben Whitelaw: 6:52

Yeah. great to hear more about that. we're gonna talk a bit about Athena shortly, but I want to kind of talk a bit about you and, your part in Resolver journey, over the, the last 20 years. you had a 15 year career in the UK diplomatic service, and you've been in the role at Resolver as head of trust and safety, for three years now. Talk us about, that transition from UK government service into this role and give us a sense of what your kind of day-to-day looks like in the broader resolver machine.

George Vlasto (Resolver): 7:24

Yeah, so, so I, sort of went into public service and, diplomacy because I wanted to work on, you know, some of the most challenging consequential issues that there were. And I was lucky to do that in various parts of the world. But it became pretty clear to me a few years ago that actually the sort of generational challenge, that we. As a society, global society, not just in the UK or, just in sort of Western countries have to grapple with is, online harm, how we harness the power of the internet, which is obviously. Exponentially improved, you know, communication, academic collaboration in some cases, political freedom, and how do we balance that against the harm, especially to vulnerable users, particularly children. So for a number of years, I was looking for an opportunity to move across and then I was very fortunate to find this opportunity, it resolver. So it was actually oddly psychologically, quite a natural transition, although it's a very different job than what I was doing before. So my day at the moment, really it entails. Creating space for our team. So as I said at the beginning, our strength comes from our ability to. Understand quickly and respond rapidly to emerging problems. And the only way we can do that is by having teams who are enabled to be pretty entrepreneurial, actually in their own areas. obviously, you know, within a structure. So my, basic job is to make space for, our engineering teams or our risk detection teams or our human intelligence specialists to focus on what they need to, to have the tools at their disposal, to have the resources and the sort of framework to operate within, which enables'em to respond really quickly on behalf of our clients and partners to these problems, which can come out of nowhere. You know, we can see these things, particularly as harm verticals merge together. We see different manifestations here. We sit with com networks at the moment. You need to be very, very agile to be able to respond to those kind of effectively. So that's really my job, is to try and create the, framework that enables agility. That's. Harder said than done, I think

Ben Whitelaw: 9:15

for sure. just explain to us what you mean now by Comm Network.'cause there might be listeners who don't fully understand what that means.

George Vlasto (Resolver): 9:22

So I think this kind of emerging trend, this, blending of, nihilistic sort of ideology, suicide and risk extortion, sextortion in some cases. this really, these really nasty groups like 7, 6 4 coming to the fore. who. Don't fall neatly. No one falls neatly in any case, but doesn't fall in turn of the traditional harm verticals that most of our partners and most of the industry look at. You know, whether it's child safety or. sextortion or suicide and self-harm, and they, cut across, they're using new techniques. They're sometimes interoperable with organized crime and money laundering. These are really, really complex challenges that have emerged very rapidly and even have kind of major law enforcement agencies kind of on the back foot trying to respond to them. They sit somewhere outside of the kind of regulatory framework in some cases. and a lot of our partners Are extremely keen, obviously, to understand how these groups operate, how they manifest on their platforms, the kind of behaviors they're using and the risks they post to their users. and that's one of those kind of examples where really that's a kind of almost a, you know, a kind of war room of people at Resolver who will come together, look at this from different angles, and try and develop a solution to the problem that enables our partners to get ahead of it before it starts to, really damage their wellbeing of the users on their platforms.

Ben Whitelaw: 10:35

Interesting. you mentioned needing to respond quickly to the problems and issues the partners bring to you. Can you give an example that kind of sticks in your mind from the time you've been a resolver so far, where that's been particularly stark or you've had to kind of move particularly quickly or, or to address, uh, how maybe the, you never saw coming.

George Vlasto (Resolver): 10:56

Yeah, so I'll give you a couple examples. It's slightly different. So, so one was, we were working with one of our partners, on, suicide and self harm prevention. and detected by our technology and then, and then reviewed by analysts in pretty much real time, we saw an indication that a school aged, individual. was planning to go to their school and, at least according to their social media post, take their own life. We saw that in real time. obviously we alerted our partner to that. but we also in parallel alerted law enforcement in the jurisdiction to which this applied, who were able to intervene in that individual's kind of journey to school. and we were able to offer that person of psychological support, and ensure that they weren't able to, go through with their plan, which had had been sort of fueled by some of that. Exposure to material online. that's, you know, we actually see that relatively frequently, that kind of, very, very rapid response to a threat online where we can work with our partners, work with law enforcement or other kind of social organizations to intervene, to stop things happening. and it's obviously incredibly powerful when it happens and it kind of resonates through the whole business or people who know about it within the business.'cause it's, it's such a. Such an acute demonstration of what we do, you know, particularly when it's at that individual scale. So that'd be one example. And then, and then another, which we had recently where I think we benefit sometimes when we work with our partners.'cause we don't just look at our partners platforms, we also look around them, the ecosystem around them. And in one particular case, recently. was an individual who was posing a risk to children on our partner's platform. but their activity on our partner's platform was, it was policy violative, but not illegal. and they, they were able to circumvent some of the countermeasures that were put in place to try and stop them using the platform. But using our sort of off platform research, we could see this person actually posed a real world threat to children. Uh, it was quite clear they were engaged in illegal activity and soliciting illegal activity elsewhere. So we were able to work in coordination with our partner to pass that information we had to law enforcement in the location that that person was, was in. and then they were able to take appropriate action. So, you know, in one case, sort of helping somebody who was, Potentially a victim of, suicide self-harm or suicide self-harm material that they've been exposed to. And on the other, looking to take action, against a potential perpetrator of, child exploitation. so those kind of examples, you know, actually occur on a remarkably regular basis actually.

Ben Whitelaw: 13:13

And so you're kind of acting as a, as the kind of eyes of a platform or a partner. in other spaces on the internet perhaps where, that partner or that platform might not have visibility. Is that fair to say?

George Vlasto (Resolver): 13:24

Exactly. Yeah. and sometimes they may not even have the latitude to act on information, you know, elsewhere because they haven't collected it, they haven't seen it, but also because it sits outside of their ecosystem. Whereas, we can sort of act as that intermediary sometimes with, know, law enforcement or, social support agencies or something if, intervention is required.

Ben Whitelaw: 13:42

And so many of the stories that Mike and I talk on the podcast feel like they start on one platform, emanate to another, move to another. And so that, is, definitely an, a kind of thread in many of the stories we discuss. And as I'm sure you're aware, we're gonna talk today, George, in a bit more detail about Athena and about child sexual abuse material, which you touched on. And again. It crops up all too frequently in the podcast each week. Its kind of proliferation as an online harm has seemingly only kind of increased over time. I'd love your thoughts on the landscape, child sexual abuse material now and how that has evolved over time. Are we seeing it get worse in the way that it feels like? What's your sense?

George Vlasto (Resolver): 14:22

So, so I think in terms of scale, certainly, in terms of severity, I think that's hard to say. that's a somewhat subjective, call to me. But certainly in terms of scale it is, and the, the way it manifests, the, the different environments that we're seeing it on, increasingly. are really concerning. And, and the advent of generative AI clearly has, of supercharged the problem. and I think there's a misperception in some quarters that, CSUN that's generated using artificial intelligence is a kind of victimless crime. There is a huge body of, academic and, social research that suggests that is, that's a fallacy. And actually it is, you know. Equally problematic, and can lead to, further offending from, child predator communities and also, I mean, just fundamentally is treated in most jurisdictions as being, you equally illegal and egregious as, real world CS a. So yeah, it is definitely getting, bigger in scale. it's becoming, More, prevalent in more places on the internet, and I think that child predators or consumers of CSA, becoming more. Experts actually how they generate, disseminate, and crucially monetize this kind of stuff. and that's, that's really kind of one of the biggest changes over the last three or four years, and we've discussed this extensively with the likes of, uh, the IWF and in Hope, and other partners. How do we tackle this monetization of, child sexual exploitation material and, and CSA in particular because it's a growing problem and financial drivers, Only exacerbate what was previously a kind of or less entirely sexual crime.

Ben Whitelaw: 15:49

Yeah. listeners who, who may be kind of, don't come across child sexual abuse material and, and maybe aren't familiar with it, can you just talk about how platforms, and, at Resolver you would identify child sexual abuse material, if it kind of came through, a moderation queue or, or a system.'cause again, I think there are a few important distinctions there. just to outline before we talk a bit about, how you are, you are using your Athena tool to, to mitigate it.

George Vlasto (Resolver): 16:16

Sure. so are kind of a, a number of basic ways to detect CS a materials. So the simplest, is human review. So a human being looking at an image and saying, this is CS A, that can either be a moderator like working within queues, for a platform or technology service or, increasingly user reporting. as more and more platforms have, implemented sophisticated user reporting mechanisms. But it ultimately revolts around a person looking at it. The second way, which is kind of fairly embedded now, across the industry, is what's called known CS a detection. So ASH matching. So any image that's submitted to. For example, ncmec or, the Internet Watch Foundation, or others is hashed. So it's given a digital signature. It's then shared, with partners. so whenever that image reappears in the ecosystem, a platform should be able to call out against hash database and get a confirmation that this image that's come through your platform is known to be. Child sexual abuse material, therefore it must be removed and reported, and so on and so forth. And then the final way, which hopefully will lead us onto a conversation about Athena is what's called unknown cs a detection. So use using machine learning technology at scale to, identify and detect previously unseen or novel child sexual abuse material, whether it's real world, cs a from, from real world abuse, or, csam that's been generated using artificial intelligence, but crucially, that's never been seen before.

Ben Whitelaw: 17:35

And my, my understanding, George, is that a lot of the kind of child sexual abuse material that, is on platforms is unknown or unmatched. And so there is a, you know, basically a kind of iceberg above the water of, you know, material that is known, is in the database and therefore is matched. But there's this kind of vast amount underneath the surface that, you know. Databases haven't seen before. Therefore, platforms can't detect. Can you explain what that means in practice for platforms and, and how that maybe changes how trust and safety teams deal with it and, and even, you know, refer to it to law enforcement as well?

George Vlasto (Resolver): 18:09

Yes, so I'd agree completely with what you just said, which is that the proportion of known versus unknown CS a is sadly heavily weighted in favor of unknown CS A. So the vast majority of Cs A that exists online has previously never been seen before, or certainly not recorded in a hash database. So effectively you have a real constraint on your ability to detect child sexual abuse material, either because you've only got a limited number of people that you can deploy to human review. You know, you can't human review every single image or. item on your, platform at, tens of billion, hundreds of billions of items scale. and if you're just relying on known csam hash matching, you are only gonna be catching a very small proportion, as you say, that top of the iceberg that's been seen before. Mm-hmm. and even relatively small changes to those images, you know, the signature of those images can throw known csam detection as well. So, at the moment, or historically it's been a really significant challenge for platforms and for law enforcement, to detect novel CSA and also to be able to differentiate between the different grades or different levels of CSA, the most severe, which might indicate an imminent harm or ongoing harm to a child that therefore law enforcement may want to prioritize their efforts in, targeting or, or investigating.

Ben Whitelaw: 19:19

And just explain for listeners, George, if you, can, that those gradings, cause I was almost unaware that there were different types of child sexual abuse material in terms of severity. And, again, that's helpful I guess as a, a kind of. When we come to talk about the now

George Vlasto (Resolver): 19:33

Yeah. And I'll be a bit cautious about how I describe it because I don't want to, go sort of too far into the dark heart of this for your listeners, but effectively that in most jurisdictions, child sexual abuse material is broken down into different categories. in the United States category 1, 3 3 in the United Kingdom and other jurisdictions, it's a to c, C being the least. egregious category that's still sexual content involving a minor, but it's the least egregious category A is the most egregious. And I won't describe exactly what those categories contain, right? But I, imagine with a leap of imagination, your listeners can, uh, work that out. But, category A is, yeah, is generally speaking, the work that is, generally treated as the most egregious, and that tracks the most severe criminal penalties. if you are, Obviously creating it or sharing it or disseminating it in any way.

Ben Whitelaw: 20:20

so let's go onto Athena now. I mean, this is a new tool that Resolver, is bringing to the market. how did it come about? What does it do and how does it differ from those other kind of technologies that exists right now and which platforms utilize?

George Vlasto (Resolver): 20:34

so again, we. We thought about this from the perspective of the problem rather than the kind of product, as I was describing earlier. So the problem for anyone trying to develop an unknown CSA classification tool is that in order to develop a computer vision model, you ideally have access to a database of images. So if you are training a machine learning classifier to differentiate between different breeds of dogs, for example, you would feed the machine learning model. however many millions of pictures of a Labrador versus a, poodle until the model could determine between the two of them. it is illegal to hold a database of child sex abuse material. Therefore, you can't apply, in most cases, that methodology to training a c sound detection model. There are technical ways around it so you can, train a model. To recognize similar signatures of images, so you can train a model based on those hash databases I was talking about before, and look for what's called an embedding model. Look for kind of similarities in signature once the image has been hashed, but it's relatively imperfect that model. And it's not a true computer vision model. So we knew this was a huge and growing problem. The proliferation of CS a online, the technology that was being deployed against unknown CS a was, not as effective as it could be because of this kind of legal and ethical constraint around CSA database images. so we. Decided we were going to try and find a way to solve that problem. and we therefore partnered with, a company called R Manor Research, who are our partner in delivering this, who do a lot of work with, in this case, the UK government. and we discovered that the UK government held a database called Cade, the Child Abuse Imagery Database, to which law enforcement and organizations such as the IWF contribute images, and it's a off system. Database. It's not connected to the internet. You need to have a very high, level of security clearance to get anywhere near it. But we were able to work with Broke and the Home Office to get access to that database to train a true computer vision model on the world's most highly curated database of child sexual abuse imagery. Which seems like an odd thing to say, but it's, it's the sort of highest quality database that exists in the world because it's all been reviewed so many times. And as a result, we were able to, spend. Considerable amount of time and development effort to to train what is by a considerable margin, the most accurate detection system for unknown child sexual abuse because it's actually being trained on real images of child sexual abuse under a very strict ethical and legal. framework governed by the UK home office in this case. So the uk interior ministry for some of your listeners.

Ben Whitelaw: 23:05

Interesting. We'll, we'll come onto the testing that you've been doing with Athena, but I'm interested in that kind of security aspects. You know, access to Cade is not something that gets given to many partners. I'm sure. I'd, I'd love just to kind of know. How that went, how that came about, and also how you made sure to kind of protect your teams as they had access to this, from a kind of personal standpoint, but also obviously from a security perspective as well.

George Vlasto (Resolver): 23:31

Well, so, so those people who had access to Cade and retain access to Cade on an ongoing basis to make sure the model is retrained regularly, all have very high level of, government security clearances. So they have to have kind of passed what's called a vetting process to be allowed to go anywhere near that database. That's not everyone in Resolver, or at Rogue, but a small number of people are able to do that. but even those people don't. Have to look at the images. so they're able to access the data to train the machine learning model, but they don't have to look at the images because fortunately for us, and unfortunately, I guess for child protection, police officers and the IWF, other people who are, Extremely expert in this have already looked at those images and graded them. so our team don't actually need to view the images themselves. They just need to have access to the data to feed through the model. And it's actually only the model that looks at the image, not our staff. but it's a pretty complex. Ethical and legal, process to go through. And obviously it has to be done on a very, very strict, guidelines and supervision, to ensure that, you know, nobody is exposed to data that they don't need to see of the most egregious type, and that no one who shouldn't be seeing or having access to this data is, is allowed to, it's, yeah, it's probably the most complicated commercial. Pathway that I've ever had to sort of chart through.

Ben Whitelaw: 24:45

Yeah, I'm sure. many, many sign offs. so, so to kind of summarize, I guess what Athena does, it provides that categorization of CS a in a way that has previously been very difficult to do, with the data that's been available. What's your expectation of what that will allow platforms to do once they have access to that?

George Vlasto (Resolver): 25:04

so when we say Athena, so there's, there's two sort of parts to this. So there's Athena, as the unknown CSUN detection model, which is based on a machine learning model called Vigil, vigil ai for platforms that already have a sophisticated trust and safety workflow and known CS sound detection in place and human moderation in place, effectively, it becomes just a plugin in the middle of that. So you can run your data through a known CSA detection database, which is still valuable. And then anything which comes back, you can obviously take as read that it's CSA, or at least it's been flagged as Cs a. Anything that doesn't, appear on that database, you can run through, Athena, for an indication that it might contain unknown CSA. And then there's, potentially a human review element at the end of that and reporting requirements to EC and other, uh, reporting bodies. So that's for sort of sophisticated trust and safety teams. That's the kind of most likely use case, either through an API call to our infrastructure, or in certain cases we can deploy this on-premises for the kind of biggest, technology companies if that's what they require. For smaller outfits or, or people who, or companies that are thinking about this perhaps for the, for the first time, and don't have those sophisticated workflows. Athena as a whole can provide all three components of that. So effectively, if you have, if you are grappling with the challenge of Csam on your platform and you have no existing measures in place, we can provide for you in a single workflow, known cs, a detection, unknown csam detection, and even human review of that csam and reporting, at the end of it. So for a platform that. Wants to deal with the problem, but doesn't have the resources or potentially the kind of, the wherewithal to set up their own workflows, that they can essentially outsource it to us and we will take care of that entire problem for them to the highest possible standard that exists at, at the moment, you know, with the technology that's available to us.

Ben Whitelaw: 26:42

Yeah. Interesting. you mentioned, George, that you've been testing with a number of different kind of platforms so far. You, you talked about the accuracy of Athena, and the models that underpin it. Can you talk a bit about that accuracy relative to, other technologies that are available and also kind of what you've learned in deploying, the models in these new scenarios.

George Vlasto (Resolver): 27:04

Sure. the kind of detailed accuracy scores for vigil, are slightly sensitive. So if, any of your listeners are interested in finding out more about it, then please do get in touch and we can share those details under sort of legal protection. But in principle, when, Athena is deployed kind of in the wild or in a live environment with our, with our, customers and clients, it achieves a level of accuracy as close to. perfect as you can really achieve with like a machine learning model. However, your second part of your question was how do we, how have we learned that this works in a live environment? And it depends on what a platform or a technology service is looking to achieve. Because with any machine learning model, there are two variables. There are basically two dials that you can turn. One is called precision, and the other is recall. If you dial up precision to the highest level, you will get almost no false positives, but you may miss some edge cases, particularly AI generated imagery that might not be sort of normal imagery that may involves like non humanoid characters, for example, and things like that. Precision set at a very high level, you may miss some of those edge cases. But you will get pretty much no false positives. If you turn up recall, we'd likely to get higher number of false positives. Still not a huge number, but a higher number of false positives. But you will miss like none of those edge cases. So effectively what we say to our clients is like, what, what are you aiming for? You're aiming for incredibly high precision, perhaps'cause you've got vast quantities of data. I false positive rates cost you a huge amount as a business to review and to triage. So you turn up precision or are you looking to remove any hint of child sexual abuse material from your platform, in which case you prioritize recall. And then the other thing which we've learned is that that categorization, from A to B2C. is really important because it allows focused attention, referrals to law enforcement, alerting to NCMEC and other organizations to happen, to the most egregious harm really, really fast. And it enables our clients to root that kind of content to very highly trained and protected teams.'cause one of the other problems is obviously Cs a being viewed by. Normal moderators who don't have psychological support around them is incredibly traumatic. So if you get a really clear indication at the beginning that this is a deeply egregious, image, you can route it in the right way. You can treat it with the urgency it requires and the sensitivity that it requires. so it's seen by the right person really quickly, and then action is taken as fast as possible. So those are the kind of principle things that we've learned as we've kind of rolled it out. And, and do you

Ben Whitelaw: 29:32

expect George, as maybe platforms increase the dial on recall, in order to comply with kind of regulations and, and obviously protect children and, users that are kind of vulnerable, do you expect that false positive, increase to be. Also an issue that platforms have to, deal with as well.'cause that's something we see in other harms as, as an ongoing problem is, is that that spike and maybe cause positives.

George Vlasto (Resolver): 29:58

I actually think this is a question for regulators as well as for platforms, because most regulation, all the regulation is really predicated on the idea that platforms and technology service providers should take all reasonable efforts to find and mitigate harm. If by setting your recall aperture so wide, it's Athena or any other type of machine learning detection system, you are generating a huge amount of noise in your system and you are incurring unreasonable cost or unreasonable, interferences with user experience. Clearly that's not. That's not in the spirit of the regulation. So I think there's a dynamic conversation that needs to take place between technology companies themselves and the regulators about, you know, what is reasonable. And I think it'll be context dependent. And certainly, you know, when you listen to Ocom or you listen to the European Commission or the Australian Needs Safety Commission's Office or other global regulators, there is a real sense that, discretion is, is required. There's no one size fits all, you know, for a very, very large online platform or a technology service provider. Those thresholds might be different than for a kind of emerging technology platform that perhaps is catering ex, you know, explicitly for children, for example. So I think regulators will take a reasonable approach to it, and I think each platform will have to determine for themselves, you know, what reasonable looks like, in that context.

Ben Whitelaw: 31:08

Yeah, most, Um. George, it's fascinating to hear a bit about Athena. I I wanted to kind of broaden out before we wrap up today, around some of the in blog posts that Resolve has put out over the last few weeks as it's celebrated its 20th birthday. one of them talks about kind of predictive online safety technology, and I wondered whether you thought in the context of our discussion today, whether we're moving towards a future where models can actually anticipate harms before they occur. is there a world. you can imagine that happening. I,

George Vlasto (Resolver): 31:38

think in certain cases, maybe, I mean, I guess it depends on how you, like, how philosophical you get about what AI is designed to do. But I mean, especially even the most sophisticated AI models are statistical probability models. so they're, their expertise lies in predicting the most likely next thing at the end of the day, like at its most simple level. And obviously what we know from, online safety, but you know, actually outside of just kind of digital safety in the world in general, sometimes the most damaging things are those that are unpredicted, the kind of black swan events that, that come along, or the, the really unexpected, evolution in human behavior or interest or misuse for technology. So I do think that AI models will get better at, learning and classifiers, generally speaking, will, you know, be closer to the margin. Until we reach a, depending what, what your viewpoint is, a halian or terrifying future of general artificial intelligence. I think that that kind of human guidance of, where AI is focused, the training data sets that they're, predicated on and that kind of empathic like human understanding of how people are actually behaving in real life will remain a really, really important part of the mix. so. Obviously we'll see incremental gains in ai, but I don't think in any foreseeable, future, I mean unforeseeable in tech terms, it's like one year. Uh, I don't think AI will ever be a panacea for, trust and safety teams.

Ben Whitelaw: 33:03

Yeah. Fascinating. that's a fairly reassuring and hopeful note to end our conversation on. George. really enjoyed talking to you today. we'll share some of the resources and, and the link to Athena and also the blog post that Olba has put out over the last few weeks in the show notes of today's episode. just wanna say thanks so much for your, your time, today and for sharing a bit more about what Silva does and what Athena, is set up to do. Really excited to see how that goes.

George Vlasto (Resolver): 33:28

Thanks Ben. It's great to talk today and thank you for the opportunity to, to come on the podcast and talk a little bit about Resolver.

Ben Whitelaw: 33:33

Brilliant. Take care. Thanks.

Announcer: 33:37

Thanks for listening to Ctrl-Alt-Speech. Subscribe now to get our weekly episodes as soon as they're released. If your company or organization is interested in sponsoring the podcast, contact us by visiting ctrlaltspeech.com. That's CT RL alt speech.com.