"If it will help them get on better dates, they are down for it"

T&S Everywhere is a new Indicator Q&A series in which we interview trust and safety leaders in charge of preventing and mitigating user-on-user misbehavior in apps used by millions of people.

Jeff Dunn has led product, operations, compliance, and the many other teams that make up trust and safety, as Chief Risk Officer at Hinge and in global roles at Google. He is COO of the Digital Trust Council, a nonprofit building an independent certification for trustworthy AI. He also runs the Trust & Safety Network, whose revamped website offers free tools, jobs, and resources for T&S workers and people hoping to get into the field.

I spoke to Jeff about what makes running T&S for a dating app like Hinge unique and how AI has changed the field. The interview took place via email and Google Meet and has been lightly edited for clarity.

Where would you place Hinge on the “content moderation stack,” and why?

I’ve been working on level 2 (platforms, search engines, apps) for more than a decade. Before that, I was at ICANN, which is close to level 5. So I’ve seen T&S from a few different levels of the stack. 

At level 2, the hard part these days isn’t scale. The T&S industry has gotten a lot better at detecting scams and fraud at scale over the years. The hard part is data quality and availability. When someone reports something that happened offline, you often can’t prove what actually happened. So the decision becomes a judgment call, and judgment calls can be subjective and prone to bias. That’s why I’ve always invested in training, check-ins, and QA, and made sure appeals go to a different human agent so you get multiple perspectives. Making enforcement decisions about IRL harm you can’t prove is still hard for dating apps, rideshare apps, delivery apps, and every other platform that connects people in person.

How is T&S different at Hinge and how is it the same?

What’s different about dating is that the harm people worry about most often happens offline, after two people meet in person. That raises the stakes on everything you do online, from verification to reporting. What’s the same is a challenge almost every T&S leader runs into: the work is spread across a lot of teams that don’t always talk to each other.

When I joined Hinge, the structure was pretty typical. Operations doing its own thing. Product doing its own thing. Risk, policy, compliance, and education all doing their own thing. I proposed a General Manager model that brought everyone together under the same OKRs, the same meetings, and the same rituals. Product built tools that made ops better, and ops fed what they learned back into the product. My job was to set the strategy, but the team had to work like a well-oiled machine to get there.

That unified structure enabled us to rebuild our tooling around the operations teams, hit compliance deadlines ahead of schedule, and build one of the strongest safety programs in online dating. It also built real trust within the team, because people were solving shared problems together. I’d recommend some version of this “General Manager” flywheel model to any T&S leader.

I think Roblox has an approach that is somewhat similar to this, where product, safety, and policy are clustered more closely. Is that the idea, or am I misunderstanding what a GM does?

That’s right. The General Manager model brings together product, engineering, research, AI, data science, all of those tech and product teams sitting right side by side and reporting up to me. That way we have all the same rituals, OKRs, and all that. But also, frankly, the same swag.

I’ll add as context for readers that in many companies the Head of T&S doesn’t have dedicated engineers or PMs but has to partner with product on a case by case. Google T&S had a small-ish engineering team that it chose go “give away” in the spirit of consolidation.

I feel like there was some regretti spaghetti about that…

Yeah… A natural follow-up here is that this may work in some companies but not others. You’ve told me in the past that a significant chunk of Hinge staff is primarily working on safety. Is the GM model easier to follow there than at a place like Google, where safety staff are a smaller share of the company?

You'd be surprised. I've been consulting and chatting with other leaders and a lot of them would really want this GM model, and they work at companies of all sizes. They pretty much all want to have the people who are building the tools that ops teams are using as close to the ops team as possible. 

The issue I’ve seen at VLOPs [ed. note: Very Large Online Platforms] and other large companies is that one tech team comes in and says, “Okay, you said you need a button that does this, so we added that button.” Then another team does the same thing, and before long you’re left with 15 buttons from 15 different teams. There’s minimal design strategy, and internal tools at a lot of companies are some of the ugliest software I’ve ever seen. So the GM model is a flexible org structure that can work at any size. You just need leaders who can actually work together, because I’ve seen personalities collide when it’s product versus non-product teams.

How much of your time is spent on digital deception as a broad bucket of policy issues versus other topics? How would you rank it as a risk your team aims to mitigate, and why?

Depending on how you define “digital deception,” I’d say the majority of time is spent trying to stop false, misleading, fraudulent, or otherwise scammy behavior. That’s largely due to the volume and increasing sophistication of these digital deceivers.

That said, telling whether an image, video, or audio clip is authentic or synthetic on a dating app is a bit different from your typical scaled abuse work. Daters have been using “digital touch-up” tools like filters for many years, long before ChatGPT and Gemini images. Some light editing has always been part of dating profiles. The line is whether your photos still look like you. 

That’s where mandatory liveness checks come in. Dating apps have been rolling them out market by market, using a video selfie to confirm you’re a real person and that you match your photos. So touching up how you look is one thing. But if you go too far by impersonating someone else or using a purely synthetic or stolen set of images, that’s where teams take action.

As on any online platform, authenticity is the key to trust. And trust is the key to growth.

To what extent is getting users to agree to a more “muscular” identity verification process like this one easier on a platform like Hinge than on Reddit or a similar app

One of the reasons I was excited to go from Google to a dating app was that Google was open to basically everybody; dating apps are 18+. And people are on Google for 5,000 different reasons, but they are on a dating app for one reason. They want to connect with someone else. Whether it's for marriage or overnight, it's pretty clear that if you can do things that will help them go on better dates, then they are down for it. And we wanted to make sure that we weren't standing in the way. 

What you find at Google, if it's a fungible throwaway account for example, users will be like “I'm not doing selfie verification and ID verification and biometrics and two-step and all these things just to leave my joke comment on YouTube.” Obviously that's a bit of a simplification, but a lot of people have throwaway accounts on Reddit and elsewhere where the bar that they want to jump over is a lot lower than the bar they’re willing to jump over to meet their partner for life on a dating app. 

But one thing that you see too on dating apps in general, and this isn't just Hinge, is that if you have a badge that says you're verified or are, say, a good responder – that actually helps you make better connections and other people want to see that. Would you care on YouTube or Reddit if someone responds quickly? 

And obviously there are other platforms in between dating apps and Reddit, things like marketplaces, the Ubers, the Etsys, Pinterests, all these different kinds of platforms.

Yeah. There's this concept that policy is downstream of product purpose, which is basically what you’re saying. The more a product has one clear use, the more a policy can be reasonably scoped and justified and then more of an ask can be made of users. 

To what extent do you view reducing deception as a product design issue versus an inevitability that needs to be mitigated?

Verification, moderation, and other mitigations are bolted onto most online products, as they should be. They can be effective if they’re properly resourced and maintained. But real safety comes from embedding a safety mindset in the people building the product in the first place. Not just the T&S team. Everyone at the company. When you get that right, safety isn’t just a product feature. It’s part of every decision, every meeting, every OKR. 

That’s what safety by design really is. It’s not a checklist a PM can pick up and tick off. It’s a way of working, and I think a lot of companies are still underusing it, because in my experience it can be a real growth driver. If safety is part of the early blue-sky ideas, you tend to end up with products that need a lot less bolted-on safety later.

This cultural commitment to safety has been under attack over the past few years (for economic and political reasons). As someone who knows this industry and its leaders well, do you see the safety by design mindset getting sidelined?