Automated content moderation - what you're required to do, what's still a guess
Your content moderation AI is probably not compliant. Here's the short version.
If your app lets users post things and an AI decides what stays up, you're regulated. Here's what that means in the US, UK and EU.
If your product lets users post things, you’re probably already doing automated content moderation. An AI flags a comment, the system removes it, the user never sees why. That process is now regulated, and three different regimes have teeth.
The EU’s Digital Services Act (the DSA, which governs online platforms operating in Europe) has required moderation transparency reports since February 2024. The UK’s Online Safety Act (OSA) started being actively enforced by Ofcom in March 2025. Neither of these is a future problem.
The trap here is assuming this only applies to big social networks. It doesn’t. If your platform hosts user-generated content and your system makes automated calls about that content (flagging, demoting, removing or restricting it), you’re in scope. That covers review sites, marketplaces, community features, forums, apps with a comment section. The size thresholds matter for some obligations, but the baseline rules catch almost everyone.
The Map
🇺🇸 USA
The US position is basically: you can do what you want, but don’t lie about it. Section 230 of the Communications Decency Act 1996 gives platforms immunity from liability for both hosting third-party content and for their moderation decisions. That means no federal law forces you to disclose how your automated system works, give users an appeal, or explain why their post was removed. Platforms have wide latitude.
The FTC Act (Section 5, the general prohibition on deceptive and unfair commercial practices) applies if your moderation is being used deceptively. If you claim your AI is neutral and it systematically targets certain users based on protected characteristics, that’s an FTC risk. It’s an enforcement framing, not a compliance checklist, and no FTC rules specific to content moderation AI exist as of May 2026.
At state level, California’s AB 587 (live since July 2023) requires large social media platforms (over one million California users) to publish semi-annual reports on their content moderation policies, including how automated tools are used. It’s enforced by the California Attorney General. Other states are watching.
What’s clear: Section 230 protects US platforms from liability for their moderation decisions, with no federal disclosure mandate in force.
What’s ambiguous: Whether state transparency laws like California’s AB 587 will survive First Amendment challenges is still being litigated.
🇬🇧 UK
The Online Safety Act 2023 requires platforms that host user content to assess the risk of illegal content appearing on their service, put in proportionate systems to deal with it, and keep records showing they’ve done this. Ofcom started enforcing these duties in March 2025. The Act doesn’t care whether you’re using humans, AI or some combination, the obligations are the same either way.
Ofcom’s Codes of Practice (the published guidance setting out what compliance looks like in practice) recommend that high-risk file-sharing services use automated detection tools for known illegal content. You’re not required to use AI. But Ofcom will look at whether your moderation setup is “adequately resourced and well trained” when it investigates, and a system that misses obvious illegal content because it was never properly built will not survive scrutiny.
The ICO published specific guidance on content moderation and data protection in February 2024. The key message: UK GDPR (the UK’s data protection law) applies to any automated processing of personal data in your moderation system. That means you need a lawful basis for processing, you can’t keep more data than you need, and you have to tell users what you’re doing with it. If the AI makes a decision with a significant effect on a user (removing their account, say, or blocking their content), Article 22 of the UK GDPR (the rule restricting fully automated decisions that significantly affect people) may require you to give them a route to human review.
What’s clear: Platforms in scope of the OSA need documented risk assessments and functioning moderation systems, and UK GDPR applies to any personal data processed in that system.
What’s ambiguous: The ICO describes its content moderation guidance as a first instalment, with more to follow, so the standard for explaining automated decisions to users isn’t fully settled yet.
🇪🇺 EU
The Digital Services Act is the main event for content moderation in Europe. Any platform that hosts user content must explain its moderation policies and use of automated tools in its terms of service, give users the right to challenge moderation decisions through an internal complaints process, and provide access to out-of-court dispute resolution. These obligations have applied since February 2024.
Very large online platforms (known as VLOPs, meaning any platform with over 45 million monthly active EU users) face heavier requirements, including annual transparency reports with data on the accuracy and error rates of their automated moderation systems. A standardised reporting format has been mandatory since July 2025. If you’re a VLOP, you already know about this. If you’re not, the baseline obligations still apply.
The EU AI Act (which came fully into force in August 2024 and applies in stages through to August 2026) doesn’t name automated content moderation as a high-risk use case. That’s actually less reassuring than it sounds. If your moderation system does biometric categorisation, processes health or political data, or makes decisions that overlap with employment or education, it may already sit inside an existing high-risk category under the Act’s Annex III. EU GDPR sits on top of all of this, as it always does.
What’s clear: DSA transparency and user redress obligations apply to all hosting services in the EU right now, with heavier reporting requirements for very large platforms.
What’s ambiguous: Whether a content moderation AI system triggers the EU AI Act’s high-risk rules depends on how it’s designed and what data it touches, and the Commission has not published guidance on this intersection.
The Practical Minimum
Write down how your moderation system works before a regulator asks. What signals trigger automated removal? What data does the system use? Is there a human in the loop, and does that human actually review things or just approve what the AI flags? In the UK, Ofcom will ask for this if they investigate, and the ICO expects a DPIA (Data Protection Impact Assessment, a documented analysis of data protection risks) to exist before the system goes anywhere near personal data.
Give users a way to appeal. The DSA makes this a hard requirement for EU users. The OSA expects it for UK users. One functional internal complaints process covers both. Make it findable and make it work. A buried form that generates no response is worse than nothing, because it shows you knew the obligation existed.
Put a plain-language description of how automated moderation works in your terms of service and privacy notice. DSA Article 14 requires this for EU platforms. UK GDPR Articles 13 and 14 require it for any personal data processing. California’s AB 587 requires large platforms to publish their moderation policies including the role of automated tools.
If your system uses biometric signals, behavioural profiling or special category data (health information, religious beliefs, political views), don’t assume it falls outside the EU AI Act’s high-risk classification. Get a legal opinion before you ship.
The Grey Zone
The EU AI Act’s silence on content moderation as a named category is a source of real confusion. A moderation system using language models trained on demographic data, image recognition, or behavioural signals could slot into an existing high-risk category (biometric categorisation, for instance) without the operator realising it. No Commission guidance addresses this overlap directly, which means you’re making a judgement call with no official safety net.
Section 230 in the US is not stable. The Supreme Court declined to narrow it in the 2023 Gonzalez v. Google and Twitter v. Taamneh cases, but congressional appetite for reform hasn’t gone away. Building your entire compliance posture around Section 230 immunity is a reasonable position today and a risky one if the law changes.
The OSA’s “proportionality” standard gives Ofcom discretion that’s hard to predict. What counts as an adequately resourced moderation system for a startup with 50,000 users is not the same as for a platform with 5 million, but the obligations apply at both scales, and Ofcom hasn’t published thresholds. Smaller platforms are largely uncharted territory in enforcement terms.
Article 22 of the UK and EU GDPR restricts “solely automated decisions” that significantly affect people. Whether your moderation system crosses that threshold turns on what your human review actually does. A moderator who clicks approve on every AI flag without reading the content doesn’t provide the meaningful oversight the law requires. Nobody has tested exactly where that line is.
Not Advice