For a full appraisal of this proposal and its reasoning, please consult the full paper above. What follows is a brief summary of our main points.
We believe there are two categories of issues which technology products can produce, which we categorize as “acute” harms and “structural” harms. Acute harms operate on the level of individual posts or discrete pieces of content, and must be addressed with moderation. Structural harms, by contrast, operate on the level of products and often are not tractable or even visible on the level of specific posts.
| Harm Type | Manifestation | Examples | Regulation | Regime |
|---|---|---|---|---|
| Acute | Individual Content | Harassment / threats, self-harm, copyright infringement, pornography | Moderation | Rules Based |
| Structural | Aggregate effects of products on users in a society | Discrimination, reduced belief in institutions, mental health harms | Product Constraints | Consequential |
While many of the largest platforms have developed robust teams and systems for addressing acute harms, we believe there has been insufficient investment in systems designed to understand and address structural harms. Potential structural harms relate to many alleged platform challenges. We believe, though, that the most pressing is reductions of mental health, especially among populations of concern like teenagers. A second pressing candidate relates to reductions of measures of interpersonal connection, specifically measures like social trust.
To build a system capable of addressing structural harms, three questions must be answered:
What form must product interventions take to have both observable effects and demonstrably mitigate harms?
Is there a sufficient consensus for intervening on particular structural harms?
What is the minimum viable architecture for a system whose behavioral restrictions would be informed by such a consensus?
We believe a successful responses to these questions will be actionable, critical, and comparable. By this, we mean that the system will engage with feedback loops in real time, will use specific metrics that are flexible and suitable to the specific needs of designers, and that they must be simple and broad enough to engage with multiple different decisions.
One way to see this dynamic is via this kind of visualization:
In this chart, feedback loops – inclusive of potential moderation – induce potential population dynamics via some sort of sociotechnical mechanism. This may seem complicated, but the basic intuition is simple: without some constraint tied to pro-social outcomes, loops of this kind can regress to “optimized” points for growth or engagement without mitigating unacceptable external effects. We can see this in another way like this:
Our main conclusion is that the middle loop – the potential policy formation one – does not currently exist for the types of mitigation strategies needed to address structural harms like mental health.
And yet, we have seen this kind of situation before.
In the 19th century, as population density rapidly increased during industrialization, new manifestations of old diseases took on fearsome new presentations, leading to the invention of the modern field of public health. And in the 20th century, following several decades of widespread use of dangerous industrial products, the environmental policy (particularly with the passage of the Clean Air and Clean Water Acts) responded to health harms on populations by imposing new protective systems.
The history offers a few clear takeaways:
- New state and economic capacities make it possible to intervene on populations and measure resulting impacts.
- New forms of evidence are mobilized that suggest causal mechanisms unimaginable within the previous political and cultural paradigm.
- Early policy responses comprise an attempt to make sense of these mechanisms, observing their limits and beginning to evaluate their consequences.
- As evidence matures and new political coalitions crystallize, mechanisms are able to be evaluated as acceptable or deficient according to the standard of population health.
The 19th century story of the rise of public health infrastructure is starkly similar to the current environment. The mechanism we are proposing, we believe, would be the most direct method for implementing a response to this parallelism.
Heading 5
A range of issues are raised by this type of framework. We do not claim to have addressed them all – nor do we claim that the specific metrics or strategies described here are comprehensive. But we do believe this structure of the problem is compelling and not sufficiently considered. We hope that product designers and government officials will consider it as they engage in their important work.
Please consult the paper for further discussions of why mental health and trust are our initial recommendations for intervention, a much deeper description of the mechanism design and history of public health, as well as a discussion of a range of other issues such as power dynamics and individual autonomy, free speech and legal considerations, and governance systems, as well as several motivating examples.
Back to Top