top of page
Institutional paradigm
Attention Economy
Accountability Infrastructure
Status of platform
Cultural technology
Environment for interventions
Maximizing user attention
Population health
Locus of harms
Output-based (acute)
Outcome-based (structural)
Governance frame
Rules-based (deontology)
Empirics-based (fallibilism)
Preferred mechanism(s)
Content moderation
Constraints on system behavior
Harm Type
Individual Content
Harassment / threats, self-harm, copyright infringement, pornography
Rules Based
Aggregate effects of products on users in a society
Discrimination, reduced belief in institutions, mental health harms
Product Constraints
Number of Monthly Users
1 million
Submitted plan for metrics and methods for evaluation of potential structural harms
10 million
Consistent data collection on potential structural harms
50 million
Quarterly, enforceable assessments on product aggregate effects on structural harms, with breakouts for key subgroups
100 million
Monthly, enforceable assessments on product aggregate effects as well as targeted assessments of specific product rollouts for any subproduct used by at least 50 million users, with breakouts for key subgroups
Step 1
Goal Setting
Decide which structural harms are worthy of inclusion in evaluation.
Step 2
Metric Definition
Define an index of metrics to gauge social impact, including standards for what effect sizes are large enough to trigger intervention, and standards for which subsamples must be included in an evaluation.
Step 3
Usage Rules
Set a floor for product usage to be subject to review, potentially with an escalating set of parameters tied to scale.
Step 4
Measurement tools
Implement product solutions capable of measuring the index, such as the use of surveys, PAM, external health devices, or in-system behavior triggers.
Step 5
Team Structure
Identify a body of evaluators independent from product developers to perform analysis, with an oversight system.
Step 6
Reporting Rules
Define transparency and reporting requirements that ensure compliance in implementation.
Step 7
Mitigation Procedures
In the event mitigation steps are needed, create a tracking system to ensure sufficient compliance.
Step 8
Escalation Procedures
Provide an escalation system in the event that mitigation steps are not followed or are not sufficient to address the structural harm.
Step 9
System Revisions
Provide for a system to update the regime inclusive of triggers for intervention, product solutions which define the index, or the addition or removal of top-level structural harms for inclusion.
bottom of page