Some exciting news—I’ve started a new organization to dramatically accelerate the work I’ve been writing about in this newsletter, and we are hiring!

Over the past year, I have been exploring the potential for a new organization focused on a primary subject of this newsletter: providing an alternative to both (a) dangerous concentrations of power and (b) the challenges of providing unfettered access to increasingly capable AI systems.

We believe that high-quality deliberative democratic processes for decision-making on AI (and with AI) can provide a path forward, addressing both core kinds of risk—and enable us to maintain democracy in a world increasingly dominated by AI.

I had helped advocate for and support this kind of work with OpenAI, Meta, and others—and now I’ve founded the AI & Democracy Foundation (AIDF) to accelerate these efforts dramatically.

We’ve raised initial funding and built out part of the team, and are now hiring for critical leadership roles.

Update: See careers.aidemocracyfoundation.org for the current open roles.

There are many other updates since my last newsletter. A few highlights include: wrapping up the initial OpenAI democratic inputs grant program (which led to a number of excellent projects and writeups), working with Metagov to set up another grant program focused on interoperable deliberative tools (using process cards as part of the application), interviews including with the New York Times, and well…several projects incubating at the AI & Democracy Foundation which I’m excited to share more about soon.

• Please forward this to people who might find it useful!

• Share, tag, and follow the new org on X/Twitter (@democraticai) and LinkedIn (AI & Democracy Foundation). You can also reshare the posts about these jobs on X/Twitter & LinkedIn).

• Stay in touch by following on any of those platforms (@metaviv and Aviv Ovadya) and subscribing if you haven’t already.

Subscribe now

I was invited to write for a ‘symposium issue’ for the Journal of Democracy, focused on AI, and I aimed to sketch answers to some of those questions. That paper, titled “Reimagining Democracy for AI” was recently published, and you can read it in full here (though this link may be better for sharing and this is the official paper link).

The abstract:

AI advances are shattering assumptions that both our democracies and our international order rely on. Reinventing our "democratic infrastructure" is thus critically necessary—and possible. Four interconnected and accelerating democratic paradigm shifts illustrate the potential: representative deliberations, AI augmentation, democracy-as-a-service, and platform democracy. Such innovations provide a viable path toward not just reimagining traditional democracies, but also enabling the transnational and even global democratic processes critical for addressing the broader challenges posed by destabilizing AI advances—including those relating to AI alignment and global agreements. We can and must rapidly invest in such democratic innovation if we are to ensure that our democratic capacity increases with our power.

The paper piece aims to summarize and contextualize many of the approaches discussed in earlier pieces (and in other outputs), except focused entirely on AI, for a general educated audience.

Below I’ll provide a few key excerpts.

Motivation

If we continue on our current course, advances in AI may take us down one of two possible paths toward a dystopian future: that of autocratic centralization, where powerful corporations or authoritarian countries unilaterally control extraordinarily powerful AI systems, or of ungovernable decentralization, where everyone has unrestricted access to those incredibly powerful systems and, because there are no guardrails, can use them to cause massive, irreversible harm.

I advocate a third path—that of combined democratic centralization and democratic decentralization—and accelerating investment in the democratic infrastructure needed to make such a path viable.

The core message is that we don’t need to settle for autocratic or ungovernable technology. Improved democratic mechanisms provide a third path.

The meat of the paper goes into the key ingredients: representative deliberations (like those discussed here; more detail), AI augmentation (e.g. in collective response systems, consensus language models), democracy-as-a-service (e.g., deliberation infrastructure providers), and platform/AI democracy. As they have mostly have already been discussed in this series, I won't excerpt those here. But what I have focused less on in my public writing is how these apply to AI alignment and AI governance.

Applications for “alignment” (heavily simplified to explain to non-experts given the very limited space)

"AI democracy" built upon augmented representative deliberations can help in developing the principles for aligning AI systems—that is, ensuring that an AI system operates according to a set of principles. […] While some of the decisions about such principles may be delegated to the direct user of an AI system, there will always be some base set of values that is encoded by default—and which may be required in order to limit severely harmful activity. Currently it is primarily the AI companies themselves […] that are deciding what generative and general-purpose AI systems should align to.

Unfortunately, if unsurprisingly, differences of perspective around such values appear to be exacerbating mistrust and geopolitical risk, as AI organizations and governments with differing values race to ensure that the most powerful systems are aligned with their values—and one casualty of this race is likely to be critical guardrails. Representative deliberation can help to address these challenges by providing a broadly acceptable mechanism for navigating across those competing values, democracy-as-a-service enables corporations to convene such deliberations while staying at arm's-length, and AI-augmentation may even enable such processes to be feasible globally.

Applications for “transnational AI governance” or “global agreements”

To further address these risks and challenges of powerful AI systems, we are likely to need some form of globally agreed-upon policies around the development, deployment, and distribution of such systems—for example, mandating that AI systems should be trained and aligned not to support the development of chemical and biological weapons.

This may sound straightforward, but it brings up a thorny issue related to open-source AI systems. Open-source systems reduce centralized corporate control of AI systems and make research easier. Unfortunately, it might not be possible to prevent people from "retraining" an open-source AI system to overcome its alignment guardrails—this has already been done with some of the most powerful opensource models. And it is impossible to "unrelease" an open system once it has been shared publicly, which means that a single actor could irreversibly impact the entire planet. Some argue that if the risks of such open releases are significant enough, we might need a global prohibition on the development or open distribution of certain types of AI systems.

There is currently significant disagreement about how to navigate such dilemmas, and meaningful consensus is exceedingly difficult to achieve, due to challenges including the speed of change; uncertainty and disagreement around the degree and direction of AI impacts; distrust among key actors; ease of replication; and the lack of a broadly trusted process for weighting conflicting ethical obligations. The same democratic innovations may be invaluable here also, providing a complement to more traditional geopolitical negotiations.

There is much more to say here, particularly in how this can be practically achieved, given our existing economic and geopolitical incentives and institutions. One thing that is increasingly likely is that “business as usual” will be insufficient—the same approaches that have so far averted catastrophe from nuclear or biological weapons, and which have not averted catastrophe in climate, are unlikely to be capable of addressing severe negative impacts of AI, given the rate of advances and extreme uncertainty.

Momentum and next steps:

In the last nine months, we have gone from having almost no recognition of the necessity to think about democratic innovations to seeing almost every major AI company begin to explore how best to incorporate aspects of deliberative democracy into their work. […]

There is incredible capacity and momentum in the democratic-innovation ecosystem, but the rate of AI advances is far faster. I have therefore been exploring the possibility of setting up a fund focused on democratic innovation to accelerate the design, testing, evaluation, and composition of such processes at increasing scale, working in partnership with civil society, academia, AI companies, and multistakeholder and multilateral organizations for implementation.

I would like to see governments around the world developing similar focused funds to ensure that we can rapidly build the capacity to run complex end-to-end processes for both alignment and policy. Corporations advancing AI should also signal their willingness to invest in democratic governance and alignment, with funds pre-allocated for running processes that can satisfy particular criteria, whether developed in-house or externally. This would create a market incentive for rapid investment in the development of implementable democratic processes.

Related to the above, I’m excited to be creating an organization with deep relevant expertise, both to: (1) advise corporations, governments, etc. on their funding and adoption of strategies for integrating AI and democracy, and (2) act itself as a grantmaker to accelerate targeted innovation and adoption in this space.

Conclusion:

It is a great gift that the same technology which is so destabilizing may also be harnessed to help overcome the problems it is creating. […]
There is a tremendous amount that we need to do right now to address present and significant risks and harms—but there is also little time to waste if we want to be ready to tackle the even more significant crises that are coming.

This paper only provides a sketch of what directions we might aim toward and how we might get there. I would be interested to hear what you or your organization find most valuable, confusing, or problematic in it—and what you would like more detail on in order to transform theory into practice. The full paper can be accessed here on my website (the original link here is now paywalled).

• Please share this with people who might find it interesting—and tag me if you share on social media: I’m @metaviv, aviv@mastodon.online, and Aviv Ovadya on LinkedIn.

• Stay in touch by following me on any of those platforms, reaching out at aviv@aviv.me, and of course, subscribing.

Subscribe now

I believe that we will need to rapidly increase our capacity to govern the impacts of technology—and that this will require democratic innovation of and with AI.

However, if we are to rapidly accelerate such democratic innovation, that requires an ecosystem of actors that can efficiently build on each other’s work. Moreover, while major actors and funders in the space are increasingly interested in applying such democratic innovations, the current approaches often don’t take into account the internal structure of democratic processes—structure that is critical for addressing the challenges around competence, alignment, and robustness. Thus, to support democratic innovators, funders, and other key actors, we need a standard way to communicate about the structure and applicability of processes and their subprocesses—to support adoption, interoperability, and evaluation. 

This piece thus introduces a set of concepts for describing democratic processes, and, at a high level, two key kinds of process documentation or “design artifacts” to support such communication. It is inspired by the work of both the democratic process community (with e.g., Participedia and PolicyKit), and the machine learning community (with e.g., model cards, data sheets, and reward reports).

This is also not just theoretical, though I focus here on concepts and documentation—applying these concepts and documentation practices at varying levels of fidelity has deeply influenced my overall work for the past few years; most especially in the past six months when advising AI companies and funders, including the OpenAI Democratic inputs grant program.

Processes at different levels of abstraction

[The next two sections focus on details of terminology; feel free to skip to ‘Key Artifacts’ if that isn’t your jam.]

There are several different concepts that one might be referring to when one talks about decision-making or information-gathering processes (whether they are focused on democratic outcomes, collective intelligence, alignment, etc.). Part of the confusion is that discussions often happen at different levels of abstraction.

The following is a rough taxonomy for distinguishing between those levels (though the lines may sometimes be blurry), going from the most concrete to the most abstract.

Process Run: An exercise, usually involving people, potentially mediated by machines, with inputs, outputs, and potentially additional state changes. 

• Examples: The 2020 presidential election; A Polis on data-driven campaigning, a citizen’s assembly on assisted dying.

• Corresponding computer science concept: Instance

Process Design: A detailed description of how to run a process, sufficient to enable a Process Run. This might be a plan for people to execute, and/or for code for a computer to run.1

• Examples: Presidential elections; Polis processes; a standardized run sheet for a specific style of a 5-day citizens assembly. 

• Relationships to other terms: Every Process Design may have many Process Runs.

• Corresponding computer science concept: Class

Process Pattern: A more abstract specification of the interfaces between the process and the external world (and potentially other processes)—intuitively, the “shape of a process” or the “contract that it satisfies”.

• Examples: Elections, Collective response systems, Citizens’ assemblies.

• Relationships to other terms: Many different Process Designs may satisfy the same Process Pattern interface and vice versa.

• Corresponding computer science concept: an interface or abstract class (in the Java sense)

In sum, one can think of a Process Run as a specific ‘event’ (e.g., 2020 presidential election); a Process Design (e.g., presidential elections) as a detailed description of how to run such events for people and/or machines; and a Process Pattern as a way of specifying a family of Process Designs with many similar properties (e.g., elections).

At each level of abstraction, these can contain one another (e.g., a process may involve many sub-processes), or feed into each other (e.g., a process may involve some output which is then used as the input into another process).

Process Properties

The next definition is somewhat self-explanatory, but still helpful to make explicit if only to distinguish between its subtypes (bolded).

Process Property: A characteristic of a process. Some process properties are designed (e.g. # of participants invited) and others must be measured (e.g., speed, resources consumed), or approximated via measurement (e.g. trust in a democratic process, quality of outputs).

• Examples: 

  • Designed: The output is a statement of up to length N. This will cost around Y dollars per represented person per year.

  • Measured: K participants dropped out.

  • Approximated: Trusted by X% of the participants according to a survey.

• Relationships: Every Process Design, Process Pattern, and Process Run has many Process Properties with particular values.

Key Artifacts: Process Cards and Run Reports

Finally, we move from theoretical concepts to more actionable terms. The following are roughly analogous to approaches to machine learning transparency and documentation that have become increasingly mainstream over the past several years, e.g., the aforementioned “model cards”. They enable practitioners to understand the properties, impacts, and tradeoffs of particular processes. They are not primarily intended for a general audience, but for the process designers, evaluators, procurers, executors, advisors, and communicators (those translating technical details to a general audience).

Run Report: A roughly standardized document providing details about a particular Process Run, including the results of any evaluations of that process.2

• Relationships: There may be many Run Reports for the same Process Card.

Process Card: A roughly standardized document providing details about a process and its appropriateness for different goals, including summary results of evaluations (roughly analogous to a model card in machine learning).

• Relationships: Complete Process Cards can document a Process Design, while Abstract Process Cards can document Process Patterns, leaving out some details. Much of a Process Card is defined in terms of Process Properties.

Process Benchmark: A specification of a (measured or approximated) Process Property (or set of properties) that can be compared across processes, through some standardized approach to evaluation.

• Relationships: Process Benchmarks can be specified for any level of abstraction from Process Runs, to Process Designs, to Process Patterns; this enables comparison across designs. Benchmarks can be referenced in Process Cards and Process Designs.

What do Process Cards and Run Reports include?

[To fully describe this with examples is outside of the scope of this excerpt, but please reach out if you are interested in creating Process Cards and Run Reports, or helping develop a standard structure for them; that is an active work-in-progress. The following is a high-level overview.]

A Process Card should include information that helps identify if a process (a Process Design or Process Pattern) is appropriate for a particular use. It should be generalized enough for anyone relevant who could be interested in running that process. A Process Card should not include topic or process run-specific information. It minimally includes: intended uses (including challenges it can and cannot address), inputs, outputs, additional impacts; and relationships to other processes. If the process has been sufficiently evaluated, it would also include a summary of evaluation results (e.g., an analysis of biases, groupthink risks, quality measures, legitimacy measures, etc.). 

A Run Report should include the specific details of a particular Process Run—one that has been run or will be run (leaving the more general process design to the Process Card). Who was involved, how were they selected, what were the specific inputs and outputs, what were the results of evaluations (including any relevant benchmarks), etc.

Combined, Run Reports (for a process and all of its subprocesses), augmented by the corresponding Process Cards (and potentially references to domain-specific resources) should provide most of the information needed for replication of a Process Run.

Why care?

The overall goal of such documentation is to improve the ecosystem by making many critical tasks easier. Somewhat standardized Process Cards and Run Reports are intended to…

Help process innovators and executors to:

• Learn from each other similar processes.

• Use each others’ processes and even combine efforts if appropriate, e.g. by enabling the identification of the best subprocesses for complex end-to-end processes (through clear descriptions of intended uses, inputs, outputs, and additional impacts).

• Avoid replication of work and enable learning from others' challenges and solutions.

• Choose similar questions and topics in order to facilitate comparisons of the results.

Help funders and advisors:

• Identify organizations that should connect with each other to overcome challenges.

• Route appropriate resources and expertise to innovators.

Help potential process adopters in corporations, government, philanthropy, civil society, etc.:

• Identify which groups and processes are useful to connect with for specific goals.

• See if existing processes already answer their questions.

Help researchers and other evaluators:

• Clarify which evaluation approaches are being used across groups and processes, enabling comparison across them and reducing duplication of effort.

• Compare outputs of processes across similar topics and questions.

• Support the development of benchmarks that enable understanding of improvements (and regressions) in process quality.

From processes to structures

While Process Cards and Run Reports can help move us toward a world of interoperable design, composition, evaluation, and adoption of democratic processes, they are only a first step. True democratic integration requires more than just processes—it requires evolving institutional structures. Many early pseudo-democratic bodies around the world started out as processes that involved the convening of temporary bodies by a king or similar figure. Over the decades and centuries, those temporary bodies became institutionalized structures and took on the forms we know today—including the British Parliament.

Such structures can be thought of as yet another layer of abstraction, connecting processes together into resilient networks, gathering and transforming knowledge, decisions, and power across space and time. Such structures were (at least in some cases) ultimately described and defined in formal documents such as the United States Constitution. As the AI ecosystem develops more experience with innovative processes, or uses of processes, it can begin to explore analogous forms of institutionalization, perhaps documented through standard ‘Structure Sheets’ that designate how processes defined in process cards interact.

Shared language and documentation practices alone cannot create innovation, but they can accelerate an ecosystem of innovation and even foster accountability. We have seen this not only in machine learning, but in fields as disparate as nutrition and mechanical design. If we want to ensure that our democracy capacity can keep up with AI advances, we will need all of the innovation and accountability we can muster!

Thanks to Andrew Konya, Jessica Yu, Shannon Hong, Matthew Prewitt, and Tantum Collins for reading drafts of earlier versions.

• Please share this with people who might find it interesting or useful—and tag me if you share on social media: I’m @metaviv, aviv@mastodon.online, and Aviv Ovadya on LinkedIn.

• Stay in touch by following me on any of those platforms, reaching out at aviv@aviv.me, and subscribing.

  Subscribe now

  Next issue I’ll be sharing a paper laying out a rough vision for the future of AI and Democracy.

This piece uses the platform frame, but all of the content fully applies to the non-state democratic governance of AI systems, ala ‘AI democracy’. It focuses on interoperability with respect to established systems and institutions—a sort of interoperability with the status quo. Future pieces will explore interoperability within the ecosystem of democratic and deliberative innovation.

Readers who are already familiar with platform democracy might want to skip the section entitled “What is platform democracy?”.

Interoperable Platform Democracy

Is there a world where corporations not only run democratic processes for their decision-making—but where such processes are actually a good thing? A world where important and controversial choices facing corporate platforms and AI organizations are decided not by leadership fiat but by a truly representative deliberation (largely outside of government)—and where this is not just ‘democracy washing’?

This piece explores what that world might look like, and how such democratic processes—potentially commissioned by corporations—might beneficially interoperate into our existing institutions of national, transnational, and global governance (hereafter referred to simply as institutions).

Such questions are particularly salient given both Meta’s concrete actions to use such processes (initially to develop greenfield policies in the Metaverse), and AI leaders’ exhortations to "align their interests to that of humanity"—where such processes might be particularly applicable.

There were several key guiding questions that led to the approach outlined below:

• Who should actually be in charge?

• Is it possible to govern tech in a way that moves power to the people being impacted—and away from both corporate leadership and oppressive governments?

• What are pragmatic approaches we can try today to rapidly improve the governance of transnational technologies—in a world where such international coordination seems increasingly difficult?

What is platform democracy?

Platform democracy: Governance of the people, by the people, for the people—except within the context of an internet platform (e.g. Facebook, YouTube, or TikTok) instead of a physical nation.

More formally, platform democracy refers to the use of democratic processes to include the populations impacted by a platform, in the governance of that platform in a representative fashion.

In particular, two approaches to such platform democracy are considered here: intensive deliberative democratic platform assembly processes for complex decisions and lighter-weight collective dialogue processes for decisions that need less context. In both cases, an organization (such as a platform) needs to make a decision that would benefit from democratic legitimacy from the decision.

Such questions might include:

• What if anything should be done about content that is not strictly false, but which is meant to be misleading?

• Under what conditions, if any, should audio or video be recorded in online spaces in order to identify potential harassment, and if so, who should have access to such recordings?

• What kinds of content, if any, should not be shown as ‘trending’?

• What kinds of outputs are acceptable from generative AI systems?

None of these are theoretical. Meta has already directly explored a version of the first two of these questions through such processes; Twitter would likely have asked the third question had there not been an acquisition, and OpenAI’s CEO has described the fourth as a question for which he would like global democratic input.

How are these questions then answered such that the processes are "democratic"?

A microcosm of the impacted population is convened and facilitated by a neutral 3rd party, such that everyone being impacted by the decision (might be e.g. the user base, or the countries the organization operates in) has roughly the same opportunity to be selected (through sortition: stratified random sampling). The selected people make the ultimate recommendation to the decision-maker—and unlike a poll, they are given the opportunity to learn from each other's perspectives (and for decisions that involve significant tradeoffs or context, they also learn from stakeholders and experts). These ‘deliberators’ are paid for their time, and ideally child care, elder care, travel, etc., to reduce the self-selection bias.

Why are these processes legitimate?

The potential democratic legitimacy of such processes comes from their representative nature—instead of every single person having an opportunity to vote, but spending fairly small amounts of time per person, a much smaller number of people vote, but they each are supported with the time and resources to make the best possible decision (without the often perverse incentives of electoral politics or corporate profit). Such processes are also not just some techno-optimistic idealistic dream but are being used by existing governments around the world. Moreover, as any individual has only a small chance of being selected, it is far more feasible to imagine such processes working across many platforms, even globally, than an electoral representation system.

Can deliberative processes work across many languages and cultures?

Such deliberative democratic processes have now been run with many languages a number of times, including several across the EU. Admittedly, interlingual and intercultural deliberation is still imperfect, but there are both process approaches and tools that can help mitigate the risks, and ongoing experimentation to develop best practices around the most challenging aspects (e.g., subtle differences in word connotations across languages).

What happens after the process is complete?

As a slight generalization, when governments run such deliberative processes, they usually serve as recommendations, which must be either implemented, or receive a response from the government about why the recommendation is not being followed. The same can apply when the commissioning organizations are companies like Meta or OpenAI; though it is also likely possible to make the results binding.

Interoperability with existing institutions

Platform democracy does not exist in isolation—it should be structured to support existing institutions instead of fighting them. There are several places where this can happen. To contextualize the options for interoperability with existing institutions, it’s useful to understand the different organizations potentially involved in such a process.

• First, there must be a commissioning organization, e.g. Meta, Twitter, Google, or OpenAI. This could also be a combination of organizations, or even organizations and governments together.

• They commission the deliberation with "deliberative infrastructure providers"— organizations that run these sorts of processes as neutral third parties for governments (and now companies) around the world. The deliberative infrastructure providers also select the members of the deliberation body using sortition and facilitate the assembly itself.

• These deliberation infrastructure providers may work with existing expert and stakeholder bodies to provide context for the deliberators or create a new temporary advisory or stakeholder body to help support the deliberation.

• Finally, the deliberation members learn from those stakeholders, experts, and each other in order to make the final recommendation.

Impacts of platform democracy outputs

The most obvious touchpoint where such a process interacts with the broader world beyond the commissioning organization relates to the impacts of the process outputs.

Media pressure

Let's not beat around the bush here—platform democracy, in its most limited form, can be considered a form of self-regulation. However, it is different from most forms of self-regulation in that the power of creating the mandate is not directly in the hands of the platform. It is instead put to people chosen at random, without any incentive to accede to the platform’s wishes, and facilitated by a 3rd party deliberation organization. (A rigorous process aiming for strong legitimacy would also use as impartial a method as possible of choosing experts and stakeholders.)

Moreover, even if the recommendation is not binding, the legitimacy of the mandate created by such a representative democratic process makes this kind of self-regulation rather awkward for a company to ignore. In plainer words, it looks very bad to the media (and the governments that follow it) if an organization convened a process to have the people tell it what they want in a democratic fashion—and the organization ignored the outputs.

Perhaps even more exciting, a sufficiently transparent and high-profile process not only educates the members of the deliberation themselves, but allows the broader public, media, and even regulators to follow along through broadcast and social media, enabling learning from experts and stakeholders alongside the members. This can potentially help elevate the overall level of conversation on issues with complex trade-offs more effectively than hearings used to score political points, and also help the public see itself through its reflection in the deliberative microcosm.

Raising the responsibility baseline

In fact, recommendations that come out of a process that is seen as broadly legitimate are likely to not only affect the organization that is convening it, but also any other organization facing similar questions and advocacy and interest groups that relate to the question (assuming it is not too specific to the convening organization). If the question is for example around potential responsibility actions, this can help create a corresponding responsibility baseline—a minimal level of action that is seen to be broadly acceptable, which may be higher than the current industry default, raising pressure to implement responsibility practices across the board.

Even if the responsibility baseline is lowered, that is potentially indicative that the impacted population does not actually believe that that level of ‘responsibility’ is warranted (for example, you can imagine a deliberative process that determines that there should actually be less content moderation around a particular issue—which would be a good thing to know).

Creating a responsibility ‘north star’

Some processes may not change the baseline, but may instead create a north star— responsibility practices that might be too difficult to fully execute on, but can be aspired to and approximated. Such north stars may also exert pressure on the entire industry of the commissioning organization.

Identifying global ‘moral high ground’

For some issues, the challenge is not around the ideal north star, or the minimal baseline for responsibility. Instead, there might be deeply competing notions of what responsibility even is. For example, some organizations developing powerful AI systems say that the responsible thing to do is to share as much as possible—maximizing openness. Others are extremely cautious and barely release any information about their research. Both sides say that they are acting for the good of humanity—in other words that they have the ‘moral high ground’. Both argue their perspective to the public and regulators with the intent of shaping perception and law. Similar differences in approach occur in many domains, including in tradeoffs between privacy and security.

A rigorous global deliberative process can create something closer to an idealized public sphere to actually identify what ‘humanity’1 believes is the moral high ground (that such companies should be aiming for). There are thus potentially strong incentives for organizations that believe that they are closer to the ‘true moral high ground of humanity’ to convene such processes, in order to have their approach validated (assuming that they are correct).

Regulatory and institutional suggestions

Such responsibility baselines, north stars, and moral high grounds may then directly impact the actions of legislators, regulators, standards bodies, multilateral bodies, multi-stakeholder bodies, trade associations, etc., in ways that may be binding. In other words, the commissioning organization is essentially fronting the cost for deep research and input gathering that can then directly feed into these existing processes, some of which may have more binding force. Concretely, this might look like, for example, the UK government, the EU, UNESCO, or the Partnership on AI developing recommendations (or, for governments, even laws) directly based on and referencing those deliberative outputs. This could be true even if the deliberative process that was originally convened by Meta or OpenAI—assuming that the process was seen as rigorously impartial and democratic.

Bindingness

There is the option that the convening organization can pre-commit to making an output binding (when otherwise legal), using the legal infrastructure of the jurisdiction(s) they are operating in. There are likely a number of legal instruments that can be used to do this depending on the relevant jurisdiction (e.g. a golden share arrangement).

Conflict with platform democracy outputs

There are some common questions about how this might play out in practice:

What happens when there is conflict with existing law or regulation?

In situations where there are conflicts between the outputs of deliberations and existing laws or regulations, the situation is roughly analogous to when a company’s strong ideological stance conflicts with that of a government. In some cases, this may be seen as good, e.g. when a company avoids sharing location information about democracy activists, thus violating the laws of an authoritarian country. In other cases this may be seen as problematic, e.g. when a ride-sharing company ignores local safety regulations. Either way, if organizations do not follow the laws of the nations they are based in, they face the consequences. The main difference is that if the legitimacy of the process used to create the deliberate outputs is higher than that used by the government (for example in an authoritarian or extremely partisan context), then there may be significant pressure, both externally and internally pushing for the more democratic outcome.

Could the governments or regulators themselves actually be involved in the process?

Definitely, though of course this can become more challenging with more global processes (and thus more governments). It's also worth noting that one of the benefits of the platform itself running a process is that the process can be specific to features that only that platform has, and it may not be worth the time for government officials to be involved with every platform in such a manner. That said, especially for processes that involve multiple platforms or industry consortia, governments may want to act as co-convenors, and platforms may want that also in order to increase the legitimacy of the outcomes.

Could there be permanent deliberative bodies?

There are many potential models beyond the simple temporary platform assembly or collective dialogue, including institutionalized permanent models built on approaches such as multibody sortition, the Ostbelgien model, the Paris model, and which could directly interact with existing institutions in much more sophisticated ways. It feels somewhat presumptuous to explore this in the context of platforms and companies without more understanding and exploration of the basic model, but it is important to know that it may be possible to have key decisive power over an entire company through such processes, as they are refined and combined. One could even imagine augmenting or replacing a traditional corporate board structure with carefully designed deliberative bodies in order to truly enable democratic governance, with no higher executive or board-level power (though feasibility might depend on the jurisdiction).

What happens if there are multiple representative deliberations with conflicting outcomes, perhaps even some run by the governments themselves?

There is no clear answer to this as this entire regime is too nascent. It is perhaps roughly analogous to having multiple treaties or non-binding agreements that are in conflict in a multilateral context. The ideal is likely that the process that is most rigorous and thus most legitimately democratic wins out—but there are many potential interpretations of rigorous, legitimate, and democratic, and no clear arbiter. This suggests that it is particularly important to create international standards for such processes in order to ensure consistent evaluation.

More generally, any time there are multiple competing decision-makers, potentially of varying quality, and no official hierarchy, there is bound to be tension, (ideally productive tension) and there is value in creating institutions to navigate those tensions.

Inputs to platform democracy

Beyond simply interoperating with other organizations through the outputs, democratic process inputs can also interact with existing institutions and organizations at other stages of the process.

These include:

• The commissioning organization could actually be a joint body involving a partnership of a platform (or platforms) with a government or even governments, multilateral institutions etc. The commissioning organization could itself be an existing multi-constituency body such as the Digital Trust and Safety Partnership.

• The expert and stakeholder body could also be an existing multi-stakeholder body such as the Partnership on AI.

• Governments could help support the actual process of sortition selection if they already have ‘sortition infrastructure’ (as e.g. Mongolia has, as illustrated by its incredible turnout for their deliberative democratic process on constitutional amendments).

Why we might want platform democracy

I might prefer a world where purely public institutions fully govern our technological developments and have kept up with the rate of technological change—change that respects no borders. But we have not evolved our existing governance institutions to take on the challenge of legislating at the speed of technology, and that is unlikely to change very quickly.

The realist question we thus face is:

"How can we practically govern an onslaught of technological disruption—and what are the consequences if we fail to do so?"

This is not theoretical—platforms like Facebook, YouTube, and TikTok have shaped society through their policies, but even more impactfully, they have shaped the incentives of society through their ranking systems. These ranking systems determine what kinds of politicians, journalists, or entertainers succeed and shape the kinds of content they produce. Our existing governance institutions, over a decade after this became clear, have done very little to improve the impact of such systems on society outside of the narrow scopes of personalization and privacy.

We can and must do better, both to tackle belated issues and the emerging governance challenges around new technologies. This is especially salient for advances in AI, such as foundation models like GPT-4 and products like ChatGPT built on top of them, which are likely to rapidly transform our lives. Perhaps deliberative democracy can help us find a way forward. Given a steady rhythm of convened processes, decisions might be made democratically, even at global scale, within months instead of years or decades.

Platform democracy alone cannot solve our problems, but it perhaps provides a useful new governance option between our status quo of platform autocracy and platform chaos.

• Please share this with people who might find it interesting—and tag me if you share on social media: I’m @metaviv, aviv@mastodon.online, and Aviv Ovadya on LinkedIn.

• Stay in touch by following me on any of those platforms, reaching out at aviv@aviv.me, and subscribing.

Subscribe now

1. They released results from the ~6500-person, 32-country, 19-language deliberative process I mentioned in my initial newsletter piece (technically their partners at Stanford released it).

2. Meta will convene another process on generative AI!

As readers of this publication know, I had been advocating for such processes and was the sole 3rd party embedded observer for the design and project execution of this first semi-global process (and part of the larger group that was able to observe the actual deliberations). I published a piece in Wired—‘Meta Ran a Giant Experiment in Governance. Now It’s Turning to AI’—that is out today with my high-level reactions to the process and the potential next steps.

In this post, I’ll first give a brief overview of what happened and my role, and then go into significant detail on some of the design decisions, benefits, and failures—focused specifically on the process design and online execution. (Consider the following an adapted excerpt from a working paper; for now at least, this is the canonical citation.) 

Overview

Why does Meta’s deliberative process matter?

We absolutely need new ways for companies, governments, and transnational bodies to make informed and democratic decisions on critical issues that move much faster than traditional transnational policy-making—and especially around the AI systems that will transform our world.

This was one of the most ambitious and global attempts so far to implement deliberative democracy processes that might make this possible.

My perspective

• I am excited to see a company give people more agency to make collective decisions about their products and policies.

• I am fairly concerned about many of the details around the process and follow-through.

• This is an incredible first step—it shows that such things are possible—but there is a lot that should be done differently going forward, both within the structure of the deliberative process itself, and its connection to product and policy.

My 'observer' role:

In my (intentionally unpaid) third-party observer role, I had a carveout to the standard Meta NDA (which I needed to sign for other areas of my work). That theoretically enables me to speak fairly freely about what I observed in the actual implementation of the deliberative ~6500-person process. (I mostly just can't share the meeting documents or anything about particular participants that are covered under privacy law.)

I was able to observe many of the meetings where the deliberation was planned, including all of the (very intense) logistics of running a process across every region of the globe—and of adopting the results across the company. I also separately spoke with many of the key actors involved in running and adopting the results of the deliberation. (Beyond being an embedded observer, I did have indirect influence on the process: informally advising on the pilots; through the questions that I asked; through briefings that I gave to the team; and through discussions I had around options and tradeoffs for different process approaches.)

This is not just about Meta

Meta has done many terrible things either through action or inaction—and I’ve called it out for this repeatedly over the past seven years. However, this approach shouldn't be judged negatively solely because it was done by Meta. Before Meta even considered this, I had reached out to many companies and organizations arguing for this approach (as did a few others as I later discovered). I've linked to this before, but if you haven't already, you can check out this working paper I put out while at Harvard Kennedy School’s Belfer Center and my last piece pulling together applications of deliberative processes to (and with) generative AI.

Meta may not be the ideal first mover—but we shouldn't judge the baby by the bathwater!

Process details

Most of the rest of this piece will get into the details of deliberative process design and execution—feel free to skip to the last few paragraphs if you're just interested in understanding the direct impacts on technology writ large (and check out the Wired article). Also, if you haven't already read my previous pieces on platform democracy and are not familiar with citizen assemblies or deliberative polls, I recommend checking that out to get a bit more context first:

Design & framing: deliberative polls versus citizen assemblies 

There are a variety of different approaches to ‘representative deliberative processes’. Meta chose a deliberative poll, which is somewhat different from the citizen assembly processes that I had previously written the most about. In order to understand the Meta process, we will dive deep into the distinctions between these two competing approaches for representative deliberation.

Deliberative polls and citizen assemblies are similar in many key respects, especially that they both:

• Prioritize considered judgments (i.e. deliberation) over gut opinions (i.e. the way that many people approach referendums, elections, traditional polls, and surveys). This deliberation leads to better-informed results.

• Involve people selected via something like sortition (representative sampling) to form a microcosm of the population—instead of self-selection (e.g. in normal participatory processes) or delegation (e.g. elections). This randomness provides some robustness against powerful interest groups and malicious actors.

While there are significant similarities, and both would be considered representative deliberations, their differences in both framing and process have very significant impacts. In theory, they are roughly analogous to differences in procedural rules and processes in different legislative bodies (e.g. filibuster versus not in the US Senate), but in practice, the differences in framing have many downstream impacts.  

In particular, deliberative polls, in practice, are primarily seen by participants, external observers, and even conveners, as a data-gathering endeavor that may inform governance, while (best-practice following) citizen assemblies are seen as themselves an alternative form of governance. In deliberative polls, the people are often called participants1 and often view it as an experiment or a study, in sharp contrast to citizen assemblies where the people are called assembly members and treated as such (the preferred word is ‘delegates’). Perhaps the lines between these should not be as distinct as they currently are throughout society—but the ramifications of in our current world are potentially significant, with implications on agency, external communications, and privacy. Moreover, as we will see, the currently implemented online system for running deliberative polls can particularly feel like a study, and is far more limited even than what I've outlined above—the process is extremely ‘cookie cutter’, with very little participant agency. 

One of the primary focuses of deliberative polls, which illustrates a difference between data gathering and governance, is that deliberative polls are significantly focused on opinion change before and after deliberation. Knowing about opinion change can be extraordinarily helpful to know as a decision-maker, is a valuable frame for deliberative democratic legitimacy, and has other important implications—but it isn't the sort of thing that people tend to focus on for a governance process such as a legislature. For the purpose of governance, people care about the ultimate decision of the legislature, not where it started. Moreover, in many cases, no one even knows what the questions that should be tracked are; those are identified in the process of creating the recommendations or legislation.

Here is a more detailed overview of the differences in their goals and structures:

• Citizen Assembly: 

  • Treated as a sort of temporary, scoped legislature (usually advisory, but often requiring an action or response).

  • Convened to recommended policy changes.

  • Outputs are recommendations drafted by assembly members and approved through ‘rough consensus’ methods (often aiming for 80% approval).

  • Participants are selected through a variety of means, particularly 2-stage sortition, focusing on demographic representativeness (and sometimes including attitudinal representativeness—i.e., opinions on the issue at hand).

  • Prioritizes agency for members and creating good conditions for collaborative problem-solving in order to develop realistic and actionable recommendations.

  • Longer deeper deliberation (5+ days), including access to the ultimate decision-makers, and often the freedom for assembly members to identify additional experts and stakeholders to bring in to inform their deliberations.

  • Non-standardized structure, albeit with best practice guidelines. There are processes using the term even though they don't live up to the best practices.

• Deliberative Poll: 

  • Treated as an improved form of polling (often referred to as an experiment or study).

  • Convened to understand how an entire population would answer a set of questions if they had more time to think about it (with statistical rigor, including control groups).

  • Outputs are the participant's answers to polling questions (both before and after the deliberation; focusing on opinion change) and recordings of the small group deliberation, for decision-makers to interpret.

  • Participants are selected through a variety of means depending on the location, including: traditional polling firms, survey panels, and GIS-based sampling; with the goal of ensuring both attitudinal and demographic representativeness.

  • Prioritizes approaches for mitigating groupthink dynamics such as domination by the more advantaged, and the common tendency of small groups to go to more extreme positions.

  • Shorter deliberations (1-2 days), with pre-chosen experts and stakeholders and limited opportunity to interact with them (but enabling some people to participate who might not be able to join otherwise; even with significant compensation).

  • Highly standardized structure, and control over the term via trademark.

This isn't a perfect description, and there are notable exceptions; e.g. citizen assemblies that did not live up to the descriptions provided above, or deliberative polls where the outputs were directly tied to decisions instead of simply informing decision-makers. However, it allows us to summarize two of the often contentious ‘camps’ within the world of deliberative democracy and representative deliberations that have very strong opinions on these two kinds of processes. A deliberative poll ‘camp’ perspective is that citizen assemblies have sample sizes that are too small, don’t confirm if their members are truly representative in terms of their perspectives, are too long to be inclusive (because of the burden of participating in longer processes), with too much control by facilitators, and with insufficient evidence for avoiding groupthink dynamics, etc. A common citizen assembly ‘camp’ perspective is that deliberative polls don't do governance at all and are close to giant focus groups (overly focused on statistical significance), don't give participants agency, don't let them actually provide anything more than poll outputs, don't involve much deliberation given their shorter length, etc.

From my perspective, deliberative polls provide significant value due to their empirically backed demonstration that one can, with a specific process design, overcome many of the challenges that have been predicted in deliberative decision-making. For example, ensuring that men, the more educated, etc. do not dominate the small group discussions. I have tremendous respect for the deliberative polling team for developing processes focused on addressing these issues. As a result, I see processes that involve key aspects and lessons from deliberative polling as being useful at the very end of a more elaborate decision-making process, to enable a sort of informed referendum of a microcosm, after a set of policies or principles has been narrowed to a small set or menu of options. However, given my observations, I believe that aspects of the current deliberative polling processes have significant weaknesses (particularly with the current online platform), and there is much that should be incorporated from the citizen assembly world even for that limited application of an ‘informed microcosm referendum’. 

Citizen assemblies provide a whole host of additional ‘capabilities’ that are not present in deliberative polls—including enabling the development of that set of options that might be decided by a microcosm referendum (and they also provide alternative ways to structure a microcosm referendum). However, they also have something to learn from the rigor by which deliberative polls have been studied, e.g. to quantify the extent to which dominant voices might impact the proceedings with different process designs. I would like to see significantly more investment in the study of assembly processes and their subcomponents to see what works and what doesn't.

In both cases, there is evidence to suggest that deliberators feel more empowered as a result of the process. Also in both cases, there is an opportunity for augmenting the very human deliberation processes both internally and externally with AI-augmented tools eliciting and distilling perspectives at scale, at specific stages where they can be particularly helpful (for example, for participatory input from the broader public beyond the deliberators, and as a way to source responses and follow-up questions within the deliberation plenaries).

In sum, I believe that deliberative polls are an effective mechanism for doing exactly what they were intended to do: ‘determine what people would decide if they had more time to think—and to consider some competing arguments in an evidence-based way.’ However, aspects of the deliberative poll framing, process design, and execution (including the current online platform) may over-optimize for statistical and scientific rigor over other factors that matter for deep deliberation, high-quality decisions, and democratic legitimacy. 

Execution

Meta ultimately chose to use a deliberative poll, working with the Deliberative Democracy Lab at Stanford, and supported by the Behavioral Insights Team; and they opted to use the online deliberative polling platform also created at Stanford. 

There are many aspects to running a deliberative process, including for example, a mechanism for governing the overall process, identifying the question or questions at hand, development of briefing material, identifying experts and stakeholders to provide context, participant recruitment, communications to participants, and the management of the deliberations themselves. Here I'll focus primarily on participant recruitment and the online platform used to run and moderate the deliberations themselves.

Participant recruitment

An ideal deliberative process would involve participants who are perfectly representative of the entire group being ‘governed’. There are several best practices that have been developed around this using sortition for influential governance processes. From what I understand, these were not used in this process in many of the regions, and instead, Meta’s process recruited participants using 14 survey panel providers across the 32 countries. There are a variety of reasons that Meta and its partners on this chose to go this direction, many of which are understandable given logistical constraints. If there had been a plug-and-play alternative for best-practice sortition processes across countries, that would have been ideal, and I see this as another critical piece of infrastructure that we need to collectively invest in.

This is particularly important, as the choice of using survey panel providers likely increased the extent to which the process was seen as an experiment or a study, as opposed to a meaningful governance exercise (I personally heard participants referring to the process as an experiment or a study).

Meta did compensate participants for their time, and some of the panel providers did provide ‘concierge’ services to help ensure that they were all set up for success. 

The online deliberative poll platform

The online deliberation system was both impressive—and had significant issues. It was developed by the Stanford Crowdsourced Democracy Team and did exactly what it is meant to do from a process perspective at scale (with just a few minor bugs here and there)—which is incredible for this space, and I hold the team that developed it in high regard. However, it is not yet at the level of polish that I believe is needed for processes of significant societal import, and there are a number of design and process decisions that make it deeply impersonal and in some cases, even dysfunctional, for meaningful deliberation. I see this not as any failure of the team, which has created something novel and valuable but as a result of our failure to be funding democratic process innovation at an appropriate level given our technological capabilities.

A core goal of the system was to be able to scale deliberative processes far beyond what would be possible with human facilitators. There still were human moderators with relevant language expertise available and watching the deliberations as they progressed, coordinating over Slack. However, they were not “at the table”  directing the conversation, but instead acting more like moderators, watching for issues, and addressing them as they came up (this is in theory somewhat similar to what excellent citizen assemblies facilitators do, e.g. Mosiac Lab; but in practice, the online platform experience did not live up to that standard). So humans were still theoretically a critical part of the process of moderating the discussions, but far less people were needed to moderate the same number of participants, which enabled the process to be run at the scale that it was given the timeline and costs.

The online deliberative poll system was essentially a heavily customized video chat system that runs in the browser. Here is a short overview of the kinds of things that worked well, some of which could be adopted in other systems:

• Functionality: 

  • It showed participants instructions for each stage of the process, including a reference to the questions being asked.

  • It allowed groups to progress to the next stage through a voting process (though there were some downsides, as I will cover below).

  • It integrated auto-playing videos to introduce new topics at each stage.

  • It directly integrates the process of writing, editing, and voting on a set of questions for the experts.

• Moderation:

  • It encouraged participants who hadn't spoken to speak up.

  • It provided a ‘speaking queue’ so people could request to speak in a structured way, eliminating interruptions by others.

  • It provided a powerful backend for non-participant moderators to see transcripts in real-time (with potentially problematic segments emphasized) and listen to segments of them, and very rapidly get a sense of the entire conversation with a well-designed expert user experience (so that they could take corrective action if need be).

However, the system, at least as executed within the processes that I observed, also had many significant flaws:

• Deliberation quality:

  • While there was some real deliberation—e.g. weighing of trade-offs across issues—the extent of deeper deliberation versus simply the sharing of opinions appeared significantly lower than I have seen in some other processes, and other facilitators that also were able to observe the process itself shared a similar perspective.

• Impersonality: 

  • In some of the small groups that I observed, many or most people were identified only by numbers, which resulted in impersonal interactions such as participants saying “I agree with 4823" (perhaps this was intentional for privacy, or due to participants not following directions to set their name).

  • The system prevented people from speaking more than 45 seconds at a time, regardless of how much time they had taken in the past, and cut them off mid-sentence. (There are good reasons to do the latter in some cases, but it should be handled much more delicately!)

  • Larger 'plenary' sessions were designed as one-way webinars, with participants having no way to interact with each other or with the experts (the small groups did come up with the questions, so they do get to hear what other groups found important; and in most deliberative polls the person who came up the first draft of the question does get to ask it).

• Lack of meaningful agency:

  • Participants could only interact with each other in the small groups that they were randomly assigned, and had no interaction across groups even within their country or region—not to mention across languages.  Beyond impacting agency, it also dramatically limited the collective intelligence of the group.

  • People were mostly randomly assigned to a new small group every session (though this was a logistical limitation that will be fixed in future sessions).

  • Participants had no way to directly suggest modifications to proposals being provided to them that they needed to choose between. (In theory, this might be picked up in the qualitative analysis of the transcripts of the conversations, but that is different in kind from agency or democratic decision-making; and it would have required that participants be very aware that this was a potential impact of their deliberations—potentially counterproductively leading to more performative deliberations.)

• Process short-circuiting:

  • Each small group deliberation was meant to be an hour and a half, but it appeared to me that in some small groups, participants realized that they could leave early (while still being compensated) if they all agreed to move to the next stage fairly quickly. I saw this happen multiple times, leading to minimal deliberation on the topic, and ending sessions in less than half of the allotted time. (In contrast to the citizen assemblies and offline deliberative polls where participants often wanted more time to make better decisions.)

• Confusion and non-participation:

  • Participants were initially thrust into the video meeting room with instructions to make introductions—but they did not always know how to use the interface. I saw groups where it took several minutes for someone to finally speak up. 

  • Some people never made an introduction. In one group I observed, only 3 people introduced themselves. 

  • I saw multiple people who were entirely silent across an entire small group deliberation, with their cameras turned off, just listening (or completely ignoring it). 

• Disruption: 

  • Some participants would repeatedly take up significant amounts of time by repeatedly adding themselves to the question queue. As an extreme case, one participant was just babbling incoherently continuously throughout the entire small group session that I witnessed, repeatedly disrupting the conversation.

This is far from a complete list of the issues that came up with the process and system. I share all this not to argue that Meta was terrible due to their use of this system—this is a system that Stanford professors and students poured their hearts and souls into long before Meta dreamed up their Community Forums. It has also enabled a new approach to group communication—with far higher quality than, for example, most Facebook groups! (And the experience was still positive enough that 82% of the participants said they would “recommend this event to Meta as a way to make decisions in the future” and ‘78% thought the members of their group “participated relatively equally”’.) Any new modality will likely start out far from perfect, and I think the general approach holds promise for specific uses within a larger process; in spite of these issues, the majority of groups did function reasonably effectively with a reasonable level of quality of deliberation. 

While there are real flaws that need to be addressed, many of these issues can be overcome given sufficient resources for implementation and operational support and the team is actively working on addressing them. I would also like to see significant resourcing to decompose systems like this into modules, enabling experimentation with a variety of processes across applications and organizations (if you are doing or are aware of solid work characterizing these sorts of ‘modules’; ideally even more general than this and this paper, please reach out). My expectation is that the best processes in the future will have some smaller-scale components with very hands-on human facilitation, and also larger-scale components with vastly improved automated facilitation. We can invest now in ensuring that even the automated aspects of facilitation that enable broader participation are also as humane as possible.

Beyond the deliberation process

The deliberation itself is only one piece of the much larger operation. There is much more to say about the gritty details of getting such a process off the ground, making it as neutral as possible on the issues at stake, and then having the desired impact on the organization and society. That will wait for another time since you've already read (or skimmed) over 3,000 words. Suffice it to say for now that I did not see Meta’s hands on the scale in any meaningful way (aside from the implicit impacts of choosing what questions to ask).

Looking forward

Meta did not just announce the results of one massive deliberative poll—it also announced that it would be running another one on generative AI. I've also been developing a set of recommendations to inform future processes for Meta and other organizations that are starting to explore this space. I hope to share more on this shortly, but for now, I’ll just throw one concrete proposal into the mix—I believe that it would be valuable to run a global high-quality deliberative process on the question of “Under what conditions should powerful foundation models be open sourced?” Meta is particularly well placed to be part of setting up such a deliberation given its central place in open-sourcing such models.

Beyond just Meta and its process, above I alluded to a set of challenges where we will need additional investment to develop better deliberative processes. I expect that you'll hear more from me on that soon, as one of my current core goals is to help scope out the critical ingredients (and resources) we will need for effective transnational governance of transformative AI.2

Thanks to Jessica Yu, Andrew Konya, Kyle Redman, James Fishkin, and others for reading drafts and providing constructive feedback, regardless of differences of perspective on the conclusions.

• Please share this with people who might find it interesting—and tag me if you share on social media: I’m @metaviv, aviv@mastodon.online, and Aviv Ovadya on LinkedIn.

• Stay in the loop by following me on any of those platforms, reaching out at aviv@aviv.me, and of course, subscribing.

Subscribe now

This piece gives an overview of my AI & democracy work and provides links to my public writing and speaking. I’ll aim to keep it somewhat updated.

There is a core guiding challenge that motivates my work:

How can we govern the impacts of technology, at the necessary speed and scale, in a way that is legitimate, high-quality, democratic, and (where necessary) global?

Why does this matter? I believe that:

The impacts of AI are being defined by unilateral decisions—either centralized decisions by corporations and autocratic nations, or decentralized decisions to open ‘source’ AI systems.
Unilateral centralization creates dangerous concentration of power and strong pressures to compete for ‘AI supremacy’—leading competitors to cut corners on ethics and safety. Unilateral decentralization, while generally more well-intentioned, is leading to increasingly destructive weaponization of AI systems.

I believe we need viable alternatives to unilateralism and its negative impacts—and that we already have many of the basic ingredients.

Addressing this challenge first involved research exploring and observing a vast array of approaches to decision-making and governance to identify and synthesize mechanisms that:

1. Can be practical and legitimate at different scales—including globally where absolutely necessary.

2. Can provide high-quality informed decisions at sufficient speeds.

3. Can be connected to influence and power.

That research led to deep analysis of two particularly promising approaches—representative deliberations (such as citizens' assemblies) and collective response processes—both of which may be able to satisfy these criteria (with sufficient support infrastructure). These can be informed and supported through e.g., multistakeholder, mass participatory, case development, and simulation processes which also provide promising pieces of the puzzle (alongside some very creative novel processes).

The applied side of my work involves helping technology companies, civil society, governments, and international organizations understand the potential benefits and limitations of such governance processes, navigate the tradeoffs to identify the most appropriate mechanisms, and support their work in applying those methods.

Much of that work is not public, but involved engaging with companies and governments to find allies who could see how this could benefit them, the public, and our critical institutions, and advising on the execution and funding of such processes.

The following are highlights from my more public work.

Overview

📃 Reimagining Democracy for AI | Journal of Democracy (overview; direct paper link; non-paywalled preprint)
This paper provides an overview of much of this work, and why it matters. The following is the abstract:

AI advances are shattering assumptions that both our democracies and our international order rely on. Reinventing our “democratic infrastructure” is thus critically necessary—and possible. Four interconnected and accelerating democratic paradigm shifts illustrate the potential: representative deliberations, AI augmentation, democracy-as-a-service, and platform democracy. Such innovations provide a viable path toward not just reimagining traditional democracies but enabling the transnational and even global democratic processes critical for addressing the broader challenges posed by destabilizing AI advances—including those relating to AI alignment and global agreements. We can and must rapidly invest in such democratic innovation if we are to ensure that our democratic capacity increases with our power.

Deliberation for AI Governance

🎤 Can ChatGPT Make This Podcast? | The New York Times [Dec 2022 • 🐦 thread]
Starting about 31 minutes in, I talk about how we can govern and align AI using democratic processes, including at global scale, building on the ideas of "platform democracy" through citizen assemblies and "generative CI". I back this up with concrete examples of transnational and global deliberations run by the EU and Meta (Facebook); and by the UN using AI to support such governance in wartorn Libya.

📰 Meta Ran a Giant Experiment in Governance. Now It’s Turning to AI | WIRED • [July 2023]
An op-ed I wrote about my interactions with and observations of Meta’s massive deliberative process and recommendations for their upcoming process on generative AI. I argue that we will need such governance innovation if we are to navigate the narrow pathway between autocratic centralization and ungovernable decentralization.

📧 Deliberative Polls, Citizen Assemblies, and an Online Deliberation Platform
A closer look into that deliberative process, including a deep dive into two different kinds of deliberative democratic process (the differences matter!) and the online system used for running the deliberation.

📃 “Democratising AI”: Multiple Meanings, Goals, and Methods | AIES
Section 5.2 describes more formally how global AI governance might work, and how 'representative deliberative processes' (like citizens' assemblies) overcome some of the critical challenges of multistakeholder and participatory processes.

📰 Can ‘we the people’ keep AI in check? | TechCrunch (🐦)
Overview of the idea of using citizens' assemblies for the governance of AI.

📰 Red Teaming GPT-4 Was Valuable. Violet Teaming Will Make It Better | WIRED (🐦 thread)
An op-ed I wrote about getting ahead of the impacts of AI—and reiterating the potential for deliberative processes for AI governance. (I also did a 📻 Marketplace Tech interview aired on NPR covering similar ground.)

📺 Generative AI Is About To Reset Everything, And, Yes It Will Change Your Life | Forbes - YouTube
Well-produced mini-documentary with close to a million views about generative AI—and where I introduce the potential for using deliberative processes for AI governance.

AI for deliberative governance

These papers are somewhat more technical but get into the details of what it looks like to augment deliberative governance with AI.

📃 'Generative CI' through Collective Response Systems | arXiv (🐦 thread)
Distills the key components of governance and sense-making tools like Polis, Remesh, PSi, etc. in a set of key properties and principles in order to help enable a richer understanding of the possibility space. In doing so it, Illustrates a potential correspondence between generative collective intelligence processes and generative AI processes.

📃 Elicitation Inference Optimization for Multi-Principal-Agent Alignment | NeurIPS FMDM
Shows how one can scale collective response processes using elicitation inference with the help of a large language model and a latent factor model (new, more precise, more readable, and updated version coming soon!).

Deliberation for social media governance (motivated by governing AI ranking systems)

One of the core motivations of this work was to provide an alternative approach for determining the objective function of recommender systems—so still definitely AI—though I focused more on policy questions as a foot in the door for company allies (as there is a more obvious set of needs for corporations there that new forms of delegated governance can help address).

📃 Towards Platform Democracy: Policymaking Beyond Corporate CEOs and Partisan Pressure | Belfer Center | Harvard (🐦 thread)
Articulates how democratic governance solves a direct pain point for tech companies and goes into detail on how one particular viable approach, the citizen assembly model, can be applied in the context of a technology platform.

🎤 ‎Techdirt: What Is Platform Democracy? on Apple Podcasts
Podcast with Mike Masnick that goes into more detail.

📰 To build trust, platforms should try a little democracy | Platformer
Well-articulated coverage of the platform democracy proposal by Casey Newton.

📧 ‘Platform Democracy’—a very different way to govern powerful tech (🐦 thread)
Alludes to some of the work I did bringing the platform assembly model to Twitter—which would have been piloted in the summer of 2022 had the acquisition bid not (unintentionally) killed it. Also introduces Meta's 32-country deliberation which I have been a formal 3rd party observer to, in addition to informally advising their pilots.

📃 Interoperable Platform Democracy
How deliberative democratic processes commissioned by corporations can interact with nation-state, multilateral, and multistakeholder decision-making

Enabling a functional information ecosystem

While one of the benefits of representative deliberations is that they can create their own internal robust and informed information ecosystem, the broader information ecosystem still matters significantly and that is where much of my prior (and some current) work has been focused. Many of the insights about the broader information ecosystem can help support representative deliberations and vice versa.

📃 Bridging-based ranking | Belfer Center | Harvard (🐦 thread)
This piece introduced the term bridging-based ranking—an alternative to engagement-based and chronological ranking, meant to overcome their incentives for division.

📃 Bridging Systems: Open Problems for Countering Destructive Divisiveness across Ranking, Recommenders, and Governance | ‘Optimizing for What?' (🐦 thread)
Articulates a research and practice area, around how to build systems that bridge divides, including recommender systems, governance systems, etc.; provides a menu of open problems for research and practice.

A positive future we can aim for

In the past few months, and especially the past month, this space has been particularly exciting as many more people and organizations have started taking concrete steps in similar directions as I describe above. There is an increasing appetite for exploring the intersections of AI for governance, and governance or alignment of AI using citizen assemblies and related deliberative processes and tools. I hope to share more of a map of the developing ecosystem in a future piece.

Finally, I want to highlight two recent pieces that are not from me: this open letter organized by AI Now (with signatories from across the AI ethics and safety “camps”) around the necessity of governance of general-purpose AI and an article by Rumman Chowdhury arguing for the need for global oversight1 of AI. I believe that we can develop ‘wise systems’—including modern reimaginings of democracy—to help with both.

Thanks to Andrew Konya and Rahmin Sarabi for reading drafts of this.

• Please share this with people who might find it interesting—and tag me if you share on social media: I’m @metaviv, aviv@mastodon.online, and Aviv Ovadya on LinkedIn.

• Stay in touch by following me on any of those platforms, reaching out at aviv@aviv.me, and of course, subscribing.

Subscribe now

1. Yes, the following also applies to AI companies and organizations that have thorny questions such as “What guardrails do we need in place to release an AI model?, “What values should the system be aligned to?”, and “How should we tradeoff openness vs. safety in AI model development and release?"
   Making this happen around AI is my primary focus right now (and was always an ultimate aim—see Governance of AI, with AI, through deliberative democracy for more on this).

2. I should also re-emphasize that this is not a ‘tell-all account’; I received permission to share details about the Facebook/Meta and Twitter efforts.

This is meant to be a general audience overview of the proposal described in the paper shown below, and an update on adoption by Twitter and Meta.

Last winter (Jan 2022+), I helped Twitter almost run a pilot of a democratic process for developing its policies—Elon’s acquisition bid killed it (unintentionally). I’ll be sharing more on that project below.

I’m Aviv Ovadya a technologist and researcher working to reimagine how we interact with technology; affiliated with Harvard’s Berkman Klein Center and Cambridge University’s Center for the Future of Intelligence.1 My goal in this newsletter is to share useful ideas and actionable hope at the messy intersection of technology and society—positive visions of a world where technology is ‘good for people’ and concrete actions that may help take us there.

In this piece, I explore one direction that I believe is both crucial and hopeful—and some exciting news about recent steps forward through both successful (and thwarted) platform adoption.

Problem: Platforms are powerful, unaccountable, and stuck

For the last six years, I've spent a lot of my time trying to get platforms like Facebook and YouTube to make changes to address challenges like misinformation and polarization (arguably resulting from their AI recommendation objectives). I’ve also kept running against the same blockers when trying to get change to happen. They aren’t what many people expect—often the biggest blocker has nothing directly to do with $$$ or advertisers or even eyeballs2—it's about fear.

Platforms are scared (for good reason) that influential stakeholders will make their life very difficult if they make changes that hurt those stakeholders or their constituents. These stakeholders include factions in political battles all over the world who run the governments where the platforms operate—and where platform decisions can impact who wins those battles. Platform stakeholders also include powerful and conflicting (and important) civil rights organizations like the American Civil Liberties Union (ACLU) which fights for free speech and the Anti-Defamation League (ADL) which fights against hate speech.

Problems like misinformation, polarization, and hate speech are just hard, and there are no 100% clear answers—so there is essentially nothing a platform can decide that will gain broad trust of powerful stakeholders or the general public. Even if Google or Meta were non-profits, it would be hard for people to trust their actions on such important issues. This leaves the platforms stuck—it means the best thing for them to do (for their self-preservation) is often to take the minimal possible action.

That’s the platform perspective. That sucks for them. But it also sucks for us.

Platforms impact billions with their decisions. Right now, those decisions are primarily in the hands of corporations and unaccountable CEOs—and are heavily influenced by pressure from those partisan politicians and authoritarian states aiming to entrench their power. Better answers to the question of “Who decides?” about platform actions are crucial, as platforms have power over what speech is allowed, how it flows, etc.—and control over these has huge impacts on the viability of democracies and economies.

So who should decide what platforms do about issues like polarization, hate speech, and misinformation? The ACLU or ADL? Republicans or Democrats? Ceos or authoritarian dictators?

I would love it if we could just chuck it all and decentralize everything instead, but it turns out that doesn’t actually help solve most problems. Instead, I’ve spent much of the last few years trying to find the holy grail—a way to get platforms unstuck in a way that benefits all of us. It turns out there is something that might just do the job!

To understand that path, let’s first take a moment to imagine a world where democracy might be working. (This world!)

Context: A reinvention of democracy

Did you know that the European Union convened 800 people, chosen by lottery to be representative from across EU countries, to help decide the future of Europe? The representatives matched the makeup of the EU as a whole (via sortition) and didn’t just spout off their gut opinions, or check boxes in a survey. They spent their weekends deliberating together with facilitation support, learning from experts and each other and drafting and proposing recommendations. People came from 27 countries, speaking 24 languages, with simultaneous translation, compensation for time, etc.—and their work may shape the future of a continent.

This new kind of one-off representative deliberative process worked so well that the EU is going to institutionalize this—to make such democratic bodies a core component of EU decision-making. Similar processes have had success around the world—combining democratic lotteries and deliberation to navigate complex issues from climate change in France to residential zoning in Australia.

These processes are often called citizen assemblies, citizen panels, or citizen juries. The video from The Economist below shows why (and these incredible resources by Claudia Chwalisz’s team at the OECD go into the weedy details). In the current age of low trust, strong democratic mandates are hard to come by—and these approaches may provide that.

A path forward: ‘Platform Democracy’

But what does this have to do with technology and governance? It provides a viable alternative to a broken status quo. What if platform decisions about speech and censorship, algorithms and policies, were not decided by CEOs but by the people who they impact? I call this general concept platform democracy—it is an ideal to strive for.

When I put it that way it perhaps sounds crazy or at least naive. But to be clear, I don’t (currently) advocate for the impractical, e.g. getting rid of tech company leadership and replacing them with “platform democracy.” What I believe is achievable is using democratic processes that have already been validated around the world to move us significantly closer to that ideal. To gradually shift power over collective decisions into the hand of the people that those decisions impact. Devolution of power can be both incremental and revolutionary at the same time.

One specific approach for implementing aspects of platform democracy is the platform assembly—this is analogous to the EU process described earlier.

The image below provides an overview of how it could work:

Why would Facebook or YouTube or Twitter do this? Because platform democracy gets them unstuck! If they can say that a decision was made by a democratic process—honestly say that because it is true—then it can make it much harder for stakeholders to attack it and it can build public trust. Yes, it means they give up some power, but for many decisions, this is still a very good trade.

You might also have a bunch of reservations about the EU convening approach itself. Everything from “Who has time for this sort of thing?” to “You think people chosen at random will make good decisions?” to even “How is this even democracy?”. All of these questions have fairly decent answers, and this would need to be book-length for me to get into all of the gory details. However, a paper I wrote as a fellow at the Harvard Kennedy School on platform democracy provides far more detail and references. (If you prefer podcasts, I go into all of the standard questions here with Mike Masnick.)

Conversations with platforms

For the past several years I’ve been in talks with platforms to get such approaches adopted, at least as a pilot, and for the past year or so I’ve also been working directly with the community of organizations that implement these processes for governments. It is frankly incredible what these organizations have been doing—essentially reinventing democracy for the modern era and getting powerful governments around the world on board.

But my goal wasn’t to get a government on board—it was to matchmake a platform facing a controversial decision where it felt stuck (e.g. “What do I do about political ads?”) with an organization that can run a democratic process that not only gets it unstuck—but does so by putting (at least some) of the power in the hands of the people.

Platform adoption

Almost everyone told me this was crazy, but within a year, two platforms got on board. (Well, two that I can talk about right now.) I teased the Twitter story early, so let me elaborate on that first, now that you finally have the context. As far as I know, this is the first public mention of Twitter’s (former) platform democracy ambition.

The Twitter tragedy

Last winter I started making headway advocating for such a process at Twitter, and after many months of back-and-forth, I was able to connect them with a citizen assembly convener, newDemocracy. Here’s an excerpt from the explanation I provided to Twitter, with a few details removed.

One of the potential questions that the platform assembly would have helped Twitter answer was:

What should we do about potential misinformation that is flagged by 3rd party checkers, given our need to balance free expression with our other values and responsibilities? 

The plan that was decided on in March was to run the pilot assembly such that it would complete its work by the end of September, in time to significantly shape the roadmap for the following year. The team in charge of this secured the necessary approvals and resources, got buy-in from cross-functional teams, etc. I was told that just about everyone who was briefed on this was enthusiastic about it. The hope was that if a small-scale regional pilot assembly worked well, that could start the wheels moving toward global assemblies tackling many of Twitter’s thorniest problems.

Unfortunately, this was when Elon Musk made the Twitter takeover bid in April—and that put everything on indefinite hold. Soon after Musk arrived, everyone involved was being laid off and I got permission to share this story. It currently seems unlikely that Elon’s Twitter will continue with this vision, at least at the moment. But Elon, if you are reading this, feel free to reach out and this can continue where it left off! More generally, I’m always happy to hear from people interested in making this happen at their organization.

The Facebook pilots

While the effort with Twitter fell flat, Meta/Facebook was exploring a similar path—and it kept going. Facebook has now run two pilots across five countries, asking participants to deliberate on the following question:

“What to do about problematic climate information on the Facebook platform?”

These pilots were not quite what I would call a “true platform assembly” (unlike what Twitter was planning to run) but they are very exciting in spite of that. They are the first example I am aware of a representative, informative, and impactful democratic process run by a global platform (representative within the countries they operated in). These pilots are also a stark contrast to Facebook’s useless referendum experiment in 2009. You can read more about them at Casey Newton's Platformer (and as an observer, I hope to share more on them in future issues).

Platformer

Facebook's big new experiment in governance

In June, I wrote that to build trust, platforms should try a little more democracy. Instead of relying solely on their own employees, advisory councils, and oversight boards, I wrote, tech companies should involve actual users in the process. Citing the work Aviv Ovadya, a technologist who recently …

Read more

4 years ago · 38 likes · 1 comment · Casey Newton

The pilots went so well that the only question was what Meta/Facebook would do next. While I had pushed for something closer to a truly global platform assembly of 300-500 people (providing full agency and much more time to assembly members) that was not to be, at least not yet. Instead, Facebook decided to go big. Very big.

Meta’s Mammoth “Community Forum”

This newsletter is being delivered to your inbox shortly after Meta announces its grand plan for scaling up these pilots. Meta will soon run a giant global deliberative process based on the Deliberative Polling approach—what it calls a “Community Forum”. It will involve almost 6000 people, representative across 32 countries, making decisions that will impact the future of the company’s biggest bet.

To me, this is incredibly exciting. Yes, I do have significant reservations about the specific approaches (which I look forward to sharing). I have been embedded in the process of setting this up and there is a lot to say—the devil of democracy is in the details. I am cautiously hopeful that this is just a first stab at the (extraordinary) operational challenges of meaningful and inclusive global platform governance.

I'll leave most of that for future pieces, both here and in other channels. For now, I think it's worth just appreciating how revolutionary this is. Nothing quite like it has ever been done before to help set policy for a company. It's not leadership, shareholders, employees, or elites who are being asked for their input. It's everyday people, matching the makeup of 32 countries, across every region of the world, supported with context, and compensated for their time.

I'm looking forward to seeing what they decide.

• Learn more at my Harvard Belfer paper on platform democracy or podcast.

• Future issues will also cover other topics I’ve worked on over the past decade including algorithms that bridge divides, complementary approaches to platform democracy (e.g. digital tools like Polis and Remesh), and especially ensuring that generative AI has a positive impact on society.

• Share this with people who might find it interesting—and tag me if you share on social media: I’m @metaviv, aviv@mastodon.online, and Aviv Ovadya on LinkedIn.

• Stay in touch by following me on any of those platforms, reaching out at aviv@aviv.me, and of course, subscribing.

Subscribe now

Thanks to far more people and communities than I can list here for their time and insights that have enabled this work, including Iain Walker, Kyle Redman, Rahmin Sarabi, Luke Thorburn, Yago Bermejo, Linn Davis, Peter MacLeod, Adam Cronkite, Claudia Chwalisz, Ieva Cesnulaityte, Helene Landemore, James Fishkin, Alice Siu, Antoine Vergne, Shahar Avin, David Krueger, Stephen Larrick, Leisel Bogan, Afsaneh Rigot, Amritha Jayanti, Nathan Schneider, Bee Cavello, Joe Edelman, Colin Megill, Andrew Konya, Robyn Caplan, Eli Pariser, and the many employees I can’t name fighting for democracy across the tech industry.