Method — evaluating assemblies and deliberative processes
Updated: 2026-10-05. Task E. Warranted: the corpus contains evaluated cases (Scotland research reports, CA UK inquiry) plus an OECD evaluation framework (SRC-0119) and KNOCA impact framework (SRC-0136).
Frames
- Process quality ≠ policy agreement ≠ political impact — a “successful” assembly is not one whose recommendations were adopted; evaluating against adoption confuses the assembly’s work with the institution’s response (SRC-0119, SRC-0117).
- OECD evaluation guidelines distinguish evaluating the process, the outputs, and the outcomes (SRC-0119).
Input evaluation
- Representativeness vs declared strata + published benchmark table (SRC-0038 model).
- Invitation/response funnel rates; attrition; demographic drift between selection and sitting (SRC-0123, SRC-0120).
- Inclusion/accessibility coverage: were support provisions actually delivered (SRC-0101).
- Corpus instance: CA UK published composition vs benchmarks; Scotland CA research report evaluated recruitment (SRC-0044).
Process evaluation
- Equality of voice; deliberative quality (measurable via observation/surveys — Bächtiger & Parkinson framework, SRC-0114).
- Facilitator influence; evidence balance; agenda integrity (did members shape the agenda within the remit?) (SRC-0102, SRC-0129).
- Participant learning — knowledge gain is the most reliably measured effect (SRC-0106).
- Corpus instance: Scotland CA member-experience research report (SRC-0044); CA Scotland research report (SRC-0045).
Output evaluation
- Recommendation clarity/actionability; recorded strength of support; minority positions captured (Leeds tiers, Ireland minority reports — SRC-0047, SRC-0017); internal consistency.
Outcome evaluation
- Institutional response (was it given? on time? substantive?); adoption; implementation (delivery, not promise — Paris droit de suite is the tracking instrument, SRC-0027); policy impact; public legitimacy; participant trust/attitudes; longer-term institutional change (e.g. CA Scotland → statutory climate assembly, SRC-0045→0041).
- KNOCA/CAST impact-evaluation framework is the practitioner instrument (SRC-0136).
Evaluation independence
| Model | Properties | Corpus instance |
|---|---|---|
| Commissioner-run | fast, resourced; conflicts obvious | Scottish Government research reports — official evaluation of own process (SRC-0044, SRC-0045) |
| External academic | independent, methodologically stronger; slower | Hansard Society external critique at Senedd 2019 CA (SRC-0137); Fournier et al comparative evaluations (SRC-0127) |
| Practitioner self-evaluation | insider knowledge; self-interest risk | organiser reports common across corpus |
| Participant surveys/mixed | member experience captures inclusion | Scotland CA research report (SRC-0044) |
| Conflict flag: designers evaluating own processes is common — KNOCA/OECD independence framing applies (SRC-0119, SRC-0136). |
Comparative closeout — evaluation evidence in the corpus
- Strong independent/official evaluation: Scotland CA (research report + member experience, SRC-0044); CA Scotland (research report, SRC-0045); CA UK (select-committee inquiry + government response, SRC-0040).
- Mainly organiser self-report: Newham, Leeds, Camden, Oxford, Devon, Assemble/HotP (organiser/commissioner documents dominate).
- Recommendation-level implementation tracking: Paris droit de suite (SRC-0027); Brussels climate assembly administration follow-up (SRC-0132); elsewhere weak/absent.
- Weak evaluation evidence: local jury cases; movement-led bodies (no independent evaluation evidenced).
Cicada relevance
- Evaluation is a recordable property: who evaluated, independent or not, at what stage, against which of the four frames. Records should never treat “recommendations adopted” as the success criterion.