There’s no standard for endpoint adjudication – and that’s the risk.

There is no regulatory mandate for endpoint adjudication. No operational regulatory guidance. No industry standard. Many sponsors and CROs running adjudication trials still tend to build processes from scratch today — often without the benefit of best practice.

Designing adjudication from scratch is like building the plane while flying it – but without instruments. And what’s at stake? Adjudicated outcomes feed DSMB data reviews, interim analyses, and the statistical analyses that determine trial conclusions on efficacy and safety. But when the adjudication process is poorly designed, the damage compounds: aging events that slow down study milestones and decisions; low data quality; high rates of rework; budget overruns; study delays; and adverse impacts to data integrity.

In a recent Xtalks webinar, I discussed adjudication best practices with Kristin Prusmack and Emily Trymbiski from the Independent Data Monitoring and Endpoint Adjudication (IDEA) Committee Management department at ICON. We took an in-depth look at process design — common pitfalls, what works and why, and the decisions that separate a well-run adjudication process from one that spends the trial in recovery mode.

When does a clinical trial need adjudication?

Not every study requires adjudication. To determine if adjudication is applicable, asses the endpoints in your protocol, specifically whether they are objective or subjective.

Objective endpoints — a lab value like HbA1c, a reproducible measurement — can be classified without complex medical judgment. The outcomes for these kinds of endpoints may need to be normalized through use of a central lab, but they do not require adjudication.

Subjective endpoints — clinical events such as death, stroke, myocardial infarction —  are classified by applying clinical criteria and medical judgement. That judgement introduces variability, and variability is a concern in clinical event classification for several reasons: medical training and practice differ across institutions and countries; definitions for the same event type differ across protocols; and the criteria themselves are complex.

The way to control for the variability inherent in investigator identification, classification and reporting of clinical events is to engage a small independent committee of experts — an Adjudication Committee — blinded to treatment even in open-label studies, who will apply protocol-specific event definitions consistently across every potential endpoint for a trial. This centralized, independent, expert review and decision-making process is what normalizes adjudicated outcomes, promoting credibility — with sponsors, statisticians and regulators.    

Design the adjudication process backward from your SAP.

Process design makes or breaks any trial, and an adjudication trial is no exception. The important lesson is that you design your adjudication process by working backward from the trial’s ultimate data analysis objectives.

Since the goal of clinical event adjudication is to obtain normalized outcomes for statistical analyses of efficacy or safety endpoints, every process design consideration should be aligned to that goal.

The SAP outlines the adjudicated endpoint outcome data that will be needed for the CSR, interim analyses, DSMB reviews. Start here. These statistical analysis requirements determine your endpoint data capture plan, from reported clinical events to adjudicated outcomes — and every other process design decision builds from there.

The endpoint data capture plan must ensure that all potential endpoints are identified for adjudication. Missed endpoints lead to data gaps that can skew efficacy and safety results. Best practice is a multi-faceted approach that starts with investigator-reporting of potential endpoints, by event category, via a designated endpoint reporting CRF and then strategically layers on event detection methods as supplemental methods where applicable. For example:

• clinical review by CRAs and medical monitors
• programmatic review of AEs and/or labs
• AI-assisted event detection

What is not recommended is detection of potential endpoints through data review and triggering alone, without direct investigator-reporting, as this can lead to significant data gaps. Detection methods can only run on available information, and the investigator has access to the most comprehensive available patient data and records.

The endpoint data capture plan must also ensure adjudicators complete CRFs are set-up to capture the outcome data statisticians will need. Ideally, the data capture on the endpoint reporting CRFs should align with and support the completion of the adjudication CRFs.

The charter comes before the system

If there is one process design principle that matters more than any other, it is this: the charter comes before the system. The charter is where process design culminates. It is the center that holds everything together.

The charter is a comprehensive operating procedure for the Adjudication Committee and the processes that support its implementation. It defines the:

• committee structure, roles and responsibilities
• events requiring adjudication
• endpoint dossier contents
• workflow details for each process from event identification to tracking to case processing and QC to dossier compilation and submission to adjudicator workflows
• event criteria for each event type
• adjudicators will reach final decisions for adjudicated outcomes

When teams start building the system before the charter is stable, every charter revision results in a system change, rework, added cost and a delay. The most common reason adjudication trials end up in recovery mode is that the project team did not start planning for adjudication early enough, resulting in a race to make up lost time. In this race, the charter tends to chase the system, the endpoint data capture plan has to be re-designed because it is not fit for purpose and the rework and delays cascade from there.

The key learning here is that the endpoint data capture plan, the committee, the charter and the system all need to be ready by or before first patient in — and certainly before the first event — making early and thorough, goal-oriented planning an essential focus on the critical path for study start-up on every adjudication trial.

Compile the leanest dossier that still informs the decision

A workflow that does not work will break down under pressure. A workflow that does not flow amplifies its friction across the trial. Optimizing your workflow should be a top priority.

Endpoint dossier workflow design is an art.  The goal is the leanest dossier possible that still gives the committee what it needs to make a complete, informed decision. Every source document added beyond that threshold increases friction and drag:

• increases site burden
• lengthens case closure cycle times
• drives up translation costs
• lengthens dossier review time
• generates more queries

Best practice is to establish site checklists, per endpoint type, of the minimal set of source document records that would typically speak to the endpoint criteria the committee needs to assess. Collect those, then perform content review to ascertain if any further documents are needed to fully address the endpoint criteria. This closes the majority of cases against a completed checklist, while tailoring document collection for the outlier cases that require case-specific document collection.

The lesson: time invested up-front on developing a strategy to compile accurate, focused and complete endpoint dossiers not only delvers what the committee needs for sound decision-making but it also sparks momentum in the adjudication workflow, enabling the faster case closure rates that drive on-time, on-budget, high quality adjudication project delivery.

True arbitration vs a third tiebreaker

The workflow you select for the committee’s decision-making process makes or breaks endpoint data quality, yet this is one of the most underestimated choices your team will make.

The most widely used structure is parallel review. At Phase 1, two independent adjudicators review each case without knowledge of each other’s assessment. If they agree, the case is closed with a final decision. If they disagree, the case escalates to Phase 2 — to a third adjudicator, a chair, or a full committee.

The two-reviewer structure works at Phase 1. Where teams frequently go wrong is the escalation path for discordant cases. Discordance signals a difficult case. Sending this kind of case to a third tie-breaking independent adjudicator — one with no additional experience to resolve the disagreement, and without even telling them the case is discordant — is false arbitration. This third reviewer is not actually resolving the disagreement; they are merely casting another vote. The final outcome is derived from a “majority wins” approach that can comprise two votes for the wrong outcome and one vote for the right one.

True arbitration works differently. At Phase 2, the discordant case is either routed to the chair or the full committee. The case is identified as discordant, and this triggers a different kind of assessment: the root cause for the discordance is identified, and a final outcome is determined, based on this insight. In both approaches, the chair can also judge whether the discordance indicates a case precedent calling for broader action, e.g. revisions to endpoint criteria or additional committee training.

True arbitration workflows leverage full discernment, promote continual process improvement and learning, and build increased consistency of decision-making across the committee over time. It produces higher quality outcomes than a third blind tiebreaker.

Why an online adjudication system beats a manual one

Adjudication workflows are inherently complex — dynamic, cross-functional, interdependent, and driven by data and documents. A common failure point is choosing a manual approach over an online adjudication system. At any trial size, the pain points add up quickly:

• manual handoffs create operational drag
• freeform steps and a proliferation of standalone trackers increase quality risks
• aging cases result in missed deliverables and delayed study milestones
• workload boluses and rework cause budget overruns

These problems only multiply at scale.

In manual adjudication, the work is strained by overhead at every step. Documents move slowly from sites to case processing teams to adjudicators without traceability or structure. Steps like PHI/PII redaction and the identification of discordant cases are manual, time-consuming and prone to human error. Teams working across a mix of email, shared drives, various trackers and multiple systems are working under conditions which neither have much needed guardrails for compliance nor the data structure to provide the required line of sight for meaningful status reporting.

An online adjudication system establishes one unified workspace. Everyone in the process — across teams, functions, organizations, and geography — can collaborate in real-time, exchange data and documents seamlessly and follow a system-driven workflow that makes the process clear, efficient, repeatable, and consistent. Because the system defines the process, governs it, and enforces compliance, the system becomes the process.

The advantages are worth naming. Process automations cut manual effort:

• system integrations eliminate double data entry and manual reconciliations
• translation-vendors integrations expedite translation of non-English source documentation
• smart forms reduce queries by ensuring data accuracy up front
• programmed alerts and reminders prompt timely completion
• consolidated automated tracking replaces multiple manual trackers
• automated identification and routing of disagreement cases ensures complete, accurate final decisions
• AI-assisted redaction keeps a human in the loop while cutting effort and improving quality and compliance

Beyond automation, the system delivers consolidated standard and custom reporting; 21 CFR Part 11, HIPPA and GDPR compliance; and full process traceability where the system itself and its audit trail become the record of who took what action when and why.

The lesson is clear. Your team needs an online adjudication system, and when you plan for the system build, establish a stable charter first, then write the system specifications. For custom builds, allow 4-6 weeks from finalized specifications to go live. Custom builds are sometimes necessary, but for most adjudication trials, consider a pre-configured template. It saves cost and time, and ensures your process design is rooted in best practice. These templates already incorporate proven workflows with core specifications, validation and testing complete – roughly half of the effort and build time than a custom build.

Plan early: experience and technology decide the outcome

The core lesson from the field: planning powered by experience and technology is everything. Engage experts who know adjudication best practice from both an operational and a system vantage point — people who have learned first-hand what works and what does not work. They know the pitfalls and what it takes to get adjudication right. They are flying that fully constructed plane, with instruments, and can get you to your destination.

Start with the end in mind. Invest the up-front planning time to develop a cohesive endpoint data capture plan, design an efficient process, empanel the committee, develop the charter, and build the system.

Ready to see what a purpose-built adjudication system looks like? See Judi in action.

For a deeper look at adjudication best practices, watch the full webinar recording.

‍

‍Cathy Tyner brings 27 years of clinical research experience with a focus on Oversight Group Management for Clinical Endpoint Committees, Data Monitoring Committees, and other operations. A recognized thought leader in endpoint adjudication best practices, Ms. Tyner has authored the industry's seminal guidance on adjudication operations and best practice.

‍