Scenario 08 · Operational Review, Organizational Learning and Value Articulation

Part 01 · Scenario boundaries

What this page covers, and what it does not

Move the familiar review up to the layer of organizational assets before any framework.

Scene description

Scenarios 02, 03 and 04 form judgements about priority, resource and team. Scenario 05 advances cross-functional decisions into execution. Scenario 06 handles goal, information and incentive alignment with external partners. Scenario 07 judges whether important quality issues are controlled and whether corrective actions are effective.

Scenario 08 picks up the outcomes left by those scenarios and asks the next question:

Once the project has produced a result, or paid a cost, how does a Clinical Operations leader judge what the organization has actually learned, and make sure the learning changes the next decision and action?

Operational review should not happen only at the end of a study. Any of the following moments may deserve one:

【1】A key governance decision has been implemented.

【2】A protocol amendment, country adjustment or vendor change has completed.

【3】A critical milestone has materially slipped or accelerated.

【4】A project has recovered from a major deviation.

【5】A recovery plan did not produce the expected effect.

【6】A near miss could have caused a serious consequence.

【7】An unconventional practice produced a repeatable result.

【8】A key person is about to leave, rotate or hand over.

Scope

This page covers how project experience enters organizational learning, including review selection, fact reconstruction, assumption calibration, tacit-knowledge elicitation, asset formation, cross-program reuse and professional value articulation.

The page does not assume that only the project commissioner can convene a review. CROs, service providers, regional teams and other clinical-operations organizations also need to convert contracted work, interface experience and internal execution into reusable capability.

This page covers

Which events justify a structured review?
How can the team avoid rewriting yesterday's decision from today's answer?
How should the team separate an action mistake, a mechanism flaw and an underlying assumption error?
Where can one project experience transfer, and where is its boundary?
How can the tacit judgement held by PMs, CRAs, site teams and external partners be made explicit?
What tools, rules, training or governance actions should a review conclusion produce?
How can the team confirm that the lesson is actually retrieved in the next study?
How can organizational learning be converted into credible, anonymized and evidence-based personal professional articulation?

Out of scope

Performance evaluation or accountability as the primary purpose of a review
Replacing deviation investigation, root cause analysis, CAPA or quality-system requirements with a review
Technical selection of an enterprise knowledge-management platform
Public disclosure of project confidential, participant or commercially sensitive information
Promoting a single project experience into a universal rule
Fabricating results or exaggerating personal contribution for hiring or promotion
A single company's fixed study close-out procedure

Expected outputs

Project After-Action Review Record
Event and decision chronology
Single-loop / double-loop learning assessment
Tacit-knowledge interview record
Lesson boundary
Updated rule, tool, template or training case
Learning conversion and cross-program reuse register
Anonymized professional capability case

Part 02 · Breaking down the problem

Core question

How can a Clinical Operations leader turn the facts, judgements, mistakes and effective practices of one project into organizational assets that change the next decision and action?

Six judgements around the core question

1. Does the event justify a structured review?

Did the event affect the critical path, participant protection, reliability of trial results, budget or an important development window?
Did the outcome differ materially from the original prediction?
Has the same friction appeared across sites, countries, service providers or projects?
Did the team come close to a serious failure but avoid the consequence through temporary action?
Did an unconventional practice produce a better-than-expected outcome?
Is important knowledge concentrated in a few people and at risk of being lost through turnover?
Is there a realistic opportunity to use the finding elsewhere?

Review capacity is limited. A COD should not turn every action item into a formal exercise. Priority belongs to events with high impact, high uncertainty, recurrence or significant learning value.

2. What actually happened at the time?

What were the original plan, success criteria and expected outcome?
What information did the team have at each critical point?
What was unknown, ignored or absent from the governance discussion?
Who made which decision, and when?
Which facts, professional judgements and assumptions supported the decision?
Which signals were already present but not recognized as risks?
Which explanations emerged only after the result?
Where did the outcome diverge from the forecast?

This judgement reduces hindsight bias. Knowing the outcome does not justify assuming that the team should have known information that emerged later.

3. At which level did the gap occur?

The findings can be separated into three levels:

【1】Action level: The direction and rules were broadly sound, but an action was not completed or was performed poorly.

【2】Mechanism level: Roles, process, measures, resources, governance cadence or collaboration interfaces made the right action difficult.

【3】Assumption level: The organization's understanding of sites, participants, service providers, complexity, timelines or its own capability was wrong.

An action-level issue may require correction, training or stronger execution. A mechanism-level issue calls for changes to process and management design. An assumption-level issue requires the organization to revisit how the problem and success were defined.

4. Where can the lesson transfer?

From which development phase, therapeutic context, country environment and delivery model did the lesson arise?
Which conditions must be present?
Which findings depend on the current team, system or service provider?
Which judgements can transfer to a related trial or adjacent situation?
Which guardrails are needed to prevent mechanical copying?
Is there a second case or other evidence?
What new evidence would invalidate the lesson?

One project is one sample. Organizational learning requires both a useful pattern and an explicit boundary around that pattern.

5. Which knowledge still exists only in people's experience?

Who was closest to the work as the event unfolded?
Which judgements never appeared in a plan, meeting record or formal report?
Which early signals were recognized by experienced PMs, CRAs, site teams or service-provider staff?
Where did informal coordination or personal relationships keep the project moving?
Which exceptions worked, and why?
Could a new team member understand the difficult decision by reading the file alone?
Which judgement would disappear if a key person left?

Many clinical operations capabilities cannot be converted directly into an SOP. Interviews and shared review should not merely collect stories. They should make judgement signals, boundary conditions and action logic explicit.

6. How will the finding change the next action?

Which rule, template, training module, governance gate or decision criterion should change?
Who will create and maintain the asset?
Which roles and projects need to use it?
At which project point should use be triggered?
How will users find and understand it?
What evidence after the next use would show that it helped?
When should the asset be revised, expanded or retired?

A review that produces only an archived document has not completed the learning process. The finding must return to project operations.

Part 03 · Judgement framework

COD framework: from project experience to the next action

Scenario 08 uses a four-part framework: Reconstruct the situation → Test the assumptions → Convert the knowledge → Return it to action. After-Action Review (AAR) provides a way into the lived situation. Double-loop learning determines the depth of inquiry. SECI helps personal experience become usable team knowledge. Cross-program reuse tests whether the organization has genuinely learned.

Reconstruct the situation → Test the assumptions → Convert the knowledge → Return it to action

1. Reconstruct the situation

Rebuild the process with information available at the time, without using today's answer to rewrite yesterday's decision

The team first creates a complete event and decision chronology, including: the original plan and success criteria; critical assumptions; important events and signals; information available at the time; formal and informal decisions; actions taken; results and deviations; external conditions outside reasonable control.

Four familiar AAR questions provide a practical entry: What was expected to happen? What actually happened? Why was there a difference? What should be retained or changed?

The questions are simple. The quality of the discussion depends on the environment. Participants must be able to offer different memories and explanations. The project leader should not announce a preferred answer before the evidence is examined, and the review should not quietly become a performance-rating meeting.

Formal records provide only part of the evidence. Meeting minutes normally capture decisions, but may omit discarded options, uncertainty and informal coordination. A strong review checks system data, planning versions, governance records, service-provider deliverables, site feedback and participant accounts against one another.

Outputs: event and decision chronology, shared fact base, original objectives and assumptions, known and unknown information, and outcome variance.

2. Test the assumptions

Determine whether the necessary change sits in an action, a management mechanism, or the organization's interpretation of the project

Single-loop learning asks: with the existing objectives and rules unchanged, how can the action be performed better next time?

Double-loop learning asks a further question: were the objective, rule, metric and underlying assumption appropriate?

After a site activation delay, "start contract follow-up earlier" is an action-level improvement. Redesigning the dependencies across contract, ethics, materials and systems is a mechanism-level change. Examining whether the organization mistakenly equates "site activated" with "site ready to screen" reaches the assumption level.

Double-loop learning does not require every review to overturn the process. Some failures do arise from execution. The COD should use evidence to choose the depth of change. "System issue" must not erase clear individual accountability, and "human error" must not hide a management environment that repeatedly produces the same outcome.

Outputs: action-, mechanism- and assumption-level findings; practices worth retaining; and management approaches to stop, change or test further.

3. Convert the knowledge

Turn individual experience into knowledge that a team can understand, debate and use

SECI describes four modes of knowledge conversion: Socialization, where people who experienced the event review, observe and compare practice, including what never entered the formal record; Externalization, where judgement cues, causal interpretations, practical rules and exceptions are expressed; Combination, where experience is integrated with data, process, other cases, existing templates and professional requirements; and Internalization, where training, simulation, tool use and practice in another study turn explicit knowledge back into working capability.

SECI is not shorthand for "write the experience down." The difficult work is to unpack "an experienced person just knows": Which signal was observed? Why did the signal matter? Which options were considered? Under which changed conditions would the original practice no longer apply? How could a new colleague make a sound judgement in a comparable situation?

Organizational assets can take several forms: a judgement checklist, integrated impact assessment, governance rule, case, training module, decision threshold, interview question, metric definition or system trigger. Not every lesson belongs in an SOP.

Outputs: tacit-knowledge record, boundary conditions, tools and rules to update, anonymized cases and training material.

4. Return the learning to action

Make the learning findable, retrievable and testable in the next study

An organizational memory needs more than storage. Each material asset should have: a clear name and purpose; intended users; conditions of use and non-use; an accountable maintainer; a location; a project trigger for retrieval; a planned next-use project; and feedback and version history after use.

Retrieval deserves explicit management. A knowledge base can contain hundreds of documents, yet none will influence a protocol, service-provider selection, site start-up or amendment unless the workflow contains a trigger to retrieve the relevant learning.

Cross-program reuse is not copy-and-paste. The next study should record what was adopted, what was adapted, what outcome followed, and which earlier assumption was supported or contradicted by new evidence. Organizational capability develops through repeated use and correction.

Outputs: Learning Conversion and Cross-Program Reuse Register, updated project tools, retrieval triggers, subsequent evidence and version record.

Eight-step organizational learning loop from project action to recalibration with new results — **The learning loop.** Project action → Actual outcome → Reconstruct the situation → Test the assumptions → Convert the knowledge → Update rules and tools → Use in the next study → Recalibrate with new results. The objective is not a larger archive of lessons learned. The objective is better judgement in the next project.

Part 04 · Worked example

How did one protocol amendment become a multi-country, multi-system implementation breakdown?

The example demonstrates the method. It does not describe a specific trial or imply one correct answer.

Starting point

An ongoing multicentre clinical trial requires a substantial protocol amendment. The amendment affects eligibility criteria, visit and assessment schedules, safety monitoring, selected data collection and informed-consent information.

The project seeks to finalize and submit the amendment quickly in order to limit the effect on the development timeline. Each function advances its assigned deliverables, but implementation begins to fragment: submission, approval and implementation dates differ across countries; the protocol, informed-consent forms, eCRF, IRT, laboratory manual, supply plan and training material are not ready at the same time; some sites have completed training while supporting systems remain unavailable; some sites have approval but no clear instruction on when to switch; participants already on study, participants in screening and newly enrolled participants require different transition arrangements; the CRO, central laboratory, IRT provider and other service providers continue to raise additional requirements and change orders; the project relies on manual tracking, temporary meetings and a small number of experienced people to control document versions; the amendment is eventually implemented, but the project absorbs repeat training, site confusion, execution deviations, data-cleaning pressure and additional cost.

The easiest review conclusion is: next time, communicate earlier, strengthen cross-functional coordination and use a more detailed tracker. The suggestions are not wrong. They do not explain why separate functions completed their assigned work while integrated implementation came close to losing control.

Step one: reconstruct the situation

The review team rebuilds the chronology from the first amendment concept to final site transition: When was the amendment need confirmed? What was the amendment intended to solve? Which content remained unstable before finalization? When did each function receive sufficiently stable input? Which systems and service providers were included in the preliminary impact assessment? When were country submission, ethics review, ICF, eCRF, IRT, supply, laboratory and training deliverables ready? Who decided that a country or site could switch? Which criteria defined "implementation-ready"? Which dependencies were known but excluded from the integrated plan? Which risks became visible only when sites began to execute?

The review cannot rely only on the project management team. Clinical Development, Clinical Operations, Regulatory, Data Management, Biostatistics, Safety, Supply, the CRO and critical service providers each saw a different process. The formal chronology is checked against change records, training evidence, system release dates, site feedback and governance material.

The reconstruction shows that the failure was not one late deliverable. The project remained for an extended period in a state of parallel protocol versions, asynchronous systems and ambiguous country and site transition criteria.

Step two: test the assumptions

The findings are separated into three levels.

Action level: Some deliverables missed their agreed dates. Several dependencies did not enter the integrated plan on time. Sites were not managed to a consistent issue-resolution standard. Training status was not linked to system and material readiness.

Mechanism level: Functions performed their own impact assessments without one integrated dependency map. The protocol final date was treated as the common start signal, preventing early preparation of long-lead activities. Functions declared their own readiness without country–site–participant integrated readiness criteria. No role held the integrated judgement for parallel versions and site transition. Critical service providers entered too late, compressing contract change and technical preparation until after protocol finalization.

Assumption level:

【1】Assumption one: implementation work can only begin after final protocol approval. Many dependencies can and should be identified once content is sufficiently stable. Waiting for a final version removes the remaining schedule margin.

【2】Assumption two: a protocol amendment is principally a Medical and Regulatory document change. An amendment changes site work, participant pathways, data flows, system configuration, supply requirements, contract scope and training. Operationally, it is a controlled change to an interconnected trial system.

【3】Assumption three: if each function completes its deliverable, integrated implementation will follow. Green functional statuses do not prove that a country, site or participant can use the new protocol safely and consistently.

【4】Assumption four: approval means ready to implement. Regulatory or ethics approval is one condition. ICFs, systems, supply, laboratories, training and site confirmation may still be incomplete.

【5】Assumption five: every country and site follows one transition path. Projects may operate with several versions for a significant period and need explicit transition rules at country, site and participant level.

The review moves from "follow up earlier next time" to a redesign of amendment governance.

Step three: convert the knowledge

The team invites the PMs, CTMs, CRAs, Data, Regulatory, Supply and service-provider representatives who handled the disruption to explain the judgements behind the work: When is amendment content stable enough to begin downstream assessment? Which changes require early notice to IRT, central laboratory and supply teams? How should approval, system readiness, site readiness and formal transition be distinguished? How should participants already on study move between versions? Which signals show that the amendment timeline is no longer feasible? Which dependencies require project-level management rather than separate functional tracking? Which temporary practices reduced risk, and which merely deferred the problem?

Their experience is combined with formal data and other project cases to produce: an integrated protocol-amendment impact assessment; a dependency map between amendment content and downstream deliverables; a country–site–participant version-status model; integrated amendment readiness criteria; amendment governance gates and escalation conditions; early-engagement rules for critical service providers; and an anonymized training case.

Step four: return the learning to action

When another study requires a protocol amendment, the project team retrieves the learning at defined points:

【1】Begin preliminary impact identification when the amendment concept forms.

【2】Run a cross-functional integrated impact assessment once content is sufficiently stable.

【3】Confirm the critical implementation path and long-lead dependencies before finalization.

【4】After country approval, continue to assess system and site transition readiness.

【5】After implementation, verify site execution, data capture and participant transition.

【6】Update the rules and tools with evidence from the new study.

The reuse record shows that some practices apply to all material protocol amendments, while certain transition rules depend on the country and system configuration. The team refines the boundary instead of copying one project model into every study.

Core judgement from the case

A protocol amendment is not only a changed document. It is a controlled change to the clinical trial operating system.

If the project manages only protocol finalization and regulatory approval, much of the implementation risk is deferred to countries, sites, systems and participants. The contribution of Scenario 08 is not to preserve the memory that "the amendment was chaotic." It is to change how the next amendment is governed.

Part 05 · Critical actions

What a Clinical Operations leader does in this scenario

Select events with sufficient impact, uncertainty, recurrence or learning value for structured review.
Reconstruct the original plan, information, assumptions, decisions, actions and outcomes before discussing causes.
Distinguish information available at the time from information discovered later.
Bring the accounts of functions, regions and external partners into one evidence-based chronology.
Separate action-, mechanism- and assumption-level findings and choose a proportionate depth of change.
Review failures, near misses and unexpectedly effective practices.
Define the context, boundary and untested aspects of every material lesson.
Use interviews, shared review and case reconstruction to elicit tacit judgement from experienced staff.
Convert learning into a rule, tool, case, training module, metric or system trigger.
Assign an asset owner, retrieval point and next-use project.
Test the learning in another study and revise it with new evidence.
Use anonymized evidence to explain personal contribution, while distinguishing team outcome, individual judgement and formal decision authority.

Part 05 · Role boundaries

Role boundaries

Coordination responsibility does not mean replacing professional functions.

Function boundaries at a glance

Clinical Operations leadership: Selects the review topic; protects access to facts and differing accounts; convenes cross-functional reconstruction and learning; carries findings into project tools, governance and training; and tracks cross-program reuse. The leader should not predefine the "correct lesson" or turn the review into a disguised performance hearing.

Project team and specialist functions: Provide information available at the time, professional judgement and action rationale; verify factual accuracy; determine whether functional rules need to change; and define boundary conditions. Medical, Statistical, Quality, Safety, Regulatory and Data conclusions remain with the authorized functions.

PMs, CTMs, CRAs and front-line practitioners: Contribute field information, early signals, exceptions and constraints absent from formal files. Practitioners are not merely interview subjects. They should help determine whether a new team could actually use the resulting knowledge.

CROs and other external partners: Where appropriate, provide delivery evidence, resource changes, interface failures and improvement ideas. Contract disputes, confidentiality obligations and quality accountability should remain distinct from the learning discussion. "Open review" must not erase formal responsibility.

Quality Assurance or organizational-learning roles: Support quality-system, investigation, knowledge-management or learning methods according to the organization's model. Formal quality investigation, CAPA and effectiveness verification remain subject to applicable procedures and cannot be replaced by a general review.

People managers and capability-development teams: Resource cross-program reuse, case-based training and knowledge maintenance. Professional value articulation may support development and promotion, but organizational learning should not be reduced to individual performance material.

Part 05 · Related capabilities

Related capabilities

Organizational learning, systems judgement, project governance, knowledge management, cross-functional facilitation, evidence-based articulation, people development, case development and knowledge transfer.

The boundary between review and performance assessment

Operational review needs psychological safety and factual accountability.

If every statement flows directly into a performance rating, participants are likely to conceal uncertainty, workarounds and early mistakes. If the review avoids responsibility altogether, the organization cannot address clear misconduct, negligence or non-compliance.

The purposes and processes should be separated:

【1】Operational review reconstructs the process, tests assumptions and improves the system;

【2】Quality investigation establishes facts, impact, cause and formal action;

【3】Performance management addresses role expectations, capability and behavioural accountability;

【4】Confirmed facts may be shared when appropriate, but the three activities should not be collapsed into one meeting.

Part 06 · Tools for this scenario

Tools for this scenario

1. Project After-Action Review Record

Use when an important decision has been implemented; a milestone has materially deviated; a protocol amendment or recovery plan has completed; a near miss occurred; or an unconventional practice produced a notable result. Problem addressed: prevents review from recording only the outcome and reconstructs the original objective, information, assumptions, actions and result. Core fields: review topic and learning purpose; original plan and success criteria; event and decision chronology; information known, unknown or overlooked at the time; critical actions and rationale; actual outcome and variance; practices to retain, change or test further.

2. Single-Loop / Double-Loop Learning Assessment

Use when the review repeatedly concludes with "communicate more," "train again," "plan earlier" or "follow up closely." Problem addressed: distinguishes an execution problem from a management mechanism or an underlying objective, measure and assumption that needs to change. Core fields: observed problem; action-level explanation and change; mechanism-level explanation and change; assumption-level explanation and change; supporting and contradictory evidence; selected depth of improvement; decision owner; verification approach.

3. Clinical Operations Tacit-Knowledge Interview Guide

Use when a project depends heavily on experienced people, formal files cannot explain key judgements, or a team is preparing for handover or turnover. Problem addressed: converts "an experienced person knows" into judgement cues that can be observed, discussed and learned. Core fields: earliest signal noticed; why the signal mattered; situation not covered by the formal process; options and trade-offs considered; effective and ineffective workarounds; conditions and exceptions; points most likely to mislead a new colleague; recommended case, tool or training output.

4. Learning Conversion and Cross-Program Reuse Register

Use when a review has produced a material finding that needs to become an organizational asset and be tested through later use. Problem addressed: connects knowledge conversion, asset maintenance, project retrieval and effectiveness evidence in one record, preventing lessons from ending in a shared folder. Core fields: source and core finding; conditions of use and non-use; organizational asset created; intended user; maintainer and location; project trigger for retrieval; next planned use; actual adoption and adaptation; result and new evidence; decision to retain, revise, scale or retire.

Supplemental professional-value record

Professional articulation does not require a fifth core tool. It can be an anonymized output from the fourth register:

Project context → My judgement → Critical decision or action advanced → Organizational change → Outcome evidence → Transferable capability.

The account should state where formal decision authority sat, distinguish team outcome from individual contribution, and remove product names, site identities, participant information, non-public data and commercially sensitive information.

Part 07 · Intellectual foundations

Foundations behind the methodology

The three primary methods address different problems. After-Action Review (AAR) provides a route into the lived situation. Double-loop learning determines the depth of inquiry. SECI explains how personal experience enters organizational knowledge. Organizational memory provides a test for the final step.

Conceptual source

After-Action Review: AAR provides a route into the lived project situation. The familiar questions concern intended and actual results, the reason for the difference, and what should be retained or changed. The value lies less in a fixed template than in evidence-based participant self-discovery. This page treats AAR as a practice developed over time in U.S. Army training and operations rather than attributing it to a single invented authority. AAR does not replace formal quality investigation and cannot work well when participants lack room to offer an honest account.

Double-loop learning: Double-loop learning determines the depth of review. Single-loop learning corrects action under existing objectives and rules. Double-loop learning examines whether the objectives, rules, metrics and underlying assumptions remain valid. Scenario 08 uses this distinction to move beyond "try harder next time." Sometimes the tracking method must change. Sometimes governance must change. Sometimes the organization must revise how it understands the project.

SECI knowledge conversion: SECI explains how personal experience enters organizational knowledge. Socialization, externalization, combination and internalization form a continuing process of knowledge creation rather than a one-time documentation exercise. The objective is not to force every tacit judgement into a document. It is to convert important judgement into an appropriate asset and allow another team to develop working capability through use.

Organizational memory: Organizational memory provides a test for the final step. Knowledge must be acquired, retained and retrieved. A large document repository is not automatically a memory. Experience enters the organization only when a project can find and use it at the point of need.

Representative thinkers

Chris Argyris & Donald Schön

Argyris and Schön developed influential accounts of organizational learning and single- and double-loop learning. Their work helps managers move from correcting actions to examining organizational objectives, rules and assumptions.

Ikujiro Nonaka & Hirotaka Takeuchi

Nonaka's theory of organizational knowledge creation and the authors' later account of the knowledge-creating company provide a framework for continued movement between tacit and explicit knowledge. SECI describes the relationship among socialization, externalization, combination and internalization.

U.S. Army AAR practice

After-Action Review developed over time through U.S. Army training and operational practice. The method emphasizes shared reconstruction of intended and actual outcomes and participant self-discovery. AAR should not be assigned to a convenient but unsupported individual "inventor."

James P. Walsh & Gerardo Rivera Ungson

Walsh and Ungson's work on organizational memory highlights acquisition, retention and retrieval. It also warns that memory can be used, misused or left inaccessible.

Part 07 · References

References

Argyris, C. (1977). Double Loop Learning in Organizations. Harvard Business Review, 55(5), 115–125.
Argyris, C., & Schön, D. A. (1978). Organizational Learning: A Theory of Action Perspective. Addison-Wesley.
Nonaka, I. (1994). A Dynamic Theory of Organizational Knowledge Creation. Organization Science, 5(1), 14–37.
Nonaka, I., & Takeuchi, H. (1995). The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation. Oxford University Press.
Walsh, J. P., & Ungson, G. R. (1991). Organizational Memory. Academy of Management Review, 16(1), 57–91.
Department of the Army. (2025). TC 7-0.1: After Action Reviews. Headquarters, Department of the Army.

Operational Review, Organizational Learning and Value Articulation