What it is

Tests & Operations is the place where Merlino moves from modelling to action. It turns technique mapping and test catalogues into executable red teaming activities by orchestrating operations in Morgana/Caldera. The module provides a structured approach to adversary emulation: define what to test, execute it safely against target machines, and capture outcomes as evidence.

Catalogue-driven execution

Merlino uses the Catalogue as the source of truth for test definitions. Tests & Operations reads the Catalogue data, groups tests logically, and prepares them for execution. This keeps validation consistent across engagements and prevents one-off, undocumented testing.

• Test normalization: align tests to techniques/sub-techniques and keep identifiers consistent.
• Structured grouping: organize tests by operation, technique set, scenario, or target scope.
• Repeatability: run the same validation cycle over time to track improvement and regression.

Morgana (Caldera) integration

Tests & Operations is fully integrated with Morgana (Caldera). Merlino synchronizes plans, operations, and execution status bidirectionally, so your workbook remains the operational control layer while Caldera performs the adversary emulation on targets.

• Operation creation: Merlino can automatically create Operations based on your selected test sets and scope.
• Agent and target alignment: operations are mapped to target machines onboarded into the emulation environment.
• Execution synchronization: start/stop status, progress, outcomes, and timings flow back into Merlino.
• Evidence collection: results are captured as structured outcomes tied to test IDs and related TTPs.

Operational monitoring and results

The module provides a live operational view of test performance. You can monitor success rates, execution counts, and runtime indicators, identify tests that need attention, and drill down into results per operation. This makes it easy to focus on what is failing, what is unstable, and what needs refinement.

• Top performers: quickly identify stable tests with strong execution success.
• Needs attention: highlight tests that are failing, timing out, or underperforming.
• Execution telemetry: view success/failure counts, time taken, and operational status per test.
• Operation history: correlate test outcomes across multiple runs and multiple environments.

How results impact posture

Test outcomes are not isolated. Results are synchronized into Merlino and are used to update workbook statistics and posture signals. This helps you move from “claimed coverage” to “validated coverage” by making success/failure measurable across technique sets.

• Coverage confidence: successful test execution supports stronger evidence for technique readiness.
• Gap visibility: failures and timeouts highlight where controls, telemetry, or detections are insufficient.
• Continuous improvement: re-run operations after changes and observe measurable improvement.
• Defensible reporting: tie outcomes back to techniques, tests, and operations with a clear evidence trail.

Typical workflow

1. Select scope: choose techniques or scenarios and derive the test set from the Catalogue.
2. Generate operations: Merlino creates one or more operations aligned to targets and execution strategy.
3. Synchronize and run: operations are synchronized with Morgana/Caldera and executed against target machines.
4. Review outcomes: analyze success rates, failures, and performance; drill into details when needed.
5. Update posture: results flow back into Merlino statistics to reflect validated coverage and quality.

Note: Tests & Operations is designed for controlled adversary emulation and repeatable validation. It connects execution evidence to Merlino’s broader methodology so results are measurable, comparable over time, and easy to communicate.