TTama
Docs/Reference

Reference

Testing and qualification contract

Qualification belongs to an exact source revision and the product built from it. Retain the command, exit status, report, supported screenshots or recordings, and the persisted or external state the run observed. A source test pass is not proof of a signed app, an installed hook release, a live provider session or the desktop. Missing dependencies and refused runs remain blocked.

#Evidence boundaries

Qualification belongs to an exact source revision and the product built from it. Retain the command, exit status, report, supported screenshots or recordings, and the persisted or external state the run observed. A source test pass is not proof of a signed app, an installed hook release, a live provider session or the desktop. Missing dependencies and refused runs remain blocked.

Tests use isolated state under the owning checkout's ignored build directory. They must not alter operator credentials, grants or selected hooks. Actual run results belong with their retained evidence, not in a permanent table of “current work” statuses.

#CLI example coverage and required evidence

The files under examples/ are executable command examples, not prose runbooks and not test evidence. Each script exposes one bounded CLI outcome and states mutation or provider risk before the command.

The CLI examples need proof at their actual boundaries:

  • CLI discovery must retain or create the correct bundled link, refuse unrelated entries, and report missing PATH configuration.
  • Catalog and session commands must agree in human and JSON output, including IDs, commands, events, scope and loaded release identity.
  • Seal verification and runtime status must read the release being qualified; declared checksums alone do not prove its consumers work.
  • Repository scans must cover clean and violating input without changing the tree. Provider-backed cleanup must preserve Git identity and finish with an independent scan.
  • Git-dispatcher installation must retain its reviewed plan, exact writes and post-install result. MCP configuration must resolve to the selected release.
  • Recovery status must remain read-only; a repair must be requested through its normal product interface and verified from the resulting state.

#Required suites

#Managed commit-message review

tests/commit-review/managed.rs installs the real dispatcher in an isolated Git fixture, preserves Git's actual message hook, creates a valid commit through Brama, refuses incident-bearing messages and formatting-only changes without advancing HEAD, then restores the original hook on uninstall. Run cargo test --manifest-path rust/Cargo.toml -p tama-cli --test commit_review_managed -- --ignored --nocapture only when qualification is permitted, with the exact TAMA_TEST_SOURCE_REVISION and real configured reviewer. Commands, exit statuses, source revision and binary digest are retained under Cargo's build directory.

#Owned pending stop assessment

node tests/hooks-running/stop-review/journey.mjs requires TAMA_TEST_EXECUTABLE, TAMA_TEST_SOURCE_REVISION and the real installed TAMA_TEST_OKO_BIN. It checks the clean candidate and retains the executable digest before invoking the real native core and stop hook. The listener uses an internal interface declared by the operating system and an OS-assigned port; it deliberately does not answer. The journey requires an immediate pending refusal, reuse of the same candidate's assessment, and caller cancellation ending the actual HTTP connection without publishing a late verdict. A fresh native inspection must find no surviving hook. A local role phase then uses real Oko set, clear and reassignment commands, verifying that an unchanged candidate cannot retain an obsolete refusal or pass; an unreadable role file must cancel its assessment. It retains the installed Oko digest separately from the candidate Tama digest. The following phase uses the caller's actual Brama configuration and the same real stop hook: it requires a pending refusal followed by a completed passing verdict, reused for the unchanged candidate. An unavailable reviewer fails qualification with its actual cause; no authored provider answer replaces it. Progress from all phases and the final result stay under .build/stop-review/.

This exercises the shipped assessment module and native transport, not OMP's automatic refusal continuation or operator-interruption lifecycle; those still require separate real harness runs. A test definition is not a pass.

#Native hook request suite

cargo test --manifest-path rust/Cargo.toml --target-dir .build/cargo -p tama-cli --test ssh_quoting

Run from the native Tama checkout. The suite under tests/tool-payload/ requires a committed source tree and launches the actual tama hook block-unauthorized-ssh command with JSON on stdin and an isolated HOME. It checks quoted evidence, escaped delimiters, substitutions, nested interpreters, here-documents, Eval code and native argument vectors. It does not execute the transport requests it submits.

An allowed literal request must exit successfully without a blocking decision. An unauthorized transport must produce the structured {"decision":"block","reason":"…"} document and its refusal exit, not merely crash. Reports under .build/tool-payload/ retain the source revision, binary checksum, each request, command, stdout, stderr, exit status and cleanup result. The isolated HOME is removed.

This suite qualifies the native decision boundary only. Live qualification must separately verify the installed and loaded release, reproduce the request through a provider session, and inspect its decision in Session and Posture. A source pass cannot stand in for that journey. Use the normal authorization controls; a guard refusal is a blocked run, not permission to alter its selection or approval records.

#Desktop contract qualification

The desktop package currently declares the Tama executable and no Swift test product. Do not invoke an undeclared test product or treat successful compilation as a contract test. Qualification must exercise the actual CLI request boundary and the corresponding desktop screen, retaining the exact source revision, commands, observed state and supported recordings.

For coverage inspection, use the real installed-configuration journey described in provider configuration, then inspect the same counts and failure causes on Coverage. A null count must remain unknown with its actual inspection cause; an inspected configuration with no mappings must show zero. Dispatcher evidence alone does not qualify the graphical result or live harness enforcement.

#Signed build contract

Command, after explicit approval and with dedicated release credentials:

TAMA_INSTALL_AFTER_BUILD=no \
WISENT_CODESIGN_IDENTITY='<dedicated identity>' \
WISENT_APP_PROVISIONING_PROFILE='<app profile path>' \
WISENT_NETWORK_FILTER_PROVISIONING_PROFILE='<network profile path>' \
sh Scripts/build-app.sh

Scope: app, daemon, System Extension, bundled hook release, manifests, code signing, and resource layout. Side effects: writes .build/Tama.app; it must not install into ~/Applications. Qualification additionally inspects signature, entitlements, embedded versions, dependency revisions, hook provenance, and release resource isolation. Qualification must exercise a supported Node executable and confirm that missing or older Node is rejected before build output is replaced. Runtime-install qualification must separately confirm pre-mutation rejection, symlink-resolved canonical executable pinning in both registry command surfaces, one sealed preload path under hooks-runtime/current shared by installed commands and supervised semantic dispatch, post-install downgrade rejection before target loading, matching installed.json.nodeExecutable and installed.json.nodeVersion provenance, the same values in the Posture release identity and Settings local enforcement, and literal handling of paths containing spaces, apostrophes, dollar signs, and backticks.

#Safe local canonical examples

Execute every example labeled read-only from its declared clean state using the signed candidate. Record exact product/source/example/hook identities, expected state shape, representative failure, and cleanup. File counts or process exit alone are insufficient.

#Clean-device onboarding and core E2E

Use a dedicated Apple-silicon macOS 14+ qualification Mac with no Tama state or credential. Exercise welcome, Wisent sign-in, explicit runtime install, explicit backend registration, macOS approval, one real supervisor, session status, scoped override, reload, and normal cleanup. This suite changes Keychain, Application Support, per-user hook entrypoints, ServiceManagement, SystemExtensions, NetworkExtension preferences, and Full Disk Access.

#Controlled recovery, provider, and release suites

Separately authorize and execute emergency disable/re-enable, cleanup-agent success/outage/ timeout/cancellation with process-tree termination and partial-edit recovery, integrity corruption/recovery, deactivate/uninstall, notarized publication including post-packaging and pre-upload signature, staple, and Gatekeeper rejection paths, exact upgrade, and compatible rollback. Use disposable repositories/accounts, dedicated Apple/GitHub credentials, bounded waits, and exact cleanup. Never target personal or production state.

#Isolation, determinism, and credential control

  • Unit suites own temporary directories and must leave no global Tama state.
  • Device suites start from a recorded clean snapshot and use one named release candidate.
  • Provider/destructive suites require explicit operator approval for their exact account, repository, device, and time window.
  • Credentials enter through Keychain, protected profile files, or provider authentication; never command arguments, fixtures, logs, screenshots, or evidence.
  • Retry occurs only after a classified result proves it is safe. No flaky-test retry policy.
  • Cleanup failure fails the suite and remains visible.
  • Two Tama versions never share a live Application Support directory concurrently.

#Evidence record

Every executed suite records: suite/example name, exact Git tag and source revision, artifact digest, hook release ID, platform/architecture, start/end time, controlled identity label, precondition snapshot, expected observable contract, bounded redacted result, failure path exercised, cleanup result, and operator approval reference. Release notes link these records without embedding secrets or personal data.

Testing is complete only when every row above has direct current evidence, all canonical examples have honest final status, suites are deterministic/isolated/diagnosable, and the signed published bytes complete clean-device onboarding plus recovery.