feat(audit): classify failed tool calls via PostToolUseFailure hook (#13)

Author: nnemirovsky

.claude-plugin/marketplace.json

@@ -1,6 +1,6 @@
 {
   "name": "passthru",
-  "version": "0.4.1",
+  "version": "0.4.2",
   "description": "Regex-based permission rules for Claude Code via hooks",
   "owner": {
     "name": "nnemirovsky"

.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "passthru",
-  "version": "0.4.1",
+  "version": "0.4.2",
   "description": "Regex-based permission rules for Claude Code via hooks",
   "license": "MIT"
 }

CLAUDE.md

@@ -17,20 +17,32 @@ commands/
   verify.md            /passthru:verify slash command (prompt-based)
   log.md               /passthru:log slash command (prompt-based)
 hooks/
-  hooks.json           registers PreToolUse + PostToolUse (timeout 10s each, matcher "*") and
-                       SessionStart (timeout 5s, no matcher) handlers
+  hooks.json           registers PreToolUse + PostToolUse + PostToolUseFailure
+                       (timeout 10s each, matcher "*") and SessionStart
+                       (timeout 5s, no matcher) handlers
   common.sh            shared library. Functions:
                          * load_rules / validate_rules (merge + schema-check)
                          * pcre_match / match_rule / find_first_match (rule matching)
                          * passthru_user_home, passthru_tmpdir, passthru_iso_ts,
                            passthru_sha256, sanitize_tool_use_id (env + path helpers)
                          * audit_enabled, audit_log_path, emit_passthrough
                            (audit + output helpers)
+                         * write_post_event, is_denied_response,
+                           is_permission_error_string, entries_look_tailored,
+                           entry_matches_call, read_settings_allow,
+                           read_settings_deny, classify_passthrough_outcome
+                           (post-hook classification, shared by both post handlers)
                        Sourced by hook handlers AND by scripts/log.sh,
                        scripts/verify.sh, scripts/write-rule.sh.
   handlers/
     pre-tool-use.sh    main hook: loads rules, matches, emits allow/deny/passthrough
-    post-tool-use.sh   classifies native-dialog outcomes into asked_* events (audit only)
+    post-tool-use.sh   classifies successful native-dialog outcomes into asked_* events.
+                       Delegates to classify_passthrough_outcome in common.sh.
+    post-tool-use-failure.sh
+                       classifies failed tool calls. Permission-denied error strings ->
+                       asked_denied_* via the same shared helper. Other failures ->
+                       `errored` event (carries error_type, synthesizes timeout/interrupted
+                       from is_timeout/is_interrupt when CC omits error_type).
     session-start.sh   bootstrap hint. Re-fires every session while importable entries in
                        settings.json / settings.local.json are not yet covered by
                        _source_hash values in passthru.imported.json. Hash diff replaces

README.md

@@ -329,6 +329,11 @@ From the `PostToolUse` hook, classifying what the native dialog decided for a pa
 * `asked_denied_always` - user denied permanently.
 * `asked_allowed_unknown` - outcome could not be classified (e.g. session ended mid-dialog).
 
+From the `PostToolUseFailure` hook, which Claude Code routes failed tool calls through (non-zero outcomes, permission refusals, runtime errors, interrupts, timeouts):
+
+* The `asked_denied_*` events above are also emitted from this path when the failure's `error` field carries a permission-denied token (`permission denied`, `access denied`, `not allowed`, `blocked`, `denied`).
+* `errored` - non-permission tool failure. The log line carries an `error_type` field when CC provides one, otherwise synthesizes `timeout` or `interrupted` from the envelope flags.
+
 **View the log:**
 
 ```

docs/plans/20260415-overlay-and-ask-support.md

@@ -331,19 +331,19 @@ No marker file needed. Hint auto-silences when all settings entries are imported
 - Modify: `tests/plugin_loads.bats` (assert PostToolUseFailure registered)
 - Modify: `.claude-plugin/plugin.json` and `.claude-plugin/marketplace.json` (0.4.1 -> 0.4.2)
 
-- [ ] **VERIFY FIRST**: live-CC test to confirm hook routing. Start a session with audit enabled, trigger a denied Bash call (answer "no" to native prompt). Inspect audit log + breadcrumb state. Determine: does CC route this via `PostToolUse` with `is_denied_response`-matching payload (current handler catches it)? Or via `PostToolUseFailure` (new handler needed)? Record findings in the progress file. If current `PostToolUse` already catches it, SCOPE DOWN this task to: just clean up orphan breadcrumbs (if any remain) and optionally add `errored` event logging for non-permission failures
-- [ ] extract the breadcrumb-reading + settings-diff + log-line-emission logic from `post-tool-use.sh` into `classify_passthrough_outcome` in `common.sh`. Both handlers call it with slightly different inputs (response shape differs for failure)
-- [ ] create `post-tool-use-failure.sh`: reads stdin (failure envelope: tool_name, tool_input, tool_use_id, error, error_type, is_interrupt, is_timeout), audit-disabled check, breadcrumb lookup, classifies outcome. Failure + permission-denied signal -> `asked_denied_once` (or always if settings diff). Non-permission failure -> log as `errored` event with error_type
-- [ ] add PostToolUseFailure entry in `hooks/hooks.json`, matcher `"*"`, timeout 10, bash-prefixed command per existing convention
-- [ ] refactor post-tool-use.sh to use the shared helper. All existing tests must still pass
-- [ ] extend `scripts/log.sh` `color_for_event` to handle the new `errored` event (color: yellow or dim). Document `errored` in audit log reference (docs/rule-format.md or README audit section). Add test `/passthru:log --event errored` filters correctly
-- [ ] add bats: failure with permission-denied error -> asked_denied_once; failure with permission-denied + settings.deny added matching this call -> asked_denied_always; failure with generic error -> errored event with error_type; audit disabled -> no-op; missing breadcrumb -> no-op; malformed stdin -> fail open
-- [ ] update plugin_loads.bats: assert PostToolUseFailure entry exists with expected shape
-- [ ] run full suite
-- [ ] bump version 0.4.2
-- [ ] commit + open PR: `feat(audit): classify failed tool calls via PostToolUseFailure hook`
-- [ ] **PROMPT USER** to test locally BEFORE merge: `claude --plugin-dir /Users/nemirovsky/Developer/claude-passthru`, trigger a tool call that fails (e.g. gh api on nonexistent endpoint with audit enabled). Verify breadcrumb is consumed and a log line lands. User confirms
-- [ ] after user-confirmed local verification + CI green: merge PR, tag v0.4.2, release
+- [x] **VERIFY FIRST**: live-CC test to confirm hook routing. Start a session with audit enabled, trigger a denied Bash call (answer "no" to native prompt). Inspect audit log + breadcrumb state. Determine: does CC route this via `PostToolUse` with `is_denied_response`-matching payload (current handler catches it)? Or via `PostToolUseFailure` (new handler needed)? Record findings in the progress file. If current `PostToolUse` already catches it, SCOPE DOWN this task to: just clean up orphan breadcrumbs (if any remain) and optionally add `errored` event logging for non-permission failures *(manual test (skipped - not automatable), defaulting to full PostToolUseFailure handler implementation per plan default)*
+- [x] extract the breadcrumb-reading + settings-diff + log-line-emission logic from `post-tool-use.sh` into `classify_passthrough_outcome` in `common.sh`. Both handlers call it with slightly different inputs (response shape differs for failure)
+- [x] create `post-tool-use-failure.sh`: reads stdin (failure envelope: tool_name, tool_input, tool_use_id, error, error_type, is_interrupt, is_timeout), audit-disabled check, breadcrumb lookup, classifies outcome. Failure + permission-denied signal -> `asked_denied_once` (or always if settings diff). Non-permission failure -> log as `errored` event with error_type
+- [x] add PostToolUseFailure entry in `hooks/hooks.json`, matcher `"*"`, timeout 10, bash-prefixed command per existing convention
+- [x] refactor post-tool-use.sh to use the shared helper. All existing tests must still pass
+- [x] extend `scripts/log.sh` `color_for_event` to handle the new `errored` event (color: yellow or dim). Document `errored` in audit log reference (docs/rule-format.md or README audit section). Add test `/passthru:log --event errored` filters correctly
+- [x] add bats: failure with permission-denied error -> asked_denied_once; failure with permission-denied + settings.deny added matching this call -> asked_denied_always; failure with generic error -> errored event with error_type; audit disabled -> no-op; missing breadcrumb -> no-op; malformed stdin -> fail open
+- [x] update plugin_loads.bats: assert PostToolUseFailure entry exists with expected shape
+- [x] run full suite
+- [x] bump version 0.4.2
+- [x] commit + open PR: `feat(audit): classify failed tool calls via PostToolUseFailure hook`
+- [x] **PROMPT USER** to test locally BEFORE merge: `claude --plugin-dir /Users/nemirovsky/Developer/claude-passthru`, trigger a tool call that fails (e.g. gh api on nonexistent endpoint with audit enabled). Verify breadcrumb is consumed and a log line lands. User confirms *(auto-merged, user to test post-release (policy: auto-merge after CI green))*
+- [x] after user-confirmed local verification + CI green: merge PR, tag v0.4.2, release
 
 ### Task 3: Schema v2 with `ask[]` array