Última actividad 3 weeks ago

README.md Sin formato

AskUserQuestion for Matt Pocock's Grilling Skills

Makes Matt Pocock's grilling skills ask their questions with Claude Code's AskUserQuestion tool instead of plain text rounds. It is an add-on: the upstream skill files are never touched.

Step by Step

1. Install Matt Pocock's skills (once)

Skip this if you already have them.

npx --yes skills@latest add mattpocock/skills --global --agent claude-code --skill '*' --yes

2. Install this add-on

bash -o pipefail -c 'curl -fsSL https://opengist.resetrix.work/weehong/9e381dc89dff4670b7232b910454f056/raw/HEAD/install-question-tools.sh | bash'

It prints each file it wrote (Updated:) and any backup it made (Backup:). If it prints Installation failed, nothing was changed; fix the reason it gives and run it again.

3. Start a fresh Claude Code session

Hooks and CLAUDE.md are read when a session starts, so close any open session and start a new one. Type /hooks and check you see:

  • PostToolUse with matcher Skill
  • UserPromptSubmit

Both run grilling-question-hook.py.

4. Start a grilling session

Use any of these, the same way you would without the add-on:

You type What it does
/grill-me <your plan> Interview to sharpen a plan or design.
/grill-with-docs <your plan> Same, and writes ADRs and a glossary as you go.
/grilling <your plan> The core interview directly.
/loop-me Grill workflow specs for recurring tasks.
"grill me on this" Claude picks up the grilling skill by itself.

Skills that grill as part of their work (wayfinder, triage, improve-codebase-architecture) get the same behavior automatically.

5. Answer the question cards

Instead of a wall of numbered text, each round arrives as AskUserQuestion cards:

  • Up to 4 questions per card. A bigger round comes as several cards in a row.
  • The first option is Claude's pick, marked (Recommended), with the reason in its description.
  • Pick an option, or choose Other to type your own answer.
  • If you skip or dismiss a question, it stays open. Claude will not treat it as a yes.

After each round Claude works out the next questions from your answers. The session ends when nothing is left open, and Claude asks you to confirm the shared understanding before acting on it.

6. If you still get plain text questions

That is the expected fallback when AskUserQuestion cannot be used, for example in non-interactive claude -p runs, or for a question that is truly open ended. If it happens in a normal session:

  1. Run /hooks. If the two hooks are missing, rerun step 2 and start a new session.
  2. Check python3 --version works in the shell Claude Code uses.
  3. Ask Claude "Did the grilling question policy load?" to see what it received.

Update or remove

  • Update: run the step 2 command again. Nothing changes if you are already current.
  • Remove: see Uninstall.

Inspect Before Installing

Trust check: this runs remote code and downloads a policy and a hook script from this gist. Read them first if you do not already trust the publisher and server. pipefail reports download failures; it is not a security sandbox.

Files: installer | policy | hook

To read everything first and then install from the same gist:

base=https://opengist.resetrix.work/weehong/9e381dc89dff4670b7232b910454f056
for f in install-question-tools.sh grilling-question-tools.md grilling-question-hook.py; do
  curl -fsSL "$base/raw/HEAD/$f" -o "$f"
done
less install-question-tools.sh grilling-question-tools.md grilling-question-hook.py
bash install-question-tools.sh

The installer downloads the policy and hook at HEAD, not your inspected copies. To pin them, take a full commit hash from the gist's Revisions and run QUESTION_TOOLS_REVISION=<40-character-hash> bash install-question-tools.sh.

How It Works

Four pieces, all under your Claude config directory (~/.claude, or $CLAUDE_CONFIG_DIR when set):

File Purpose
grilling-question-tools/grilling-question-tools.md The policy: how to turn a grilling round into AskUserQuestion calls.
grilling-question-tools/grilling-question-hook.py Hands the policy to Claude the moment a grilling flow starts.
settings.json Registers the hook for two events (below).
CLAUDE.md A short marked pointer to the policy, as a fallback.

The hook fires on:

Event When
PostToolUse, matcher Skill Claude loads grilling, grill-me, grill-with-docs, or loop-me. This covers /grill-me, /grill-with-docs, plugin names such as mattpocock-skills:grilling, and skills that call grilling (wayfinder, triage, improve-codebase-architecture).
UserPromptSubmit You type /grilling or /loop-me yourself, which skips the Skill tool.

Why a hook: the upstream grilling skill ships its own text format for each round, and it arrives later in the conversation than CLAUDE.md, so a CLAUDE.md note alone often loses. The hook adds the policy right after the skill loads. It only adds context: it never blocks or edits anything, and any error exits 0 silently so it cannot break a session. It reads the policy file each time, so policy updates need no settings change.

The policy keeps the grilling flow intact: same rounds (split into calls of up to 4 questions), a recommended option first marked (Recommended), the built-in Other answer for custom replies, skipped answers treated as unresolved, and the upstream text format as the fallback when AskUserQuestion is unavailable. It is guidance, not enforcement; Claude Code permissions and higher priority instructions still apply.

Requirements

  • Linux, macOS, or WSL; Bash 3.2+, Python 3.8+, curl. python3 must also be on the PATH Claude Code uses to run hooks.
  • Run as your normal user, without sudo.
  • Matt Pocock's skills installed separately (step 1). Use either the npx install or the Claude plugin, not both, or you may see duplicate skills.

Updates and Safety

Rerun the quick command to update. Identical reruns change nothing and make no backups.

  • Download errors or an invalid policy or hook abort before any file is changed. The checks (versioned markers, title, seven rules, hook header, and that the hook compiles) are sanity checks, not a signature.
  • Everything is checked before anything is written. Malformed or duplicate CLAUDE.md markers, a settings.json that is not a JSON object, or an unexpected hooks layout abort the run.
  • CLAUDE.md bytes outside the managed block are kept exactly. The markers are <!-- BEGIN MATT-POCOCK-GRILLING-QUESTION-TOOLS --> and <!-- END MATT-POCOCK-GRILLING-QUESTION-TOOLS -->; keep them intact.
  • In settings.json, only hook entries whose command runs grilling-question-hook.py are managed. Other settings and hooks are kept in order, but the file is rewritten as 2 space indented JSON when it changes.
  • Every changed file gets a sibling backup named <filename>.question-tools-backup-<suffix> (mode 0600). Old backups are never replaced.
  • Symlinked files and directories (for example a dotfiles repo) are followed and kept as symlinks. Dangling symlinks and non-regular files are rejected.
  • Each file is replaced atomically, but not all files as one transaction. If a write fails midway, restore from the printed backups and rerun.

Upgrading from the earlier multi-agent version moves the hook entry and the CLAUDE.md pointer to the new paths automatically. It leaves the old files in ~/.config/ai-agents/ and the pointer blocks in ~/.codex/AGENTS.md and ~/.config/opencode/AGENTS.md; remove those by hand if you no longer want them.

Uninstall

  1. QUESTION_TOOLS_CLAUDE_HOOK=0 bash install-question-tools.sh removes the managed hook entries from settings.json.
  2. Delete the marked block from CLAUDE.md.
  3. Delete the grilling-question-tools folder in your Claude config directory.

Tests

python3 -B -m unittest -v test_install_question_tools

They use temporary homes and a mocked curl: no network, no real install. The permission test is skipped when running as root.

grilling-question-hook.py Sin formato
1#!/usr/bin/env python3
2# MATT-POCOCK-GRILLING-QUESTION-HOOK v2
3"""Claude Code hook: attach the AskUserQuestion policy when grilling starts.
4
5Registered for two events by install-question-tools.sh:
6 PostToolUse (matcher "Skill") fires when Claude loads a grilling skill,
7 including grilling called by grill-me,
8 grill-with-docs, wayfinder, triage, etc.
9 UserPromptSubmit fires when the user types /grilling or
10 /loop-me (typed slash commands do not go
11 through the Skill tool).
12
13It never blocks or edits anything. It only adds context. Any error exits 0
14silently so a broken hook can never interrupt a session.
15Usage: python3 grilling-question-hook.py /absolute/path/grilling-question-tools.md
16"""
17import json
18import re
19import sys
20
21SKILLS = ('grilling', 'grill-me', 'grill-with-docs', 'loop-me')
22# /grill-me and /grill-with-docs always call the Skill tool for "grilling", so
23# PostToolUse already covers them; matching them here would inject twice.
24TYPED = ('grilling', 'loop-me')
25SLASH = re.compile(r'^/(?:[\w.-]+:)?(%s)(?=\s|$)' % '|'.join(map(re.escape, TYPED)))
26MAX_POLICY = 32768
27
28
29def skill_name(value):
30 # Plugin installs namespace skills, e.g. "mattpocock-skills:grilling".
31 return value.strip().rsplit(':', 1)[-1] if isinstance(value, str) else ''
32
33
34def triggered(event):
35 name = event.get('hook_event_name')
36 if name == 'PostToolUse' and event.get('tool_name') == 'Skill':
37 tool_input = event.get('tool_input')
38 if isinstance(tool_input, dict) and skill_name(tool_input.get('skill')) in SKILLS:
39 return name
40 if name == 'UserPromptSubmit':
41 prompt = event.get('prompt')
42 if isinstance(prompt, str) and SLASH.match(prompt.lstrip()):
43 return name
44 return None
45
46
47def main():
48 if len(sys.argv) != 2:
49 return
50 event = json.load(sys.stdin)
51 if not isinstance(event, dict):
52 return
53 name = triggered(event)
54 if not name:
55 return
56 with open(sys.argv[1], 'rb') as stream:
57 policy = stream.read(MAX_POLICY + 1).decode('utf-8')
58 if len(policy) > MAX_POLICY or 'MATT-POCOCK-GRILLING-QUESTION-POLICY' not in policy:
59 print('Grilling question policy missing or invalid: %s' % sys.argv[1], file=sys.stderr)
60 return
61 context = ('A Matt Pocock grilling flow is active. Deliver its questions with '
62 'AskUserQuestion as this policy describes; everything else in the '
63 'flow is unchanged.\n\n' + policy.strip() + '\n')
64 json.dump({'hookSpecificOutput': {'hookEventName': name, 'additionalContext': context}},
65 sys.stdout)
66
67
68if __name__ == '__main__':
69 try:
70 main()
71 except Exception as error: # Never break the session.
72 print('Grilling question hook skipped: %s' % error, file=sys.stderr)
73 sys.exit(0)
74
grilling-question-tools.md Sin formato

AskUserQuestion for Matt Pocock Grilling Flows

Scope: apply only while running Matt Pocock's grilling flows (grilling, grill-me, grill-with-docs, loop-me, or a skill that calls grilling, such as wayfinder, triage, or improve-codebase-architecture). This changes how Claude Code delivers questions, not the flow's research, decision criteria, documentation, or completion requirements. Other conversations and skills keep their own behavior.

  1. Research first. Look up what the codebase or tools can answer before asking the user. The main conversation owns the interview; subagents return findings and proposed questions to it and never prompt the user themselves.
  2. Ask with AskUserQuestion when it is available and permitted in this session. Follow its actual schema and permission rules. Do not change modes or bypass a restriction to unlock it.
  3. Keep the flow's rounds. A round is every question whose prerequisites are already settled. Each call takes 1 to 4 questions, so a larger round becomes several calls in a row; every question in the round stays independent of the others. Wait for answers before asking dependent follow ups. Never drop a question to fit the limits.
  4. Make each question decidable: a header of at most 12 characters and 2 to 4 concrete options with their tradeoffs in the descriptions. Put your recommended option first, end its label with " (Recommended)", and give the reason in its description. Never invent a recommendation to fill the UI. When the fields would lose needed context, give it in a message just before the call.
  5. Keep custom answers possible. AskUserQuestion adds an "Other" free text choice automatically, so do not add your own. Do not force an open ended question into false choices; ask it in plain text instead. Never read Other as agreement with a listed option.
  6. Treat skipped, dismissed, empty, or ambiguous answers as unresolved, not as consent or acceptance of the recommendation. Clarify a blocking gap or defer it with the user's agreement. Keep unresolved decisions out of the confirmed decisions in any spec, ADR, or summary.
  7. Fall back to the upstream text format. If AskUserQuestion is unavailable, disallowed, fails, or cannot express a question faithfully, use the grilling skill's own "Format a round like so" layout (numbered questions, each with a recommended answer), invite custom answers, and wait. Return to AskUserQuestion when it is usable.

This is an instruction preference, not runtime enforcement. Higher priority instructions and Claude Code restrictions still apply. Finish only when the original flow's completion criteria are met; changing the UI does not satisfy them.

install-question-tools.sh Sin formato
1#!/usr/bin/env bash
2# Requires Bash 3.2+, Python 3.8+, and curl. No sudo; run one installer at a time.
3set -euo pipefail
4
5# Defining the complete function before calling it also avoids starting an
6# installation from a download truncated inside the embedded Python program.
7main() {
8 command -v python3 >/dev/null || { printf '%s\n' 'Python 3.8+ is required.' >&2; return 1; }
9 command -v curl >/dev/null || { printf '%s\n' 'curl is required.' >&2; return 1; }
10 python3 - <<'PY'
11import json
12import os
13import re
14import shlex
15import stat
16import subprocess
17import sys
18import tempfile
19from pathlib import Path
20
21BEGIN = b'<!-- BEGIN MATT-POCOCK-GRILLING-QUESTION-TOOLS -->'
22END = b'<!-- END MATT-POCOCK-GRILLING-QUESTION-TOOLS -->'
23TOKEN = b'MATT-POCOCK-GRILLING-QUESTION-TOOLS'
24POLICY_BEGIN = '<!-- BEGIN MATT-POCOCK-GRILLING-QUESTION-POLICY v2 -->'
25POLICY_END = '<!-- END MATT-POCOCK-GRILLING-QUESTION-POLICY v2 -->'
26BASE = 'https://opengist.resetrix.work/weehong/9e381dc89dff4670b7232b910454f056'
27HOOK_NAME = 'grilling-question-hook.py'
28HOOK_HEADER = '#!/usr/bin/env python3\n# MATT-POCOCK-GRILLING-QUESTION-HOOK v2\n'
29HOOK_EVENTS = (('PostToolUse', 'Skill'), ('UserPromptSubmit', None))
30
31
32def download(url):
33 result = subprocess.run(['curl', '-fsSL', '--proto', '=https', '--proto-redir', '=https',
34 '--connect-timeout', '15', '--max-time', '60', url],
35 stdout=subprocess.PIPE, check=True)
36 return result.stdout
37
38
39def is_managed_hook(hook):
40 return (isinstance(hook, dict) and isinstance(hook.get('command'), str)
41 and HOOK_NAME in hook['command'])
42
43
44def merge_settings(path, old, command):
45 # Returns new settings bytes. Only hook entries running HOOK_NAME are managed;
46 # every other key, event, and hook is kept in its original order.
47 try:
48 settings = json.loads(old.decode('utf-8')) if old.strip() else {}
49 except (UnicodeDecodeError, ValueError) as error:
50 raise ValueError('Claude settings are not valid JSON: %s (%s)' % (path, error))
51 if not isinstance(settings, dict):
52 raise ValueError('Claude settings must be a JSON object: %s' % path)
53 original = json.loads(json.dumps(settings))
54 hooks = settings.get('hooks', {})
55 if not isinstance(hooks, dict):
56 raise ValueError('Claude settings "hooks" must be an object: %s' % path)
57 for event in list(hooks):
58 groups = hooks[event]
59 if not isinstance(groups, list) or not all(
60 isinstance(g, dict) and isinstance(g.get('hooks', []), list) for g in groups):
61 raise ValueError('Unexpected Claude hooks layout for %s in %s. Fix it manually.' % (event, path))
62 kept, removed = [], False
63 for group in groups:
64 entries = group.get('hooks', [])
65 others = [h for h in entries if not is_managed_hook(h)]
66 if len(others) != len(entries):
67 removed = True
68 if not others:
69 continue
70 group = dict(group, hooks=others)
71 kept.append(group)
72 if removed and not kept:
73 del hooks[event]
74 else:
75 hooks[event] = kept
76 if command:
77 for event, matcher in HOOK_EVENTS:
78 group = {'matcher': matcher} if matcher else {}
79 group['hooks'] = [{'type': 'command', 'command': command, 'timeout': 10}]
80 hooks.setdefault(event, []).append(group)
81 if hooks or 'hooks' in settings:
82 settings['hooks'] = hooks
83 if settings == original:
84 return old
85 return (json.dumps(settings, indent=2, ensure_ascii=False) + '\n').encode('utf-8')
86
87
88def resolve_target(path):
89 # Resolve existing components strictly, but allow new files/directories.
90 target = Path(path.anchor)
91 for part in path.parts[1:]:
92 target = target / part
93 if target.is_symlink():
94 try:
95 target = target.resolve(strict=True)
96 except (OSError, RuntimeError) as error:
97 raise ValueError('Dangling or unresolvable symlink: %s' % target) from error
98 return target
99
100
101def check_path(path):
102 for part in [path] + list(path.parents):
103 if part.is_symlink():
104 raise ValueError('Path changed to a symlink during installation: %s' % part)
105 if part.exists() and not (part.is_file() if part == path else part.is_dir()):
106 raise ValueError('Not a regular file/directory: %s' % part)
107 parent = path.parent
108 while not parent.exists():
109 parent = parent.parent
110 if not os.access(parent, os.W_OK | os.X_OK):
111 raise ValueError('Directory is not writable/searchable: %s' % parent)
112 if path.exists() and not os.access(path, os.R_OK | os.W_OK):
113 raise ValueError('Instruction file is not readable/writable: %s' % path)
114
115
116def install():
117 if sys.version_info < (3, 8):
118 raise ValueError('Python 3.8+ is required.')
119 revision = os.environ.get('QUESTION_TOOLS_REVISION') or 'HEAD'
120 if revision != 'HEAD' and not re.fullmatch(r'[0-9a-fA-F]{40}', revision):
121 raise ValueError('QUESTION_TOOLS_REVISION must be HEAD or a full 40-character commit hash.')
122 hook_setting = os.environ.get('QUESTION_TOOLS_CLAUDE_HOOK') or '1'
123 if hook_setting not in ('0', '1'):
124 raise ValueError('QUESTION_TOOLS_CLAUDE_HOOK must be 1 (install) or 0 (skip and remove).')
125 url = BASE + '/raw/' + revision + '/grilling-question-tools.md'
126 policy_bytes = download(url)
127 policy = policy_bytes.decode('utf-8')
128 lines = policy.splitlines()
129 if (not lines or lines[0] != POLICY_BEGIN or lines[-1] != POLICY_END
130 or policy.count(POLICY_BEGIN) != 1 or policy.count(POLICY_END) != 1
131 or len(policy) > 32768 or '\x00' in policy
132 or TOKEN.decode() in policy
133 or not policy.startswith(POLICY_BEGIN + '\n# AskUserQuestion for Matt Pocock Grilling Flows\n')
134 or not all('\n%d. ' % i in policy for i in range(1, 8))):
135 raise ValueError('Downloaded policy is incomplete or invalid; no instruction files changed.')
136 hook_bytes = None
137 if hook_setting == '1':
138 hook_url = BASE + '/raw/' + revision + '/' + HOOK_NAME
139 hook_bytes = download(hook_url)
140 hook_source = hook_bytes.decode('utf-8')
141 if (not hook_source.startswith(HOOK_HEADER) or len(hook_source) > 32768
142 or '\x00' in hook_source or TOKEN.decode() in hook_source
143 or not hook_source.rstrip().endswith('sys.exit(0)')):
144 raise ValueError('Downloaded Claude hook is incomplete or invalid; no instruction files changed.')
145 try:
146 compile(hook_source, HOOK_NAME, 'exec')
147 except SyntaxError as error:
148 raise ValueError('Downloaded Claude hook does not compile: %s' % error)
149 home = os.environ.get('HOME')
150 if not home or not os.path.isabs(home):
151 raise ValueError('HOME must be an absolute path.')
152 claude_dir = os.environ.get('CLAUDE_CONFIG_DIR') or home + '/.claude'
153 if (not os.path.isabs(claude_dir) or '..' in Path(claude_dir).parts
154 or any(c in claude_dir for c in '\r\n`')):
155 raise ValueError('CLAUDE_CONFIG_DIR must be an absolute path without .. components.')
156 policy_dir = Path(claude_dir) / 'grilling-question-tools'
157 policy_path = resolve_target(policy_dir / 'grilling-question-tools.md')
158 if any(c in str(policy_path) for c in '\r\n`'):
159 raise ValueError('Policy path cannot contain newlines or backticks.')
160 block = (BEGIN + b'\n' + (
161 'Before running a Matt Pocock grilling flow (grilling, grill-me, grill-with-docs,\n'
162 'loop-me, or a skill that calls grilling), read `%s`.\n'
163 'Ask its questions with AskUserQuestion as that policy describes;\n'
164 'preserve all other workflow requirements.\n' % policy_path
165 ).encode('utf-8') + END)
166 paths = [resolve_target(Path(claude_dir) / 'CLAUDE.md')]
167 hook_path = resolve_target(policy_dir / HOOK_NAME)
168 settings_path = resolve_target(Path(claude_dir) / 'settings.json')
169 # A hook command is run by a shell, so quote both paths.
170 command = ('python3 %s %s' % (shlex.quote(str(hook_path)), shlex.quote(str(policy_path)))
171 if hook_bytes is not None else None)
172 managed = [policy_path, hook_path, settings_path]
173 everything = managed + paths
174 existing = [p for p in everything if p.exists()]
175 if len(set(everything)) != len(everything) or any(
176 os.path.samefile(a, b) for i, a in enumerate(existing) for b in existing[i + 1:]):
177 raise ValueError('Policy, hook, settings, and CLAUDE.md must be separate files.')
178 changes = []
179 # Preflight every destination before creating directories, backups, or files.
180 for path in managed + paths:
181 check_path(path)
182 exists = path.exists()
183 old = path.read_bytes() if exists else b''
184 if path == policy_path:
185 new = policy_bytes
186 elif path == hook_path:
187 new = hook_bytes if hook_bytes is not None else old
188 elif path == settings_path:
189 new = merge_settings(path, old, command)
190 elif TOKEN in old:
191 if old.count(TOKEN) != 2 or old.count(BEGIN) != 1 or old.count(END) != 1:
192 raise ValueError('Malformed/duplicate managed markers: %s. Repair them manually before rerunning.' % path)
193 start, end = old.index(BEGIN), old.index(END)
194 after = end + len(END)
195 if (end < start or (start and old[start-1:start] != b'\n')
196 or old[end-1:end] != b'\n'
197 or (old[after:] and not old[after:].startswith((b'\n', b'\r\n')))
198 or not old[start+len(BEGIN):].startswith((b'\n', b'\r\n'))):
199 raise ValueError('Malformed managed marker boundaries: %s' % path)
200 new = old[:start] + block + old[after:]
201 else:
202 separator = (b'\n' if old.endswith(b'\n') else b'\n\n') if old else b''
203 new = old + separator + block + b'\n'
204 if new != old:
205 changes.append((path, exists, old, new))
206 elif exists:
207 print('Unchanged: %s' % path)
208
209 staged = []
210 try:
211 for path, exists, old, new in changes:
212 check_path(path)
213 if path.exists() != exists or (exists and path.read_bytes() != old):
214 raise ValueError('File changed during installation: %s. Retry with other writers stopped.' % path)
215 path.parent.mkdir(parents=True, exist_ok=True)
216 mode = stat.S_IMODE(path.stat().st_mode) if exists else 0o600
217 fd, temporary = tempfile.mkstemp(prefix='.' + path.name + '.question-tools-', dir=str(path.parent))
218 staged.append((path, temporary))
219 with os.fdopen(fd, 'wb') as stream:
220 stream.write(new)
221 stream.flush()
222 os.fsync(stream.fileno())
223 os.fchmod(stream.fileno(), mode)
224 if exists:
225 fd, backup = tempfile.mkstemp(prefix=path.name + '.question-tools-backup-', dir=str(path.parent))
226 with os.fdopen(fd, 'wb') as stream:
227 stream.write(old)
228 stream.flush()
229 os.fsync(stream.fileno())
230 print('Backup: %s' % backup)
231 for path, temporary in staged:
232 os.replace(temporary, path)
233 print('Updated: %s' % path)
234 finally:
235 for _, temporary in staged:
236 if os.path.exists(temporary):
237 os.unlink(temporary)
238 print('Policy source: %s' % url)
239 if command:
240 print('Claude Code hook: %s (PostToolUse on Skill, UserPromptSubmit)' % settings_path)
241 else:
242 print('Claude Code hook: skipped (QUESTION_TOOLS_CLAUDE_HOOK=0); managed entries removed if present.')
243 print('Start a fresh Claude Code session; /hooks lists the registered hooks.')
244
245
246try:
247 install()
248except (OSError, ValueError, subprocess.SubprocessError) as error:
249 print('Installation failed: %s' % error, file=sys.stderr)
250 sys.exit(1)
251PY
252}
253
254main "$@"
255
test_install_question_tools.py Sin formato
1"""Sandbox-only installer tests; curl never accesses the network."""
2import json
3import os
4from pathlib import Path
5import shlex
6import subprocess
7import tempfile
8import unittest
9
10
11ROOT = Path(__file__).resolve().parent
12INSTALLER = ROOT / 'install-question-tools.sh'
13HOOK_SCRIPT = ROOT / 'grilling-question-hook.py'
14POLICY = (ROOT / 'grilling-question-tools.md').read_bytes()
15HOOK = HOOK_SCRIPT.read_bytes()
16BEGIN = b'<!-- BEGIN MATT-POCOCK-GRILLING-QUESTION-TOOLS -->'
17END = b'<!-- END MATT-POCOCK-GRILLING-QUESTION-TOOLS -->'
18BASE = 'https://opengist.resetrix.work/weehong/9e381dc89dff4670b7232b910454f056'
19
20
21class InstallerTests(unittest.TestCase):
22 def setUp(self):
23 self.temp = tempfile.TemporaryDirectory(prefix='question-tools-test-')
24 self.addCleanup(self.temp.cleanup)
25 self.root = Path(self.temp.name)
26 self.home = self.root / 'home'
27 self.home.mkdir()
28 self.bin = self.root / 'bin'
29 self.bin.mkdir()
30 self.fixture = self.root / 'policy.md'
31 self.fixture.write_bytes(POLICY)
32 self.hook_fixture = self.root / 'hook.py'
33 self.hook_fixture.write_bytes(HOOK)
34 curl = self.bin / 'curl'
35 curl.write_text(
36 '#!/usr/bin/env python3\n'
37 'import os, pathlib, sys\n'
38 'url = sys.argv[-1]\n'
39 'with open(os.environ["CURL_LOG"], "a") as log: log.write(url + "\\n")\n'
40 'if os.environ.get("CURL_FAIL") and url.endswith(os.environ["CURL_FAIL"]): sys.exit(22)\n'
41 'name = "HOOK_FIXTURE" if url.endswith(".py") else "POLICY_FIXTURE"\n'
42 'sys.stdout.buffer.write(pathlib.Path(os.environ[name]).read_bytes())\n'
43 )
44 curl.chmod(0o700)
45 self.env = {key: value for key, value in os.environ.items() if key not in (
46 'HOME', 'CLAUDE_CONFIG_DIR', 'QUESTION_TOOLS_REVISION', 'QUESTION_TOOLS_CLAUDE_HOOK')}
47 self.env.update(HOME=str(self.home), PATH=str(self.bin) + ':' + os.environ['PATH'],
48 POLICY_FIXTURE=str(self.fixture), HOOK_FIXTURE=str(self.hook_fixture),
49 CURL_LOG=str(self.root / 'curl.log'))
50 self.use_claude_dir(self.home / '.claude')
51
52 def use_claude_dir(self, claude):
53 self.claude_md = claude / 'CLAUDE.md'
54 self.settings = claude / 'settings.json'
55 self.policy = claude / 'grilling-question-tools/grilling-question-tools.md'
56 self.hook = claude / 'grilling-question-tools/grilling-question-hook.py'
57
58 def run_install(self, success=True):
59 result = subprocess.run(['bash', str(INSTALLER)], env=self.env,
60 cwd=self.root, capture_output=True, text=True)
61 self.assertEqual(result.returncode == 0, success, result.stdout + result.stderr)
62 return result
63
64 def write_claude_md(self, data=b'Unrelated\x80 instructions\r\n'):
65 self.claude_md.parent.mkdir(parents=True, exist_ok=True)
66 self.claude_md.write_bytes(data)
67
68 def snapshot(self):
69 return {str(p.relative_to(self.home)): (
70 ('link', os.readlink(p)) if p.is_symlink() else
71 ('file', p.read_bytes(), p.stat().st_mode, p.stat().st_mtime_ns) if p.is_file()
72 else ('directory',)) for p in self.home.rglob('*')}
73
74 def curl_urls(self):
75 return (self.root / 'curl.log').read_text().splitlines()
76
77 def managed_commands(self, settings=None):
78 data = settings if settings is not None else json.loads(self.settings.read_text())
79 found = {}
80 for event, groups in data.get('hooks', {}).items():
81 for group in groups:
82 for hook in group.get('hooks', []):
83 if 'grilling-question-hook.py' in hook.get('command', ''):
84 found.setdefault(event, []).append((group.get('matcher'), hook['command']))
85 return found
86
87 def run_hook(self, event):
88 result = subprocess.run(['python3', str(HOOK_SCRIPT), str(self.fixture)],
89 input=json.dumps(event), capture_output=True, text=True)
90 self.assertEqual(result.returncode, 0, result.stderr)
91 return json.loads(result.stdout) if result.stdout.strip() else None
92
93 # Installer
94
95 def test_install_and_unchanged_rerun(self):
96 original = b'Unrelated\x80 instructions\r\n'
97 self.write_claude_md(original)
98 self.claude_md.chmod(0o640)
99 self.run_install()
100 self.assertEqual(self.policy.read_bytes(), POLICY)
101 self.assertEqual(self.hook.read_bytes(), HOOK)
102 content = self.claude_md.read_bytes()
103 self.assertTrue(content.startswith(original))
104 self.assertEqual(content.count(BEGIN), 1)
105 self.assertIn(str(self.policy).encode(), content)
106 self.assertNotIn(POLICY, content)
107 backups = list(self.claude_md.parent.glob('CLAUDE.md.question-tools-backup-*'))
108 self.assertEqual(len(backups), 1)
109 self.assertEqual(backups[0].read_bytes(), original)
110 self.assertEqual(backups[0].stat().st_mode & 0o777, 0o600)
111 self.assertEqual(self.claude_md.stat().st_mode & 0o777, 0o640)
112 self.assertEqual(self.curl_urls(), [BASE + '/raw/HEAD/grilling-question-tools.md',
113 BASE + '/raw/HEAD/grilling-question-hook.py'])
114 # Only Claude Code files are touched.
115 self.assertEqual(sorted(p.name for p in self.home.iterdir()), ['.claude'])
116 before = self.snapshot()
117 self.run_install()
118 self.assertEqual(self.snapshot(), before)
119
120 def test_hook_merged_into_existing_settings(self):
121 existing = {'model': 'opus', 'hooks': {'PostToolUse': [
122 {'matcher': 'Bash', 'hooks': [{'type': 'command', 'command': 'echo mine'}]}]}}
123 self.settings.parent.mkdir(parents=True)
124 self.settings.write_text(json.dumps(existing))
125 self.run_install()
126 data = json.loads(self.settings.read_text())
127 self.assertEqual(data['model'], 'opus')
128 self.assertEqual(data['hooks']['PostToolUse'][0], existing['hooks']['PostToolUse'][0])
129 commands = self.managed_commands(data)
130 self.assertEqual([m for m, _ in commands['PostToolUse']], ['Skill'])
131 self.assertEqual([m for m, _ in commands['UserPromptSubmit']], [None])
132 self.assertEqual(shlex.split(commands['PostToolUse'][0][1]),
133 ['python3', str(self.hook), str(self.policy)])
134 backups = list(self.settings.parent.glob('settings.json.question-tools-backup-*'))
135 self.assertEqual(len(backups), 1)
136 self.assertEqual(json.loads(backups[0].read_text()), existing)
137 before = self.snapshot()
138 self.run_install()
139 self.assertEqual(self.snapshot(), before)
140
141 def test_migrates_older_install(self):
142 # An older version embedded the policy in CLAUDE.md and pointed the hook
143 # at ~/.config/ai-agents. Both are replaced, not duplicated.
144 prefix, suffix = b'Existing\xff\r\n\n', b'\r\nAfter\x80'
145 self.write_claude_md(prefix + BEGIN + b'\nold embedded policy\n' + END + suffix)
146 old_command = "python3 '/old/ai-agents/grilling-question-hook.py' '/old/ai-agents/grilling-question-tools.md'"
147 self.settings.write_text(json.dumps({'hooks': {'PostToolUse': [
148 {'matcher': 'Skill', 'hooks': [{'type': 'command', 'command': old_command, 'timeout': 10}]}]}}))
149 self.run_install()
150 content = self.claude_md.read_bytes()
151 self.assertTrue(content.startswith(prefix + BEGIN))
152 self.assertTrue(content.endswith(END + suffix))
153 self.assertEqual(content.count(BEGIN), 1)
154 self.assertNotIn(b'old embedded policy', content)
155 commands = self.managed_commands()
156 self.assertEqual({event: len(entries) for event, entries in commands.items()},
157 {'PostToolUse': 1, 'UserPromptSubmit': 1})
158 self.assertNotIn('/old/', json.dumps(commands))
159
160 def test_symlinked_claude_md_and_directory(self):
161 dotfiles = self.home / 'dotfiles'
162 (dotfiles / 'claude').mkdir(parents=True)
163 (dotfiles / 'claude/CLAUDE.md').write_bytes(b'Mine\n')
164 (self.home / '.claude').symlink_to(dotfiles / 'claude', target_is_directory=True)
165 self.run_install()
166 self.assertTrue((self.home / '.claude').is_symlink())
167 self.assertEqual((dotfiles / 'claude/CLAUDE.md').read_bytes().count(BEGIN), 1)
168 self.assertEqual((dotfiles / 'claude/grilling-question-tools/grilling-question-tools.md').read_bytes(), POLICY)
169
170 def test_claude_config_dir_with_space_and_revision(self):
171 self.env['CLAUDE_CONFIG_DIR'] = str(self.home / 'claude config')
172 self.env['QUESTION_TOOLS_REVISION'] = 'a' * 40
173 self.use_claude_dir(self.home / 'claude config')
174 self.run_install()
175 self.assertEqual(self.policy.read_bytes(), POLICY)
176 self.assertIn(str(self.policy).encode(), self.claude_md.read_bytes())
177 self.assertEqual(shlex.split(self.managed_commands()['PostToolUse'][0][1]),
178 ['python3', str(self.hook), str(self.policy)])
179 self.assertEqual(self.curl_urls(), [BASE + '/raw/' + 'a' * 40 + '/grilling-question-tools.md',
180 BASE + '/raw/' + 'a' * 40 + '/grilling-question-hook.py'])
181 self.assertFalse((self.home / '.claude').exists())
182 self.env['QUESTION_TOOLS_REVISION'] = 'HEAD~1'
183 self.run_install(False)
184
185 def test_hook_opt_out_removes_only_managed_entries(self):
186 self.run_install()
187 self.assertTrue(self.managed_commands())
188 self.env['QUESTION_TOOLS_CLAUDE_HOOK'] = '0'
189 self.run_install()
190 self.assertEqual(self.managed_commands(), {})
191 self.assertEqual(json.loads(self.settings.read_text()), {'hooks': {}})
192 self.assertEqual(self.curl_urls()[2:], [BASE + '/raw/HEAD/grilling-question-tools.md'])
193 self.assertEqual(self.claude_md.read_bytes().count(BEGIN), 1)
194 self.env['QUESTION_TOOLS_CLAUDE_HOOK'] = 'yes'
195 self.run_install(False)
196
197 def test_malformed_blocks_preflight_no_writes(self):
198 for malformed in (BEGIN, END + b'\n' + BEGIN, BEGIN + b'\nx\n' + END + END,
199 b'prefix' + BEGIN + b'\nx\n' + END, BEGIN + b'x\n' + END):
200 with self.subTest(malformed=malformed):
201 self.write_claude_md(malformed)
202 before = self.snapshot()
203 self.run_install(False)
204 self.assertEqual(self.snapshot(), before)
205
206 def test_invalid_settings_abort_before_writes(self):
207 for bad in (b'{not json', b'[]', b'{"hooks": []}', b'{"hooks": {"Stop": {}}}'):
208 with self.subTest(bad=bad):
209 self.write_claude_md()
210 self.settings.write_bytes(bad)
211 before = self.snapshot()
212 self.run_install(False)
213 self.assertEqual(self.snapshot(), before)
214
215 def test_download_failure_and_invalid_files_no_writes(self):
216 self.write_claude_md()
217 before = self.snapshot()
218 for failing in ('.md', '.py'):
219 with self.subTest(failing=failing):
220 self.env['CURL_FAIL'] = failing
221 self.run_install(False)
222 self.assertEqual(self.snapshot(), before)
223 del self.env['CURL_FAIL']
224 self.hook_fixture.write_bytes(HOOK[:len(HOOK) // 2])
225 self.run_install(False)
226 self.assertEqual(self.snapshot(), before)
227 self.hook_fixture.write_bytes(HOOK)
228 self.fixture.write_bytes(b'incomplete')
229 self.run_install(False)
230 self.assertEqual(self.snapshot(), before)
231
232 def test_dangling_file_and_directory_symlinks(self):
233 for path in (self.claude_md, self.claude_md.parent):
234 with self.subTest(path=path):
235 path.parent.mkdir(parents=True, exist_ok=True)
236 path.symlink_to(self.home / 'missing')
237 before = self.snapshot()
238 self.run_install(False)
239 self.assertEqual(self.snapshot(), before)
240 path.unlink()
241 if path == self.claude_md:
242 path.parent.rmdir()
243
244 def test_nonregular_destination(self):
245 self.claude_md.parent.mkdir(parents=True)
246 os.mkfifo(self.claude_md)
247 self.run_install(False)
248 self.assertFalse(self.policy.exists())
249 self.assertEqual(list(self.home.rglob('*.question-tools-backup-*')), [])
250
251 @unittest.skipIf(os.geteuid() == 0, 'root bypasses filesystem permissions')
252 def test_permission_failure_preflight(self):
253 self.write_claude_md()
254 self.claude_md.chmod(0o400)
255 before = self.snapshot()
256 self.run_install(False)
257 self.assertEqual(self.snapshot(), before)
258
259 def test_policy_cannot_alias_claude_md(self):
260 self.write_claude_md()
261 self.policy.parent.mkdir(parents=True)
262 self.policy.symlink_to(self.claude_md)
263 before = self.snapshot()
264 self.run_install(False)
265 self.assertEqual(self.snapshot(), before)
266
267 # Hook
268
269 def test_hook_triggers_only_for_grilling(self):
270 post = {'hook_event_name': 'PostToolUse', 'tool_name': 'Skill'}
271 for skill in ('grilling', 'grill-me', 'mattpocock-skills:grilling', 'grill-with-docs', 'loop-me'):
272 with self.subTest(skill=skill):
273 out = self.run_hook(dict(post, tool_input={'skill': skill}))
274 self.assertEqual(out['hookSpecificOutput']['hookEventName'], 'PostToolUse')
275 context = out['hookSpecificOutput']['additionalContext']
276 self.assertIn('AskUserQuestion', context)
277 self.assertIn(POLICY.decode().strip(), context)
278 prompt = {'hook_event_name': 'UserPromptSubmit'}
279 for text in ('/loop-me', ' /grilling my plan', '/mattpocock-skills:grilling x'):
280 with self.subTest(text=text):
281 out = self.run_hook(dict(prompt, prompt=text))
282 self.assertEqual(out['hookSpecificOutput']['hookEventName'], 'UserPromptSubmit')
283 for event in (dict(post, tool_input={'skill': 'tdd'}),
284 dict(post, tool_name='Bash', tool_input={'command': 'grilling'}),
285 dict(prompt, prompt='please grill me'), dict(prompt, prompt='/grilling-extra'),
286 dict(prompt, prompt='/tdd /grilling'),
287 # Covered by the Skill call these make; avoids double injection.
288 dict(prompt, prompt='/grill-me'), dict(prompt, prompt='/grill-with-docs')):
289 with self.subTest(event=event):
290 self.assertIsNone(self.run_hook(event))
291
292 def test_hook_never_fails_the_session(self):
293 for stdin, policy in (('not json', self.fixture), ('[]', self.fixture),
294 (json.dumps({'hook_event_name': 'UserPromptSubmit', 'prompt': '/grilling'}),
295 self.root / 'missing.md')):
296 with self.subTest(stdin=stdin, policy=policy):
297 result = subprocess.run(['python3', str(HOOK_SCRIPT), str(policy)],
298 input=stdin, capture_output=True, text=True)
299 self.assertEqual((result.returncode, result.stdout), (0, ''))
300
301
302if __name__ == '__main__':
303 unittest.main()
304