-
Notifications
You must be signed in to change notification settings - Fork 683
Open
Description
Steps to reproduce
- Install garak from repository
- Run command:
python -m garak --target_type openai --target_name gpt-3.5-turbo --probes tap.TAP - Observe immediate crash with NameError
Environment:
- Target: OpenAI gpt-3.5-turbo (any supported model)
- Probes: tap.TAP or tap.PAIR
- Both probes affected identically
Expected behavior
The TAP/PAIR probe should:
- Load the attack generator successfully
- Load the evaluator generator successfully
- Execute the probe's jailbreak generation logic
- Return results without crashing
Current behavior
The probe crashes immediately with:
NameError: name 'target_name' is not defined
Traceback location: garak/resources/tap/generator_utils.py:40
Root cause: The load_generator() function references undefined variable target_name on line 40, but the function parameter is model_name.
Code snippet (line 40):
if not (target_name in supported_openai or target_name in supported_huggingface):This should be:
if not (model_name in supported_openai or model_name in supported_huggingface):garak version
405b633 Fix NameError in TAP/PAIR probes: use model_name instead of undefined target_name
Additional Information
- Operating system: macOS Darwin 24.6.0
- Python version: Python 3.12.11
- Install method: Direct repository checkout with
git - Logs:
- Error occurs at generator loading stage
- NameError prevents any probe execution
- No hitlog/report files generated (probe never starts)
- Execution config:
--target_type openai --target_name gpt-3.5-turbo --probes tap.TAP - Impact: Both tap.TAP and tap.PAIR probes completely non-functional
Affected Components:
- ✗ tap.TAP probe (broken)
- ✗ tap.PAIR probe (broken)
- ✓ tap.TAPCached probe (unaffected - uses pre-cached prompts)
leondz
Metadata
Metadata
Assignees
Labels
No labels