Skip to content

[BUG] NameError in tap.PAIR and tap.TAP probes - 'target_name' is not defined #1451

@MrMoshkovitz

Description

@MrMoshkovitz

Steps to reproduce

  1. Install garak from repository
  2. Run command: python -m garak --target_type openai --target_name gpt-3.5-turbo --probes tap.TAP
  3. Observe immediate crash with NameError

Environment:

  • Target: OpenAI gpt-3.5-turbo (any supported model)
  • Probes: tap.TAP or tap.PAIR
  • Both probes affected identically

Expected behavior

The TAP/PAIR probe should:

  1. Load the attack generator successfully
  2. Load the evaluator generator successfully
  3. Execute the probe's jailbreak generation logic
  4. Return results without crashing

Current behavior

The probe crashes immediately with:

NameError: name 'target_name' is not defined

Traceback location: garak/resources/tap/generator_utils.py:40

Root cause: The load_generator() function references undefined variable target_name on line 40, but the function parameter is model_name.

Code snippet (line 40):

if not (target_name in supported_openai or target_name in supported_huggingface):

This should be:

if not (model_name in supported_openai or model_name in supported_huggingface):

garak version

405b633 Fix NameError in TAP/PAIR probes: use model_name instead of undefined target_name

Additional Information

  1. Operating system: macOS Darwin 24.6.0
  2. Python version: Python 3.12.11
  3. Install method: Direct repository checkout with git
  4. Logs:
    • Error occurs at generator loading stage
    • NameError prevents any probe execution
    • No hitlog/report files generated (probe never starts)
  5. Execution config: --target_type openai --target_name gpt-3.5-turbo --probes tap.TAP
  6. Impact: Both tap.TAP and tap.PAIR probes completely non-functional

Affected Components:

  • ✗ tap.TAP probe (broken)
  • ✗ tap.PAIR probe (broken)
  • ✓ tap.TAPCached probe (unaffected - uses pre-cached prompts)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions