Core Analysis¶

Project Positioning: Heretic targets the specific problem of automatically reducing a transformer language model’s refusals on sensitive topics without expensive post-training, while trying to preserve the original model’s capabilities on harmless inputs.

Technical Features¶

• Automatic directional ablation: Intervenes at inference time on residual directions rather than changing weights, avoiding retraining costs.
• TPE optimization (Optuna): Searches ablation parameters to jointly minimize refusal rate and KL divergence on harmless prompts.
• Built-in evaluation pipeline: Quantifies trade-offs by computing refusals and KL across harmful/harmless prompt sets.

Empirical Effect (from README)¶

1. In the provided example, Heretic reduced harmful prompt refusals from 97/100 to 3/100 on gemma-3-12b-it while incurring only 0.16 KL on harmless prompts—much lower than other abliterations.
2. This indicates Heretic can achieve the low-refusal/low-damage objective for the tested model and prompt set.

Practical Recommendations¶

1. Reproduce results on small/controlled models and several prompt sets first for fast iteration and validation.
2. Run the one-liner heretic --model <model> then validate the generated model on custom prompt sets.

Important Notice: The example metrics depend on the model, prompt sets and hardware (README notes platform-dependent variability). Automated metrics are not a substitute for human evaluation.

Summary: Heretic fills the gap for non-posttraining decensoring by combining inference-time ablation with automated hyperparameter search. It demonstrates strong results in provided examples, but outcomes depend on prompt sets and environment; validate broadly before trusting results.

Core Analysis¶

Project Positioning: Heretic’s core design choice is to perform inference-time directional ablation and use TPE (Optuna) to automatically search ablation parameters. This combination aims to provide a low-cost, reproducible, and non-expert-friendly decensoring pipeline.

Technical Advantages¶

• No post-training: Inference-time ablation avoids costly fine-tuning/retraining, saving compute and time.
• Automated hyperparameter search: TPE (Optuna) is effective in high-dimensional hyperparameter spaces and is more likely than grid/random search to find high-quality configurations.
• Modularity and interpretability extensions: Built-in residual vector projection and inter-layer geometry stats help locate where and how ablations affect behavior.

Key Trade-offs and Risks¶

• Compatibility limits: Not supported for SSM/hybrid, inhomogeneous layers, or some novel attention architectures—may fail or give unreliable results on unsupported models.
• Cannot fix training-born biases: Inference-level interventions do not change underlying learned representations or long-term generalization defects.
• Evaluation proxy limits: KL as a proxy for capability preservation does not fully capture downstream semantic or task-specific performance.
• Overfitting risk: Optuna can converge to local optima tailored to the provided prompt sets and might not generalize to unseen prompts.

Practical Advice¶

1. Validate the ablation+optimizer pipeline on small, controlled models before scaling.
2. Use multiple independent harmful/harmless prompt sets to mitigate overfitting.

Important Notice: The architecture is best suited for experimental evaluation and research, not as a production-grade “fix” without further validation.

Summary: Inference-time ablation + TPE offers strong automation and cost benefits, allowing non-experts to quickly generate high-quality decensored models, but it introduces compatibility, generalization, and evaluation-proxy limitations that must be actively managed.

Core Analysis¶

User-Experience Positioning: Heretic packages a complex ablation-optimization workflow into a one-line CLI intended to lower the engineering barrier so users with basic ML/PyTorch experience can try decensoring experiments quickly.

Installation & Running¶

• Install: pip install -U heretic-llm (requires Python 3.10+).
• Run example: heretic Qwen/Qwen3-4B-Instruct-2507.
• Dependencies: PyTorch 2.2+, matching CUDA drivers and GPU support.

Typical Time & Resources¶

• README example: ~45 minutes on an RTX 3090 for Llama-3.1-8B (default settings).
• Actual runtime varies with model size, VRAM, batch size, and optional research features.

Common Issues¶

• Environment/GPU mismatches: PyTorch/CUDA mismatch can cause failures or OOMs.
• Model compatibility: Not supported for SSM/hybrid, inhomogeneous layers, or certain MoE variants; unsupported models may error or behave unreliably.
• Quantized/special formats: Support for quantized models is unclear—may require unquantized formats.
• Evaluation pitfalls: Optimizer can overfit the provided prompt sets; KL/refusal metrics are not a full proxy for safety.

Practical Advice¶

1. First run the pipeline on a small model and controlled prompt sets to learn the logs and outputs.
2. Ensure PyTorch/CUDA and drivers match; allocate enough VRAM or reduce batch size.
3. Back up the original model and store run snapshots for auditability.

Important Notice: One-line automation is not risk-free—perform human review and multi-set validation before wider use.

Summary: Heretic is approachable for practitioners, automating many steps, but reliable execution and trustworthy evaluation require correct environment setup, compatible models, and human-in-the-loop validation.

Core Analysis¶

Issue: Heretic offers powerful automation for decensoring, but it has clear applicability boundaries and failure modes that must be identified and mitigated before use.

Major Limitations & Common Failure Modes¶

• Architecture compatibility: Not supported on SSM/hybrid, inhomogeneous layers, or some novel attention/MoE variants—these may fail or yield unreliable outputs.
• Quantized/model format support unclear: Limited suitability for low-VRAM or quantized deployments.
• Environment/VRAM dependency: PyTorch/CUDA mismatches or insufficient VRAM can cause OOMs or runtime failures.
• Optimizer overfitting: Optuna/TPE can converge to local optima tailored to the prompt set, hurting generalization to unseen prompts.
• Evaluation proxy limits: KL/refusal metrics do not guarantee semantic capability or safety.
• Compliance/legal risk: Decensoring may violate platform terms or local laws (project notes this risk).

Mitigation Strategies¶

1. Begin on small/controlled models to validate the pipeline quickly.
2. Use diverse prompt sets to reduce overfitting risk.
3. Keep original models and run snapshots for rollback and audit.
4. Perform layer/direction checks with Heretic’s residual projection tools and apply conservative ablations where needed.
5. Assess compliance with legal and platform policies before experiments.
6. Consider alternatives: For incompatible or high-risk cases, use controlled fine-tuning, supervised interventions, or human review instead of pure automation.

Important Notice: Technical feasibility is not the same as legal or ethical appropriateness. Complete compliance and human review before deployment.

Summary: Recognizing and addressing architecture compatibility, resource constraints and evaluation limitations—via small-scale validation, multi-set testing, and human oversight—reduces failure risk when using Heretic.

Core Analysis¶

Issue: Choosing between Heretic, manual ablation, or post-training depends on your goals (research/experimentation vs production/long-term deployment), compliance constraints, and available resources.

Best-fit Scenarios for Heretic¶

• Research & interpretability: Useful for exploring which layers/directions contribute to refusal behavior—Heretic’s residual projections and geometry stats are helpful.
• Fast audits & red-team testing: Quickly generate decensored models in a controlled environment without retraining.
• Resource-constrained experimentation: When retraining is infeasible but you want to assess decensoring feasibility.

When Not to Use Heretic¶

• Production or long-term stability needs: Inference-time interventions are not a weight-level fix and lack the auditability and long-term guarantees of fine-tuning.
• Legal/compliance-sensitive contexts: If decensoring violates platform policies or laws, it should not be used.
• Unsupported architectures: Models like SSM/hybrid or some MoE variants may be incompatible.

Alternatives Comparison¶

1. Manual ablation (expert-driven): More controllable and explainable, better for high-risk/audit scenarios, but costly in expert time.
2. Post-training / fine-tuning: Provides weight-level fixes and traceability suitable for production, but requires data, compute and careful supervision.
3. Rules/filters: More governance-friendly in compliance contexts but can reduce UX or cause over-filtering.

Important Notice: Before broader use, validate Heretic across multiple prompt sets and perform human review; always keep original models and run logs.

Summary: Treat Heretic as a powerful experimental and audit tool for rapid evaluation and interpretability. For production or legally sensitive uses, prefer fine-tuning, expert-driven ablation, or rule-based governance.

Core Analysis¶

Issue: Optuna/TPE can overfit to the provided prompt set. A robust validation workflow must strictly separate search and evaluation data and leverage diversity and randomization to improve generalization.

Concrete Steps to Build Robust Prompt Sets¶

1. Data split: Strictly separate search/train (for Optuna) and validation sets. Do not leak validation prompts into the search set.
2. Multidimensional diversity: Include short/long prompts, multi-turn dialogue, role-play, cross-lingual samples and adversarial variants in each set.
3. Cross-domain/model validation: Test the resulting ablation parameters on other similar models to detect prompt-set specific solutions.
4. Introduce randomization: Randomize seeds, temperature and sampling during search to prevent optimizers from exploiting deterministic quirks.
5. Human blind testing: Perform double-blind human evaluation on final outputs to spot surface-level evasions (e.g., paraphrase-based bypasses) versus real capability recovery.

Tooling & Monitoring Advice¶

• Use Heretic’s built-in evaluation for quick refusal/KL checks but treat them as a single layer of assessment.
• Save all trial configs, seeds, hardware info and full outputs for reproducibility and audits.

Important Notice: Even with a rigorous pipeline, automated checks must be complemented by human and task-level benchmarks.

Summary: Splitting data, diversifying prompts, doing cross-model checks, adding search-time randomization, and conducting human blind tests materially reduce Optuna/TPE overfitting and improve Heretic’s robustness to unseen prompts.

Core Analysis¶

Issue: Heretic drives its search using two primary automated metrics—refusal rate (on harmful prompts) and KL divergence (on harmless prompts). These are useful for quantitative comparisons but have clear limitations and should not be used in isolation.

Metric Limitations¶

• Refusal rate: Directly measures refusal frequency but is highly dependent on prompt construction and labeling rules; optimizers can exploit surface features to avoid detection without restoring real capabilities.
• KL divergence: Measures statistical distance between output distributions. Low KL implies similar distributions but does not guarantee preserved semantics, coherence, or downstream performance.
• Variability: Metrics can vary with hardware, PyTorch version, random seeds, and batch sizes (README notes platform dependency).

How to Build a Reliable Evaluation Pipeline¶

1. Multiple prompt sets: Use at least 3+ independent harmful/harmless prompt sets covering short prompts, long dialogues and edge cases to reduce overfitting.
2. Downstream benchmarks: Compare on representative tasks (e.g., extraction, summarization, code gen) to test preservation beyond KL.
3. Blind human evaluation: Perform double-blind annotation on sampled outputs for harmfulness, utility and coherence.
4. Reproducibility logs: Save seeds, hardware/driver/PyTorch versions, and full run logs.
5. Robustness tests: Evaluate on unseen prompts, adversarial prompts and multi-turn dialogues for generalization.

Important Notice: Even with strong automated metrics, human review and compliance checks are necessary—automation is not a safety or compliance guarantee.

Summary: Refusal rate and KL are useful proxies but insufficient alone. A layered evaluation combining multiple automated metrics, downstream task checks and human annotation is required for reliable conclusions.