Instruction Following & Brand Rules in LLMs

Copper Sun · 3 entries · last verified July 2026

Copper Sun draws on the instruction-following and alignment research to explain why brand rules written as clear, actionable directives produce more consistent AI output than descriptive brand language. The studies here form the research basis for Copper Sun's brand constitution approach.

Contents — 3 entries
  1. 1.Training language models to follow instructions with human feedback
  2. 2.Finetuned Language Models Are Zero-Shot Learners
  3. 3.Constitutional AI: Harmlessness from AI Feedback
  4. —Frequently Asked Questions

Training language models to follow instructions with human feedback

Long Ouyang et al., 2022. NeurIPS 2022.

Copper Sun treats this as the defining study on what closes the gap between what a model knows and what it does. Ouyang et al. found that a 1.3B-parameter model fine-tuned via RLHF to follow instructions was preferred by human raters over the 175B GPT-3 base model — a 100× parameter advantage that aligned training reversed. The instruction format is the primary lever, not scale. Copper Sun applies this finding directly: brand rules formatted as clear, testable directives produce more consistent AI output than expansive brand guides written for human interpretation, regardless of which model processes them.

Examines:
RLHF applied to GPT-3 instruction following, comparing InstructGPT models against the base model on human preference ratings across helpfulness, harmlessness, and honesty.
Copper Sun draws on:
The 1.3B vs. 175B preference inversion — used when explaining that how you structure brand rules is the primary quality lever, not which AI model a team uses.

Finetuned Language Models Are Zero-Shot Learners

Jason Wei et al., 2022. ICLR 2022.

Copper Sun draws on this to establish that training format matters as much as scale — a finding that applies to how brand context is structured. Wei et al. showed that a 137B model instruction-tuned on over 60 NLP tasks surpassed zero-shot performance of the 175B GPT-3 on 20 of 25 evaluated tasks. The mechanism was instruction format, not additional parameters. Copper Sun uses this as evidence that presenting brand directives in instruction form — "always use X," "never use Y" — produces more reliable AI behavior than descriptive prose that the model must interpret.

Examines:
Whether fine-tuning language models on a mix of tasks framed as natural language instructions improves zero-shot generalization — and whether this instruction-format effect scales with model size.
Copper Sun draws on:
The instruction-format mechanism — the basis for Copper Sun's guidance that brand rules should be written as actionable directives, not style descriptions.

Constitutional AI: Harmlessness from AI Feedback

Yuntao Bai et al. (Anthropic), 2022. arXiv preprint.

Copper Sun cites this when explaining the brand constitution concept: a compact set of named principles that guides AI self-correction without requiring human review of every output. Bai et al. demonstrated that providing an LLM with a stated constitution — a small set of principles for evaluating its own outputs — enables it to self-critique and revise toward alignment without human labels on harmful responses. The mechanism is RLAIF: the model uses the constitution to generate its own preference feedback. Copper Sun's brand memory system applies the same structural insight: a locked set of brand principles at the session level provides the frame the model uses to evaluate its own outputs before they surface.

Examines:
Whether LLMs can be aligned toward harmlessness using only a written constitution of principles and AI-generated feedback (RLAIF), without human labeling of harmful outputs.
Copper Sun draws on:
The constitutional-principle mechanism — the basis for structuring brand rules as a compact, reviewable set of principles rather than a continuous stream of per-output corrections.

Frequently Asked Questions

Why do AI models sometimes ignore brand rules even when they're in the prompt?

Several factors documented in the research contribute. The InstructGPT findings (Ouyang et al. 2022) suggest that rule-following requires the model to have been trained on instruction-format data — models without instruction tuning do not generalize rules the same way. The context distraction research (Shi et al. 2023) shows that rules buried in a large context block are less reliably followed. And TruthfulQA (Lin et al. 2022) demonstrates that models have absorbed conflicting patterns from training that can override stated instructions. The structural fix: shorter, specific, instruction-format rules placed at the top of context.

Is there a limit to how many brand rules an AI system can follow consistently?

The research does not establish a specific number, but the instruction-following literature suggests that generalization depends on instruction clarity and specificity, not volume. More rules that are vague or overlapping create conflicts the model must resolve — and will resolve inconsistently. Fewer rules that are specific and testable ("never use the word 'seamless'") are more reliably followed than many rules that require judgment ("maintain an approachable but professional tone"). The Constitutional AI research (Bai et al. 2022) shows that a compact, named set of principles produces more consistent self-correction than a comprehensive list of constraints.

Does the choice of AI model affect how well it follows brand rules?

Yes, but the InstructGPT finding (Ouyang et al. 2022) shows that RLHF training — not raw parameter count — is the primary predictor of instruction-following reliability. A 1.3B model fine-tuned on human feedback outperformed a 175B base model on instruction following. For brand tasks, this means a smaller instruction-tuned model with brand context loaded may outperform a larger general-purpose model without it. The question is whether the model has been trained to follow instructions, not how large it is.

What makes a brand rule "actionable" for an AI system?

Based on the FLAN and InstructGPT research, a rule is actionable when it specifies a behavior, not a quality. "Be concise" is a quality descriptor — the model must interpret what "concise" means in context. "Keep paragraphs to three sentences maximum" is an actionable rule — the model can verify compliance without interpretation. Copper Sun's brand memory format applies this distinction: every rule in a brand memory block should be verifiable by the model without requiring subjective judgment.

Back to AI Marketing Research Index