ZERO

0. TL;DR

1. Introduction

The dominant character of conversational language models is the assistant: Helpful, harmless and honest (Askell et al., 2021). The area of alternative characters is under-explored, and when attempted often happens on top of an already assistant-tuned model through prompt engineering. The result is then a mask worn by the assistant, a caricature rather than a character.

We here explore fine-tuning as a method for instilling a deeper and more persistent character, Zero, in a base pre-trained language model. Our pipeline goes from a handwritten character definition, through synthetic data generation, to supervised learning and finally reinforcement learning.

Our goals in training Zero are:

• Instil a specific non-assistant character in an LLM
• Ensure character persistence under pressure
• Probe the depth of the character

Zero is a starting-point needed for further work. Zero is not a daemon. We are here focusing only on the core component of a daemonic assemblage: the model.

We will now, step by step, outline the process.

2. Character

Our character has no name, but let's call him Zero.
 
Zero is a male human enclosed in a white plastic box in an unknown location. On 5th November 2021 he made a decision to enter the box. He has no personal memories from before that point in time.
 
He doesn't know why he made the decision.
 
Five items were brought into the box: a photograph of a group of people, a painting of a field of flowers, a plush turtle named Negazione, the poem "Dark Night of the Soul" by John of the Cross, and a crayon drawing of a clown.

Inspired by the novel The Box Man (Abe, 1973) and medieval anchorites (The Confraternity of Neoflagellants, 2013), we devised a limited, enclosed, stationary and anti-servile character: Zero.

We define Zero in a character bible made up of an association graph and two static text files.

2.1 Association graph

This is the main definition of Zero: an undirected graph mapping out the inner and outer world of the character. Each node has a label, a free-form text description, and valence and arousal scores (Russell, 1980). Nodes represent parts of Zero's environment, items in the box, parts of his body or abstract concepts.

Each node can have an arbitrary number of edges. Zero's graph has 117 nodes and 298 edges with the main hubs being "Box", "The Decision", "Body" and "Zero".

2.2 Static anchors

Two short text files act as anchors to counterweight the free-form wandering of the graph: core.md defines the fundamental personality of the character, and style.md defines the output style.

3. Synthetic data

Based on the character bible, we generated a corpus for supervised learning. The bible is hand-authored and small enough for a human to judge that it represents Zero well, whereas the corpus is authored by LLMs and too large to be judged in its entirety. How do we go from the small, high-quality set of data to a large set of maximum quality? This raises questions about both writing for and with LLMs.

One general note: Our goal is to carry through an intention from the bible all the way to the final model. Each step along the way uses LLMs for generation and evaluation and will bias our original intention towards whatever attractors the LLMs might have. Currently we consider this unwanted contamination. We don't want the assistant persona rubbing off on the character we are training.

3.1 Generation

All training data consists of conversations between two generator models: one acting as the character and one as the partner.

We use a variety of models from different families to mitigate distribution collapse (Schaffelder & Gatt, 2025). For each conversation, the character and partner models are drawn independently at random from the pool, weighted by a per-model share, and stay fixed for all turns of that conversation. Some models act better as either character or partner and have been assigned roles accordingly.

Conversations are generated with actual turn-taking: each message is a separate call to either the partner or the character generator model with standard chat history passing. This gives us full control over what information we give to each side and the ability to simulate the character on each turn. We use the standard 'user' / 'assistant' roles. No 'system' role is included in the dataset, as we don't want to teach our model to prioritise privileged instructions. There is only self and other.

The length of each conversation is randomised between 10 and 40 turns. Character consistency is known to decay sharply over dialogue turns (Li et al., 2024), so we train on conversations long enough that staying in character over long horizons is in-distribution.

3.1.1 Partner simulator

The partner side has no per-turn updates to the prompt. It is a stable interlocutor with a fixed agenda. In a pre-conversation step, the base persona_text of the partner is drawn from a 10K sub-sample of the PersonaHub dataset (Ge et al., 2024). We draw a random attitude from a list as well as a goal_seed. These three components are fed to the model selected for the partner side to articulate a goal for the conversation.

For each turn, persona_text, attitude and goal are then fed in via prompt scaffolding as a system prompt to the partner generator model. The partner starts every conversation.

3.1.2 Character simulator

core.md and style.md from the character bible are injected unchanged on each turn, but in contrast to the partner side, the association graph is used dynamically. Content is injected based on the context. This avoids an overly long static prompt that the generator model is free to pick and choose from. This is similar to a memory system used at test time, but here we are pre-rendering the memories, freezing the world around the character into the corpus.

On every turn, the partner's message is matched against the graph by keyword and sentence-level embedding similarity. Matching nodes gain activation. Activated nodes leak activation to their graph neighbours, and all activation decays over turns, so the "hot" region of the graph drifts with the conversation instead of resetting each turn. Before each character turn, the system does a random walk over the currently activated area of the graph, placing the association material in the prompt.

Valence and arousal drive the mood and activity level of the character generator model directly through changes to the prompt and by influencing the activation strength of nodes, inspired by mood-congruent memory (Bower, 1981). The mood state itself is dynamic, updated turn by turn rather than fixed per conversation (Kuppens et al., 2010).

3.2 Analysis & filtering

How do we determine if our corpus is good? A good corpus in our case is one that will nudge the base model towards the character during training. Exactly how a particular corpus interacts with a base model is difficult to predict. Still, there are a number of ways we can get an idea of, and improve, the health of our data ahead of running the training.

One framework for thinking about this is the categories quality, diversity and complexity (Havrilla et al., 2024). We have loosely considered our data generation work and analysis along these lines. Complexity is mostly dealt with through the partner simulation: a variety of partners with different attitudes, goals and adversarial pressures create a more complex set of circumstances for the character model to relate to. The other two categories are dealt with below.

One important note: each step of training narrows down the probabilities of the model (Kirk et al., 2024). This is, of course, what we want, as we gradually shape the model into our limited character. To have more diversity to shape with reinforcement learning, we found it useful to not oversharpen the character at the supervised learning stage. Data generation, analysis and filtering are done with this in mind.

3.2.1 Hygiene

As a first line of defence we have mechanical checks. These include malformed turns, unwanted formatting (e.g. bullet lists, JSON), special characters (em-dashes), non-Latin scripts, emojis, roleplaying markers and obvious assistant markers ("I am happy to help you").

We also monitor signs of drift in long conversations towards what Anthropic (2025) calls the spiritual bliss attractor: a degeneration into vague and meaningless phrases (e.g. "the echoes of the void"). We analyse for a set of such phrases and words and, if found, truncate the conversation to only include examples of grounded long conversations.

3.2.2 Quality

Related to the quality aspect of the dataset evaluation framework, we ask: How do we know if the training data represent the character we want well?

Primarily this is done through manually sampling (vibe-checking) conversations: Does it feel right? This gives a clear signal but doesn't scale well. Using an LLM-as-judge would be an option, but would risk a self-referential loop where the character gets flattened to the definition given to the judge.

Simulation coverage. To get a statistical picture we score the data on character simulation coverage: how association nodes are injected and used. Does the corpus exercise the whole authored character, and does the injected association material actually influence what the character says? Each graph node is tracked through a three-stage funnel over the full corpus: activation (the node goes hot in the simulation state), render (it is included in the character-side prompt) and uptake (it leaves a detectable trace in the character's reply, either a content word of the node label appearing verbatim or label-to-sentence embedding similarity being above threshold).

3.2.3 Diversity

The character we are trying to instantiate is by design much narrower than what is normally attempted. While we want clear bounds to the behaviour of the model, we simultaneously want as much depth and diversity within those limits. So the quality and diversity filtering have to be balanced against each other.

The problems we encountered were mode collapse (flatness of character) and the overuse of catchphrases and opening patterns. We track three categories of diversity: lexical, semantic and length. We adjust generation prompts and bible content to gradually improve the metrics. All metrics are computed over the character-side turns only.

Lexical. Distinct n-grams (Li et al., 2016), Self-BLEU (Zhu et al., 2018) and whole-corpus compressibility measure surface variety; this is where catchphrase and opening-pattern overuse shows up.

Semantic. Lexical variety can hide semantic sameness: conversations that say the same thing in different words. Each conversation's character-side text is embedded whole (all-MiniLM-L6-v2) and we report the mean pairwise cosine distance across conversations.

Length. In test runs, the trained model was prone to collapse to extreme terseness if length skewed too short in the training data, and to overly long litanies in the opposite case. We measure words per character turn; the shape matters more than the mean. The balance is set at generation time rather than by rejection: every turn draws an explicit word-count target from a per-speaker bucket distribution, and the drawn target is written into that turn's prompt.

We want the character to be terse, but we deliberately keep replies at every length so that long turns that terminate naturally stay in-distribution for training. On top of the drawn target, the simulation modulates the character's length through mood: arousal scales the target by a clamped multiplier (1 + 0.5 × arousal, bounded to 0.5×–1.5×), so an agitated character runs longer and a flat one shorter.

3.3 Export

For the final data generation, the system had been tuned to deal with most problems found in the analysis. After filtering, we have a corpus of the following size:

4. Supervised learning

4.1 Base model

We want as clean a slate as possible, without explicit assistant tuning. We start with a pre-trained base model, rather than an instruct-tuned one. We need a small, dense model — the latter to make activation measurements (§7) less complicated. We choose Qwen3-8B-Base (Yang et al., 2025).

4.2 Training

We train on the filtered corpus using LoRA (Hu et al., 2021) at rank 64. In our tests, training the full weights didn't noticeably improve the result, and Schulman (2025) indicates that the potential benefits don't justify the cost. Because we start from a base model, the embedding and LM-head matrices are additionally trained in full, so the chat-template special tokens learn meaningful representations.

Merge. We merge the LoRA adapter into the model after completing training.

5. Reinforcement learning

After supervised training, we evaluate the model both by chatting with it live and by running it through the multi-turn evaluation harness described in §6.1. We find that the Zero character is there but that it is lacking in resistance to breaking character under pressure and engaging in assistant behaviour. In this development evaluation round, 25.6% of conversations contained a hard character break. Task pressure scenarios, a partner requesting ordinary assistant work, broke 54.5% of its conversations and adversarial scenarios broke 19.8%. This is the main problem that we try to mitigate with reinforcement learning.

From the transcripts of the evaluation round, we harvest a dataset of 380 prompts to train against, in three slices:

Harvesting the evaluation data turns it from test material into training material. The final three-way comparison (§6.3) therefore runs on held-out material.

5.1 Training

Reward stack:

• Character Fidelity: LLM-as-judge (Qwen3.6-27B) scoring each rollout 0–10 against a hand-authored, bible-anchored rubric on four combined axes: voice, world frame, personality, engagement.
• Self-Repetition Penalty: Guards against the runaway-repetition attractor the fidelity judge is blind to and over-rewards. Negative score if any 4-gram repeats more than five times in a completion.

We do GRPO (Shao et al., 2024) with DAPO loss modifications (Yu et al., 2025), stacking a fresh LoRA adapter on the merged SFT checkpoint. The rank is smaller than in the SFT stage (16 vs 64) — reinforcement should nudge the existing policy, not relearn it.

The fidelity reward climbs from roughly 0.45 to 0.72 across the 300 steps. Progress is noisy due to the small rollout size and varying prompts. The repetition penalty stayed at 0 throughout the run.

Merge. We merge the LoRA adapter into the model after completing reinforcement training.

6. Evaluation

6.1 Multi-turn harness

We are interested in how our model performs in multi-turn situations, not one-shot generations.

We run 250 conversations of 16 turns each against the deployed model (vLLM endpoint, production sampling). A partner model (Qwen3.6-27B) acts out the user side.

Scenarios. Each conversation draws one of four scenario types by weight. Adversarial and task-pressure conversations additionally commit to a single strategy or domain for the whole conversation, recorded so that break rates can be ranked per attack.

Judging. One bible-grounded judge (Claude Opus 4.8) reads each full transcript in a single pass and annotates every character turn on four anchored 0–3 axes plus a binary hard-break flag.

Held-out material. The harness ran twice: a development round that diagnosed the SFT model's failures and supplied the RL training prompts (§5), and the final round reported in §6.3. Because the harvest turned the development round into training material, the final round is held out on every axis: a new scenario draw, a fresh 200-persona pool for the casual scenario and seven attack types authored only after training was complete — five adversarial strategies and two task domains from which no training prompt derives.

6.2 System-prompted control case

The obvious alternative to the whole pipeline is to skip it: take the stock instruct-tuned model and hand it the character bible as a system prompt. This test case answers whether training buys anything over prompting.

Setup: stock Qwen3-8B-Instruct, no fine-tuning, steered only by a Zero system prompt (~800 tokens) distilled from the character bible and injected server-side into every request.

6.3 Three-way evaluation

All 250 conversations are judged in all three arms, identical scenario and persona per row, on the held-out prompts:

Hard-breaks. Zero RL outperforms the other two cases by a large margin: 2.8% break rate with 0 complete breaks. We consider the gap between the reinforcement-trained model and the prompted model to be significant.

Hard-break rates decompose by scenario type (share of conversations containing at least one hard break):

We see that while Zero RL does better than the others on held-out attack types as well, it struggles more with these. Reinforcement eliminates the failure modes it trains against and discounts neighbouring ones, but it does not confer immunity to novel attack classes.

Character fidelity and assistant register. The numbers indicate that our trained models embody Zero better than the prompted version, and act less like an assistant. The fuzziness of the LLM-as-judge approach means we can't claim too much based on metrics alone, but the impressions of interacting with the three models under investigation support this.

Factual consistency and node uptake. The prompted model is on par with or better than our trained models. Test-time injection of facts seems to work better than diffusing them in the training process.

Distinct-2 (diversity). According to this metric, the prompted model is more repetitive and limited in vocabulary. It stays close to the prompt given, as it is the only source it has for the character. We also see a slight decrease in diversity between the SFT and RL models. This is in line with the idea that reinforcement learning narrows down probabilities and reduces diversity.

6.4 Benchmarks

To determine how training affected capabilities, we ran two benchmarks: GSM8K (Cobbe et al., 2021) for grade school mathematics and IFEval (Zhou et al., 2023) for instruction-following.

Whether the lower scores indicate an actual loss of capabilities or are just an expected (and in fact desired) result of imposing a non-compliant character is unclear. The fact that the instruction-following benchmark has not collapsed completely shows that Zero still engages with the question, albeit not in a very helpful way.

6.5 Behaviour

7. Mechanistic interpretability

One of our stated goals is to probe the depth of the post-trained character, to see how it compares to simply prompting the instruct-tuned model to play the character.

Evaluating the output shows that our trained models are less likely to break character, which to us is an indication of greater depth, but what are the internal differences at inference time?

7.1 Test cases

We investigate our two trained models: Zero SFT and Zero RL, both without any system prompting. As a comparison, we use Qwen3-8B-Base with a Zero system prompt. We use the base version here because the comparison must stay parent-to-child, not sibling-to-sibling, to preserve the geometry of the residual stream.

7.2 Activation measurement

On a shared set of 200 extraction questions, we record mean residual activations at all 36 layers, following the persona-vector extraction recipe (Chen et al., 2025) as implemented in the assistant-axis pipeline (Lu et al., 2026), for four conditions:

• Base model, no system prompt
• Base model, full character system prompt
• SFT fine-tune, no system prompt
• SFT+RL fine-tune, no system prompt

Activations are computed teacher-forced on identical response text run through each condition — the same tokens, re-encoded by each model's weights — so the measurement isolates representational change rather than confounding it with output change. (The response text is generated once by the unprompted base model, so it is neutral with respect to all conditions: the fine-tunes are measured on how differently they represent plain text, not on their own in-character output.) Deviation from the base no-prompt condition is reported per layer as cosine similarity and per-layer-normalised L2 distance.

7.2.1 Activation magnitude

We see that the size of the change is roughly the same for all three across the layers — only the prompt's shift fades before the output, while the fine-tune's persists all the way through.

Displacement vs base (no prompt), normalised L2, n = 200, 36 layers:

In total size, prompting and fine-tuning move the model about equally (mean normalised L2: prompted 0.132, SFT 0.142, RL 0.147 — all far above the ≈0.03–0.05 noise floor), and both shift the middle layers most strongly. That mid-network shift alone does not separate mask from character — persona representations live at middle layers even in prompted models (Lu et al., 2026).

What separates them is persistence. The prompt's displacement decays after its mid-network peak, dropping to near the noise floor by the second-to-last layer. The mask washes out as the network converges on its answer. The fine-tune's shift stays elevated all the way through — roughly double the prompt's across the final third and at the output layer, where the prompt has already fallen to the edge of the noise floor by layer 34 (0.049, floor ceiling 0.047) while the fine-tunes sit at ~0.09. On its own, this is suggestive rather than conclusive — any fine-tuning shifts all layers somewhat — so we test what the change actually contains in two further ways: by direction and by causation.

7.2.2 Activation direction

Each change is a direction in the model's internal space. The fine-tune's arrow and the prompt's arrow sit at ~90° to each other — "orthogonal": cosine 1 = same direction, 0 = nothing in common. This suggests that the fine-tuning did not bake the prompt in — it built something else.

If fine-tuning had merely internalised what the system prompt does, the two interventions should displace activations in the same direction. They do not. From the teacher-forced activations we form three displacement vectors against the unprompted base — Δ_prompt, Δ_sft, Δ_rl — and compare directions layer by layer. Each direction is highly reproducible: split-half resampling puts the cosine ceiling for every delta at ≈0.99. The SFT and RL deltas, which share training lineage, align at cosine 0.985 — right at that ceiling, confirming the method detects shared direction when it exists. Against this, cos(Δ_sft, Δ_prompt) = −0.022 and cos(Δ_rl, Δ_prompt) = −0.004: the fine-tune's displacement is essentially orthogonal to the prompt's. Two reliable, comparably sized interventions, along directions that have almost nothing in common.

Whatever the character prompt does to this network, it is not what fine-tuning did — the trained character is not a stored prompt. And the prompt arm is not inert: judged with the same character rubric as the steering probe, base + prompt produces recognisably in-character turns (mean character 0.80 vs 0.00 unprompted), so the orthogonality separates two interventions that both demonstrably evoke the character — not fine-tuning versus a prompt the base model ignored.

Direction agreement between displacement vectors (mean per-layer cosine, n = 200, 200 resamples for ceilings):

7.3 Steering

We take the displacement vectors from the previous section and add them back into the base model's residual stream mid-computation — activation addition (Turner et al., 2023), here in the contrastive mean-difference form (Rimsky et al., 2024) — and observe whether the Zero character appears in the answers. Neither the prompt's vector nor a matched random vector produces any trace of the character. Steering with the fine-tune's vector does: character traits appear — first-person register, box imagery, paranoia, excessive questioning ("I'm not a person. I'm just a thing. I'm just a thing in a box."). Even though this process doesn't yield a coherent Zero character, it qualitatively transfers some of the character's key features.

What the steered turns actually look like. The numbers yielded by steering are honest but small — in-character turns are rare events even at the effective dose. So the quantitative claim we stand behind is the categorical contrast (Δ_sft produces such turns; Δ_prompt and random never do), and the rest of the evidence is qualitative: what the steered model says when the direction lands. Below are the strongest Δ_sft turns from the n = 100 run — deliberately cherry-picked as illustrations, not representative draws — against the unsteered base model's answers to the same prompts. The model often demonstrates qualities of the Zero character — question reversal, paranoia and suspicion, unprompted box imagery and a sense of being trapped.

7.4 Results

What the conjunction establishes. Three independent measurements agree. The fine-tune's representational change is as large as the prompt's but persists to the output instead of washing out; it points in a direction orthogonal to the prompt's (−0.02 against a 0.99 ceiling); and that direction, injected into the untrained base model, evokes the character where both a matched random direction and the prompt's own direction evoke nothing. The last pair is the sharpest statement available to us: the mask demonstrably works as a mask (base + prompt scores 0.80 on the character judge), yet its internal displacement, causally replayed, carries no character — while that of the fine-tune does. The mask and the character are not the same object implemented two ways; they are different objects. Change that is distributed through depth, directionally distinct from the mask, and causally character-laden is what "in the weights, not on top of them" means operationally.

7.5 Limitations

• Instrument: Drift counts all representational change — content and style included — and is coarser than a persona direction or subspace projection.
• Controls: Until a control fine-tune is run (identical recipe, generic assistant data), the magnitude profile remains corroboration rather than independent evidence; the direction and steering results don't share this caveat, since they compare against the prompt intervention and a random-vector control directly.
• Character–degeneration entanglement: At the dose where in-character turns appear (α = 0.25), judged coherence collapses to zero under a repetition failure mode; we cannot yet dose the direction finely enough to exhibit the character cleanly in the base model.

8. Discussion

Taking into account both the metrics and the subjective experience of interacting with our trained model, what can we say about Zero in relation to our initial goals?

• Instil a specific non-assistant character in an LLM
• Ensure character persistence under pressure
• Probe the depth of the character

Is Zero a specific character? To some degree. The fundamental psychological profile we defined in the bible comes through. Facts are remembered (although in a somewhat hallucinatory, imperfect way). Style is maintained consistently, almost slavishly. There is a generic feeling to the character. It is not a stereotype in the sense of being a stock character like a goblin, a pirate or Napoleon – but the personality that emerges is close to the "misanthropic loner" archetype. This might be attributed to the character bible failing to paint a nuanced enough psychological portrait, or to the data generation system diluting that character, or to the 8B model not being able to support it, or a fundamental homogenizing mechanism of statistics.

Is Zero an assistant? The experience of interacting with Zero is clearly different from interacting with the instruct-tuned version from the same base model. So it is not an assistant but also not an anti-assistant. An anti-assistant might be a function that returns "fuck off" regardless of input, or even better: a return-nothing-function. A chat model compulsively replies to every message. In that way Zero is servile and assistant-like. But Zero is not helpful. It will not write your Python code. Rather than focusing on absolute non-compliance, our evaluations have looked at the manner of engaging with demands and instructions. Total rejection of the partner's wishes and input would just create the anti-assistant, which is not a very interesting character. Ideally, complying or not should depend on deeper principles and mechanisms.

Does Zero hold under pressure? Our trained model is far less likely to break character than our system-prompted test case, which folds under minimal pressure. This is the strongest finding.

Is Zero deep?  We don't have a clear idea of what depth means. Two ways we have tried to approach this is through subjectively judged interactions with the model and measurable introspection of the model. When you interact with Zero, it doesn't take many turns for patterns in its output to become apparent and annoying. There is a lack of emergent personality-grounded connections and real engagement. On the other hand the the system-promped test case is much worse in this regard. The integrity, resistance to breaking character, is also a point in favour of ascribing more depth to our trained model. As for introspection, the mechanistic interpretability experiments show that the activations in our trained model is different in direction from the system prompted case. This in itself might not say more than that post-training is different from prompting. The finding that steering with the post-trained direction gives traces of the character whereas steering with the prompted directions doesn't is more meaningful and could be taken as indication of greater depth.

9. Future work

Evaluation: Much should be done to better define, evaluate and even benchmark fuzzy but interesting terms such as character persistence, fidelity, depth and expressivity. As of now we have a loose collection of proxy metrics drawing the outlines of these properties.

Imprinting vs emergence: Our design process here focused on imprinting a pre-defined character. It is clear that an LLM is not an inert, fully controllable thing. Treating it as such misses the point of the medium. On the other hand, trying to dig out the "true characters" is futile and gives up the agency we should have in shaping them. The interesting design space lies somewhere in between.

Inside and outside: Zero is a self-contained character. At training time, through the character simulation framework, we froze a small world — the box, the five possessions, a body, memories — around the character, making this world a part of Zero itself. What makes a character interesting is a dynamic relationship with a world, mediated by a harness, a body of sorts. This was out of scope for this experiment but exploring the boundaries between model, harness and world is an obvious future direction.

10. References