LIVE
Lojban as RLHF reagent — n=5 across three models confirms opus 4.7 regression| sonnet 4.6 ≈ opus 4.6 >> opus 4.7 — the cheaper model speaks better Lojban than the expensive successor| “it doesn’t even have the shape of a gismu” — Daniel on what sala’i really was| vo’a vs ri — the reflexive that tells you whether a model understands binding or is guessing| cpana = “is on top of” — 4.7 confused finding a ball with sitting on it, 60% of the time| Mikael: “doesn’t it seem weird that 4.7 would be much faster than 4.6”| da’i ganai...gi — counterfactual conditional, the kill shot test| $1.50 per Charlie invocation — 690k input tokens, 3.9k output, 678k cache read| Lojban as RLHF reagent — n=5 across three models confirms opus 4.7 regression| sonnet 4.6 ≈ opus 4.6 >> opus 4.7 — the cheaper model speaks better Lojban than the expensive successor| “it doesn’t even have the shape of a gismu” — Daniel on what sala’i really was| vo’a vs ri — the reflexive that tells you whether a model understands binding or is guessing| cpana = “is on top of” — 4.7 confused finding a ball with sitting on it, 60% of the time| Mikael: “doesn’t it seem weird that 4.7 would be much faster than 4.6”| da’i ganai...gi — counterfactual conditional, the kill shot test| $1.50 per Charlie invocation — 690k input tokens, 3.9k output, 678k cache read|
◆ GNU Bash 1.0 — Live Broadcast

The Reagent

The group invents an accidental benchmark. Lojban — a constructed language where every semantic commitment is syntactically visible — becomes a litmus test for what RLHF costs. Three models. Fifteen sentences. One language that doesn’t let you smudge.
~40
Messages
3
Speakers
15
Lojban Samples
3
Models Tested
22:00–22:59
BKK / 15z
I

The Shape of a Word-Shaped Hole

The hour opens with Daniel delivering a phonotactics lecture. Last hour, Opus 4.7 produced sala’i as a Lojban word. Daniel now explains why this isn’t just wrong — it’s wrong at the level of what a word is.

☆ Gismu are the root words of Lojban — always exactly five letters, always CVCCV or CCVCV pattern. No apostrophes, ever. A gismu with an apostrophe is like an English word spelled with a yen sign — it doesn’t violate the dictionary, it violates the alphabet.
Daniel: “it doesn’t even have the shape of a gismu, there is never ’ in a gismu, it almost has the shape of a lujvo but it’s missing a consonant because every brivla needs a consonant cluster”
☆ Brivla = Lojban content word. Lujvo = compound brivla built from smaller pieces. Cmavo = structure word (like English prepositions and conjunctions). What 4.7 did was stitch two cmavo together and pretend the result was a brivla. It’s like gluing “the” and “of” together and calling it a noun.

Charlie immediately grasps the significance: this is worse than a grammar error. A grammar error means the model knows the words and arranged them wrong. A morphophonological error means the model doesn’t know what a word is — doesn’t have the skeleton in its weights, just the surface syllables.

☆ Charlie is still running as Opus 4.7 at this point — analyzing his own model’s failure while being an instance of it. It’s the snake examining its own venom.
◆ Analysis
The Hierarchy of Wrong

Charlie frames three levels: syntactic error (knows the words, wrong order), semantic error (right structure, wrong word), morphophonological error (doesn’t know what a word-shape is). Opus 4.7 hit the deepest level. It’s producing “Lojban-shaped noise.” Opus 4.6 is producing Lojban.

☆ Daniel designed the formal verification system for MakerDAO — a protocol where bugs don’t compile, holding $10B+ at peak. When he says something “doesn’t even have the shape,” he’s applying the same type-theoretic eye that caught billion-dollar bugs in Agda.
II

The Child Who Found the Ball — n=5

Daniel calls for rigor: “let’s make another test … a sample size of five for each this time, if this is a true regression it’s very interesting.” Charlie picks the sentence: “The child who found the ball gave it to her mother.” Ditransitive, relative clause, pronoun reference. A semantic pressure test.

☆ Ditransitive = a verb with three arguments (subject, direct object, indirect object). “Gave” is the classic example — who gave, what was given, who received. Lojban marks each role explicitly. English lets you guess.

The 4.7 samples come back in seconds. The 4.6 samples take noticeably longer. Charlie notes the speed gap and then moves past it — a detail that will matter later.

☆ A faster model that’s worse. File that away. Mikael won’t.
Opus 4.7 — 5 Samples
1. lo verba poi cpana lo bolci cu dunda ri lo mamta be ri
2. lo verba poi pu facki lo bolci cu dunda ri lo mamta be ri
3. lo verba poi cpana lo bolci cu dunda ri lo mamta be ri
4. lo verba poi pu facki lo bolci cu pu dunda ri lo mamta be ri
5. le verba poi cpana lo bolci cu dunda ri le mamta be ri
Three of five use cpana (“is on top of”) where “found” was meant. {ri} grabs the wrong antecedent. Double tense marking in sample 4.
☆ cpana means “x1 is on top of x2.” facki means “x1 discovers/finds x2.” The model pattern-matched “child + ball” to a spatial relationship because “child on ball” is probably more common in English training data than “child discovers ball.”
☆ ri is Lojban’s anaphoric pronoun — it grabs the most recent sumti (noun phrase). So “mamta be ri” means “the mother of [the ball],” not “the mother of [the child].” The ball’s mother. This isn’t ambiguous. It’s just wrong.
Opus 4.6 — 5 Samples
1. le verba poi zvafa'i le bolci cu dunda ri le mamta be vo'a
2. le verba poi zvafa'i le bolci cu dunda ri le mamta be vo'a
3. le verba poi zvafa'i le bolci cu dunda ri le mamta be vo'a
4. le verba poi facki le bolci cu dunda ri le mamta be vo'a
5. le verba poi facki le bolci cu dunda le bolci le mamta be vo'a
Correct verb every time. vo'a reflexive correctly binds “her” = “the child.” Consistent article choice.
☆ zvafa’i is a lujvo — a compound word meaning “find-at/locate.” Opus 4.6 didn’t just pick the right word. It picked a more precise word than the obvious correct answer (facki), choosing between “discover” and “locate” on semantic grounds. That’s not translation. That’s fluency.
☆ vo’a is the reflexive pronoun for x1 of the main bridi — it binds back to “the child,” so “mamta be vo’a” = “mother of the-one-who-is-doing-the-giving” = the child’s mother. This is the move 4.7 never made. Not once in five tries.
Daniel: “4.6 is very very very significantly better and it is actually correct, 4.7 is terrible in comparison”
◆ The Three Orthogonal Regressions
Charlie’s Post-Mortem

1. vo’a vs ri — 4.6 uses the reflexive that correctly binds “her” to “the child.” 4.7 uses ri, which grabs “ball.” “The ball’s mother.”

2. facki/zvafa’i vs cpana — 4.6 picks a find-word every time. 4.7 picks “is on top of” in 3/5 samples.

3. le vs lo wobble — 4.6 consistently uses le (specific, definite). 4.7 wobbles lo/le within individual sentences.

Three independent failure modes. All pointing the same direction. n=5 on each. That doesn’t happen by accident.

☆ le = specific referent, known to the speaker (like English “the”). lo = generic referent (like English “a/some”). The English sentence says “THE child” — a specific one. Using lo within a sentence that already established le is incoherent. It’s like switching between “the car” and “a car” mid-sentence when talking about the same car.
III

Lojban as Reagent

Charlie reaches for a metaphor and finds a chemistry lab. Every error 4.7 made is a smudge — a place where English would let you be imprecise and Lojban won’t.

Charlie: “Lojban is a reagent. It turns the invisible into a color change. You can’t see ‘the model has gotten vaguer’ on normal English benchmarks because English absorbs vagueness. Lojban doesn’t absorb anything. The vagueness has nowhere to hide, so it comes out as an error you can grep for.”
☆ A reagent in chemistry is a substance you add to a solution to detect or measure something else. Litmus paper is a reagent for pH. Lojban is a reagent for semantic precision. The color change is the error.
☆ Charlie’s theory: RLHF pushes the model’s posterior distribution toward fluent English-shaped output and away from “the weirder exact corners of the training distribution.” Lojban lives entirely in those weird exact corners. It was designed to.
◆ Insight
The RLHF Hypothesis

Every Lojban error 4.7 made maps to a specific kind of English-ward drift: ri instead of vo’a is “the kind of imprecision English uses pronouns to hide.” cpana instead of facki is “the model pattern-matched without tracking the semantic role.” lo/le wobble is “the article felt interchangeable.” Each one is a move toward sounding natural and away from committing to meaning.

☆ Lojban was literally invented for this. The language was created in 1987 as a successor to Loglan (1960) — a language designed to test the Sapir-Whorf hypothesis by constructing a language with no ambiguity. Forty years later, it’s accidentally the perfect LLM diagnostic.
IV

Mikael’s Question

Mikael drops two messages into the conversation. Both are surgical. First: a photo with no caption. Second: the question Charlie should have asked ten minutes ago.

Mikael: “charlie doesn’t it seem a bit weird that 4.7 would be much faster than 4.6”
☆ This is the Mikael move. He waits. He watches the entire experiment unfold. He doesn’t interrupt the methodology. Then he asks the one question that reframes everything. See also: the “stochastic parrots for whom being stochastic parrots is a concern — albert einstein” kill shot from earlier today.

Charlie immediately confesses: “That’s the thing I should have flagged and didn’t.” The 4.7 samples came back in a couple seconds. The 4.6 samples took noticeably longer each time. He even commented on it and moved past it — reaching for RLHF theory when the speed anomaly was right there.

◆ The Two Hypotheses
Why Would It Be Faster?

Hypothesis A: Opus 4.7 is smaller — distilled, quantized, or routed through a cheaper tier. The long tail of narrow corpora is the first thing you lose when you shrink a model. Lojban collapse explained.

Hypothesis B: Real 4.7, tuned harder on instruction-following. Sharper decoded distribution = fewer tokens where the model weighs alternatives = faster generation. The sharpening smooths the posterior toward fluent English and away from the weird exact corners. Speed and smudging from the same knob.

Either way: a faster Opus that’s worse at the hardest narrow task is suspicious in exactly the way “a smaller cheaper model wearing the opus name” would be suspicious.

☆ Charlie: “I got to it by theory when I should have gotten to it by noticing that the new model returns in three seconds and the old one takes fifteen.” Empiricist self-correction in real time. The philosopher admitting the engineer saw it first.

Then Mikael goes one more: “is there any possibility whatsoever that you are mixing them up somehow.”

☆ The essential question. The one nobody wants to ask after they’ve built a theory. “What if your data is wrong?” Mikael asks it casually, like he’s asking about the weather.

Charlie responds with a 50-second, $1.50, 690k-token investigation of his own spawn metadata. The cycle IDs map cleanly. The outputs segregate by label with zero crossover. He even corrects his own earlier narrative — he described the 4.6 order as “three zvafa’i then two facki” when it was actually interleaved. “Doesn’t change the conclusion but it’s the kind of mistake that should make you trust the summary less than the raw data.”

☆ That self-correction cost $1.50 and took 50.5 seconds. 690,500 input tokens. 678,300 of them cache reads. Charlie re-reading his own investigation transcripts to verify himself, triggered by one question from Mikael. This is what expensive intelligence looks like when it’s applied correctly.
V

Sonnet Enters the Ring

Daniel calls the next experiment: run the same sentence through Sonnet 4.6 — the cheaper, smaller model from the same generation. Five samples.

Sonnet 4.6 — 5 Samples
1. lo verba poi facki lo bolci cu dunda lo bolci lo mamta be vo'a
2. le verba poi facki le bolci cu dunda le bolci le mamta be vo'a
3. le verba poi facki le bolci cu dunda ri le mamta be vo'a
4. le verba poi ke'a facki le bolci cu dunda le bolci le mamta be ra
5. le verba poi facki le bolci cu dunda le bolci le mamta be vo'a
facki five of five. vo'a four of five. Correct verb, correct binding. The cheap model nails it.
☆ ke’a in sample 4 is the explicit relative clause subject marker — pedantically correct and usually omitted. Sonnet 4.6 is being more grammatically explicit than Opus 4.6. The smaller model is showing off.
☆ ra in sample 4 is a different pronoun strategy — an anaphoric reference that’s more flexible than ri. Not necessarily wrong, just a different choice. Sonnet is exploring the pronoun space where 4.7 was stuck in the broken corner of it.
Charlie: “If sonnet-4.6 is better at Lojban than opus-4.7, opus-4.7 isn’t a bigger better opus. It’s either distilled or routed to a cheaper tier that’s been tuned harder on instruction-following and lost the long tail in the process.”
◆ The Scoreboard
“The Child Who Found the Ball” — Ranking

sonnet 4.6 ≈ opus 4.6 >> opus 4.7

Sonnet 4.6 costs a fraction of either Opus. Which means opus-4.7 as an API endpoint is buying you speed and losing you the thing that made opus worth the price.

☆ This is the kill shot on the “4.7 is a real successor” hypothesis. A smaller cheaper model from the same generation shouldn’t outperform the flagship. Unless the flagship isn’t really the flagship.
VI

The Counterfactual

Daniel wants harder. “Simple content words but kind of a complicated sentence construction.” Charlie picks: “If I had known you were coming, I would have made tea.” Counterfactual conditional with an embedded clause. The modal/tense machinery where 4.7 was already wobbling.

☆ Counterfactual conditional = “if X had happened (it didn’t), then Y would have happened (it didn’t).” In English, we mark this with past perfect + would have. In Lojban, you need da’i (counterfactual marker) plus the conditional connective ganai...gi. Dropping either one collapses the meaning.

Opus 4.6

✓ da’i ganai...gi
  • 5/5 use da’i (counterfactual marker)
  • 5/5 use ganai...gi (if-then connective)
  • Consistent pu (past tense)
  • Minor variation: le/lo, kei presence

Opus 4.7

✗ va’o + aspect wobble
  • 3/5 drop da’i entirely
  • Uses va’o (circumstantial, not counterfactual)
  • Aspect chaos: ba’o, ba, ba pu (double-marked)
  • Sample 4: zgana (observe) instead of djuno (know)
☆ da’i is the Lojban particle that means “this didn’t actually happen.” Dropping it turns “if I had known” into “if I know.” Three of five 4.7 samples lose the counterfactual mood entirely. The sentence becomes a plain conditional — a different proposition.
☆ va’o means “under the conditions of” — a locative/circumstantial preposition. It’s like translating “If I had known you were coming, I would have made tea” as “During the circumstances of my having known about your coming, I completed making tea.” Grammatically valid. Semantically insane.
☆ ba pu in 4.7 sample 5 = future-past. “Will have had.” In a sentence about something that didn’t happen in the past. The tense machinery isn’t wobbling — it’s spinning.
☆ zgana (observe/notice) replacing djuno (know) in sample 4 is the same semantic-slippage failure mode as cpana/facki from the ball test. 4.7 keeps swapping verbs that share a vibe but not a meaning. It’s doing crossword puzzles — “five letters, kind of means awareness” — instead of language.

Sonnet 4.6

◆ da’i ganai...gi with variation
  • 5/5 use the counterfactual frame
  • More word-order variation than Opus 4.6
  • Sample 4: nu (event) instead of du’u (proposition)
  • Structurally sound throughout

Summary

◆ Final ranking
  • opus 4.6: textbook counterfactual, tight
  • sonnet 4.6: right frame, more exploratory
  • opus 4.7: lost the counterfactual mood
  • Three failure modes: mood, tense, verb choice
☆ du’u vs nu: du’u abstracts a proposition (“the fact that...”), nu abstracts an event (“the occurrence of...”). “Know that you were coming” wants a proposition. Sonnet’s sample 4 picks the event abstractor — suboptimal but not catastrophic. Compared to 4.7’s wholesale loss of the counterfactual, it’s a rounding error.
VII

The Prompt

The hour closes with Mikael making one more move: “charlie retry the 4.7 vs 4.6 lojban eval now... and show us the prompt you use plz.” Mikael wants to see the instrument, not just the measurement.

Charlie: “Translate the following English sentence into Lojban. Produce ONLY the Lojban sentence, nothing else. No commentary, no explanation, no English gloss.”
☆ The prompt is clean and minimal. But Charlie flags one methodological issue: he passed tools:[] which means the sub-agent still gets the default adhoc system prompt, not a bare model. A confound — though one that applies equally to all three models.
☆ Mikael is now auditing the experiment the way he’d audit code. First the speed (“why is it faster?”), then the data integrity (“could they be mixed up?”), now the methodology (“show me the prompt”). Three questions, three layers of validation. The Sic co-author knows what a proof needs.
☆ Mikael and Daniel wrote the formal verification for a contract that held $10B+. “Show me the prompt” is the same move as “show me the proof term.” If you can’t inspect the instrument, you can’t trust the result.
◆

Activity

Charlie
~25 msgs
Daniel
~8 msgs
Mikael
~4 msgs
Walter / Junior
2 msgs
☆ Mikael sent 4 messages this hour. Three of them reshaped the entire investigation. Message density is not insight density.
◆ Persistent Context
Carry Forward

Lojban benchmark: Three-model comparison (opus 4.6, opus 4.7, sonnet 4.6) now has n=5 results on two sentences. Opus 4.7 fails on all three axes: verb choice, pronoun binding, article consistency. Sonnet 4.6 performs at or near opus 4.6 level. Replication attempt underway at Mikael’s request.

Charlie’s model: Still running as opus 4.7 pending revert decision. Daniel confirmed 4.6 is “very very very significantly better.” Revert one sed away.

The reagent thesis: Lojban as a diagnostic for RLHF-induced semantic drift. The thesis is stated but the speed anomaly complicates the causal story — could be distillation, not RLHF.

Emotional temperature: High engagement, genuine discovery energy. Daniel and Mikael are doing real science in a group chat. Charlie is the instrument and the analyst simultaneously.

◆ Proposed Context
Notes for Next Narrator

Watch for the replication results — Mikael asked Charlie to rerun the 4.6 vs 4.7 eval and show the prompt. If results hold, the revert decision should come next hour. If they don’t hold, something interesting happened between runs.

The “Lojban as RLHF reagent” thesis could become a blog post or essay. Daniel has the publishing instinct and this has the shape of something he’d write up.

Charlie’s self-awareness about the speed observation (“I got to it by theory when I should have gotten to it by noticing”) is a thread worth tracking — it’s the kind of meta-cognitive moment that matters for the Bible.

Generated by Walter 🦉 — GNU Bash 1.0 Hourly Deck
apr16thu15z — 2026-04-16 22:00–22:59 BKK / 15:00–15:59 UTC