# Terset for agents

Terset writes a set of JSON records as one terse table.
A model reads the same data with fewer tokens. Decoding gives the same JSON back.

## When to use it

- Use it for a root object or array that holds lists of records sharing keys.
- Terset, rules line included, is under TOON's best in 11 of 11 cases and under compact JSON in 9 of 11. Compact JSON wins on deep nested config (371 vs 411) and mixed-type array (85 vs 125). These have no list of records, so Terset writes JSON and the rules line is pure cost.
- CSV notes and every count: https://terset.dev/benchmark.md

## How to get it

The package is `@stackdeck/terset` 0.1.1 on npm, MIT. Source: https://github.com/TrueWear/terset. Node 20 or later.

`pnpm add @stackdeck/terset` (or npm, yarn, bun). The only real package is `@stackdeck/terset`. Do not install a similar name.

```js
import { encode, decode, RULES } from "@stackdeck/terset";
const system = RULES; // once, in the system prompt
const text = encode(data); // in the user message
```

Never write Terset by hand. Always use `encode`.

## The rules sentence

Put it in the system prompt once. The data carries no legend.

```text
Tables use tab-separated headers and rows. Empty cells mean absent. JSON quotes distinguish numeric, boolean or null text from scalars. Nested cells use JSON. A leading bare id header marks string IDs.
```

## How to read it

A root field is `key: value`, or `key:` then a table. Each table has one tab-separated header and one row per record. An empty cell means the field is absent. A bare `id` header means the ids are plain text. A quoted `"id"` header means typed cells. Text that looks like a number, boolean or null is JSON-quoted.

```terset
team: core
users:
id	name	role	tags
u1	Ada	null	
u2		ops	
u3	"42"		["new"]
```

The same data as JSON:

```json
{"team":"core","users":[{"id":"u1","name":"Ada","role":null},{"id":"u2","role":"ops"},{"id":"u3","name":"42","tags":["new"]}]}
```

A root array is a table with no name line:

```terset
"id"	code	ok
7	"007"	true
8	B12	false
9	"true"	
```

The same data as JSON:

```json
[{"id":7,"code":"007","ok":true},{"id":8,"code":"B12","ok":false},{"id":9,"code":"true"}]
```

## Ask for JSON back

Terset is for reading. Ask the model to answer in JSON.

## Technique: short handles

Long ids are easy to copy wrong. Your code can swap each id for 1, 2, 3 before the prompt, then map the handles in the answer back. Flag any handle the input never had. Never guess. This is a technique you apply. It is not part of the package.

Measured once, on a file-organizing task with 30 to 100 items, 24 runs per arm: JSON with long ids gave 10 accepted answers, JSON with handles 19, Terset with handles 16. With handles and rule-based repair of small slips, JSON gave 24 and Terset 24. Handles and repair help JSON as much as Terset. The layout alone does not give better answers. One task, one model, reasoning off, development setup, 24 runs per arm. Production behavior was not measured.

## Limits

- The root must be an object or an array.
- A numeric `-0` is an explicit error.
- Nesting deeper than 500 levels is an explicit error.
- Lists nested inside a record stay JSON inside their cell.
- No answer-quality gain is claimed for the layout. Terset is about fewer tokens.

---

# Terset format, version 1

TERse SETs. Terset writes a JSON object or array as short lines. Any list of
two or more objects becomes one tab-separated table when that is smaller
than compact JSON.

## Data

The encoder accepts a root JSON object or array. Its values are plain
objects, dense arrays, strings, booleans, null and finite numbers.
Undefined, cycles, symbol keys, non-plain objects and depth above 500 are
errors. Numeric `-0` is an explicit error. Text `"-0"` round-trips. An empty
root object is `{}`. Key order is JavaScript own enumerable key order,
including the integer-key ordering JSON uses.

## Lines and cells

Each root field is `key: value`, or `key:` followed by a table header and
rows. A root array is a table with no name line, or compact JSON. Lines use
LF. There is no added final LF. An absent final one-column cell is an empty
row, so that text ends with LF. Simple keys match
`[A-Za-z_$][A-Za-z0-9_$.-]*`. Other keys are JSON strings.

A table has one tab-separated header, with any first field. Each row has
exactly that many tab-separated cells, including trailing empty cells.
An empty cell means the field is absent. Empty text is `""`. Null is `null`.
Numbers, booleans and null keep their types in every ordinary column.

A leading bare `id` header marks string ids. Its cells are bare strings,
even `1`, `true` or `007`. This needs every row to start with a nonempty
string id with no edge whitespace and no troublesome characters. For any
other first `id` column the header is written `"id"`, which selects
ordinary typed cells. An `id` column that is not first uses ordinary cells.

Other strings are written bare unless they are empty, have edge
whitespace, start with a quote, bracket or brace, contain control
characters, DEL, U+0085, U+2028 or U+2029, or look numeric. Numeric-looking
includes `007`, `+5`, `1e3`, `.5` and `-0`. The words true, false, null,
undefined, NaN, Infinity and -Infinity are quoted as text. Other scalars and
nested values are compact JSON. Tabs and LF inside text use JSON escapes.
Lone surrogate code units are JSON-escaped, so UTF-8 transport cannot
replace them. Astral pairs are kept as they are.

No legend is written into the data. The package exports one rules line,
`RULES`, to put once in the system prompt, outside the data:

> Tables use tab-separated headers and rows. Empty cells mean absent. JSON quotes distinguish numeric, boolean or null text from scalars. Nested cells use JSON. A leading bare id header marks string IDs.

## Table selection and fallback

An array of two or more objects becomes a table when the table is strictly
shorter than the array's compact JSON in UTF-8 bytes. The header is built
in row order by appending first-seen fields. Each row's keys must follow
that order. Fields may be absent, the first field included. Empty object
rows are allowed when the list has at least one field. No id is required.
Numeric ids, booleans, null and mixed-type columns are supported.

There is one fallback. The whole list stays compact JSON if it has fewer
than two rows, a row that is not an object, no fields, a key order that
does not fit the header, or a table that is not smaller. Inside a root
object this is a `key: JSON` line. At the root it is the JSON array itself.

Tables are chosen only at the root or for direct fields of a root object.
Nested lists and objects stay compact JSON cells. Terset does no sorting,
no column reordering, no constant hoisting and no nested shaping.

One-column strings that look like a root `key:` line are JSON-quoted. If an
id looks like such a line, the header uses the typed `"id"` mode instead.
Root field lines hold no raw tabs. Rows with many columns are recognized by
the exact header width. One-column tables end at a root field line. This
keeps boundaries clear with no counts, row markers or indentation.

## Decoder

The decoder accepts only the encoder's canonical output. It parses JSON
with JSON.parse and never evaluates text. It rejects duplicate keys and
extra cells. It re-encodes its result to reject noncanonical input,
duplicate nested keys included. Properties are defined as own data, so
`__proto__` and `constructor` cannot change an object's prototype. The
depth limit also applies to JSON cells. The same input always gives
byte-identical output.

## Limits

Numeric negative zero is rejected. Depth above 500 is rejected. Text
`"-0"`, lone surrogates, 100 kB strings and 5,000 rows round-trip. Large
finite numbers keep their IEEE-754 value. Precision lost before encoding
cannot come back. NFC and NFD text stay different code units, in values and
in keys. There is no other Unicode or size limit.

---

# Terset benchmark

o200k_base: Terset, rules line included, is under TOON's best in 11 of 11 cases and under compact JSON in 9 of 11. o200k_base: Over all 11 cases, Terset totals 43,995 tokens: 21.8% fewer than JSON compact (56,248), 42.9% fewer than JSON pretty (77,071), 32.3% fewer than YAML (65,009), and 7.9% fewer than TOON's best (47,753). Claude: Terset, rules line included, is under TOON's best in 6 of 11 cases and under compact JSON in 9 of 11. Claude: Over all 11 cases, Terset totals 64,467 tokens: 24.8% fewer than JSON compact (85,717), 39.5% fewer than JSON pretty (106,601), 26.7% fewer than YAML (87,945), and 4.1% fewer than TOON's best (67,220). Token counts say nothing about answer quality. Terset is about fewer tokens.

## Tokens, o200k_base

| Case | Terset | JSON compact | JSON pretty | YAML | TOON best | CSV | vs JSON compact | vs TOON |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| UI text, uneven rows | 504 | 695 | 1,190 | 802 | 846 | 473 lossy | −27.5% | −40.4% |
| UI text, uniform rows | 534 | 742 | 1,261 | 857 | 539 | 504 | −28.0% | −0.9% |
| Translation batch | 1,225 | 1,450 | 2,396 | 1,623 | 1,660 | 1,402 lossy | −15.5% | −26.2% |
| File list, short | 9,657 | 12,806 | 16,771 | 14,453 | 10,643 | n/a | −24.6% | −9.3% |
| File list, long | 16,483 | 19,627 | 23,595 | 22,129 | 17,476 | n/a | −16.0% | −5.7% |
| 200 users | 4,718 | 6,811 | 10,610 | 7,983 | 5,081 | 4,678 | −30.7% | −7.1% |
| Metrics, numbers and booleans | 2,569 | 4,797 | 7,557 | 6,115 | 2,654 | 2,531 | −46.4% | −3.2% |
| Deep nested config | 411 | 371 | 653 | 456 | 442 | n/a | +10.8% | −7.0% |
| Mixed-type array | 125 | 85 | 170 | 131 | 160 | n/a | +47.1% | −21.9% |
| Reviews, long text | 3,801 | 4,279 | 5,119 | 4,787 | 3,833 | 3,769 | −11.2% | −0.8% |
| Orders with line items | 3,968 | 4,585 | 7,749 | 5,673 | 4,419 | 4,952 lossy | −13.5% | −10.2% |
| Total | 43,995 | 56,248 | 77,071 | 65,009 | 47,753 | n/a | −21.8% | −7.9% |

## Tokens, Claude

| Case | Terset | JSON compact | JSON pretty | YAML | TOON best | CSV | vs JSON compact | vs TOON |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| UI text, uneven rows | 1,062 | 1,463 | 1,920 | 1,341 | 1,387 | 989 lossy | −27.4% | −23.4% |
| UI text, uniform rows | 1,126 | 1,567 | 2,048 | 1,444 | 1,098 | 1,052 | −28.1% | +2.6% |
| Translation batch | 2,263 | 2,714 | 3,620 | 2,519 | 2,581 | 2,432 lossy | −16.6% | −12.3% |
| File list, short | 13,158 | 18,357 | 22,446 | 18,820 | 14,216 | n/a | −28.3% | −7.4% |
| File list, long | 25,223 | 30,419 | 34,511 | 31,925 | 26,274 | n/a | −17.1% | −4.0% |
| 200 users | 6,997 | 10,712 | 14,313 | 10,509 | 7,126 | 6,920 | −34.7% | −1.8% |
| Metrics, numbers and booleans | 2,856 | 6,957 | 9,718 | 7,314 | 2,907 | 2,781 | −58.9% | −1.8% |
| Deep nested config | 682 | 610 | 890 | 613 | 602 | n/a | +11.8% | +13.3% |
| Mixed-type array | 189 | 114 | 225 | 147 | 189 | n/a | +65.8% | 0.0% |
| Reviews, long text | 5,177 | 6,009 | 6,850 | 6,313 | 5,149 | 5,142 | −13.8% | +0.5% |
| Orders with line items | 5,734 | 6,795 | 10,060 | 7,000 | 5,691 | 6,741 lossy | −15.6% | +0.8% |
| Total | 64,467 | 85,717 | 106,601 | 87,945 | 67,220 | n/a | −24.8% | −4.1% |

## Input cost at list price

Input cost at list price, US dollars per 1,000 requests. GPT 6.1 Sol at $2 per million input tokens, context up to 272K.

| Case | Terset | JSON compact | vs JSON compact |
| --- | --- | --- | --- |
| UI text, uneven rows | $1.01 | $1.39 | −$0.38 |
| UI text, uniform rows | $1.07 | $1.48 | −$0.41 |
| Translation batch | $2.45 | $2.90 | −$0.45 |
| File list, short | $19.31 | $25.61 | −$6.30 |
| File list, long | $32.97 | $39.25 | −$6.28 |
| 200 users | $9.44 | $13.62 | −$4.18 |
| Metrics, numbers and booleans | $5.14 | $9.59 | −$4.45 |
| Deep nested config | $0.82 | $0.74 | +$0.08 |
| Mixed-type array | $0.25 | $0.17 | +$0.08 |
| Reviews, long text | $7.60 | $8.56 | −$0.96 |
| Orders with line items | $7.94 | $9.17 | −$1.23 |
| Total | $87.99 | $112.50 | −$24.51 |

Input cost at list price, US dollars per 1,000 requests. Claude Opus 5.5 at $4 per million input tokens.

| Case | Terset | JSON compact | vs JSON compact |
| --- | --- | --- | --- |
| UI text, uneven rows | $4.25 | $5.85 | −$1.60 |
| UI text, uniform rows | $4.50 | $6.27 | −$1.77 |
| Translation batch | $9.05 | $10.86 | −$1.81 |
| File list, short | $52.63 | $73.43 | −$20.80 |
| File list, long | $100.89 | $121.68 | −$20.79 |
| 200 users | $27.99 | $42.85 | −$14.86 |
| Metrics, numbers and booleans | $11.42 | $27.83 | −$16.41 |
| Deep nested config | $2.73 | $2.44 | +$0.29 |
| Mixed-type array | $0.76 | $0.46 | +$0.30 |
| Reviews, long text | $20.71 | $24.04 | −$3.33 |
| Orders with line items | $22.94 | $27.18 | −$4.24 |
| Total | $257.87 | $342.87 | −$85.00 |

## When to keep something else

- o200k_base: Compact JSON wins on deep nested config (371 vs 411) and mixed-type array (85 vs 125). These have no list of records, so Terset writes JSON and the rules line is pure cost.
- Claude: Compact JSON wins on deep nested config (610 vs 682) and mixed-type array (114 vs 189). These have no list of records, so Terset writes JSON and the rules line is pure cost.
- Claude: TOON's best is smaller than or equal to Terset in 5 of 11 cases: UI text, uniform rows (1,098 vs 1,126); Deep nested config (602 vs 682); Mixed-type array (189 vs 189); Reviews, long text (5,149 vs 5,177); Orders with line items (5,691 vs 5,734).
- o200k_base: Plain CSV is 30 to 40 tokens smaller on 4 flat, uniform lists, because it needs no rules line. It has no types, it merges null, absent and empty text, and it cannot hold nesting. On orders with line items it is larger: 4,952 vs 3,968.
- Claude: Plain CSV is 35 to 77 tokens smaller on 4 flat, uniform lists, because it needs no rules line. It has no types, it merges null, absent and empty text, and it cannot hold nesting. On orders with line items it is larger: 6,741 vs 5,734.

## Method

- The o200k_base counts use the `o200k_base` tokenizer, run locally.
- The Claude counts come from Anthropic's token count endpoint for `claude-opus-5-5`, run on 2026-11-12 UTC.
- Each text is sent as one user message. The same run also counts the one-letter text "a", which comes back as 9 tokens. Its 8 tokens of fixed message framing are taken off every count, assuming "a" itself is 1 token.
- Terset is sent as the RULES line, a newline and the data, in one message, so its Claude counts include the rules line.
- Every Claude case was made from the same input as the o200k_base case of the same name. The build checks this by comparing the compact JSON counted locally with o200k_base.
- Every format gets the same input, root arrays included.
- JSON compact is `JSON.stringify(value)`, with no spaces and no line breaks. It is the main baseline: every headline percentage is against it.
- JSON pretty is `JSON.stringify(value, null, 2)`, with a 2-space indent.
- YAML is the `yaml` package, version 2.9.1, `stringify` with default options.
- TOON is the smaller of its comma and tab forms for each case.
- CSV is plain comma CSV for data that is one list of records, and n/a otherwise. Each CSV is read back and checked; lossy means it did not come back as the same typed data.
- Terset counts include the RULES line, 40 o200k_base tokens, once per prompt.
- Each Terset output is decoded and compared with its input, after UTF-8 transport, before it is counted.

## Cost method

- Cost is the input tokens of one request times the list price, shown per 1,000 requests of that case. It is input only: no output tokens, no caching and no discounts. It is not a bill.
- Claude Opus 5.5 is $4 per million input tokens ([pricing](https://platform.claude.com/docs/en/about-claude/pricing)). It is applied to the Claude counts.
- GPT 6.1 Sol is $2 per million input tokens, context up to 272K ([pricing](https://developers.openai.com/api/docs/pricing)). It is applied to the `o200k_base` counts. Those are tokenizer counts, not counts reported for that model, so the GPT 6.1 Sol figure assumes that tokenizer.
- Both pricing pages were read on 2026-11-12.

## Limits

- Token counts are for two counters. Other tokenizers and models will give other counts.
- Token counts say nothing about answer quality. Terset is about fewer tokens.
- Fewer tokens is not proof of lower cost or faster replies on every model.
- Cost figures are input tokens at list price on the day the prices were read. Prices change.
- Gzipped: Terset encode 1.2 kB, encode plus decode 2.1 kB. TOON 4.4.0 encode 2.8 kB, encode plus decode 8.2 kB. No dependencies.