Zig implementation of TOON (Token-Oriented Object Notation)
A spec-compliant Zig library for encoding and decoding TOON format — a compact, human-readable, line-oriented format for structured data, optimized for LLM prompts with 30-60% token reduction vs JSON.
Features • Installation • Usage • API Reference • Contributing
- Full TOON v3.0 Specification Compliance — Implements all normative requirements
- Encoder: JSON → TOON with configurable delimiters and indentation
- Decoder: TOON → JSON with strict mode validation
- Zero Dependencies — Pure Zig, no external dependencies
- Memory Safe — Explicit allocator-based memory management
- v1.5 Features — Key folding and path expansion support
JSON (40 bytes):
{"users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]}TOON (28 bytes) — 30% smaller:
users[2]{id,name}:
1,Alice
2,Bob
Add to your build.zig.zon:
.dependencies = .{
.toon = .{
.url = "https://github.com/copyleftdev/toon-zig/archive/refs/tags/v0.1.0.tar.gz",
.hash = "...",
},
},Then in build.zig:
const toon = b.dependency("toon", .{
.target = target,
.optimize = optimize,
});
exe.root_module.addImport("toon", toon.module("toon"));const std = @import("std");
const toon = @import("toon");
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();
const allocator = gpa.allocator();
// Create a value
var obj = toon.JsonValue.initObject(allocator);
defer obj.deinit(allocator);
const name_key = try allocator.dupe(u8, "name");
try obj.asObject().?.put(name_key, toon.JsonValue.initString("Alice"));
const age_key = try allocator.dupe(u8, "age");
try obj.asObject().?.put(age_key, toon.JsonValue.initInteger(30));
// Encode to TOON
const encoded = try toon.encode(allocator, obj, .{});
defer allocator.free(encoded);
std.debug.print("{s}\n", .{encoded});
// Output:
// name: Alice
// age: 30
}const std = @import("std");
const toon = @import("toon");
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
defer _ = gpa.deinit();
const allocator = gpa.allocator();
const input =
\\users[2]{id,name}:
\\ 1,Alice
\\ 2,Bob
;
var decoded = try toon.decode(allocator, input, .{});
defer decoded.deinit(allocator);
// Access the data
const users = decoded.asConstObject().?.get("users").?;
const first_user = users.asConstArray().?.items[0];
const name = first_user.asConstObject().?.get("name").?.asString().?;
std.debug.print("First user: {s}\n", .{name}); // Alice
}const options = toon.EncodeOptions{
.indent = 4, // Spaces per level (default: 2)
.delimiter = .tab, // .comma (default), .tab, or .pipe
.key_folding = .safe, // .off (default) or .safe
.flatten_depth = 3, // Max folding depth (default: max)
};
const encoded = try toon.encode(allocator, value, options);const options = toon.DecodeOptions{
.indent = 2, // Expected indent size (default: 2)
.strict = true, // Strict validation (default: true)
.expand_paths = .safe, // .off (default) or .safe
};
var decoded = try toon.decode(allocator, input, options);TOON is designed for efficient structured data in LLM prompts:
name: Alice
age: 30
active: true
user:
id: 123
profile:
name: Ada
tags[3]: admin,ops,dev
users[2]{id,name,role}:
1,Alice,admin
2,Bob,user
matrix[2]:
- [3]: 1,2,3
- [3]: 4,5,6
items[3]:
- 42
- name: widget
- hello
# Tab-delimited
data[2 ]{id name}:
1 Alice
2 Bob
# Pipe-delimited
tags[3|]: a|b|c
JsonValue— Union type representing JSON values (null, bool, integer, float, string, array, object)JsonArray—ArrayList(JsonValue)JsonObject—StringArrayHashMap(JsonValue)with preserved insertion orderDelimiter—.comma,.tab,.pipeToonError— Error type for encoding/decoding failures
encode(allocator, value, options)— Encode JsonValue to TOON stringdecode(allocator, input, options)— Decode TOON string to JsonValueencodeDefault(allocator, value)— Encode with default optionsdecodeDefault(allocator, input)— Decode with default options
initNull(),initBool(b),initInteger(i),initFloat(f),initString(s)— Create primitivesinitArray(allocator),initObject(allocator)— Create containersclone(allocator)— Deep copydeinit(allocator)— Free memoryeql(other)— Deep equality comparisonparseJson(allocator, json_str)— Parse from JSON stringtoJsonString(allocator)— Serialize to JSON string
# Build library
zig build
# Run unit tests
zig build test
# Run fixture tests (requires test fixtures)
zig build test-fixtures
# Run all tests
zig build test-allThis implementation targets TOON Specification v3.0 and implements:
- ✅ Canonical number formatting (no exponent, no trailing zeros)
- ✅ String escaping (only
\\,\",\n,\r,\t) - ✅ Quoting rules per §7.2
- ✅ Key encoding per §7.3
- ✅ Object encoding with preserved key order
- ✅ Primitive arrays (inline)
- ✅ Tabular arrays with field lists
- ✅ Arrays of arrays (expanded)
- ✅ Mixed arrays (expanded)
- ✅ Objects as list items per §10
- ✅ Delimiter handling (comma, tab, pipe)
- ✅ Strict mode validation
- ✅ Key folding (encoder)
- ✅ Path expansion (decoder)
Contributions are welcome! Please ensure:
- All tests pass (
zig build test-all) - Code follows Zig style conventions
- New features include tests
- Spec compliance is maintained
MIT License — see LICENSE for details.