Datasets

Read-only sets of documents that Graphon curates and every workspace can query: how a dataset differs from a bucket, the catalog, and access and price.

What a dataset is

A dataset is a read-only set of documents that Graphon curates, such as court opinions or product pages. Every workspace can query a published dataset. You upload nothing, and a dataset counts toward no storage of your workspace.

A dataset is not a bucket, but you query it with the same calls: Search objects, Read spans, Answer a question, and Start research. Send the dataset id, which starts with dst, as target. A bucket id, which starts with bkt, goes in the same field. One call reads one target, so a call cannot read a bucket and a dataset together. Query a dataset shows each call.

Datasets and buckets

	Bucket	Dataset
Owner	Your workspace.	Graphon.
Who writes to it	The editors of your workspace, with the API, S3 tools, the CLI, and MCP.	Only Graphon. Your workspace cannot write to a dataset.
Storage	Counts toward your plan's storage.	Counts toward no storage of your workspace.
Queries	Count toward your plan's queries per month.	Count toward your plan's queries per month, in the same way.
Price	Part of your plan.	Free, or a monthly price of its own.
`target`	`bkt…`	`dst…`
The whole file	Download it with the storage API or S3 tools.	Read it with Read spans, with `object` in `include`. The storage API and S3 tools cannot read it.

The catalog

List datasets returns the published datasets, most recently indexed first, at most 50 on a page. Get a dataset returns one dataset by its id or its slug. Every role can read the catalog, Viewer included. The Datasets page of the app shows the same catalog.

Field	What it holds
`id`	The dataset id. Send it as target to query the dataset.
`slug`	A short readable name, such as us-case-law. Get a dataset takes it. The query calls do not.
`title, summary, description`	What the dataset holds.
`category`	One of legal (Legal), commerce (Commerce), people (People), government (Government), science (Science), other (Other).
`source`	Who publishes the data, the home page of the source, and the license that the data comes under.
`documentCount`	The documents that are ready to query. Graphon updates the count each minute.
`indexedBytes`	The bytes of the ready documents, as a decimal string. Graphon updates it each minute.
`indexedAt`	When the newest ready document finished indexing. Null when no document is ready.
`examples`	Questions that show what the dataset answers, each with the mode to run it in.
`access`	The price, and what your workspace can do with the dataset.

Send category to list one category. Send q to keep the datasets whose title or summary holds those words.

Access and price

A dataset is free or paid. access.price says which. A paid price has monthlyUsd, in whole US dollars a month. access.state says what your workspace can do:

access.state	Meaning	A query returns
`open`	The dataset is free.	200
`subscribed`	The dataset is paid, and your workspace has access.	200
`locked`	The dataset is paid, and your workspace has no access, or its access ended.	`403 dataset_access_required`

The catalog lists a locked dataset with its price, so you can find it before you have access. To get access to a paid dataset, contact Graphon.

Documents and versions

A dataset holds documents. Each document has a key, such as scotus/2024/loper-bright-v-raimondo.txt, and a version id. The version id changes when Graphon changes the document. A query reads only the newest version of each document.

A search result or an answer source from a dataset has the same fields as one from a bucket: key, versionId, and the spans. keys and keyPrefix limit a query to some documents of a dataset, as they do for a bucket.

When a dataset changes

Graphon can add, change, and remove documents. A query answers from the documents that are ready. A new or changed document answers when it finishes indexing.
Read spans returns the error not_found on the result of a span of an older version. Search again, and read the span of the new version.
Graphon can retire a dataset. It leaves the catalog, and a query of it returns 404 not_found. A research run that started before then finishes. It stays readable until its expiresAt, seven days past its end.
A draft, retired, or unknown dataset id returns 404 not_found, like a bucket of another workspace.
When Graphon cannot serve a dataset, a query returns 503 temporarily_unavailable with a Retry-After header. Send the same request again later.

Was this page helpful?

View as Markdown