bluzen/aurora-full

aurora-full

The complete Aurora corpus: 18.4 million deduplicated illustrations with tag, caption, aesthetic and quality-band annotations, plus source attribution for every row.

The rule this dataset is built on

No filter deletes anything. Every pass writes a column instead. The curated slice is therefore a query rather than a separate artefact:

SELECT * FROM aurora
WHERE aesthetic > 0.62
  AND dedup_cluster_rank = 1
  AND resolution_min >= 768

If you disagree with our thresholds, write a different query. Nobody should have to re-crawl the internet to argue with us.

Loading it

from datasets import load_dataset

ds = load_dataset("bluzen/aurora-full", split="train", streaming=True)

Streaming is strongly recommended. The full set does not fit on your laptop and will not fit on the next one either.

Known biases

The source boorus over-represent recent work, popular characters and a narrow band of styles. The source, year and style_cluster columns exist so you can measure that rather than take our word for it.