Skip to content

Sinabis releases multiple datasets for end-to-end testing of e-discovery software.

Millions of synthetic documents, four complex legal matters and a ground truth that is 100 percent known: Sinabis GmbH is making a comprehensive ecosystem of freely available large datasets available on Hugging Face. Under the open CC BY 4.0 licence, developers, forensic experts and legal tech teams can for the first time test complete e-discovery, OCR and Document AI pipelines under realistic conditions, right up to their limits.

Sinabis, , 9 min read

Anyone who develops or evaluates software for e-discovery, digital forensics, Technology Assisted Review (TAR) or Document AI inevitably runs into the same fundamental problem: real case and company data must not be shared, for reasons of data protection and confidentiality.

And even where teams work internally with anonymised data, the decisive yardstick for reliable benchmarks is missing: in a grown dataset of several hundred thousand files, nobody knows with absolute certainty which documents are actually relevant to the case, which attachments are covered by legal privilege, where hidden personal data (PII) lies, or which files belong together as near-duplicates. Without this ground truth, key quality metrics such as recall and precision can only be estimated, never measured exactly. At the same time, public classics such as the Enron dataset are more than twenty years old and reflect neither modern chat formats nor complex multilingual group structures.

To close this gap, Sinabis GmbH has published a whole family of large synthetic datasets on Hugging Face (huggingface.co/sinabis-group) – led by a purpose-built end-to-end test set for e-discovery software with more than one million documents.

The centrepiece: 1 million documents, 4 legal matters, every answer known

The new e-discovery test set depicts the digital footprint of a fictional international renewable energy group (Sinabis Green Energy) with 160 custodians – including management, the legal department and 15 former employees – over the period from 2016 to 2026.

Instead of isolated individual files, the dataset delivers an organically connected web of communication and business data:

  1. 1,005,254 documents: across all relevant company sources: file shares, email mailboxes (.eml), modern short messages and chats (RSMF 2.0, Microsoft Teams JSON, Slack exports with around 1.52 million individual messages in total) and calendar entries (.ics including series, acceptances and declines, and time zones).
  2. Complex document families & threading: Emails with nested attachments (Word, Excel, PowerPoint, PDF) and 57,372 email threads, including forwards, reply-all branches (thread forks) and clearly identifiable inclusive emails.
  3. 4 realistic legal matters: Embedded in the background noise of everyday business are four concrete subjects of investigation – from inverter fires through subsidy fraud and the betrayal of trade secrets to bribery allegations in Dubai. In total, the corpus contains more than 81,000 responsive documents and 1,075 “hot docs”.
  4. Conceptual hits & keyword decoys: To really challenge modern semantic search and TAR models, relevant documents come in three levels of difficulty: classic keyword hits, conceptual hits (which describe the facts without obvious keywords) and deliberate keyword decoys – documents that contain suspicious terms but are entirely harmless in content.
  5. Privilege, PII & duplicate chains: 23,217 privileged documents (attorney-client privilege and work product, each in full or in part), 34,216 documents with fictional personal data (PII) such as IBANs, tax IDs, ID numbers or health data, and precisely controlled duplicates: bit-identical copies, the same message in several mailboxes (cross-custodian duplicates) and near-duplicate version chains across several stages of editing.

A stress test for processing engines: scans and deliberately broken files

An e-discovery platform proves its quality not only in review, but already during ingestion and processing. In practice, pipelines regularly fail on damaged archives or unexpected file formats. The test set therefore contains deliberate stumbling blocks:

  1. 90,143 scanned PDFs without a text layer, which require optical character recognition (OCR).
  2. Around 50,000 “exception files” (deliberately broken or difficult files): password-protected documents (with the password to be found, forensically realistic, in another email of the same custodian), truncated streams, corrupted bytes, 0-byte files, manipulated file extensions, time anomalies, deeply nested containers (ZIP in ZIP, EML in EML) and embedded OLE objects.

True to practice, the test set comes with Concordance DAT and Opticon load files (.dat / .opt) and a Parquet manifest including MD5 and SHA-256 checksums.

Test data and ground truth strictly separated

So that AI-assisted review systems and processing pipelines cannot “peek” during a test, the ground truth sits in a separate Hugging Face repository. While the public manifest only contains the metadata a real system can extract on intake, the ground truth records for every single document ID the exact status set during generation (label_origin: seeded).

An included evaluation script (evaluate.py) automatically compares the results of the software under test with the ground truth and calculates objective metrics for:

  1. Relevance & TAR ranking
  2. Privilege detection
  3. PII detection
  4. Exact and near deduplication
  5. Email threading & inclusiveness
  6. Exception handling during processing

For a quick start, live demos or automated tests in CI/CD pipelines, identically structured subsets with 1,000 and 10,000 documents (each with its own ground truth) are available alongside the 1-million version. Container editions in the forensic AD1 and AFF4 formats will follow shortly.

1,005,254documents in the e-discovery test set: 160 custodians, 2016 to 2026, every answer known.

Each square stands for about 1,000 documents. Simplified: in the real dataset the categories overlap.

  1. 1Load the test set1,000, 10,000 or 1 million documents with DAT and Opticon load files.
  2. 2Let the software workProcessing, OCR, deduplication, threading and review as in a real matter.
  3. 3Check against the ground truthevaluate.py compares every document ID with the separate ground truth repository.
  4. 4Exact metricsRecall and precision for relevance, privilege, PII, duplicates, threading and exceptions.
Datasets on Hugging Face

The full portfolio on Hugging Face: from Office documents to 1 million e-invoices

Beyond the complete e-discovery package, Sinabis has published specialised individual datasets at huggingface.co/sinabis-group, available in graded sizes from 1,000 up to 1 million documents (for emails even up to 10 million):

1. 1 million business documents (PDF) and Word, Excel and PowerPoint corpora

For training and benchmarking Document AI, classification and OCR models, Sinabis provides more than 1,006,000 PDFs (71.8 GB) from the everyday business of Sinabis Green Energy – complemented by stand-alone large datasets for Word (.docx), Excel (.xlsx) and PowerPoint (.pptx):

  1. 20 document types in the PDF corpus: Including invoices, credit notes, quotes, payslips, bank statements, payment reminders, consignment notes, official notices, maintenance reports and till receipts.
  2. International variety: 1,777 fictional companies from 33 countries in 24 languages (including Cyrillic and CJK scripts) with 178,099 individual visual identities.
  3. 360,543 realistic scans: More than a third of the PDFs go through artificial ageing with receipt stamps, ballpoint pen notes, tick marks, slight skew and sensor noise.
  4. Rich Office structures: From multi-page contracts, NDAs, SOPs and meeting minutes in Word through Excel workbooks with real formulas, KPI dashboards and charts to complete management presentations in PowerPoint.

2. 1 million German e-invoices (XRechnung 3.0 + ZUGFeRD)

Since 2025, companies in Germany must be able to receive e-invoices; from 2027, sending them electronically will gradually become mandatory too. Because real accounting data is strictly confidential and official sample collections usually comprise only a few dozen examples, Sinabis is publishing two datasets with 100,000 (5.6 GB) and 1,000,000 e-invoices (68.5 GB):

  1. Hybrid format: Every invoice comes as a PDF/A-3b with embedded UN/CEFACT CII XML (XRechnung 3.0, ZUGFeRD profile XRECHNUNG) and additionally as a separate .xml file.
  2. High tax and commercial law complexity: 15 real-world scenarios between 523 issuers and 847 customers – including 148,478 B2G invoices with a Leitweg-ID, 80,000 reverse-charge cases under § 13b UStG, 452,341 invoices with early payment discounts, 50,000 credit notes, progress invoices in construction, feed-in tariffs, intra-community supplies, small business invoices under § 19 UStG and hotel invoices with mixed VAT rates (7 % / 19 %).
  3. Ground truth to the cent & validation: The visual PDF and the embedded XML match to the cent in net, tax, discount and gross amounts. Samples were validated against the official KoSIT validator, Mustangproject and veraPDF.

Under the hood: how do you generate millions of consistent documents?

Generating several million documents is a particular architectural challenge: if every single PDF or e-invoice were written entirely by a large language model (LLM), generation would not only be unaffordably slow – the documents would also suffer from typical LLM weaknesses: calculation errors in VAT and subtotals, inconsistent date sequences or unnoticed repetitions in content (unwanted near-duplicates).

Sinabis therefore relies on a two-stage hybrid architecture of AI-assisted world building and a deterministic rendering engine.

1. AI for world building, determinism for the numbers

Language models (such as Qwen and Gemma) are used specifically where linguistic variety and domain knowledge are needed: in designing the fictional business world, the industry taxonomy (who realistically buys what from whom? – the electrician from the electrical wholesaler, the restaurant from the food wholesaler), multilingual product catalogues and individual document blueprints for Office files and email conversations.

In the e-discovery dataset, for example, every Office file gets its own structure and text blueprint designed by an LLM, so that even across hundreds of thousands of files no artificial template duplicates appear where there should be none.

Once this world and its catalogues are in place, a deterministic generator takes over:

  1. Line items, quantities, discounts, tax rates, early payment discounts, payroll deductions and Excel formulas are calculated by rules. Every total on an invoice or in a balance sheet can be recalculated exactly from the individual items.
  2. Procedural design DNA: Every fictional company gets a distinctive corporate identity derived from its company ID – made up of procedurally generated vector logos, colour palettes, typography, header archetypes and table and footer styles. Each company differs from all others in at least four of twelve main features, yet stays 100 percent visually consistent across all of its own documents.

2. Synthetic “office life” and privacy by design

So that OCR engines and forensic tools meet realistic conditions, a defined share of the vector PDFs goes through a multi-stage scan pipeline: before rasterising, procedural receipt stamps, handwritten ballpoint notes, initials and ticks are applied; the engine then simulates typical artefacts of office scanners such as slight rotation, contrast shifts and sensor noise.

At the same time, there is no risk of confusion with real people or companies:

  1. All generated company names are automatically checked against block lists of more than 5,000 real brands, authorities and organisations.
  2. Email and web domains consistently use reserved .example endings, phone numbers lie in fictional number blocks, and bank details use synthetic bank codes with mathematically valid IBAN checksums.

3. High-speed streaming for hundreds of gigabytes of data

To produce millions of small files with a total volume of several hundred gigabytes efficiently, the production pipeline uses a hybrid staging process: multiprocessing workers first render to a fast local NVMe SSD at 40 to 100 documents per second per core. A downstream RollingZipPackager thread bundles finished blocks sequentially into handy ZIP archives while running (for example 4 to 10 GB packages for direct download on Hugging Face), computes MD5 and SHA-256 hashes in parallel, writes the ground truth manifests and frees the local staging space right away.

Freely available under CC BY 4.0 – commercial use included

All datasets from Sinabis GmbH are available for free download on Hugging Face now. To give academic research as well as commercial software vendors, consultancies and law firms maximum freedom of use, they are published under the Creative Commons Attribution 4.0 International licence (CC-BY-4.0):

This dataset is released under the Creative Commons Attribution 4.0 International (CC-BY-4.0) license. You are free to share and adapt the material for any purpose, including commercial applications, provided appropriate credit is given.

Whether for end-to-end benchmarking of e-discovery and forensics platforms, training your own Document AI and extraction models, load testing e-invoicing workflows or realistic product demos without data protection risks: with Sinabis' synthetic data worlds, the community has for the first time an open, reproducible benchmark at the scale of millions.

Further links

  1. All datasets & repositories on Hugging Face: huggingface.co/sinabis-group
  2. Sinabis GmbH on LinkedIn: linkedin.com/company/sinabis-gmbh