TABLET
A large-scale Visual Table Understanding dataset with
4 million examples
across 21 tasks,
grounded in 2 million unique tables where 88% preserve
original visualizations.
Paper accepted at ICLR 2026.
About the dataset
While table understanding increasingly relies on pixel-only settings where tables are processed as visual representations, current benchmarks predominantly use synthetic renderings that lack the complexity and visual diversity of real-world tables. Additionally, existing visual table understanding (VTU) datasets offer fixed examples with single visualizations and pre-defined instructions, providing no access to underlying serialized data for reformulation. We introduce TABLET, a large-scale VTU dataset with 4 million examples across 20 tasks, grounded in 2 million unique tables where 88% preserve original visualizations. Each example includes paired image-HTML representations, comprehensive metadata, and provenance information linking back to the source datasets. By preserving original visualizations and maintaining example traceability in a unified large-scale collection, TABLET establishes a foundation for robust training and extensible evaluation of future VTU models.
Previous datasets render table images from serialized tables, losing original visual details. In contrast, TABLET locates and retrieves the original table visualizations across 14 tabular datasets, resulting in 4M examples grounded in 2M unique tables.
Here is a breakdown of examples per task and their source dataset.
Tasks & Data Sources
| Task | Seeds | Examples |
|---|---|---|
| ent_link | TURL | 1,523,904 37.5% |
| col_type | TURL | 628,396 15.4% |
| struct_aware_parse | PubTabNet, | 523,699 12.9% |
| wikibio | WikiBIO | 728,321 17.9% |
| hybridqa | HybridQA | 69,599 1.7% |
| fetaqa | ToTTo | 4,662 0.1% |
| hitab | NSF, | 10,671 0.3% |
| infotabs | InfoTabs | 23,738 0.6% |
| tabfact | TabFact | 112,432 2.8% |
| tabmwp | TabMWP | 38,431 0.9% |
| tat-qa | TAT-QA | 2,756 0.1% |
| totto | ToTTo | 125,095 3.1% |
| wikitq | WikiTableQuestions | 22,033 0.5% |
| rel_extraction | TURL | 64,790 1.6% |
| table_instruction | InfoTabs, | 136,944 3.4% |
| row_column_extraction | InfoTabs, | 8,678 0.2% |
| table_cell_extraction | InfoTabs, | 8,693 0.2% |
| table_cell_location | InfoTabs, | 8,664 0.2% |
| table_recognition | InfoTabs, | 7,839 0.2% |
| table_size_detection | InfoTabs, | 8,750 0.2% |
| merged_cell_detection | InfoTabs, | 8,450 0.2% |
| visual_table_qa | ToTTo, | 306 0.1% |
| Total | 4,066,851 |
Visual Table QA Benchmark
To evaluate whether models can truly combine visual perception with table understanding, we introduce VisualTableQA, a manually curated benchmark of 306 examples. Human annotators selected tables with high visual complexity and formulated questions that can only be answered by jointly attending to visual cues and tabular structure, for instance, identifying a car by its color or a historical figure by visual attributes in an accompanying image.
You can find VisualTableQA benchmark alongside other tasks in TABLET-test or you can download it directly here.
Authors
License
TABLET's original contributions — the table image renderings, unified annotation format, task instructions, and accompanying metadata and scripts produced by the TABLET authors — are released under the Creative Commons Attribution 4.0 International (CC BY 4.0) license, except where the license of a source dataset requires otherwise (see below).
TABLET is a meta-dataset aggregating 14 existing table understanding datasets. Each TABLET subset remains subject to the license and terms of use of its original source. By downloading or using TABLET, you agree to comply with the license of every source dataset you use, in addition to the terms above. Where a source license conflicts with CC BY 4.0, the source license prevails for that subset.
| Subset | Source license | Notes |
|---|---|---|
| TURL | Apache 2.0 | Tables originate from Wikipedia (CC BY-SA 3.0). |
| ToTTo | CC BY-SA 3.0 | ShareAlike: this subset is redistributed under CC BY-SA 3.0. |
| TabFact | MIT | Tables originate from Wikipedia (CC BY-SA 3.0). |
| WikiTableQuestions | CC BY-SA 4.0 | ShareAlike: this subset is redistributed under CC BY-SA 4.0. |
| HybridQA | MIT | Tables and passages originate from Wikipedia (CC BY-SA 3.0). |
| HiTab | C-UDA v1.0 | Microsoft Computational Use of Data Agreement: computational use only; any redistribution of this subset must include the C-UDA agreement. |
| PubTabNet | CDLA-P 1.0 | Annotations under CDLA-Permissive 1.0. Table images are from the PubMed Central Open Access Subset and remain subject to the PMC OA terms of use; individual article licenses vary. |
| TabMWP | CC BY-NC-SA 4.0 | Non-commercial use only. ShareAlike: this subset is redistributed under CC BY-NC-SA 4.0. The MIT license in the repository covers code only. |
| TAT-QA | CC BY 4.0 | — |
| InfoTabs | Apache 2.0 | Tables originate from Wikipedia infoboxes (CC BY-SA 3.0). See the dataset's datasheet for non-academic usage. |
| WikiBIO | CC BY-SA 3.0 | ShareAlike: this subset is redistributed under CC BY-SA 3.0. |
| FeTaQA | CC BY-SA 4.0 | ShareAlike: this subset is redistributed under CC BY-SA 4.0. |
| MMTab | MIT | MMTab is itself constructed from 14 public table datasets; the licenses of its underlying sources also apply. Code repository is Apache 2.0. |
| DocStruct4M | Apache 2.0 | Built from publicly available text-rich image datasets; the licenses of its underlying sources also apply. |
Usage restrictions to be aware of
- Non-commercial subset. The TabMWP subset is licensed CC BY-NC-SA 4.0 and may not be used for commercial purposes. If you require a fully commercial-compatible corpus, exclude this subset.
- ShareAlike subsets. The subsets derived from ToTTo, WikiTableQuestions, FeTaQA, and WikiBIO carry ShareAlike terms: adaptations of these subsets must be distributed under the same (or a compatible) license, not plain CC BY.
- HiTab. Use and redistribution of the HiTab subset is governed by the C-UDA v1.0; results of computational analysis (e.g., trained models) are not restricted, but the data itself may only be redistributed together with the C-UDA terms.
- PubTabNet images. Images remain subject to the PMC Open Access Subset terms; check individual article licenses if you redistribute them outside a research context.
- Attribution. Whatever subsets you use, please attribute the original datasets and cite the corresponding papers, in addition to citing TABLET.
License information above was collected from each project's official repository or dataset card. It is provided in good faith and does not constitute legal advice; if in doubt, consult the original license texts linked in the table.
Cite
If you find this dataset useful, please cite it using the following format:
@misc{alonso2025tabletlargescaledatasetrobust,
title={TABLET: A Large-Scale Dataset for Robust Visual Table Understanding},
author={Iñigo Alonso and Imanol Miranda and Eneko Agirre and Mirella Lapata},
year={2025},
eprint={2509.21205},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2509.21205},
}Contact
If you need help accessing data or have questions, please contact Iñigo Alonso.