Skip to content

Column-based tabular validation

Maturity labels

  • Now: Stable and supported in current releases.
  • Preview: Usable today, but behavior and APIs may evolve.
  • Planned: Not yet implemented.

Note

Status: Preview

Column-oriented patterns are usable today, but this area is still maturing.

This guide shows how to work with columnar and record-style tables in Omnipy while keeping typed validation in the flow.

Columnar input to row-wise records

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/envs/latest/lib/python3.10/site-packages/markdown_exec/_internal/formatters/python.py", line 71, in _run_python
    exec_python(code, code_block_id, exec_globals)
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/envs/latest/lib/python3.10/site-packages/markdown_exec/_internal/formatters/_exec_python.py", line 8, in exec_python
    exec(compiled, exec_globals)  # noqa: S102
  File "<code block: n15>", line 2, in <module>
    columnar = om.ColumnWiseTableWithColNamesModel({
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/checkouts/latest/src/omnipy/data/model.py", line 535, in __init__
    self._primary_validation(super_kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/checkouts/latest/src/omnipy/data/model.py", line 551, in _primary_validation
    super().__init__(**super_kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/envs/latest/lib/python3.10/site-packages/pydantic/v1/main.py", line 364, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 5 validation errors for ColumnWiseTableWithColNamesModel
__root__ -> sample -> __root__
  [Omnipy] none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> count -> __root__
  [Omnipy] none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> __root__
  value is not a valid list (type=type_error.list)
__root__ -> __root__
   (type=assertion_error)
__root__
   (type=assertion_error)

Add typed validation on top of records

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/envs/latest/lib/python3.10/site-packages/markdown_exec/_internal/formatters/python.py", line 71, in _run_python
    exec_python(code, code_block_id, exec_globals)
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/envs/latest/lib/python3.10/site-packages/markdown_exec/_internal/formatters/_exec_python.py", line 8, in exec_python
    exec(compiled, exec_globals)  # noqa: S102
  File "<code block: n16>", line 7, in <module>
    columnar = om.ColumnWiseTableWithColNamesModel({'sample': ['s1', 's2'], 'count': ['10', '20']})
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/checkouts/latest/src/omnipy/data/model.py", line 535, in __init__
    self._primary_validation(super_kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/checkouts/latest/src/omnipy/data/model.py", line 551, in _primary_validation
    super().__init__(**super_kwargs)
  File "/home/docs/checkouts/readthedocs.org/user_builds/omnipy/envs/latest/lib/python3.10/site-packages/pydantic/v1/main.py", line 364, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 5 validation errors for ColumnWiseTableWithColNamesModel
__root__ -> sample -> __root__
  [Omnipy] none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> count -> __root__
  [Omnipy] none is not an allowed value (type=type_error.none.not_allowed)
__root__ -> __root__
  value is not a valid list (type=type_error.list)
__root__ -> __root__
   (type=assertion_error)
__root__
   (type=assertion_error)

With interactive mode enabled (runtime.config.data.model.interactive = True), failed assignments do not leave partial invalid state in the container.

Omnipy vs Pandera (honest comparison)

Where Omnipy has advantages

  • End-to-end typed flow from nested/JSON/remote ingestion to table-style records.
  • Continuous validation behavior integrated with model/dataset operations.
  • Practical handling of nested structures before or alongside table conversion.

Where Pandera wins today

  • More mature dedicated dataframe validation ecosystem.
  • Richer dataframe-native schema checks and integrations.
  • Generally better raw speed for pure dataframe-only validation workloads.

When to choose which

  • Choose Omnipy when tabular validation is one part of a broader typed transformation pipeline.
  • Choose Pandera when your core problem is dataframe-first validation at scale.
  • Use both when needed: Omnipy for ingestion/transforms, Pandera for dataframe policy gates.