Skip to content

Tutorial 3: Dataset batch

Start with Dataset + Model batch parsing, then apply transformations without explicit loops.

Setup

>>> import omnipy as om

Batch parsing with Dataset

>>> readings = om.Dataset[om.Model[int]]({'sensor_a': '1', 'sensor_b': 2.0, 'sensor_c': 3})
>>> readings

╭───┬────────────────┬────────────┬────────┬──────────────────╮
│ # │ Data file name │    Type    │ Length │ Size (in memory) │
│   │                │            │        │                  │
│ 0 │ sensor_a       │ Model[int] │      - │        589 Bytes │
│ 1 │ sensor_b       │ Model[int] │      - │        589 Bytes │
│ 2 │ sensor_c       │ Model[int] │      - │        589 Bytes │
╰───┴────────────────┴────────────┴────────┴──────────────────╯

No-for-loop batch transform pattern

>>> incremented = readings.do(lambda value: int(value) + 1)

Hierarchical datasets

>>> from omnipy import Dataset, Model
>>> Inner = Dataset[Model[int]]
>>> Outer = Dataset[Inner]
>>> grouped = Outer({'group1': {'a': 1, 'b': 2}, 'group2': {'a': 10}})
>>> grouped

╭───┬────────────────┬─────────────────────┬────────┬──────────────────╮
│ # │ Data file name │        Type         │ Length │ Size (in memory) │
│   │                │                     │        │                  │
│ 0 │ group1         │ Dataset[Model[int]] │      2 │           1.9 kB │
│ 1 │ group2         │ Dataset[Model[int]] │      1 │           1.4 kB │
╰───┴────────────────┴─────────────────────┴────────┴──────────────────╯

>>> grouped_incremented = grouped.do(lambda dataset: dataset.do(lambda value: int(value) + 1))

You get batch behavior and hierarchy handling without writing explicit for loops.

What you learned

  • Dataset[Model[int]](...) parses and batches many values in one typed container.
  • Dataset.do(...) lets you apply per-item transforms without explicit loops.
  • Nested datasets can be transformed hierarchically while preserving structure.

Common pitfalls

  • Forgetting to convert values inside lambdas when needed. Use int(value) in mixed parsed inputs.

Next steps