omnipy.components.pandas.datasets

Datasets that map file names to pandas-backed table models.

CLASS	DESCRIPTION
`ListOfPandasDatasetsWithSameNumberOfFiles`	Validate a list of aligned `PandasDataset` instances.
`PandasDataset`	Store named pandas-backed tables as `PandasModel` data files.

ListOfPandasDatasetsWithSameNumberOfFiles

Bases: Model[list[PandasDataset]]


              flowchart BT
              omnipy.components.pandas.datasets.ListOfPandasDatasetsWithSameNumberOfFiles[ListOfPandasDatasetsWithSameNumberOfFiles]
              omnipy.data.model.Model[Model]
              omnipy.data._mixins.display.ModelDisplayMixin[ModelDisplayMixin]
              omnipy.data._mixins.display.BaseDisplayMixin[BaseDisplayMixin]
              omnipy.data._data_class_creator.DataClassBase[DataClassBase]

                              omnipy.data.model.Model --> omnipy.components.pandas.datasets.ListOfPandasDatasetsWithSameNumberOfFiles
                                omnipy.data._mixins.display.ModelDisplayMixin --> omnipy.data.model.Model
                                omnipy.data._mixins.display.BaseDisplayMixin --> omnipy.data._mixins.display.ModelDisplayMixin
                

                omnipy.data._data_class_creator.DataClassBase --> omnipy.data.model.Model
                
                omnipy.util.pydantic.GenericModel --> omnipy.data.model.Model
                



              click omnipy.components.pandas.datasets.ListOfPandasDatasetsWithSameNumberOfFiles href "" "omnipy.components.pandas.datasets.ListOfPandasDatasetsWithSameNumberOfFiles"
              click omnipy.data.model.Model href "" "omnipy.data.model.Model"
              click omnipy.data._mixins.display.ModelDisplayMixin href "" "omnipy.data._mixins.display.ModelDisplayMixin"
              click omnipy.data._mixins.display.BaseDisplayMixin href "" "omnipy.data._mixins.display.BaseDisplayMixin"
              click omnipy.data._data_class_creator.DataClassBase href "" "omnipy.data._data_class_creator.DataClassBase"

Validate a list of aligned PandasDataset instances.

The list is expected to contain at least two datasets and each dataset must contain one or more files.

Used by:

Reference Code reference
- omnipy concat_dataframes_across_datasets
- omnipy components pandas tasks concat_dataframes_across_datasets

Source code in src/omnipy/components/pandas/datasets.py

class ListOfPandasDatasetsWithSameNumberOfFiles(Model[list[PandasDataset]]):
    """Validate a list of aligned ``PandasDataset`` instances.

    The list is expected to contain at least two datasets and each dataset
    must contain one or more files.
    """
    @classmethod
    def _parse_data(cls, data: list[PandasDataset]) -> Any:
        dataset_list = data
        assert len(dataset_list) >= 2
        assert all(len(dataset) for dataset in dataset_list)

PandasDataset

Bases: Dataset[PandasModel]


              flowchart BT
              omnipy.components.pandas.datasets.PandasDataset[PandasDataset]
              omnipy.data.dataset.Dataset[Dataset]
              omnipy.data._mixins.display.DatasetDisplayMixin[DatasetDisplayMixin]
              omnipy.data._mixins.display.BaseDisplayMixin[BaseDisplayMixin]
              omnipy.data._mixins.task.TaskDatasetMixin[TaskDatasetMixin]
              omnipy.data._data_class_creator.DataClassBase[DataClassBase]

                              omnipy.data.dataset.Dataset --> omnipy.components.pandas.datasets.PandasDataset
                                omnipy.data._mixins.display.DatasetDisplayMixin --> omnipy.data.dataset.Dataset
                                omnipy.data._mixins.display.BaseDisplayMixin --> omnipy.data._mixins.display.DatasetDisplayMixin
                

                omnipy.data._mixins.task.TaskDatasetMixin --> omnipy.data.dataset.Dataset
                
                omnipy.data._data_class_creator.DataClassBase --> omnipy.data.dataset.Dataset
                
                omnipy.util.pydantic.GenericModel --> omnipy.data.dataset.Dataset
                



              click omnipy.components.pandas.datasets.PandasDataset href "" "omnipy.components.pandas.datasets.PandasDataset"
              click omnipy.data.dataset.Dataset href "" "omnipy.data.dataset.Dataset"
              click omnipy.data._mixins.display.DatasetDisplayMixin href "" "omnipy.data._mixins.display.DatasetDisplayMixin"
              click omnipy.data._mixins.display.BaseDisplayMixin href "" "omnipy.data._mixins.display.BaseDisplayMixin"
              click omnipy.data._mixins.task.TaskDatasetMixin href "" "omnipy.data._mixins.task.TaskDatasetMixin"
              click omnipy.data._data_class_creator.DataClassBase href "" "omnipy.data._data_class_creator.DataClassBase"

Store named pandas-backed tables as PandasModel data files.

This dataset maps each data file name to a table represented by :class:~omnipy.components.pandas.models.PandasModel.

Example

dataset = PandasDataset({'table': [{'id': 1, 'value': 'a'}]}) tuple(dataset.keys()) ('table',)

Returned by:

Reference Code reference
- omnipy
- omnipy components pandas
  - serializers PandasDatasetToTarFileSerializer deserialize
  - tasks

Subclassed by:

Reference Code reference omnipy components pandas
- datasets ListOfPandasDatasetsWithSameNumberOfFiles
- serializers PandasDatasetToTarFileSerializer

Used by:

Reference Code reference
- omnipy
  - convert_dataset_pandas_to_csv
  - extract_columns_as_files
- omnipy components pandas
  - serializers PandasDatasetToTarFileSerializer serialize
  - tasks
    - convert_dataset_pandas_to_csv
    - extract_columns_as_files

Source code in src/omnipy/components/pandas/datasets.py

class PandasDataset(Dataset[PandasModel]):
    """Store named pandas-backed tables as ``PandasModel`` data files.

    This dataset maps each data file name to a table represented by
    :class:`~omnipy.components.pandas.models.PandasModel`.

    Example:
        >>> dataset = PandasDataset({'table': [{'id': 1, 'value': 'a'}]})
        >>> tuple(dataset.keys())
        ('table',)
    """

    ...