Key Components

Key Components

1. DataPipeline

The DataPipeline is the core component that allows you to define a sequence of data processing tasks. It can include both transformations and validations.

Example usage:

from dataDisk import DataPipeline, Transformation, Validator

# Create a DataPipeline
pipeline = DataPipeline()

# Add transformation tasks or validators
pipeline.add_task(Transformation(double))
***

pipeline.add_task(Transformation(square))
pipeline.add_task(Validator(is_even))

# Process data through the pipeline
data = [1, 2, 3, 4, 5]
result = pipeline.process(data)
print(result)