Interface IntegrationPartition.Builder

All Superinterfaces:
Buildable, CopyableBuilder<IntegrationPartition.Builder,IntegrationPartition>, SdkBuilder<IntegrationPartition.Builder,IntegrationPartition>, SdkPojo
Enclosing class:
IntegrationPartition
@Mutable @NotThreadSafe public static interface IntegrationPartition.Builder extends SdkPojo, CopyableBuilder<IntegrationPartition.Builder,IntegrationPartition>

  • Method Details

    • fieldName

      IntegrationPartition.Builder fieldName(String fieldName)

      The field name used to partition data on the target. Avoid using columns that have unique values for each row (for example, `LastModifiedTimestamp`, `SystemModTimeStamp`) as the partition column. These columns are not suitable for partitioning because they create a large number of small partitions, which can lead to performance issues.

      Parameters:
      fieldName - The field name used to partition data on the target. Avoid using columns that have unique values for each row (for example, `LastModifiedTimestamp`, `SystemModTimeStamp`) as the partition column. These columns are not suitable for partitioning because they create a large number of small partitions, which can lead to performance issues.
      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • functionSpec

      IntegrationPartition.Builder functionSpec(String functionSpec)

      Specifies the function used to partition data on the target. The accepted values for this parameter are:

      • identity - Uses source values directly without transformation

      • year - Extracts the year from timestamp values (e.g., 2023)

      • month - Extracts the month from timestamp values (e.g., 2023-01)

      • day - Extracts the day from timestamp values (e.g., 2023-01-15)

      • hour - Extracts the hour from timestamp values (e.g., 2023-01-15-14)

      Parameters:
      functionSpec - Specifies the function used to partition data on the target. The accepted values for this parameter are:

      • identity - Uses source values directly without transformation

      • year - Extracts the year from timestamp values (e.g., 2023)

      • month - Extracts the month from timestamp values (e.g., 2023-01)

      • day - Extracts the day from timestamp values (e.g., 2023-01-15)

      • hour - Extracts the hour from timestamp values (e.g., 2023-01-15-14)

      Returns:
      Returns a reference to this object so that method calls can be chained together.

    • conversionSpec

      IntegrationPartition.Builder conversionSpec(String conversionSpec)

      Specifies the timestamp format of the source data. Valid values are:

      • epoch_sec - Unix epoch timestamp in seconds

      • epoch_milli - Unix epoch timestamp in milliseconds

      • iso - ISO 8601 formatted timestamp

      Only specify ConversionSpec when using timestamp-based partition functions (year, month, day, or hour). Glue Zero-ETL uses this parameter to correctly transform source data into timestamp format before partitioning.

      Do not use high-cardinality columns with the identity partition function. High-cardinality columns include:

      • Primary keys

      • Timestamp fields (such as LastModifiedTimestamp, CreatedDate)

      • System-generated timestamps

      Using high-cardinality columns with identity partitioning creates many small partitions, which can significantly degrade ingestion performance.

      Parameters:
      conversionSpec - Specifies the timestamp format of the source data. Valid values are:

      • epoch_sec - Unix epoch timestamp in seconds

      • epoch_milli - Unix epoch timestamp in milliseconds

      • iso - ISO 8601 formatted timestamp

      Only specify ConversionSpec when using timestamp-based partition functions (year, month, day, or hour). Glue Zero-ETL uses this parameter to correctly transform source data into timestamp format before partitioning.

      Do not use high-cardinality columns with the identity partition function. High-cardinality columns include:

      • Primary keys

      • Timestamp fields (such as LastModifiedTimestamp, CreatedDate)

      • System-generated timestamps

      Using high-cardinality columns with identity partitioning creates many small partitions, which can significantly degrade ingestion performance.

      Returns:
      Returns a reference to this object so that method calls can be chained together.