Class JobUpdate

java.lang.Object
software.amazon.awssdk.services.glue.model.JobUpdate
All Implemented Interfaces:
Serializable, SdkPojo, ToCopyableBuilder<JobUpdate.Builder,JobUpdate>
@Generated("software.amazon.awssdk:codegen") public final class JobUpdate extends Object implements SdkPojo, Serializable, ToCopyableBuilder<JobUpdate.Builder,JobUpdate>

Specifies information used to update an existing job definition. The previous job definition is completely overwritten by this information.

See Also:

  • Method Details

    • jobMode

      public final JobMode jobMode()

      A mode that describes how a job was created. Valid values are:

      • SCRIPT - The job was created using the Glue Studio script editor.

      • VISUAL - The job was created using the Glue Studio visual editor.

      • NOTEBOOK - The job was created using an interactive sessions notebook.

      When the JobMode field is missing or null, SCRIPT is assigned as the default value.

      If the service returns an enum value that is not available in the current SDK version, jobMode will return JobMode.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from jobModeAsString().

      Returns:
      A mode that describes how a job was created. Valid values are:

      • SCRIPT - The job was created using the Glue Studio script editor.

      • VISUAL - The job was created using the Glue Studio visual editor.

      • NOTEBOOK - The job was created using an interactive sessions notebook.

      When the JobMode field is missing or null, SCRIPT is assigned as the default value.

      See Also:

    • jobModeAsString

      public final String jobModeAsString()

      A mode that describes how a job was created. Valid values are:

      • SCRIPT - The job was created using the Glue Studio script editor.

      • VISUAL - The job was created using the Glue Studio visual editor.

      • NOTEBOOK - The job was created using an interactive sessions notebook.

      When the JobMode field is missing or null, SCRIPT is assigned as the default value.

      If the service returns an enum value that is not available in the current SDK version, jobMode will return JobMode.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from jobModeAsString().

      Returns:
      A mode that describes how a job was created. Valid values are:

      • SCRIPT - The job was created using the Glue Studio script editor.

      • VISUAL - The job was created using the Glue Studio visual editor.

      • NOTEBOOK - The job was created using an interactive sessions notebook.

      When the JobMode field is missing or null, SCRIPT is assigned as the default value.

      See Also:

    • jobRunQueuingEnabled

      public final Boolean jobRunQueuingEnabled()

      Specifies whether job run queuing is enabled for the job runs for this job.

      A value of true means job run queuing is enabled for the job runs. If false or not populated, the job runs will not be considered for queueing.

      If this field does not match the value set in the job run, then the value from the job run field will be used.

      Returns:
      Specifies whether job run queuing is enabled for the job runs for this job.

      A value of true means job run queuing is enabled for the job runs. If false or not populated, the job runs will not be considered for queueing.

      If this field does not match the value set in the job run, then the value from the job run field will be used.

    • description

      public final String description()

      Description of the job being defined.

      Returns:
      Description of the job being defined.

    • logUri

      public final String logUri()

      This field is reserved for future use.

      Returns:
      This field is reserved for future use.

    • role

      public final String role()

      The name or Amazon Resource Name (ARN) of the IAM role associated with this job (required).

      Returns:
      The name or Amazon Resource Name (ARN) of the IAM role associated with this job (required).

    • executionProperty

      public final ExecutionProperty executionProperty()

      An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job.

      Returns:
      An ExecutionProperty specifying the maximum number of concurrent runs allowed for this job.

    • command

      public final JobCommand command()

      The JobCommand that runs this job (required).

      Returns:
      The JobCommand that runs this job (required).

    • hasDefaultArguments

      public final boolean hasDefaultArguments()
      For responses, this returns true if the service returned a value for the DefaultArguments property. This DOES NOT check that the value is non-empty (for which, you should check the isEmpty() method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.

    • defaultArguments

      public final Map<String,String> defaultArguments()

      The default arguments for every run of this job, specified as name-value pairs.

      You can specify arguments here that your own job-execution script consumes, as well as arguments that Glue itself consumes.

      Job arguments may be logged. Do not pass plaintext secrets as arguments. Retrieve secrets from a Glue Connection, Secrets Manager or other secret management mechanism if you intend to keep them within the Job.

      For information about how to specify and consume your own Job arguments, see the Calling Glue APIs in Python topic in the developer guide.

      For information about the arguments you can provide to this field when configuring Spark jobs, see the Special Parameters Used by Glue topic in the developer guide.

      For information about the arguments you can provide to this field when configuring Ray jobs, see Using job parameters in Ray jobs in the developer guide.

      Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

      This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the hasDefaultArguments() method.

      Returns:
      The default arguments for every run of this job, specified as name-value pairs.

      You can specify arguments here that your own job-execution script consumes, as well as arguments that Glue itself consumes.

      Job arguments may be logged. Do not pass plaintext secrets as arguments. Retrieve secrets from a Glue Connection, Secrets Manager or other secret management mechanism if you intend to keep them within the Job.

      For information about how to specify and consume your own Job arguments, see the Calling Glue APIs in Python topic in the developer guide.

      For information about the arguments you can provide to this field when configuring Spark jobs, see the Special Parameters Used by Glue topic in the developer guide.

      For information about the arguments you can provide to this field when configuring Ray jobs, see Using job parameters in Ray jobs in the developer guide.

    • hasNonOverridableArguments

      public final boolean hasNonOverridableArguments()
      For responses, this returns true if the service returned a value for the NonOverridableArguments property. This DOES NOT check that the value is non-empty (for which, you should check the isEmpty() method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.

    • nonOverridableArguments

      public final Map<String,String> nonOverridableArguments()

      Arguments for this job that are not overridden when providing job arguments in a job run, specified as name-value pairs.

      Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

      This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the hasNonOverridableArguments() method.

      Returns:
      Arguments for this job that are not overridden when providing job arguments in a job run, specified as name-value pairs.

    • connections

      public final ConnectionsList connections()

      The connections used for this job.

      Returns:
      The connections used for this job.

    • maxRetries

      public final Integer maxRetries()

      The maximum number of times to retry this job if it fails.

      Returns:
      The maximum number of times to retry this job if it fails.

    • allocatedCapacity

      @Deprecated public final Integer allocatedCapacity()
      Deprecated.
      This property is deprecated, use MaxCapacity instead.

      This field is deprecated. Use MaxCapacity instead.

      The number of Glue data processing units (DPUs) to allocate to this job. You can allocate a minimum of 2 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.

      Returns:
      This field is deprecated. Use MaxCapacity instead.

      The number of Glue data processing units (DPUs) to allocate to this job. You can allocate a minimum of 2 DPUs; the default is 10. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.

    • timeout

      public final Integer timeout()

      The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.

      Jobs must have timeout values less than 7 days or 10080 minutes. Otherwise, the jobs will throw an exception.

      When the value is left blank, the timeout is defaulted to 2880 minutes.

      Any existing Glue jobs that had a timeout value greater than 7 days will be defaulted to 7 days. For instance if you have specified a timeout of 20 days for a batch job, it will be stopped on the 7th day.

      For streaming jobs, if you have set up a maintenance window, it will be restarted during the maintenance window after 7 days.

      Returns:
      The job timeout in minutes. This is the maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.

      Jobs must have timeout values less than 7 days or 10080 minutes. Otherwise, the jobs will throw an exception.

      When the value is left blank, the timeout is defaulted to 2880 minutes.

      Any existing Glue jobs that had a timeout value greater than 7 days will be defaulted to 7 days. For instance if you have specified a timeout of 20 days for a batch job, it will be stopped on the 7th day.

      For streaming jobs, if you have set up a maintenance window, it will be restarted during the maintenance window after 7 days.

    • maxCapacity

      public final Double maxCapacity()

      For Glue version 1.0 or earlier jobs, using the standard worker type, the number of Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.

      For Glue version 2.0+ jobs, you cannot specify a Maximum capacity. Instead, you should specify a Worker type and the Number of workers.

      Do not set MaxCapacity if using WorkerType and NumberOfWorkers.

      The value that can be allocated for MaxCapacity depends on whether you are running a Python shell job, an Apache Spark ETL job, or an Apache Spark streaming ETL job:

      • When you specify a Python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.

      • When you specify an Apache Spark ETL job (JobCommand.Name="glueetl") or Apache Spark streaming ETL job (JobCommand.Name="gluestreaming"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

      Returns:
      For Glue version 1.0 or earlier jobs, using the standard worker type, the number of Glue data processing units (DPUs) that can be allocated when this job runs. A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory. For more information, see the Glue pricing page.

      For Glue version 2.0+ jobs, you cannot specify a Maximum capacity. Instead, you should specify a Worker type and the Number of workers.

      Do not set MaxCapacity if using WorkerType and NumberOfWorkers.

      The value that can be allocated for MaxCapacity depends on whether you are running a Python shell job, an Apache Spark ETL job, or an Apache Spark streaming ETL job:

      • When you specify a Python shell job (JobCommand.Name="pythonshell"), you can allocate either 0.0625 or 1 DPU. The default is 0.0625 DPU.

      • When you specify an Apache Spark ETL job (JobCommand.Name="glueetl") or Apache Spark streaming ETL job (JobCommand.Name="gluestreaming"), you can allocate from 2 to 100 DPUs. The default is 10 DPUs. This job type cannot have a fractional DPU allocation.

    • workerType

      public final WorkerType workerType()

      The type of predefined worker that is allocated when a job runs. Accepts a value of G.1X, G.2X, G.4X, G.8X or G.025X for Spark jobs. Accepts the value Z.2X for Ray jobs. For more information, see Defining job properties for Spark jobs

      If the service returns an enum value that is not available in the current SDK version, workerType will return WorkerType.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from workerTypeAsString().

      Returns:
      The type of predefined worker that is allocated when a job runs. Accepts a value of G.1X, G.2X, G.4X, G.8X or G.025X for Spark jobs. Accepts the value Z.2X for Ray jobs. For more information, see Defining job properties for Spark jobs
      See Also:

    • workerTypeAsString

      public final String workerTypeAsString()

      The type of predefined worker that is allocated when a job runs. Accepts a value of G.1X, G.2X, G.4X, G.8X or G.025X for Spark jobs. Accepts the value Z.2X for Ray jobs. For more information, see Defining job properties for Spark jobs

      If the service returns an enum value that is not available in the current SDK version, workerType will return WorkerType.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from workerTypeAsString().

      Returns:
      The type of predefined worker that is allocated when a job runs. Accepts a value of G.1X, G.2X, G.4X, G.8X or G.025X for Spark jobs. Accepts the value Z.2X for Ray jobs. For more information, see Defining job properties for Spark jobs
      See Also:

    • numberOfWorkers

      public final Integer numberOfWorkers()

      The number of workers of a defined workerType that are allocated when a job runs.

      Returns:
      The number of workers of a defined workerType that are allocated when a job runs.

    • securityConfiguration

      public final String securityConfiguration()

      The name of the SecurityConfiguration structure to be used with this job.

      Returns:
      The name of the SecurityConfiguration structure to be used with this job.

    • notificationProperty

      public final NotificationProperty notificationProperty()

      Specifies the configuration properties of a job notification.

      Returns:
      Specifies the configuration properties of a job notification.

    • glueVersion

      public final String glueVersion()

      In Spark jobs, GlueVersion determines the versions of Apache Spark and Python that Glue available in a job. The Python version indicates the version supported for jobs of type Spark.

      Ray jobs should set GlueVersion to 4.0 or greater. However, the versions of Ray, Python and additional libraries available in your Ray job are determined by the Runtime parameter of the Job command.

      For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

      Jobs that are created without specifying a Glue version default to Glue 0.9.

      Returns:
      In Spark jobs, GlueVersion determines the versions of Apache Spark and Python that Glue available in a job. The Python version indicates the version supported for jobs of type Spark.

      Ray jobs should set GlueVersion to 4.0 or greater. However, the versions of Ray, Python and additional libraries available in your Ray job are determined by the Runtime parameter of the Job command.

      For more information about the available Glue versions and corresponding Spark and Python versions, see Glue version in the developer guide.

      Jobs that are created without specifying a Glue version default to Glue 0.9.

    • hasCodeGenConfigurationNodes

      public final boolean hasCodeGenConfigurationNodes()
      For responses, this returns true if the service returned a value for the CodeGenConfigurationNodes property. This DOES NOT check that the value is non-empty (for which, you should check the isEmpty() method on the property). This is useful because the SDK will never return a null collection or map, but you may need to differentiate between the service returning nothing (or null) and the service returning an empty collection or map. For requests, this returns true if a value for the property was specified in the request builder, and false if a value was not specified.

    • codeGenConfigurationNodes

      public final Map<String,CodeGenConfigurationNode> codeGenConfigurationNodes()

      The representation of a directed acyclic graph on which both the Glue Studio visual component and Glue Studio code generation is based.

      Attempts to modify the collection returned by this method will result in an UnsupportedOperationException.

      This method will never return null. If you would like to know whether the service returned this field (so that you can differentiate between null and empty), you can use the hasCodeGenConfigurationNodes() method.

      Returns:
      The representation of a directed acyclic graph on which both the Glue Studio visual component and Glue Studio code generation is based.

    • executionClass

      public final ExecutionClass executionClass()

      Indicates whether the job is run with a standard or flexible execution class. The standard execution-class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.

      The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.

      Only jobs with Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX. The flexible execution class is available for Spark jobs.

      If the service returns an enum value that is not available in the current SDK version, executionClass will return ExecutionClass.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from executionClassAsString().

      Returns:
      Indicates whether the job is run with a standard or flexible execution class. The standard execution-class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.

      The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.

      Only jobs with Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX. The flexible execution class is available for Spark jobs.

      See Also:

    • executionClassAsString

      public final String executionClassAsString()

      Indicates whether the job is run with a standard or flexible execution class. The standard execution-class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.

      The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.

      Only jobs with Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX. The flexible execution class is available for Spark jobs.

      If the service returns an enum value that is not available in the current SDK version, executionClass will return ExecutionClass.UNKNOWN_TO_SDK_VERSION. The raw value returned by the service is available from executionClassAsString().

      Returns:
      Indicates whether the job is run with a standard or flexible execution class. The standard execution-class is ideal for time-sensitive workloads that require fast job startup and dedicated resources.

      The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary.

      Only jobs with Glue version 3.0 and above and command type glueetl will be allowed to set ExecutionClass to FLEX. The flexible execution class is available for Spark jobs.

      See Also:

    • sourceControlDetails

      public final SourceControlDetails sourceControlDetails()

      The details for a source control configuration for a job, allowing synchronization of job artifacts to or from a remote repository.

      Returns:
      The details for a source control configuration for a job, allowing synchronization of job artifacts to or from a remote repository.

    • maintenanceWindow

      public final String maintenanceWindow()

      This field specifies a day of the week and hour for a maintenance window for streaming jobs. Glue periodically performs maintenance activities. During these maintenance windows, Glue will need to restart your streaming jobs.

      Glue will restart the job within 3 hours of the specified maintenance window. For instance, if you set up the maintenance window for Monday at 10:00AM GMT, your jobs will be restarted between 10:00AM GMT to 1:00PM GMT.

      Returns:
      This field specifies a day of the week and hour for a maintenance window for streaming jobs. Glue periodically performs maintenance activities. During these maintenance windows, Glue will need to restart your streaming jobs.

      Glue will restart the job within 3 hours of the specified maintenance window. For instance, if you set up the maintenance window for Monday at 10:00AM GMT, your jobs will be restarted between 10:00AM GMT to 1:00PM GMT.

    • toBuilder

      public JobUpdate.Builder toBuilder()
      Description copied from interface: ToCopyableBuilder
      Take this object and create a builder that contains all of the current property values of this object.
      Specified by:
      toBuilder in interface ToCopyableBuilder<JobUpdate.Builder,JobUpdate>
      Returns:
      a builder for type T

    • builder

      public static JobUpdate.Builder builder()

    • serializableBuilderClass

      public static Class<? extends JobUpdate.Builder> serializableBuilderClass()

    • hashCode

      public final int hashCode()
      Overrides:
      hashCode in class Object

    • equals

      public final boolean equals(Object obj)
      Overrides:
      equals in class Object

    • equalsBySdkFields

      public final boolean equalsBySdkFields(Object obj)
      Description copied from interface: SdkPojo
      Indicates whether some other object is "equal to" this one by SDK fields. An SDK field is a modeled, non-inherited field in an SdkPojo class, and is generated based on a service model.

      If an SdkPojo class does not have any inherited fields, equalsBySdkFields and equals are essentially the same.

      Specified by:
      equalsBySdkFields in interface SdkPojo
      Parameters:
      obj - the object to be compared with
      Returns:
      true if the other object equals to this object by sdk fields, false otherwise.

    • toString

      public final String toString()
      Returns a string representation of this object. This is useful for testing and debugging. Sensitive data will be redacted from this string using a placeholder value.
      Overrides:
      toString in class Object

    • getValueForField

      public final <T> Optional<T> getValueForField(String fieldName, Class<T> clazz)

    • sdkFields

      public final List<SdkField<?>> sdkFields()
      Specified by:
      sdkFields in interface SdkPojo
      Returns:
      List of SdkField in this POJO. May be empty list but should never be null.

    • sdkFieldNameToField

      public final Map<String,SdkField<?>> sdkFieldNameToField()
      Specified by:
      sdkFieldNameToField in interface SdkPojo
      Returns:
      The mapping between the field name and its corresponding field.