Compute¶

These dataclasses are used in the SDK to represent API requests and responses for services in the databricks.sdk.service.compute module.

class databricks.sdk.service.compute.AddInstanceProfile¶

instance_profile_arn: str¶: The AWS ARN of the instance profile to register with Databricks. This field is required.

iam_role_arn: str | None = None¶

The AWS IAM role ARN of the role associated with the instance profile. This field is required if your role name and instance profile name do not match and you want to use the instance profile with [Databricks SQL Serverless].

Otherwise, this field is optional.

[Databricks SQL Serverless]: https://docs.databricks.com/sql/admin/serverless.html

is_meta_instance_profile: bool | None = None¶: Boolean flag indicating whether the instance profile should only be used in credential passthrough scenarios. If true, it means the instance profile contains an meta IAM role which could assume a wide range of roles. Therefore it should always be used with authorization. This field is optional, the default value is false.

skip_validation: bool | None = None¶: By default, Databricks validates that it has sufficient permissions to launch instances with the instance profile. This validation uses AWS dry-run mode for the RunInstances API. If validation fails with an error message that does not indicate an IAM related permission issue, (e.g. “Your requested instance type is not supported in your requested availability zone”), you can pass this flag to skip the validation and forcibly add the instance profile.

as_dict() → dict¶: Serializes the AddInstanceProfile into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the AddInstanceProfile into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → AddInstanceProfile¶: Deserializes the AddInstanceProfile from a dictionary.

class databricks.sdk.service.compute.AddResponse¶

as_dict() → dict¶: Serializes the AddResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the AddResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → AddResponse¶: Deserializes the AddResponse from a dictionary.

class databricks.sdk.service.compute.Adlsgen2Info¶

A storage location in Adls Gen2

destination: str¶: abfss destination, e.g. abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>.

as_dict() → dict¶: Serializes the Adlsgen2Info into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Adlsgen2Info into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Adlsgen2Info¶: Deserializes the Adlsgen2Info from a dictionary.

class databricks.sdk.service.compute.AutoScale¶

max_workers: int | None = None¶: The maximum number of workers to which the cluster can scale up when overloaded. Note that max_workers must be strictly greater than min_workers.

min_workers: int | None = None¶: The minimum number of workers to which the cluster can scale down when underutilized. It is also the initial number of workers the cluster will have after creation.

as_dict() → dict¶: Serializes the AutoScale into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the AutoScale into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → AutoScale¶: Deserializes the AutoScale from a dictionary.

class databricks.sdk.service.compute.AwsAttributes¶

Attributes set during cluster creation which are related to Amazon Web Services.

availability: AwsAvailability | None = None¶

Availability type used for all subsequent nodes past the first_on_demand ones.

Note: If first_on_demand is zero, this availability type will be used for the entire cluster.

ebs_volume_count: int | None = None¶

The number of volumes launched for each instance. Users can choose up to 10 volumes. This feature is only enabled for supported node types. Legacy node types cannot specify custom EBS volumes. For node types with no instance store, at least one EBS volume needs to be specified; otherwise, cluster creation will fail.

These EBS volumes will be mounted at /ebs0, /ebs1, and etc. Instance store volumes will be mounted at /local_disk0, /local_disk1, and etc.

If EBS volumes are attached, Databricks will configure Spark to use only the EBS volumes for scratch storage because heterogenously sized scratch devices can lead to inefficient disk utilization. If no EBS volumes are attached, Databricks will configure Spark to use instance store volumes.

Please note that if EBS volumes are specified, then the Spark configuration spark.local.dir will be overridden.

ebs_volume_iops: int | None = None¶: If using gp3 volumes, what IOPS to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used.

ebs_volume_size: int | None = None¶: The size of each EBS volume (in GiB) launched for each instance. For general purpose SSD, this value must be within the range 100 - 4096. For throughput optimized HDD, this value must be within the range 500 - 4096.

ebs_volume_throughput: int | None = None¶: If using gp3 volumes, what throughput to use for the disk. If this is not set, the maximum performance of a gp2 volume with the same volume size will be used.

ebs_volume_type: EbsVolumeType | None = None¶: The type of EBS volumes that will be launched with this cluster.

first_on_demand: int | None = None¶: The first first_on_demand nodes of the cluster will be placed on on-demand instances. If this value is greater than 0, the cluster driver node in particular will be placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, first_on_demand nodes will be placed on on-demand instances and the remainder will be placed on availability instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster.

instance_profile_arn: str | None = None¶

Nodes for this cluster will only be placed on AWS instances with this instance profile. If ommitted, nodes will be placed on instances without an IAM instance profile. The instance profile must have previously been added to the Databricks environment by an account administrator.

This feature may only be available to certain customer plans.

spot_bid_price_percent: int | None = None¶: The bid price for AWS spot instances, as a percentage of the corresponding instance type’s on-demand price. For example, if this field is set to 50, and the cluster needs a new r3.xlarge spot instance, then the bid price is half of the price of on-demand r3.xlarge instances. Similarly, if this field is set to 200, the bid price is twice the price of on-demand r3.xlarge instances. If not specified, the default value is 100. When spot instances are requested for this cluster, only spot instances whose bid price percentage matches this field will be considered. Note that, for safety, we enforce this field to be no more than 10000.

zone_id: str | None = None¶

Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like “us-west-2a”. The provided availability zone must be in the same region as the Databricks deployment. For example, “us-west-2a” is not a valid zone id if the Databricks deployment resides in the “us-east-1” region. This is an optional field at cluster creation, and if not specified, a default zone will be used. If the zone specified is “auto”, will try to place cluster in a zone with high availability, and will retry placement in a different AZ if there is not enough capacity.

The list of available zones as well as the default value can be found by using the List Zones method.

as_dict() → dict¶: Serializes the AwsAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the AwsAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → AwsAttributes¶: Deserializes the AwsAttributes from a dictionary.

class databricks.sdk.service.compute.AwsAvailability¶

Availability type used for all subsequent nodes past the first_on_demand ones. Note: If first_on_demand is zero, this availability type will be used for the entire cluster.

ON_DEMAND = "ON_DEMAND"¶

SPOT = "SPOT"¶

SPOT_WITH_FALLBACK = "SPOT_WITH_FALLBACK"¶

class databricks.sdk.service.compute.AzureAttributes¶

Attributes set during cluster creation which are related to Microsoft Azure.

availability: AzureAvailability | None = None¶: Availability type used for all subsequent nodes past the first_on_demand ones. Note: If first_on_demand is zero, this availability type will be used for the entire cluster.

first_on_demand: int | None = None¶: The first first_on_demand nodes of the cluster will be placed on on-demand instances. This value should be greater than 0, to make sure the cluster driver node is placed on an on-demand instance. If this value is greater than or equal to the current cluster size, all nodes will be placed on on-demand instances. If this value is less than the current cluster size, first_on_demand nodes will be placed on on-demand instances and the remainder will be placed on availability instances. Note that this value does not affect cluster size and cannot currently be mutated over the lifetime of a cluster.

log_analytics_info: LogAnalyticsInfo | None = None¶: Defines values necessary to configure and run Azure Log Analytics agent

spot_bid_max_price: float | None = None¶: The max bid price to be used for Azure spot instances. The Max price for the bid cannot be higher than the on-demand price of the instance. If not specified, the default value is -1, which specifies that the instance cannot be evicted on the basis of price, and only on the basis of availability. Further, the value should > 0 or -1.

as_dict() → dict¶: Serializes the AzureAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the AzureAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → AzureAttributes¶: Deserializes the AzureAttributes from a dictionary.

class databricks.sdk.service.compute.AzureAvailability¶

Availability type used for all subsequent nodes past the first_on_demand ones. Note: If first_on_demand is zero, this availability type will be used for the entire cluster.

ON_DEMAND_AZURE = "ON_DEMAND_AZURE"¶

SPOT_AZURE = "SPOT_AZURE"¶

SPOT_WITH_FALLBACK_AZURE = "SPOT_WITH_FALLBACK_AZURE"¶

class databricks.sdk.service.compute.CancelCommand¶

cluster_id: str | None = None¶

command_id: str | None = None¶

context_id: str | None = None¶

as_dict() → dict¶: Serializes the CancelCommand into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CancelCommand into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CancelCommand¶: Deserializes the CancelCommand from a dictionary.

class databricks.sdk.service.compute.CancelResponse¶

as_dict() → dict¶: Serializes the CancelResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CancelResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CancelResponse¶: Deserializes the CancelResponse from a dictionary.

class databricks.sdk.service.compute.ChangeClusterOwner¶

cluster_id: str¶

owner_username: str¶: New owner of the cluster_id after this RPC.

as_dict() → dict¶: Serializes the ChangeClusterOwner into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ChangeClusterOwner into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ChangeClusterOwner¶: Deserializes the ChangeClusterOwner from a dictionary.

class databricks.sdk.service.compute.ChangeClusterOwnerResponse¶

as_dict() → dict¶: Serializes the ChangeClusterOwnerResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ChangeClusterOwnerResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ChangeClusterOwnerResponse¶: Deserializes the ChangeClusterOwnerResponse from a dictionary.

class databricks.sdk.service.compute.ClientsTypes¶

jobs: bool | None = None¶: With jobs set, the cluster can be used for jobs

notebooks: bool | None = None¶: With notebooks set, this cluster can be used for notebooks

as_dict() → dict¶: Serializes the ClientsTypes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClientsTypes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClientsTypes¶: Deserializes the ClientsTypes from a dictionary.

class databricks.sdk.service.compute.CloneCluster¶

source_cluster_id: str¶: The cluster that is being cloned.

as_dict() → dict¶: Serializes the CloneCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CloneCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CloneCluster¶: Deserializes the CloneCluster from a dictionary.

class databricks.sdk.service.compute.CloudProviderNodeInfo¶

status: List[CloudProviderNodeStatus] | None = None¶: Status as reported by the cloud provider

as_dict() → dict¶: Serializes the CloudProviderNodeInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CloudProviderNodeInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CloudProviderNodeInfo¶: Deserializes the CloudProviderNodeInfo from a dictionary.

class databricks.sdk.service.compute.CloudProviderNodeStatus¶

NOT_AVAILABLE_IN_REGION = "NOT_AVAILABLE_IN_REGION"¶

NOT_ENABLED_ON_SUBSCRIPTION = "NOT_ENABLED_ON_SUBSCRIPTION"¶

class databricks.sdk.service.compute.ClusterAccessControlRequest¶

group_name: str | None = None¶: name of the group

permission_level: ClusterPermissionLevel | None = None¶: Permission level

service_principal_name: str | None = None¶: application ID of a service principal

user_name: str | None = None¶: name of the user

as_dict() → dict¶: Serializes the ClusterAccessControlRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterAccessControlRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterAccessControlRequest¶: Deserializes the ClusterAccessControlRequest from a dictionary.

class databricks.sdk.service.compute.ClusterAccessControlResponse¶

all_permissions: List[ClusterPermission] | None = None¶: All permissions.

display_name: str | None = None¶: Display name of the user or service principal.

group_name: str | None = None¶: name of the group

service_principal_name: str | None = None¶: Name of the service principal.

user_name: str | None = None¶: name of the user

as_dict() → dict¶: Serializes the ClusterAccessControlResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterAccessControlResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterAccessControlResponse¶: Deserializes the ClusterAccessControlResponse from a dictionary.

class databricks.sdk.service.compute.ClusterAttributes¶

Common set of attributes set during cluster creation. These attributes cannot be changed over the lifetime of a cluster.

spark_version: str¶: The Spark version of the cluster, e.g. 3.3.x-scala2.11. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

autotermination_minutes: int | None = None¶: Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination.

aws_attributes: AwsAttributes | None = None¶: Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used.

azure_attributes: AzureAttributes | None = None¶: Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used.

cluster_log_conf: ClusterLogConf | None = None¶: The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every 5 mins. The destination of driver logs is $destination/$clusterId/driver, while the destination of executor logs is $destination/$clusterId/executor.

cluster_name: str | None = None¶: Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs.

custom_tags: Dict[str, str] | None = None¶

Additional tags for cluster resources. Databricks will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags
Clusters can only reuse cloud resources if the resources’ tags are a subset of the cluster

tags

data_security_mode: DataSecurityMode | None = None¶

Data security mode decides what data governance model to use when accessing data from a cluster.

The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER.

The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. *

LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

docker_image: DockerImage | None = None¶: Custom docker image BYOC

driver_instance_pool_id: str | None = None¶: The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned.

driver_node_type_id: str | None = None¶

The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as node_type_id defined above.

This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

enable_local_disk_encryption: bool | None = None¶: Whether to enable LUKS on cluster VMs’ local disks

gcp_attributes: GcpAttributes | None = None¶: Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used.

init_scripts: List[InitScriptInfo] | None = None¶: The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

instance_pool_id: str | None = None¶: The optional ID of the instance pool to which the cluster belongs.

is_single_node: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

When set to true, Databricks will automatically set single node related custom_tags, spark_conf, and num_workers

kind: Kind | None = None¶

The kind of compute described by this compute specification.

Depending on kind, different validations and default values will be applied.

Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD

By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW.

[simple form]: https://docs.databricks.com/compute/simple-form.html

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

runtime_engine: RuntimeEngine | None = None¶

Determines the cluster’s runtime engine, either standard or Photon.

This field is not compatible with legacy spark_version values that contain -photon-. Remove -photon- from the spark_version and set runtime_engine to PHOTON.

If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used.

single_user_name: str | None = None¶: Single user name if data_security_mode is SINGLE_USER

spark_conf: Dict[str, str] | None = None¶: An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions respectively.

spark_env_vars: Dict[str, str] | None = None¶

An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., export X=’Y’) while launching the driver and workers.

In order to specify an additional set of SPARK_DAEMON_JAVA_OPTS, we recommend appending them to $SPARK_DAEMON_JAVA_OPTS as shown in the example below. This ensures that all default databricks managed environmental variables are included as well.

Example Spark environment variables: {“SPARK_WORKER_MEMORY”: “28000m”, “SPARK_LOCAL_DIRS”: “/local_disk0”} or {“SPARK_DAEMON_JAVA_OPTS”: “$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true”}

ssh_public_keys: List[str] | None = None¶: SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

use_ml_runtime: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

effective_spark_version is determined by spark_version (DBR release), this field use_ml_runtime, and whether node_type_id is gpu node or not.

workload_type: WorkloadType | None = None¶: Cluster Attributes showing for clusters workload types.

as_dict() → dict¶: Serializes the ClusterAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterAttributes¶: Deserializes the ClusterAttributes from a dictionary.

class databricks.sdk.service.compute.ClusterCompliance¶

cluster_id: str¶: Canonical unique identifier for a cluster.

is_compliant: bool | None = None¶: Whether this cluster is in compliance with the latest version of its policy.

violations: Dict[str, str] | None = None¶: An object containing key-value mappings representing the first 200 policy validation errors. The keys indicate the path where the policy validation error is occurring. The values indicate an error message describing the policy validation error.

as_dict() → dict¶: Serializes the ClusterCompliance into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterCompliance into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterCompliance¶: Deserializes the ClusterCompliance from a dictionary.

class databricks.sdk.service.compute.ClusterDetails¶

Describes all of the metadata about a single Spark cluster in Databricks.

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

autotermination_minutes: int | None = None¶: Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination.

aws_attributes: AwsAttributes | None = None¶: Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used.

azure_attributes: AzureAttributes | None = None¶: Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used.

cluster_cores: float | None = None¶: Number of CPU cores available for this cluster. Note that this can be fractional, e.g. 7.5 cores, since certain node types are configured to share cores between Spark nodes on the same instance.

cluster_id: str | None = None¶: Canonical identifier for the cluster. This id is retained during cluster restarts and resizes, while each new cluster has a globally unique id.

cluster_log_conf: ClusterLogConf | None = None¶: The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every 5 mins. The destination of driver logs is $destination/$clusterId/driver, while the destination of executor logs is $destination/$clusterId/executor.

cluster_log_status: LogSyncStatus | None = None¶: Cluster log delivery status.

cluster_memory_mb: int | None = None¶: Total amount of cluster memory, in megabytes

cluster_name: str | None = None¶: Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs.

cluster_source: ClusterSource | None = None¶: Determines whether the cluster was created by a user through the UI, created by the Databricks Jobs Scheduler, or through an API request.

creator_user_name: str | None = None¶: Creator user name. The field won’t be included in the response if the user has already been deleted.

custom_tags: Dict[str, str] | None = None¶

Additional tags for cluster resources. Databricks will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags
Clusters can only reuse cloud resources if the resources’ tags are a subset of the cluster

tags

data_security_mode: DataSecurityMode | None = None¶

Data security mode decides what data governance model to use when accessing data from a cluster.

The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER.

The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. *

LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

default_tags: Dict[str, str] | None = None¶

Tags that are added by Databricks regardless of any custom_tags, including:

Vendor: Databricks
Creator: <username_of_creator>
ClusterName: <name_of_cluster>
ClusterId: <id_of_cluster>
Name: <Databricks internal use>

docker_image: DockerImage | None = None¶: Custom docker image BYOC

driver: SparkNode | None = None¶: Node on which the Spark driver resides. The driver node contains the Spark master and the Databricks application that manages the per-notebook Spark REPLs.

driver_instance_pool_id: str | None = None¶: The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned.

driver_node_type_id: str | None = None¶

The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as node_type_id defined above.

This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

enable_local_disk_encryption: bool | None = None¶: Whether to enable LUKS on cluster VMs’ local disks

executors: List[SparkNode] | None = None¶: Nodes on which the Spark executors reside.

gcp_attributes: GcpAttributes | None = None¶: Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used.

init_scripts: List[InitScriptInfo] | None = None¶: The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

instance_pool_id: str | None = None¶: The optional ID of the instance pool to which the cluster belongs.

is_single_node: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

When set to true, Databricks will automatically set single node related custom_tags, spark_conf, and num_workers

jdbc_port: int | None = None¶: Port on which Spark JDBC server is listening, in the driver nod. No service will be listeningon on this port in executor nodes.

kind: Kind | None = None¶

The kind of compute described by this compute specification.

Depending on kind, different validations and default values will be applied.

Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD

By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW.

[simple form]: https://docs.databricks.com/compute/simple-form.html

last_restarted_time: int | None = None¶: the timestamp that the cluster was started/restarted

last_state_loss_time: int | None = None¶: Time when the cluster driver last lost its state (due to a restart or driver failure).

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

runtime_engine: RuntimeEngine | None = None¶

Determines the cluster’s runtime engine, either standard or Photon.

This field is not compatible with legacy spark_version values that contain -photon-. Remove -photon- from the spark_version and set runtime_engine to PHOTON.

If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used.

single_user_name: str | None = None¶: Single user name if data_security_mode is SINGLE_USER

spark_conf: Dict[str, str] | None = None¶: An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions respectively.

spark_context_id: int | None = None¶: A canonical SparkContext identifier. This value does change when the Spark driver restarts. The pair (cluster_id, spark_context_id) is a globally unique identifier over all Spark contexts.

spark_env_vars: Dict[str, str] | None = None¶

An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., export X=’Y’) while launching the driver and workers.

In order to specify an additional set of SPARK_DAEMON_JAVA_OPTS, we recommend appending them to $SPARK_DAEMON_JAVA_OPTS as shown in the example below. This ensures that all default databricks managed environmental variables are included as well.

Example Spark environment variables: {“SPARK_WORKER_MEMORY”: “28000m”, “SPARK_LOCAL_DIRS”: “/local_disk0”} or {“SPARK_DAEMON_JAVA_OPTS”: “$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true”}

spark_version: str | None = None¶: The Spark version of the cluster, e.g. 3.3.x-scala2.11. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

spec: ClusterSpec | None = None¶: The spec contains a snapshot of the latest user specified settings that were used to create/edit the cluster. Note: not included in the response of the ListClusters API.

ssh_public_keys: List[str] | None = None¶: SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

start_time: int | None = None¶: Time (in epoch milliseconds) when the cluster creation request was received (when the cluster entered a PENDING state).

state: State | None = None¶: Current state of the cluster.

state_message: str | None = None¶: A message associated with the most recent state transition (e.g., the reason why the cluster entered a TERMINATED state).

terminated_time: int | None = None¶: Time (in epoch milliseconds) when the cluster was terminated, if applicable.

termination_reason: TerminationReason | None = None¶: Information about why the cluster was terminated. This field only appears when the cluster is in a TERMINATING or TERMINATED state.

use_ml_runtime: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

effective_spark_version is determined by spark_version (DBR release), this field use_ml_runtime, and whether node_type_id is gpu node or not.

workload_type: WorkloadType | None = None¶: Cluster Attributes showing for clusters workload types.

as_dict() → dict¶: Serializes the ClusterDetails into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterDetails into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterDetails¶: Deserializes the ClusterDetails from a dictionary.

class databricks.sdk.service.compute.ClusterEvent¶

cluster_id: str¶

data_plane_event_details: DataPlaneEventDetails | None = None¶

details: EventDetails | None = None¶

timestamp: int | None = None¶: The timestamp when the event occurred, stored as the number of milliseconds since the Unix epoch. If not provided, this will be assigned by the Timeline service.

type: EventType | None = None¶

as_dict() → dict¶: Serializes the ClusterEvent into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterEvent into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterEvent¶: Deserializes the ClusterEvent from a dictionary.

class databricks.sdk.service.compute.ClusterLibraryStatuses¶

cluster_id: str | None = None¶: Unique identifier for the cluster.

library_statuses: List[LibraryFullStatus] | None = None¶: Status of all libraries on the cluster.

as_dict() → dict¶: Serializes the ClusterLibraryStatuses into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterLibraryStatuses into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterLibraryStatuses¶: Deserializes the ClusterLibraryStatuses from a dictionary.

class databricks.sdk.service.compute.ClusterLogConf¶

Cluster log delivery config

dbfs: DbfsStorageInfo | None = None¶: destination needs to be provided. e.g. { “dbfs” : { “destination” : “dbfs:/home/cluster_log” } }

s3: S3StorageInfo | None = None¶: destination and either the region or endpoint need to be provided. e.g. { “s3”: { “destination” : “s3://cluster_log_bucket/prefix”, “region” : “us-west-2” } } Cluster iam role is used to access s3, please make sure the cluster iam role in instance_profile_arn has permission to write data to the s3 destination.

volumes: VolumesStorageInfo | None = None¶: destination needs to be provided, e.g. { “volumes”: { “destination”: “/Volumes/catalog/schema/volume/cluster_log” } }

as_dict() → dict¶: Serializes the ClusterLogConf into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterLogConf into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterLogConf¶: Deserializes the ClusterLogConf from a dictionary.

class databricks.sdk.service.compute.ClusterPermission¶

inherited: bool | None = None¶

inherited_from_object: List[str] | None = None¶

permission_level: ClusterPermissionLevel | None = None¶: Permission level

as_dict() → dict¶: Serializes the ClusterPermission into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPermission into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPermission¶: Deserializes the ClusterPermission from a dictionary.

class databricks.sdk.service.compute.ClusterPermissionLevel¶

Permission level

CAN_ATTACH_TO = "CAN_ATTACH_TO"¶

CAN_MANAGE = "CAN_MANAGE"¶

CAN_RESTART = "CAN_RESTART"¶

class databricks.sdk.service.compute.ClusterPermissions¶

access_control_list: List[ClusterAccessControlResponse] | None = None¶

object_id: str | None = None¶

object_type: str | None = None¶

as_dict() → dict¶: Serializes the ClusterPermissions into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPermissions into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPermissions¶: Deserializes the ClusterPermissions from a dictionary.

class databricks.sdk.service.compute.ClusterPermissionsDescription¶

description: str | None = None¶

permission_level: ClusterPermissionLevel | None = None¶: Permission level

as_dict() → dict¶: Serializes the ClusterPermissionsDescription into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPermissionsDescription into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPermissionsDescription¶: Deserializes the ClusterPermissionsDescription from a dictionary.

class databricks.sdk.service.compute.ClusterPermissionsRequest¶

access_control_list: List[ClusterAccessControlRequest] | None = None¶

cluster_id: str | None = None¶: The cluster for which to get or manage permissions.

as_dict() → dict¶: Serializes the ClusterPermissionsRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPermissionsRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPermissionsRequest¶: Deserializes the ClusterPermissionsRequest from a dictionary.

class databricks.sdk.service.compute.ClusterPolicyAccessControlRequest¶

group_name: str | None = None¶: name of the group

permission_level: ClusterPolicyPermissionLevel | None = None¶: Permission level

service_principal_name: str | None = None¶: application ID of a service principal

user_name: str | None = None¶: name of the user

as_dict() → dict¶: Serializes the ClusterPolicyAccessControlRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPolicyAccessControlRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPolicyAccessControlRequest¶: Deserializes the ClusterPolicyAccessControlRequest from a dictionary.

class databricks.sdk.service.compute.ClusterPolicyAccessControlResponse¶

all_permissions: List[ClusterPolicyPermission] | None = None¶: All permissions.

display_name: str | None = None¶: Display name of the user or service principal.

group_name: str | None = None¶: name of the group

service_principal_name: str | None = None¶: Name of the service principal.

user_name: str | None = None¶: name of the user

as_dict() → dict¶: Serializes the ClusterPolicyAccessControlResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPolicyAccessControlResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPolicyAccessControlResponse¶: Deserializes the ClusterPolicyAccessControlResponse from a dictionary.

class databricks.sdk.service.compute.ClusterPolicyPermission¶

inherited: bool | None = None¶

inherited_from_object: List[str] | None = None¶

permission_level: ClusterPolicyPermissionLevel | None = None¶: Permission level

as_dict() → dict¶: Serializes the ClusterPolicyPermission into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPolicyPermission into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPolicyPermission¶: Deserializes the ClusterPolicyPermission from a dictionary.

class databricks.sdk.service.compute.ClusterPolicyPermissionLevel¶

Permission level

CAN_USE = "CAN_USE"¶

class databricks.sdk.service.compute.ClusterPolicyPermissions¶

access_control_list: List[ClusterPolicyAccessControlResponse] | None = None¶

object_id: str | None = None¶

object_type: str | None = None¶

as_dict() → dict¶: Serializes the ClusterPolicyPermissions into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPolicyPermissions into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPolicyPermissions¶: Deserializes the ClusterPolicyPermissions from a dictionary.

class databricks.sdk.service.compute.ClusterPolicyPermissionsDescription¶

description: str | None = None¶

permission_level: ClusterPolicyPermissionLevel | None = None¶: Permission level

as_dict() → dict¶: Serializes the ClusterPolicyPermissionsDescription into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPolicyPermissionsDescription into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPolicyPermissionsDescription¶: Deserializes the ClusterPolicyPermissionsDescription from a dictionary.

class databricks.sdk.service.compute.ClusterPolicyPermissionsRequest¶

access_control_list: List[ClusterPolicyAccessControlRequest] | None = None¶

cluster_policy_id: str | None = None¶: The cluster policy for which to get or manage permissions.

as_dict() → dict¶: Serializes the ClusterPolicyPermissionsRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterPolicyPermissionsRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterPolicyPermissionsRequest¶: Deserializes the ClusterPolicyPermissionsRequest from a dictionary.

class databricks.sdk.service.compute.ClusterSettingsChange¶

Represents a change to the cluster settings required for the cluster to become compliant with its policy.

field: str | None = None¶: The field where this change would be made.

new_value: str | None = None¶: The new value of this field after enforcing policy compliance (either a number, a boolean, or a string) converted to a string. This is intended to be read by a human. The typed new value of this field can be retrieved by reading the settings field in the API response.

previous_value: str | None = None¶: The previous value of this field before enforcing policy compliance (either a number, a boolean, or a string) converted to a string. This is intended to be read by a human. The type of the field can be retrieved by reading the settings field in the API response.

as_dict() → dict¶: Serializes the ClusterSettingsChange into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterSettingsChange into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterSettingsChange¶: Deserializes the ClusterSettingsChange from a dictionary.

class databricks.sdk.service.compute.ClusterSize¶

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

as_dict() → dict¶: Serializes the ClusterSize into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterSize into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterSize¶: Deserializes the ClusterSize from a dictionary.

class databricks.sdk.service.compute.ClusterSource¶

Determines whether the cluster was created by a user through the UI, created by the Databricks Jobs Scheduler, or through an API request. This is the same as cluster_creator, but read only.

API = "API"¶

JOB = "JOB"¶

MODELS = "MODELS"¶

PIPELINE = "PIPELINE"¶

PIPELINE_MAINTENANCE = "PIPELINE_MAINTENANCE"¶

SQL = "SQL"¶

UI = "UI"¶

class databricks.sdk.service.compute.ClusterSpec¶

Contains a snapshot of the latest user specified settings that were used to create/edit the cluster.

apply_policy_default_values: bool | None = None¶: When set to true, fixed and default values from the policy will be used for fields that are omitted. When set to false, only fixed values from the policy will be applied.

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

autotermination_minutes: int | None = None¶: Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination.

aws_attributes: AwsAttributes | None = None¶: Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used.

azure_attributes: AzureAttributes | None = None¶: Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used.

cluster_log_conf: ClusterLogConf | None = None¶: The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every 5 mins. The destination of driver logs is $destination/$clusterId/driver, while the destination of executor logs is $destination/$clusterId/executor.

cluster_name: str | None = None¶: Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs.

custom_tags: Dict[str, str] | None = None¶

Additional tags for cluster resources. Databricks will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags
Clusters can only reuse cloud resources if the resources’ tags are a subset of the cluster

tags

data_security_mode: DataSecurityMode | None = None¶

Data security mode decides what data governance model to use when accessing data from a cluster.

The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER.

The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. *

LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

docker_image: DockerImage | None = None¶: Custom docker image BYOC

driver_instance_pool_id: str | None = None¶: The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned.

driver_node_type_id: str | None = None¶

The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as node_type_id defined above.

This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

enable_local_disk_encryption: bool | None = None¶: Whether to enable LUKS on cluster VMs’ local disks

gcp_attributes: GcpAttributes | None = None¶: Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used.

init_scripts: List[InitScriptInfo] | None = None¶: The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

instance_pool_id: str | None = None¶: The optional ID of the instance pool to which the cluster belongs.

is_single_node: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

When set to true, Databricks will automatically set single node related custom_tags, spark_conf, and num_workers

kind: Kind | None = None¶

The kind of compute described by this compute specification.

Depending on kind, different validations and default values will be applied.

Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD

By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW.

[simple form]: https://docs.databricks.com/compute/simple-form.html

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

runtime_engine: RuntimeEngine | None = None¶

Determines the cluster’s runtime engine, either standard or Photon.

This field is not compatible with legacy spark_version values that contain -photon-. Remove -photon- from the spark_version and set runtime_engine to PHOTON.

If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used.

single_user_name: str | None = None¶: Single user name if data_security_mode is SINGLE_USER

spark_conf: Dict[str, str] | None = None¶: An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions respectively.

spark_env_vars: Dict[str, str] | None = None¶

An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., export X=’Y’) while launching the driver and workers.

In order to specify an additional set of SPARK_DAEMON_JAVA_OPTS, we recommend appending them to $SPARK_DAEMON_JAVA_OPTS as shown in the example below. This ensures that all default databricks managed environmental variables are included as well.

Example Spark environment variables: {“SPARK_WORKER_MEMORY”: “28000m”, “SPARK_LOCAL_DIRS”: “/local_disk0”} or {“SPARK_DAEMON_JAVA_OPTS”: “$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true”}

spark_version: str | None = None¶: The Spark version of the cluster, e.g. 3.3.x-scala2.11. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

ssh_public_keys: List[str] | None = None¶: SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

use_ml_runtime: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

effective_spark_version is determined by spark_version (DBR release), this field use_ml_runtime, and whether node_type_id is gpu node or not.

workload_type: WorkloadType | None = None¶: Cluster Attributes showing for clusters workload types.

as_dict() → dict¶: Serializes the ClusterSpec into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ClusterSpec into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ClusterSpec¶: Deserializes the ClusterSpec from a dictionary.

class databricks.sdk.service.compute.Command¶

cluster_id: str | None = None¶: Running cluster id

command: str | None = None¶: Executable code

context_id: str | None = None¶: Running context id

language: Language | None = None¶

as_dict() → dict¶: Serializes the Command into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Command into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Command¶: Deserializes the Command from a dictionary.

class databricks.sdk.service.compute.CommandStatus¶

CANCELLED = "CANCELLED"¶

CANCELLING = "CANCELLING"¶

ERROR = "ERROR"¶

FINISHED = "FINISHED"¶

QUEUED = "QUEUED"¶

RUNNING = "RUNNING"¶

class databricks.sdk.service.compute.CommandStatusResponse¶

id: str | None = None¶

results: Results | None = None¶

status: CommandStatus | None = None¶

as_dict() → dict¶: Serializes the CommandStatusResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CommandStatusResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CommandStatusResponse¶: Deserializes the CommandStatusResponse from a dictionary.

class databricks.sdk.service.compute.ContextStatus¶

ERROR = "ERROR"¶

PENDING = "PENDING"¶

RUNNING = "RUNNING"¶

class databricks.sdk.service.compute.ContextStatusResponse¶

id: str | None = None¶

status: ContextStatus | None = None¶

as_dict() → dict¶: Serializes the ContextStatusResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ContextStatusResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ContextStatusResponse¶: Deserializes the ContextStatusResponse from a dictionary.

class databricks.sdk.service.compute.CreateCluster¶

spark_version: str¶: The Spark version of the cluster, e.g. 3.3.x-scala2.11. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

apply_policy_default_values: bool | None = None¶: When set to true, fixed and default values from the policy will be used for fields that are omitted. When set to false, only fixed values from the policy will be applied.

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

autotermination_minutes: int | None = None¶: Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination.

aws_attributes: AwsAttributes | None = None¶: Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used.

azure_attributes: AzureAttributes | None = None¶: Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used.

clone_from: CloneCluster | None = None¶: When specified, this clones libraries from a source cluster during the creation of a new cluster.

cluster_log_conf: ClusterLogConf | None = None¶: The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every 5 mins. The destination of driver logs is $destination/$clusterId/driver, while the destination of executor logs is $destination/$clusterId/executor.

cluster_name: str | None = None¶: Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs.

custom_tags: Dict[str, str] | None = None¶

Additional tags for cluster resources. Databricks will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags
Clusters can only reuse cloud resources if the resources’ tags are a subset of the cluster

tags

data_security_mode: DataSecurityMode | None = None¶

Data security mode decides what data governance model to use when accessing data from a cluster.

The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER.

The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. *

LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

docker_image: DockerImage | None = None¶: Custom docker image BYOC

driver_instance_pool_id: str | None = None¶: The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned.

driver_node_type_id: str | None = None¶

The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as node_type_id defined above.

This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

enable_local_disk_encryption: bool | None = None¶: Whether to enable LUKS on cluster VMs’ local disks

gcp_attributes: GcpAttributes | None = None¶: Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used.

init_scripts: List[InitScriptInfo] | None = None¶: The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

instance_pool_id: str | None = None¶: The optional ID of the instance pool to which the cluster belongs.

is_single_node: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

When set to true, Databricks will automatically set single node related custom_tags, spark_conf, and num_workers

kind: Kind | None = None¶

The kind of compute described by this compute specification.

Depending on kind, different validations and default values will be applied.

Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD

By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW.

[simple form]: https://docs.databricks.com/compute/simple-form.html

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

runtime_engine: RuntimeEngine | None = None¶

Determines the cluster’s runtime engine, either standard or Photon.

This field is not compatible with legacy spark_version values that contain -photon-. Remove -photon- from the spark_version and set runtime_engine to PHOTON.

If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used.

single_user_name: str | None = None¶: Single user name if data_security_mode is SINGLE_USER

spark_conf: Dict[str, str] | None = None¶: An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions respectively.

spark_env_vars: Dict[str, str] | None = None¶

An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., export X=’Y’) while launching the driver and workers.

In order to specify an additional set of SPARK_DAEMON_JAVA_OPTS, we recommend appending them to $SPARK_DAEMON_JAVA_OPTS as shown in the example below. This ensures that all default databricks managed environmental variables are included as well.

Example Spark environment variables: {“SPARK_WORKER_MEMORY”: “28000m”, “SPARK_LOCAL_DIRS”: “/local_disk0”} or {“SPARK_DAEMON_JAVA_OPTS”: “$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true”}

ssh_public_keys: List[str] | None = None¶: SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

use_ml_runtime: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

effective_spark_version is determined by spark_version (DBR release), this field use_ml_runtime, and whether node_type_id is gpu node or not.

workload_type: WorkloadType | None = None¶: Cluster Attributes showing for clusters workload types.

as_dict() → dict¶: Serializes the CreateCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreateCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreateCluster¶: Deserializes the CreateCluster from a dictionary.

class databricks.sdk.service.compute.CreateClusterResponse¶

cluster_id: str | None = None¶

as_dict() → dict¶: Serializes the CreateClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreateClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreateClusterResponse¶: Deserializes the CreateClusterResponse from a dictionary.

class databricks.sdk.service.compute.CreateContext¶

cluster_id: str | None = None¶: Running cluster id

language: Language | None = None¶

as_dict() → dict¶: Serializes the CreateContext into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreateContext into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreateContext¶: Deserializes the CreateContext from a dictionary.

class databricks.sdk.service.compute.CreateInstancePool¶

instance_pool_name: str¶: Pool name requested by the user. Pool name must be unique. Length must be between 1 and 100 characters.

node_type_id: str¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

aws_attributes: InstancePoolAwsAttributes | None = None¶: Attributes related to instance pools running on Amazon Web Services. If not specified at pool creation, a set of default values will be used.

azure_attributes: InstancePoolAzureAttributes | None = None¶: Attributes related to instance pools running on Azure. If not specified at pool creation, a set of default values will be used.

custom_tags: Dict[str, str] | None = None¶

Additional tags for pool resources. Databricks will tag all pool resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags

disk_spec: DiskSpec | None = None¶: Defines the specification of the disks that will be attached to all spark containers.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this instances in this pool will dynamically acquire additional disk space when its Spark workers are running low on disk space. In AWS, this feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

gcp_attributes: InstancePoolGcpAttributes | None = None¶: Attributes related to instance pools running on Google Cloud Platform. If not specified at pool creation, a set of default values will be used.

idle_instance_autotermination_minutes: int | None = None¶: Automatically terminates the extra instances in the pool cache after they are inactive for this time in minutes if min_idle_instances requirement is already met. If not set, the extra pool instances will be automatically terminated after a default timeout. If specified, the threshold must be between 0 and 10000 minutes. Users can also set this value to 0 to instantly remove idle instances from the cache if min cache size could still hold.

max_capacity: int | None = None¶: Maximum number of outstanding instances to keep in the pool, including both instances used by clusters and idle instances. Clusters that require further instance provisioning will fail during upsize requests.

min_idle_instances: int | None = None¶: Minimum number of idle instances to keep in the instance pool

preloaded_docker_images: List[DockerImage] | None = None¶: Custom Docker Image BYOC

preloaded_spark_versions: List[str] | None = None¶: A list containing at most one preloaded Spark image version for the pool. Pool-backed clusters started with the preloaded Spark version will start faster. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

as_dict() → dict¶: Serializes the CreateInstancePool into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreateInstancePool into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreateInstancePool¶: Deserializes the CreateInstancePool from a dictionary.

class databricks.sdk.service.compute.CreateInstancePoolResponse¶

instance_pool_id: str | None = None¶: The ID of the created instance pool.

as_dict() → dict¶: Serializes the CreateInstancePoolResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreateInstancePoolResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreateInstancePoolResponse¶: Deserializes the CreateInstancePoolResponse from a dictionary.

class databricks.sdk.service.compute.CreatePolicy¶

definition: str | None = None¶

Policy definition document expressed in [Databricks Cluster Policy Definition Language].

[Databricks Cluster Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

description: str | None = None¶: Additional human-readable description of the cluster policy.

libraries: List[Library] | None = None¶: A list of libraries to be installed on the next cluster restart that uses this policy. The maximum number of libraries is 500.

max_clusters_per_user: int | None = None¶: Max number of clusters per user that can be active using this policy. If not present, there is no max limit.

name: str | None = None¶: Cluster Policy name requested by the user. This has to be unique. Length must be between 1 and 100 characters.

policy_family_definition_overrides: str | None = None¶

Policy definition JSON document expressed in [Databricks Policy Definition Language]. The JSON document must be passed as a string and cannot be embedded in the requests.

You can use this to customize the policy definition inherited from the policy family. Policy rules specified here are merged into the inherited policy definition.

[Databricks Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

policy_family_id: str | None = None¶

ID of the policy family. The cluster policy’s policy definition inherits the policy family’s policy definition.

Cannot be used with definition. Use policy_family_definition_overrides instead to customize the policy definition.

as_dict() → dict¶: Serializes the CreatePolicy into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreatePolicy into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreatePolicy¶: Deserializes the CreatePolicy from a dictionary.

class databricks.sdk.service.compute.CreatePolicyResponse¶

policy_id: str | None = None¶: Canonical unique identifier for the cluster policy.

as_dict() → dict¶: Serializes the CreatePolicyResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreatePolicyResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreatePolicyResponse¶: Deserializes the CreatePolicyResponse from a dictionary.

class databricks.sdk.service.compute.CreateResponse¶

script_id: str | None = None¶: The global init script ID.

as_dict() → dict¶: Serializes the CreateResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CreateResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CreateResponse¶: Deserializes the CreateResponse from a dictionary.

class databricks.sdk.service.compute.Created¶

id: str | None = None¶

as_dict() → dict¶: Serializes the Created into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Created into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Created¶: Deserializes the Created from a dictionary.

class databricks.sdk.service.compute.CustomPolicyTag¶

key: str¶: The key of the tag. - Must be unique among all custom tags of the same policy - Cannot be “budget-policy-name”, “budget-policy-id” or “budget-policy-resolution-result” - these tags are preserved.

value: str | None = None¶: The value of the tag.

as_dict() → dict¶: Serializes the CustomPolicyTag into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the CustomPolicyTag into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → CustomPolicyTag¶: Deserializes the CustomPolicyTag from a dictionary.

class databricks.sdk.service.compute.DataPlaneEventDetails¶

event_type: DataPlaneEventDetailsEventType | None = None¶

executor_failures: int | None = None¶

host_id: str | None = None¶

timestamp: int | None = None¶

as_dict() → dict¶: Serializes the DataPlaneEventDetails into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DataPlaneEventDetails into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DataPlaneEventDetails¶: Deserializes the DataPlaneEventDetails from a dictionary.

class databricks.sdk.service.compute.DataPlaneEventDetailsEventType¶

NODE_BLACKLISTED = "NODE_BLACKLISTED"¶

NODE_EXCLUDED_DECOMMISSIONED = "NODE_EXCLUDED_DECOMMISSIONED"¶

class databricks.sdk.service.compute.DataSecurityMode¶

Data security mode decides what data governance model to use when accessing data from a cluster. The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER. The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited. The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions: * LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. * LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

DATA_SECURITY_MODE_AUTO = "DATA_SECURITY_MODE_AUTO"¶

DATA_SECURITY_MODE_DEDICATED = "DATA_SECURITY_MODE_DEDICATED"¶

DATA_SECURITY_MODE_STANDARD = "DATA_SECURITY_MODE_STANDARD"¶

LEGACY_PASSTHROUGH = "LEGACY_PASSTHROUGH"¶

LEGACY_SINGLE_USER = "LEGACY_SINGLE_USER"¶

LEGACY_SINGLE_USER_STANDARD = "LEGACY_SINGLE_USER_STANDARD"¶

LEGACY_TABLE_ACL = "LEGACY_TABLE_ACL"¶

NONE = "NONE"¶

SINGLE_USER = "SINGLE_USER"¶

USER_ISOLATION = "USER_ISOLATION"¶

class databricks.sdk.service.compute.DbfsStorageInfo¶

A storage location in DBFS

destination: str¶: dbfs destination, e.g. dbfs:/my/path

as_dict() → dict¶: Serializes the DbfsStorageInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DbfsStorageInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DbfsStorageInfo¶: Deserializes the DbfsStorageInfo from a dictionary.

class databricks.sdk.service.compute.DeleteCluster¶

cluster_id: str¶: The cluster to be terminated.

as_dict() → dict¶: Serializes the DeleteCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeleteCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeleteCluster¶: Deserializes the DeleteCluster from a dictionary.

class databricks.sdk.service.compute.DeleteClusterResponse¶

as_dict() → dict¶: Serializes the DeleteClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeleteClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeleteClusterResponse¶: Deserializes the DeleteClusterResponse from a dictionary.

class databricks.sdk.service.compute.DeleteInstancePool¶

instance_pool_id: str¶: The instance pool to be terminated.

as_dict() → dict¶: Serializes the DeleteInstancePool into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeleteInstancePool into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeleteInstancePool¶: Deserializes the DeleteInstancePool from a dictionary.

class databricks.sdk.service.compute.DeleteInstancePoolResponse¶

as_dict() → dict¶: Serializes the DeleteInstancePoolResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeleteInstancePoolResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeleteInstancePoolResponse¶: Deserializes the DeleteInstancePoolResponse from a dictionary.

class databricks.sdk.service.compute.DeletePolicy¶

policy_id: str¶: The ID of the policy to delete.

as_dict() → dict¶: Serializes the DeletePolicy into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeletePolicy into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeletePolicy¶: Deserializes the DeletePolicy from a dictionary.

class databricks.sdk.service.compute.DeletePolicyResponse¶

as_dict() → dict¶: Serializes the DeletePolicyResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeletePolicyResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeletePolicyResponse¶: Deserializes the DeletePolicyResponse from a dictionary.

class databricks.sdk.service.compute.DeleteResponse¶

as_dict() → dict¶: Serializes the DeleteResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DeleteResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DeleteResponse¶: Deserializes the DeleteResponse from a dictionary.

class databricks.sdk.service.compute.DestroyContext¶

cluster_id: str¶

context_id: str¶

as_dict() → dict¶: Serializes the DestroyContext into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DestroyContext into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DestroyContext¶: Deserializes the DestroyContext from a dictionary.

class databricks.sdk.service.compute.DestroyResponse¶

as_dict() → dict¶: Serializes the DestroyResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DestroyResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DestroyResponse¶: Deserializes the DestroyResponse from a dictionary.

class databricks.sdk.service.compute.DiskSpec¶

Describes the disks that are launched for each instance in the spark cluster. For example, if the cluster has 3 instances, each instance is configured to launch 2 disks, 100 GiB each, then Databricks will launch a total of 6 disks, 100 GiB each, for this cluster.

disk_count: int | None = None¶

The number of disks launched for each instance: - This feature is only enabled for supported node types. - Users can choose up to the limit of the disks supported by the node type. - For node types with no OS disk, at least one disk must be specified; otherwise, cluster creation will fail.

If disks are attached, Databricks will configure Spark to use only the disks for scratch storage, because heterogenously sized scratch devices can lead to inefficient disk utilization. If no disks are attached, Databricks will configure Spark to use instance store disks.

Note: If disks are specified, then the Spark configuration spark.local.dir will be overridden.

Disks will be mounted at: - For AWS: /ebs0, /ebs1, and etc. - For Azure: /remote_volume0, /remote_volume1, and etc.

disk_iops: int | None = None¶

disk_size: int | None = None¶

The size of each disk (in GiB) launched for each instance. Values must fall into the supported range for a particular instance type.

For AWS: - General Purpose SSD: 100 - 4096 GiB - Throughput Optimized HDD: 500 - 4096 GiB

For Azure: - Premium LRS (SSD): 1 - 1023 GiB - Standard LRS (HDD): 1- 1023 GiB

disk_throughput: int | None = None¶

disk_type: DiskType | None = None¶: The type of disks that will be launched with this cluster.

as_dict() → dict¶: Serializes the DiskSpec into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DiskSpec into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DiskSpec¶: Deserializes the DiskSpec from a dictionary.

class databricks.sdk.service.compute.DiskType¶

Describes the disk type.

azure_disk_volume_type: DiskTypeAzureDiskVolumeType | None = None¶: All Azure Disk types that Databricks supports. See https://docs.microsoft.com/en-us/azure/storage/storage-about-disks-and-vhds-linux#types-of-disks

ebs_volume_type: DiskTypeEbsVolumeType | None = None¶: All EBS volume types that Databricks supports. See https://aws.amazon.com/ebs/details/ for details.

as_dict() → dict¶: Serializes the DiskType into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DiskType into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DiskType¶: Deserializes the DiskType from a dictionary.

class databricks.sdk.service.compute.DiskTypeAzureDiskVolumeType¶

All Azure Disk types that Databricks supports. See https://docs.microsoft.com/en-us/azure/storage/storage-about-disks-and-vhds-linux#types-of-disks

PREMIUM_LRS = "PREMIUM_LRS"¶

STANDARD_LRS = "STANDARD_LRS"¶

class databricks.sdk.service.compute.DiskTypeEbsVolumeType¶

All EBS volume types that Databricks supports. See https://aws.amazon.com/ebs/details/ for details.

GENERAL_PURPOSE_SSD = "GENERAL_PURPOSE_SSD"¶

THROUGHPUT_OPTIMIZED_HDD = "THROUGHPUT_OPTIMIZED_HDD"¶

class databricks.sdk.service.compute.DockerBasicAuth¶

password: str | None = None¶: Password of the user

username: str | None = None¶: Name of the user

as_dict() → dict¶: Serializes the DockerBasicAuth into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DockerBasicAuth into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DockerBasicAuth¶: Deserializes the DockerBasicAuth from a dictionary.

class databricks.sdk.service.compute.DockerImage¶

basic_auth: DockerBasicAuth | None = None¶: Basic auth with username and password

url: str | None = None¶: URL of the docker image.

as_dict() → dict¶: Serializes the DockerImage into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the DockerImage into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → DockerImage¶: Deserializes the DockerImage from a dictionary.

class databricks.sdk.service.compute.EbsVolumeType¶

All EBS volume types that Databricks supports. See https://aws.amazon.com/ebs/details/ for details.

GENERAL_PURPOSE_SSD = "GENERAL_PURPOSE_SSD"¶

THROUGHPUT_OPTIMIZED_HDD = "THROUGHPUT_OPTIMIZED_HDD"¶

class databricks.sdk.service.compute.EditCluster¶

cluster_id: str¶: ID of the cluster

spark_version: str¶: The Spark version of the cluster, e.g. 3.3.x-scala2.11. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

apply_policy_default_values: bool | None = None¶: When set to true, fixed and default values from the policy will be used for fields that are omitted. When set to false, only fixed values from the policy will be applied.

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

autotermination_minutes: int | None = None¶: Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination.

aws_attributes: AwsAttributes | None = None¶: Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used.

azure_attributes: AzureAttributes | None = None¶: Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used.

cluster_log_conf: ClusterLogConf | None = None¶: The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every 5 mins. The destination of driver logs is $destination/$clusterId/driver, while the destination of executor logs is $destination/$clusterId/executor.

cluster_name: str | None = None¶: Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs.

custom_tags: Dict[str, str] | None = None¶

Additional tags for cluster resources. Databricks will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags
Clusters can only reuse cloud resources if the resources’ tags are a subset of the cluster

tags

data_security_mode: DataSecurityMode | None = None¶

Data security mode decides what data governance model to use when accessing data from a cluster.

The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER.

The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. *

LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

docker_image: DockerImage | None = None¶: Custom docker image BYOC

driver_instance_pool_id: str | None = None¶: The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned.

driver_node_type_id: str | None = None¶

The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as node_type_id defined above.

This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

enable_local_disk_encryption: bool | None = None¶: Whether to enable LUKS on cluster VMs’ local disks

gcp_attributes: GcpAttributes | None = None¶: Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used.

init_scripts: List[InitScriptInfo] | None = None¶: The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

instance_pool_id: str | None = None¶: The optional ID of the instance pool to which the cluster belongs.

is_single_node: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

When set to true, Databricks will automatically set single node related custom_tags, spark_conf, and num_workers

kind: Kind | None = None¶

The kind of compute described by this compute specification.

Depending on kind, different validations and default values will be applied.

Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD

By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW.

[simple form]: https://docs.databricks.com/compute/simple-form.html

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

runtime_engine: RuntimeEngine | None = None¶

Determines the cluster’s runtime engine, either standard or Photon.

This field is not compatible with legacy spark_version values that contain -photon-. Remove -photon- from the spark_version and set runtime_engine to PHOTON.

If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used.

single_user_name: str | None = None¶: Single user name if data_security_mode is SINGLE_USER

spark_conf: Dict[str, str] | None = None¶: An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions respectively.

spark_env_vars: Dict[str, str] | None = None¶

An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., export X=’Y’) while launching the driver and workers.

In order to specify an additional set of SPARK_DAEMON_JAVA_OPTS, we recommend appending them to $SPARK_DAEMON_JAVA_OPTS as shown in the example below. This ensures that all default databricks managed environmental variables are included as well.

Example Spark environment variables: {“SPARK_WORKER_MEMORY”: “28000m”, “SPARK_LOCAL_DIRS”: “/local_disk0”} or {“SPARK_DAEMON_JAVA_OPTS”: “$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true”}

ssh_public_keys: List[str] | None = None¶: SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

use_ml_runtime: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

effective_spark_version is determined by spark_version (DBR release), this field use_ml_runtime, and whether node_type_id is gpu node or not.

workload_type: WorkloadType | None = None¶: Cluster Attributes showing for clusters workload types.

as_dict() → dict¶: Serializes the EditCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditCluster¶: Deserializes the EditCluster from a dictionary.

class databricks.sdk.service.compute.EditClusterResponse¶

as_dict() → dict¶: Serializes the EditClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditClusterResponse¶: Deserializes the EditClusterResponse from a dictionary.

class databricks.sdk.service.compute.EditInstancePool¶

instance_pool_id: str¶: Instance pool ID

instance_pool_name: str¶: Pool name requested by the user. Pool name must be unique. Length must be between 1 and 100 characters.

node_type_id: str¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

custom_tags: Dict[str, str] | None = None¶

Additional tags for pool resources. Databricks will tag all pool resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags

idle_instance_autotermination_minutes: int | None = None¶: Automatically terminates the extra instances in the pool cache after they are inactive for this time in minutes if min_idle_instances requirement is already met. If not set, the extra pool instances will be automatically terminated after a default timeout. If specified, the threshold must be between 0 and 10000 minutes. Users can also set this value to 0 to instantly remove idle instances from the cache if min cache size could still hold.

max_capacity: int | None = None¶: Maximum number of outstanding instances to keep in the pool, including both instances used by clusters and idle instances. Clusters that require further instance provisioning will fail during upsize requests.

min_idle_instances: int | None = None¶: Minimum number of idle instances to keep in the instance pool

as_dict() → dict¶: Serializes the EditInstancePool into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditInstancePool into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditInstancePool¶: Deserializes the EditInstancePool from a dictionary.

class databricks.sdk.service.compute.EditInstancePoolResponse¶

as_dict() → dict¶: Serializes the EditInstancePoolResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditInstancePoolResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditInstancePoolResponse¶: Deserializes the EditInstancePoolResponse from a dictionary.

class databricks.sdk.service.compute.EditPolicy¶

policy_id: str¶: The ID of the policy to update.

definition: str | None = None¶

Policy definition document expressed in [Databricks Cluster Policy Definition Language].

[Databricks Cluster Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

description: str | None = None¶: Additional human-readable description of the cluster policy.

libraries: List[Library] | None = None¶: A list of libraries to be installed on the next cluster restart that uses this policy. The maximum number of libraries is 500.

max_clusters_per_user: int | None = None¶: Max number of clusters per user that can be active using this policy. If not present, there is no max limit.

name: str | None = None¶: Cluster Policy name requested by the user. This has to be unique. Length must be between 1 and 100 characters.

policy_family_definition_overrides: str | None = None¶

Policy definition JSON document expressed in [Databricks Policy Definition Language]. The JSON document must be passed as a string and cannot be embedded in the requests.

You can use this to customize the policy definition inherited from the policy family. Policy rules specified here are merged into the inherited policy definition.

[Databricks Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

policy_family_id: str | None = None¶

ID of the policy family. The cluster policy’s policy definition inherits the policy family’s policy definition.

Cannot be used with definition. Use policy_family_definition_overrides instead to customize the policy definition.

as_dict() → dict¶: Serializes the EditPolicy into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditPolicy into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditPolicy¶: Deserializes the EditPolicy from a dictionary.

class databricks.sdk.service.compute.EditPolicyResponse¶

as_dict() → dict¶: Serializes the EditPolicyResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditPolicyResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditPolicyResponse¶: Deserializes the EditPolicyResponse from a dictionary.

class databricks.sdk.service.compute.EditResponse¶

as_dict() → dict¶: Serializes the EditResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EditResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EditResponse¶: Deserializes the EditResponse from a dictionary.

class databricks.sdk.service.compute.EnforceClusterComplianceRequest¶

cluster_id: str¶: The ID of the cluster you want to enforce policy compliance on.

validate_only: bool | None = None¶: If set, previews the changes that would be made to a cluster to enforce compliance but does not update the cluster.

as_dict() → dict¶: Serializes the EnforceClusterComplianceRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EnforceClusterComplianceRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EnforceClusterComplianceRequest¶: Deserializes the EnforceClusterComplianceRequest from a dictionary.

class databricks.sdk.service.compute.EnforceClusterComplianceResponse¶

changes: List[ClusterSettingsChange] | None = None¶: A list of changes that have been made to the cluster settings for the cluster to become compliant with its policy.

has_changes: bool | None = None¶: Whether any changes have been made to the cluster settings for the cluster to become compliant with its policy.

as_dict() → dict¶: Serializes the EnforceClusterComplianceResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EnforceClusterComplianceResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EnforceClusterComplianceResponse¶: Deserializes the EnforceClusterComplianceResponse from a dictionary.

class databricks.sdk.service.compute.Environment¶

The environment entity used to preserve serverless environment side panel, jobs’ environment for non-notebook task, and DLT’s environment for classic and serverless pipelines. In this minimal environment spec, only pip dependencies are supported.

client: str | None = None¶: Use environment_version instead.

dependencies: List[str] | None = None¶: List of pip dependencies, as supported by the version of pip in this environment. Each dependency is a valid pip requirements file line per https://pip.pypa.io/en/stable/reference/requirements-file-format/. Allowed dependencies include a requirement specifier, an archive URL, a local project path (such as WSFS or UC Volumes in Databricks), or a VCS project URL.

environment_version: str | None = None¶: Required. Environment version used by the environment. Each version comes with a specific Python version and a set of Python packages. The version is a string, consisting of an integer.

jar_dependencies: List[str] | None = None¶: List of jar dependencies, should be string representing volume paths. For example: /Volumes/path/to/test.jar.

as_dict() → dict¶: Serializes the Environment into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Environment into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Environment¶: Deserializes the Environment from a dictionary.

class databricks.sdk.service.compute.EventDetails¶

attributes: ClusterAttributes | None = None¶

For created clusters, the attributes of the cluster. * For edited clusters, the new attributes

of the cluster.

cause: EventDetailsCause | None = None¶: The cause of a change in target size.

cluster_size: ClusterSize | None = None¶: The actual cluster size that was set in the cluster creation or edit.

current_num_vcpus: int | None = None¶: The current number of vCPUs in the cluster.

current_num_workers: int | None = None¶: The current number of nodes in the cluster.

did_not_expand_reason: str | None = None¶

disk_size: int | None = None¶: Current disk size in bytes

driver_state_message: str | None = None¶: More details about the change in driver’s state

enable_termination_for_node_blocklisted: bool | None = None¶: Whether or not a blocklisted node should be terminated. For ClusterEventType NODE_BLACKLISTED.

free_space: int | None = None¶

init_scripts: InitScriptEventDetails | None = None¶: List of global and cluster init scripts associated with this cluster event.

instance_id: str | None = None¶: Instance Id where the event originated from

job_run_name: str | None = None¶: Unique identifier of the specific job run associated with this cluster event * For clusters created for jobs, this will be the same as the cluster name

previous_attributes: ClusterAttributes | None = None¶: The cluster attributes before a cluster was edited.

previous_cluster_size: ClusterSize | None = None¶: The size of the cluster before an edit or resize.

previous_disk_size: int | None = None¶: Previous disk size in bytes

reason: TerminationReason | None = None¶: A termination reason: * On a TERMINATED event, this is the reason of the termination. * On a RESIZE_COMPLETE event, this indicates the reason that we failed to acquire some nodes.

target_num_vcpus: int | None = None¶: The targeted number of vCPUs in the cluster.

target_num_workers: int | None = None¶: The targeted number of nodes in the cluster.

user: str | None = None¶: The user that caused the event to occur. (Empty if it was done by the control plane.)

as_dict() → dict¶: Serializes the EventDetails into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the EventDetails into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → EventDetails¶: Deserializes the EventDetails from a dictionary.

class databricks.sdk.service.compute.EventDetailsCause¶

The cause of a change in target size.

AUTORECOVERY = "AUTORECOVERY"¶

AUTOSCALE = "AUTOSCALE"¶

REPLACE_BAD_NODES = "REPLACE_BAD_NODES"¶

USER_REQUEST = "USER_REQUEST"¶

class databricks.sdk.service.compute.EventType¶

ADD_NODES_FAILED = "ADD_NODES_FAILED"¶

AUTOMATIC_CLUSTER_UPDATE = "AUTOMATIC_CLUSTER_UPDATE"¶

AUTOSCALING_BACKOFF = "AUTOSCALING_BACKOFF"¶

AUTOSCALING_FAILED = "AUTOSCALING_FAILED"¶

AUTOSCALING_STATS_REPORT = "AUTOSCALING_STATS_REPORT"¶

CLUSTER_MIGRATED = "CLUSTER_MIGRATED"¶

CREATING = "CREATING"¶

DBFS_DOWN = "DBFS_DOWN"¶

DID_NOT_EXPAND_DISK = "DID_NOT_EXPAND_DISK"¶

DRIVER_HEALTHY = "DRIVER_HEALTHY"¶

DRIVER_NOT_RESPONDING = "DRIVER_NOT_RESPONDING"¶

DRIVER_UNAVAILABLE = "DRIVER_UNAVAILABLE"¶

EDITED = "EDITED"¶

EXPANDED_DISK = "EXPANDED_DISK"¶

FAILED_TO_EXPAND_DISK = "FAILED_TO_EXPAND_DISK"¶

INIT_SCRIPTS_FINISHED = "INIT_SCRIPTS_FINISHED"¶

INIT_SCRIPTS_STARTED = "INIT_SCRIPTS_STARTED"¶

METASTORE_DOWN = "METASTORE_DOWN"¶

NODES_LOST = "NODES_LOST"¶

NODE_BLACKLISTED = "NODE_BLACKLISTED"¶

NODE_EXCLUDED_DECOMMISSIONED = "NODE_EXCLUDED_DECOMMISSIONED"¶

PINNED = "PINNED"¶

RESIZING = "RESIZING"¶

RESTARTING = "RESTARTING"¶

RUNNING = "RUNNING"¶

SPARK_EXCEPTION = "SPARK_EXCEPTION"¶

STARTING = "STARTING"¶

TERMINATING = "TERMINATING"¶

UNPINNED = "UNPINNED"¶

UPSIZE_COMPLETED = "UPSIZE_COMPLETED"¶

class databricks.sdk.service.compute.GcpAttributes¶

Attributes set during cluster creation which are related to GCP.

availability: GcpAvailability | None = None¶: This field determines whether the spark executors will be scheduled to run on preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable.

boot_disk_size: int | None = None¶: Boot disk size in GB

google_service_account: str | None = None¶: If provided, the cluster will impersonate the google service account when accessing gcloud services (like GCS). The google service account must have previously been added to the Databricks environment by an account administrator.

local_ssd_count: int | None = None¶

If provided, each node (workers and driver) in the cluster will have this number of local SSDs attached. Each local SSD is 375GB in size. Refer to [GCP documentation] for the supported number of local SSDs for each instance type.

[GCP documentation]: https://cloud.google.com/compute/docs/disks/local-ssd#choose_number_local_ssds

use_preemptible_executors: bool | None = None¶: This field determines whether the spark executors will be scheduled to run on preemptible VMs (when set to true) versus standard compute engine VMs (when set to false; default). Note: Soon to be deprecated, use the ‘availability’ field instead.

zone_id: str | None = None¶: Identifier for the availability zone in which the cluster resides. This can be one of the following: - “HA” => High availability, spread nodes across availability zones for a Databricks deployment region [default]. - “AUTO” => Databricks picks an availability zone to schedule the cluster on. - A GCP availability zone => Pick One of the available zones for (machine type + region) from https://cloud.google.com/compute/docs/regions-zones.

as_dict() → dict¶: Serializes the GcpAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GcpAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GcpAttributes¶: Deserializes the GcpAttributes from a dictionary.

class databricks.sdk.service.compute.GcpAvailability¶

This field determines whether the instance pool will contain preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable.

ON_DEMAND_GCP = "ON_DEMAND_GCP"¶

PREEMPTIBLE_GCP = "PREEMPTIBLE_GCP"¶

PREEMPTIBLE_WITH_FALLBACK_GCP = "PREEMPTIBLE_WITH_FALLBACK_GCP"¶

class databricks.sdk.service.compute.GcsStorageInfo¶

A storage location in Google Cloud Platform’s GCS

destination: str¶: GCS destination/URI, e.g. gs://my-bucket/some-prefix

as_dict() → dict¶: Serializes the GcsStorageInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GcsStorageInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GcsStorageInfo¶: Deserializes the GcsStorageInfo from a dictionary.

class databricks.sdk.service.compute.GetClusterComplianceResponse¶

is_compliant: bool | None = None¶: Whether the cluster is compliant with its policy or not. Clusters could be out of compliance if the policy was updated after the cluster was last edited.

violations: Dict[str, str] | None = None¶: An object containing key-value mappings representing the first 200 policy validation errors. The keys indicate the path where the policy validation error is occurring. The values indicate an error message describing the policy validation error.

as_dict() → dict¶: Serializes the GetClusterComplianceResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetClusterComplianceResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetClusterComplianceResponse¶: Deserializes the GetClusterComplianceResponse from a dictionary.

class databricks.sdk.service.compute.GetClusterPermissionLevelsResponse¶

permission_levels: List[ClusterPermissionsDescription] | None = None¶: Specific permission levels

as_dict() → dict¶: Serializes the GetClusterPermissionLevelsResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetClusterPermissionLevelsResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetClusterPermissionLevelsResponse¶: Deserializes the GetClusterPermissionLevelsResponse from a dictionary.

class databricks.sdk.service.compute.GetClusterPolicyPermissionLevelsResponse¶

permission_levels: List[ClusterPolicyPermissionsDescription] | None = None¶: Specific permission levels

as_dict() → dict¶: Serializes the GetClusterPolicyPermissionLevelsResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetClusterPolicyPermissionLevelsResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetClusterPolicyPermissionLevelsResponse¶: Deserializes the GetClusterPolicyPermissionLevelsResponse from a dictionary.

class databricks.sdk.service.compute.GetEvents¶

cluster_id: str¶: The ID of the cluster to retrieve events about.

end_time: int | None = None¶: The end time in epoch milliseconds. If empty, returns events up to the current time.

event_types: List[EventType] | None = None¶: An optional set of event types to filter on. If empty, all event types are returned.

limit: int | None = None¶

Deprecated: use page_token in combination with page_size instead.

The maximum number of events to include in a page of events. Defaults to 50, and maximum allowed value is 500.

offset: int | None = None¶

Deprecated: use page_token in combination with page_size instead.

The offset in the result set. Defaults to 0 (no offset). When an offset is specified and the results are requested in descending order, the end_time field is required.

order: GetEventsOrder | None = None¶: The order to list events in; either “ASC” or “DESC”. Defaults to “DESC”.

page_size: int | None = None¶: The maximum number of events to include in a page of events. The server may further constrain the maximum number of results returned in a single page. If the page_size is empty or 0, the server will decide the number of results to be returned. The field has to be in the range [0,500]. If the value is outside the range, the server enforces 0 or 500.

page_token: str | None = None¶: Use next_page_token or prev_page_token returned from the previous request to list the next or previous page of events respectively. If page_token is empty, the first page is returned.

start_time: int | None = None¶: The start time in epoch milliseconds. If empty, returns events starting from the beginning of time.

as_dict() → dict¶: Serializes the GetEvents into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetEvents into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetEvents¶: Deserializes the GetEvents from a dictionary.

class databricks.sdk.service.compute.GetEventsOrder¶

ASC = "ASC"¶

DESC = "DESC"¶

class databricks.sdk.service.compute.GetEventsResponse¶

events: List[ClusterEvent] | None = None¶

next_page: GetEvents | None = None¶

Deprecated: use next_page_token or prev_page_token instead.

The parameters required to retrieve the next page of events. Omitted if there are no more events to read.

next_page_token: str | None = None¶: This field represents the pagination token to retrieve the next page of results. If the value is “”, it means no further results for the request.

prev_page_token: str | None = None¶: This field represents the pagination token to retrieve the previous page of results. If the value is “”, it means no further results for the request.

total_count: int | None = None¶

Deprecated: Returns 0 when request uses page_token. Will start returning zero when request uses offset/limit soon.

The total number of events filtered by the start_time, end_time, and event_types.

as_dict() → dict¶: Serializes the GetEventsResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetEventsResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetEventsResponse¶: Deserializes the GetEventsResponse from a dictionary.

class databricks.sdk.service.compute.GetInstancePool¶

instance_pool_id: str¶: Canonical unique identifier for the pool.

aws_attributes: InstancePoolAwsAttributes | None = None¶: Attributes related to instance pools running on Amazon Web Services. If not specified at pool creation, a set of default values will be used.

azure_attributes: InstancePoolAzureAttributes | None = None¶: Attributes related to instance pools running on Azure. If not specified at pool creation, a set of default values will be used.

custom_tags: Dict[str, str] | None = None¶

Additional tags for pool resources. Databricks will tag all pool resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags

default_tags: Dict[str, str] | None = None¶

Tags that are added by Databricks regardless of any custom_tags, including:

Vendor: Databricks
InstancePoolCreator: <user_id_of_creator>
InstancePoolName: <name_of_pool>
InstancePoolId: <id_of_pool>

disk_spec: DiskSpec | None = None¶: Defines the specification of the disks that will be attached to all spark containers.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this instances in this pool will dynamically acquire additional disk space when its Spark workers are running low on disk space. In AWS, this feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

gcp_attributes: InstancePoolGcpAttributes | None = None¶: Attributes related to instance pools running on Google Cloud Platform. If not specified at pool creation, a set of default values will be used.

idle_instance_autotermination_minutes: int | None = None¶: Automatically terminates the extra instances in the pool cache after they are inactive for this time in minutes if min_idle_instances requirement is already met. If not set, the extra pool instances will be automatically terminated after a default timeout. If specified, the threshold must be between 0 and 10000 minutes. Users can also set this value to 0 to instantly remove idle instances from the cache if min cache size could still hold.

instance_pool_name: str | None = None¶: Pool name requested by the user. Pool name must be unique. Length must be between 1 and 100 characters.

max_capacity: int | None = None¶: Maximum number of outstanding instances to keep in the pool, including both instances used by clusters and idle instances. Clusters that require further instance provisioning will fail during upsize requests.

min_idle_instances: int | None = None¶: Minimum number of idle instances to keep in the instance pool

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

preloaded_docker_images: List[DockerImage] | None = None¶: Custom Docker Image BYOC

preloaded_spark_versions: List[str] | None = None¶: A list containing at most one preloaded Spark image version for the pool. Pool-backed clusters started with the preloaded Spark version will start faster. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

state: InstancePoolState | None = None¶: Current state of the instance pool.

stats: InstancePoolStats | None = None¶: Usage statistics about the instance pool.

status: InstancePoolStatus | None = None¶: Status of failed pending instances in the pool.

as_dict() → dict¶: Serializes the GetInstancePool into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetInstancePool into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetInstancePool¶: Deserializes the GetInstancePool from a dictionary.

class databricks.sdk.service.compute.GetInstancePoolPermissionLevelsResponse¶

permission_levels: List[InstancePoolPermissionsDescription] | None = None¶: Specific permission levels

as_dict() → dict¶: Serializes the GetInstancePoolPermissionLevelsResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetInstancePoolPermissionLevelsResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetInstancePoolPermissionLevelsResponse¶: Deserializes the GetInstancePoolPermissionLevelsResponse from a dictionary.

class databricks.sdk.service.compute.GetSparkVersionsResponse¶

versions: List[SparkVersion] | None = None¶: All the available Spark versions.

as_dict() → dict¶: Serializes the GetSparkVersionsResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GetSparkVersionsResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GetSparkVersionsResponse¶: Deserializes the GetSparkVersionsResponse from a dictionary.

class databricks.sdk.service.compute.GlobalInitScriptCreateRequest¶

name: str¶: The name of the script

script: str¶: The Base64-encoded content of the script.

enabled: bool | None = None¶: Specifies whether the script is enabled. The script runs only if enabled.

position: int | None = None¶

The position of a global init script, where 0 represents the first script to run, 1 is the second script to run, in ascending order.

If you omit the numeric position for a new global init script, it defaults to last position. It will run after all current scripts. Setting any value greater than the position of the last script is equivalent to the last position. Example: Take three existing scripts with positions 0, 1, and 2. Any position of (3) or greater puts the script in the last position. If an explicit position value conflicts with an existing script value, your request succeeds, but the original script at that position and all later scripts have their positions incremented by 1.

as_dict() → dict¶: Serializes the GlobalInitScriptCreateRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GlobalInitScriptCreateRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GlobalInitScriptCreateRequest¶: Deserializes the GlobalInitScriptCreateRequest from a dictionary.

class databricks.sdk.service.compute.GlobalInitScriptDetails¶

created_at: int | None = None¶: Time when the script was created, represented as a Unix timestamp in milliseconds.

created_by: str | None = None¶: The username of the user who created the script.

enabled: bool | None = None¶: Specifies whether the script is enabled. The script runs only if enabled.

name: str | None = None¶: The name of the script

position: int | None = None¶: The position of a script, where 0 represents the first script to run, 1 is the second script to run, in ascending order.

script_id: str | None = None¶: The global init script ID.

updated_at: int | None = None¶: Time when the script was updated, represented as a Unix timestamp in milliseconds.

updated_by: str | None = None¶: The username of the user who last updated the script

as_dict() → dict¶: Serializes the GlobalInitScriptDetails into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GlobalInitScriptDetails into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GlobalInitScriptDetails¶: Deserializes the GlobalInitScriptDetails from a dictionary.

class databricks.sdk.service.compute.GlobalInitScriptDetailsWithContent¶

created_at: int | None = None¶: Time when the script was created, represented as a Unix timestamp in milliseconds.

created_by: str | None = None¶: The username of the user who created the script.

enabled: bool | None = None¶: Specifies whether the script is enabled. The script runs only if enabled.

name: str | None = None¶: The name of the script

position: int | None = None¶: The position of a script, where 0 represents the first script to run, 1 is the second script to run, in ascending order.

script: str | None = None¶: The Base64-encoded content of the script.

script_id: str | None = None¶: The global init script ID.

updated_at: int | None = None¶: Time when the script was updated, represented as a Unix timestamp in milliseconds.

updated_by: str | None = None¶: The username of the user who last updated the script

as_dict() → dict¶: Serializes the GlobalInitScriptDetailsWithContent into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GlobalInitScriptDetailsWithContent into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GlobalInitScriptDetailsWithContent¶: Deserializes the GlobalInitScriptDetailsWithContent from a dictionary.

class databricks.sdk.service.compute.GlobalInitScriptUpdateRequest¶

name: str¶: The name of the script

script: str¶: The Base64-encoded content of the script.

enabled: bool | None = None¶: Specifies whether the script is enabled. The script runs only if enabled.

position: int | None = None¶

The position of a script, where 0 represents the first script to run, 1 is the second script to run, in ascending order. To move the script to run first, set its position to 0.

To move the script to the end, set its position to any value greater or equal to the position of the last script. Example, three existing scripts with positions 0, 1, and 2. Any position value of 2 or greater puts the script in the last position (2).

If an explicit position value conflicts with an existing script, your request succeeds, but the original script at that position and all later scripts have their positions incremented by 1.

script_id: str | None = None¶: The ID of the global init script.

as_dict() → dict¶: Serializes the GlobalInitScriptUpdateRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the GlobalInitScriptUpdateRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → GlobalInitScriptUpdateRequest¶: Deserializes the GlobalInitScriptUpdateRequest from a dictionary.

class databricks.sdk.service.compute.InitScriptEventDetails¶

cluster: List[InitScriptInfoAndExecutionDetails] | None = None¶: The cluster scoped init scripts associated with this cluster event.

global_: List[InitScriptInfoAndExecutionDetails] | None = None¶: The global init scripts associated with this cluster event.

reported_for_node: str | None = None¶

The private ip of the node we are reporting init script execution details for (we will select the execution details from only one node rather than reporting the execution details from every node to keep these event details small)

This should only be defined for the INIT_SCRIPTS_FINISHED event

as_dict() → dict¶: Serializes the InitScriptEventDetails into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InitScriptEventDetails into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InitScriptEventDetails¶: Deserializes the InitScriptEventDetails from a dictionary.

class databricks.sdk.service.compute.InitScriptExecutionDetailsInitScriptExecutionStatus¶

Result of attempted script execution

FAILED_EXECUTION = "FAILED_EXECUTION"¶

FAILED_FETCH = "FAILED_FETCH"¶

FUSE_MOUNT_FAILED = "FUSE_MOUNT_FAILED"¶

NOT_EXECUTED = "NOT_EXECUTED"¶

SKIPPED = "SKIPPED"¶

SUCCEEDED = "SUCCEEDED"¶

UNKNOWN = "UNKNOWN"¶

class databricks.sdk.service.compute.InitScriptInfo¶

Config for an individual init script Next ID: 11

abfss: Adlsgen2Info | None = None¶: destination needs to be provided, e.g. abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>

dbfs: DbfsStorageInfo | None = None¶: destination needs to be provided. e.g. { “dbfs”: { “destination” : “dbfs:/home/cluster_log” } }

file: LocalFileInfo | None = None¶: destination needs to be provided, e.g. { “file”: { “destination”: “file:/my/local/file.sh” } }

gcs: GcsStorageInfo | None = None¶: destination needs to be provided, e.g. { “gcs”: { “destination”: “gs://my-bucket/file.sh” } }

s3: S3StorageInfo | None = None¶: destination and either the region or endpoint need to be provided. e.g. { “s3”: { “destination”: “s3://cluster_log_bucket/prefix”, “region”: “us-west-2” } } Cluster iam role is used to access s3, please make sure the cluster iam role in instance_profile_arn has permission to write data to the s3 destination.

volumes: VolumesStorageInfo | None = None¶: destination needs to be provided. e.g. { “volumes” : { “destination” : “/Volumes/my-init.sh” } }

workspace: WorkspaceStorageInfo | None = None¶: destination needs to be provided, e.g. { “workspace”: { “destination”: “/cluster-init-scripts/setup-datadog.sh” } }

as_dict() → dict¶: Serializes the InitScriptInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InitScriptInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InitScriptInfo¶: Deserializes the InitScriptInfo from a dictionary.

class databricks.sdk.service.compute.InitScriptInfoAndExecutionDetails¶

abfss: Adlsgen2Info | None = None¶: destination needs to be provided, e.g. abfss://<container-name>@<storage-account-name>.dfs.core.windows.net/<directory-name>

dbfs: DbfsStorageInfo | None = None¶: destination needs to be provided. e.g. { “dbfs”: { “destination” : “dbfs:/home/cluster_log” } }

error_message: str | None = None¶: Additional details regarding errors (such as a file not found message if the status is FAILED_FETCH). This field should only be used to provide additional information to the status field, not duplicate it.

execution_duration_seconds: int | None = None¶: The number duration of the script execution in seconds

file: LocalFileInfo | None = None¶: destination needs to be provided, e.g. { “file”: { “destination”: “file:/my/local/file.sh” } }

gcs: GcsStorageInfo | None = None¶: destination needs to be provided, e.g. { “gcs”: { “destination”: “gs://my-bucket/file.sh” } }

s3: S3StorageInfo | None = None¶: destination and either the region or endpoint need to be provided. e.g. { “s3”: { “destination”: “s3://cluster_log_bucket/prefix”, “region”: “us-west-2” } } Cluster iam role is used to access s3, please make sure the cluster iam role in instance_profile_arn has permission to write data to the s3 destination.

status: InitScriptExecutionDetailsInitScriptExecutionStatus | None = None¶: The current status of the script

volumes: VolumesStorageInfo | None = None¶: destination needs to be provided. e.g. { “volumes” : { “destination” : “/Volumes/my-init.sh” } }

workspace: WorkspaceStorageInfo | None = None¶: destination needs to be provided, e.g. { “workspace”: { “destination”: “/cluster-init-scripts/setup-datadog.sh” } }

as_dict() → dict¶: Serializes the InitScriptInfoAndExecutionDetails into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InitScriptInfoAndExecutionDetails into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InitScriptInfoAndExecutionDetails¶: Deserializes the InitScriptInfoAndExecutionDetails from a dictionary.

class databricks.sdk.service.compute.InstallLibraries¶

cluster_id: str¶: Unique identifier for the cluster on which to install these libraries.

libraries: List[Library]¶: The libraries to install.

as_dict() → dict¶: Serializes the InstallLibraries into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstallLibraries into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstallLibraries¶: Deserializes the InstallLibraries from a dictionary.

class databricks.sdk.service.compute.InstallLibrariesResponse¶

as_dict() → dict¶: Serializes the InstallLibrariesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstallLibrariesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstallLibrariesResponse¶: Deserializes the InstallLibrariesResponse from a dictionary.

class databricks.sdk.service.compute.InstancePoolAccessControlRequest¶

group_name: str | None = None¶: name of the group

permission_level: InstancePoolPermissionLevel | None = None¶: Permission level

service_principal_name: str | None = None¶: application ID of a service principal

user_name: str | None = None¶: name of the user

as_dict() → dict¶: Serializes the InstancePoolAccessControlRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolAccessControlRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolAccessControlRequest¶: Deserializes the InstancePoolAccessControlRequest from a dictionary.

class databricks.sdk.service.compute.InstancePoolAccessControlResponse¶

all_permissions: List[InstancePoolPermission] | None = None¶: All permissions.

display_name: str | None = None¶: Display name of the user or service principal.

group_name: str | None = None¶: name of the group

service_principal_name: str | None = None¶: Name of the service principal.

user_name: str | None = None¶: name of the user

as_dict() → dict¶: Serializes the InstancePoolAccessControlResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolAccessControlResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolAccessControlResponse¶: Deserializes the InstancePoolAccessControlResponse from a dictionary.

class databricks.sdk.service.compute.InstancePoolAndStats¶

aws_attributes: InstancePoolAwsAttributes | None = None¶: Attributes related to instance pools running on Amazon Web Services. If not specified at pool creation, a set of default values will be used.

azure_attributes: InstancePoolAzureAttributes | None = None¶: Attributes related to instance pools running on Azure. If not specified at pool creation, a set of default values will be used.

custom_tags: Dict[str, str] | None = None¶

Additional tags for pool resources. Databricks will tag all pool resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags

default_tags: Dict[str, str] | None = None¶

Tags that are added by Databricks regardless of any custom_tags, including:

Vendor: Databricks
InstancePoolCreator: <user_id_of_creator>
InstancePoolName: <name_of_pool>
InstancePoolId: <id_of_pool>

disk_spec: DiskSpec | None = None¶: Defines the specification of the disks that will be attached to all spark containers.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this instances in this pool will dynamically acquire additional disk space when its Spark workers are running low on disk space. In AWS, this feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

gcp_attributes: InstancePoolGcpAttributes | None = None¶: Attributes related to instance pools running on Google Cloud Platform. If not specified at pool creation, a set of default values will be used.

idle_instance_autotermination_minutes: int | None = None¶: Automatically terminates the extra instances in the pool cache after they are inactive for this time in minutes if min_idle_instances requirement is already met. If not set, the extra pool instances will be automatically terminated after a default timeout. If specified, the threshold must be between 0 and 10000 minutes. Users can also set this value to 0 to instantly remove idle instances from the cache if min cache size could still hold.

instance_pool_id: str | None = None¶: Canonical unique identifier for the pool.

instance_pool_name: str | None = None¶: Pool name requested by the user. Pool name must be unique. Length must be between 1 and 100 characters.

max_capacity: int | None = None¶: Maximum number of outstanding instances to keep in the pool, including both instances used by clusters and idle instances. Clusters that require further instance provisioning will fail during upsize requests.

min_idle_instances: int | None = None¶: Minimum number of idle instances to keep in the instance pool

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

preloaded_docker_images: List[DockerImage] | None = None¶: Custom Docker Image BYOC

preloaded_spark_versions: List[str] | None = None¶: A list containing at most one preloaded Spark image version for the pool. Pool-backed clusters started with the preloaded Spark version will start faster. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

state: InstancePoolState | None = None¶: Current state of the instance pool.

stats: InstancePoolStats | None = None¶: Usage statistics about the instance pool.

status: InstancePoolStatus | None = None¶: Status of failed pending instances in the pool.

as_dict() → dict¶: Serializes the InstancePoolAndStats into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolAndStats into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolAndStats¶: Deserializes the InstancePoolAndStats from a dictionary.

class databricks.sdk.service.compute.InstancePoolAwsAttributes¶

Attributes set during instance pool creation which are related to Amazon Web Services.

availability: InstancePoolAwsAttributesAvailability | None = None¶: Availability type used for the spot nodes.

spot_bid_price_percent: int | None = None¶: Calculates the bid price for AWS spot instances, as a percentage of the corresponding instance type’s on-demand price. For example, if this field is set to 50, and the cluster needs a new r3.xlarge spot instance, then the bid price is half of the price of on-demand r3.xlarge instances. Similarly, if this field is set to 200, the bid price is twice the price of on-demand r3.xlarge instances. If not specified, the default value is 100. When spot instances are requested for this cluster, only spot instances whose bid price percentage matches this field will be considered. Note that, for safety, we enforce this field to be no more than 10000.

zone_id: str | None = None¶: Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like “us-west-2a”. The provided availability zone must be in the same region as the Databricks deployment. For example, “us-west-2a” is not a valid zone id if the Databricks deployment resides in the “us-east-1” region. This is an optional field at cluster creation, and if not specified, a default zone will be used. The list of available zones as well as the default value can be found by using the List Zones method.

as_dict() → dict¶: Serializes the InstancePoolAwsAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolAwsAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolAwsAttributes¶: Deserializes the InstancePoolAwsAttributes from a dictionary.

class databricks.sdk.service.compute.InstancePoolAwsAttributesAvailability¶

The set of AWS availability types supported when setting up nodes for a cluster.

ON_DEMAND = "ON_DEMAND"¶

SPOT = "SPOT"¶

class databricks.sdk.service.compute.InstancePoolAzureAttributes¶

Attributes set during instance pool creation which are related to Azure.

availability: InstancePoolAzureAttributesAvailability | None = None¶: Availability type used for the spot nodes.

spot_bid_max_price: float | None = None¶: With variable pricing, you have option to set a max price, in US dollars (USD) For example, the value 2 would be a max price of $2.00 USD per hour. If you set the max price to be -1, the VM won’t be evicted based on price. The price for the VM will be the current price for spot or the price for a standard VM, which ever is less, as long as there is capacity and quota available.

as_dict() → dict¶: Serializes the InstancePoolAzureAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolAzureAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolAzureAttributes¶: Deserializes the InstancePoolAzureAttributes from a dictionary.

class databricks.sdk.service.compute.InstancePoolAzureAttributesAvailability¶

The set of Azure availability types supported when setting up nodes for a cluster.

ON_DEMAND_AZURE = "ON_DEMAND_AZURE"¶

SPOT_AZURE = "SPOT_AZURE"¶

class databricks.sdk.service.compute.InstancePoolGcpAttributes¶

Attributes set during instance pool creation which are related to GCP.

gcp_availability: GcpAvailability | None = None¶: This field determines whether the instance pool will contain preemptible VMs, on-demand VMs, or preemptible VMs with a fallback to on-demand VMs if the former is unavailable.

local_ssd_count: int | None = None¶

If provided, each node in the instance pool will have this number of local SSDs attached. Each local SSD is 375GB in size. Refer to [GCP documentation] for the supported number of local SSDs for each instance type.

[GCP documentation]: https://cloud.google.com/compute/docs/disks/local-ssd#choose_number_local_ssds

zone_id: str | None = None¶

Identifier for the availability zone/datacenter in which the cluster resides. This string will be of a form like “us-west1-a”. The provided availability zone must be in the same region as the Databricks workspace. For example, “us-west1-a” is not a valid zone id if the Databricks workspace resides in the “us-east1” region. This is an optional field at instance pool creation, and if not specified, a default zone will be used.

This field can be one of the following: - “HA” => High availability, spread nodes across availability zones for a Databricks deployment region - A GCP availability zone => Pick One of the available zones for (machine type + region) from https://cloud.google.com/compute/docs/regions-zones (e.g. “us-west1-a”).

If empty, Databricks picks an availability zone to schedule the cluster on.

as_dict() → dict¶: Serializes the InstancePoolGcpAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolGcpAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolGcpAttributes¶: Deserializes the InstancePoolGcpAttributes from a dictionary.

class databricks.sdk.service.compute.InstancePoolPermission¶

inherited: bool | None = None¶

inherited_from_object: List[str] | None = None¶

permission_level: InstancePoolPermissionLevel | None = None¶: Permission level

as_dict() → dict¶: Serializes the InstancePoolPermission into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolPermission into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolPermission¶: Deserializes the InstancePoolPermission from a dictionary.

class databricks.sdk.service.compute.InstancePoolPermissionLevel¶

Permission level

CAN_ATTACH_TO = "CAN_ATTACH_TO"¶

CAN_MANAGE = "CAN_MANAGE"¶

class databricks.sdk.service.compute.InstancePoolPermissions¶

access_control_list: List[InstancePoolAccessControlResponse] | None = None¶

object_id: str | None = None¶

object_type: str | None = None¶

as_dict() → dict¶: Serializes the InstancePoolPermissions into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolPermissions into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolPermissions¶: Deserializes the InstancePoolPermissions from a dictionary.

class databricks.sdk.service.compute.InstancePoolPermissionsDescription¶

description: str | None = None¶

permission_level: InstancePoolPermissionLevel | None = None¶: Permission level

as_dict() → dict¶: Serializes the InstancePoolPermissionsDescription into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolPermissionsDescription into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolPermissionsDescription¶: Deserializes the InstancePoolPermissionsDescription from a dictionary.

class databricks.sdk.service.compute.InstancePoolPermissionsRequest¶

access_control_list: List[InstancePoolAccessControlRequest] | None = None¶

instance_pool_id: str | None = None¶: The instance pool for which to get or manage permissions.

as_dict() → dict¶: Serializes the InstancePoolPermissionsRequest into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolPermissionsRequest into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolPermissionsRequest¶: Deserializes the InstancePoolPermissionsRequest from a dictionary.

class databricks.sdk.service.compute.InstancePoolState¶

The state of a Cluster. The current allowable state transitions are as follows: - ACTIVE -> STOPPED - ACTIVE -> DELETED - STOPPED -> ACTIVE - STOPPED -> DELETED

ACTIVE = "ACTIVE"¶

DELETED = "DELETED"¶

STOPPED = "STOPPED"¶

class databricks.sdk.service.compute.InstancePoolStats¶

idle_count: int | None = None¶: Number of active instances in the pool that are NOT part of a cluster.

pending_idle_count: int | None = None¶: Number of pending instances in the pool that are NOT part of a cluster.

pending_used_count: int | None = None¶: Number of pending instances in the pool that are part of a cluster.

used_count: int | None = None¶: Number of active instances in the pool that are part of a cluster.

as_dict() → dict¶: Serializes the InstancePoolStats into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolStats into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolStats¶: Deserializes the InstancePoolStats from a dictionary.

class databricks.sdk.service.compute.InstancePoolStatus¶

pending_instance_errors: List[PendingInstanceError] | None = None¶: List of error messages for the failed pending instances. The pending_instance_errors follows FIFO with maximum length of the min_idle of the pool. The pending_instance_errors is emptied once the number of exiting available instances reaches the min_idle of the pool.

as_dict() → dict¶: Serializes the InstancePoolStatus into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstancePoolStatus into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstancePoolStatus¶: Deserializes the InstancePoolStatus from a dictionary.

class databricks.sdk.service.compute.InstanceProfile¶

instance_profile_arn: str¶: The AWS ARN of the instance profile to register with Databricks. This field is required.

iam_role_arn: str | None = None¶

The AWS IAM role ARN of the role associated with the instance profile. This field is required if your role name and instance profile name do not match and you want to use the instance profile with [Databricks SQL Serverless].

Otherwise, this field is optional.

[Databricks SQL Serverless]: https://docs.databricks.com/sql/admin/serverless.html

is_meta_instance_profile: bool | None = None¶: Boolean flag indicating whether the instance profile should only be used in credential passthrough scenarios. If true, it means the instance profile contains an meta IAM role which could assume a wide range of roles. Therefore it should always be used with authorization. This field is optional, the default value is false.

as_dict() → dict¶: Serializes the InstanceProfile into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the InstanceProfile into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → InstanceProfile¶: Deserializes the InstanceProfile from a dictionary.

class databricks.sdk.service.compute.Kind¶

The kind of compute described by this compute specification. Depending on kind, different validations and default values will be applied. Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW. [simple form]: https://docs.databricks.com/compute/simple-form.html

CLASSIC_PREVIEW = "CLASSIC_PREVIEW"¶

class databricks.sdk.service.compute.Language¶

PYTHON = "PYTHON"¶

SCALA = "SCALA"¶

SQL = "SQL"¶

class databricks.sdk.service.compute.Library¶

cran: RCranLibrary | None = None¶: Specification of a CRAN library to be installed as part of the library

egg: str | None = None¶: Deprecated. URI of the egg library to install. Installing Python egg files is deprecated and is not supported in Databricks Runtime 14.0 and above.

jar: str | None = None¶: URI of the JAR library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { “jar”: “/Workspace/path/to/library.jar” }, { “jar” : “/Volumes/path/to/library.jar” } or { “jar”: “s3://my-bucket/library.jar” }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.

maven: MavenLibrary | None = None¶: Specification of a maven library to be installed. For example: { “coordinates”: “org.jsoup:jsoup:1.7.2” }

pypi: PythonPyPiLibrary | None = None¶: Specification of a PyPi library to be installed. For example: { “package”: “simplejson” }

requirements: str | None = None¶: URI of the requirements.txt file to install. Only Workspace paths and Unity Catalog Volumes paths are supported. For example: { “requirements”: “/Workspace/path/to/requirements.txt” } or { “requirements” : “/Volumes/path/to/requirements.txt” }

whl: str | None = None¶: URI of the wheel library to install. Supported URIs include Workspace paths, Unity Catalog Volumes paths, and S3 URIs. For example: { “whl”: “/Workspace/path/to/library.whl” }, { “whl” : “/Volumes/path/to/library.whl” } or { “whl”: “s3://my-bucket/library.whl” }. If S3 is used, please make sure the cluster has read access on the library. You may need to launch the cluster with an IAM role to access the S3 URI.

as_dict() → dict¶: Serializes the Library into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Library into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Library¶: Deserializes the Library from a dictionary.

class databricks.sdk.service.compute.LibraryFullStatus¶

The status of the library on a specific cluster.

is_library_for_all_clusters: bool | None = None¶: Whether the library was set to be installed on all clusters via the libraries UI.

library: Library | None = None¶: Unique identifier for the library.

messages: List[str] | None = None¶: All the info and warning messages that have occurred so far for this library.

status: LibraryInstallStatus | None = None¶: Status of installing the library on the cluster.

as_dict() → dict¶: Serializes the LibraryFullStatus into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the LibraryFullStatus into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → LibraryFullStatus¶: Deserializes the LibraryFullStatus from a dictionary.

class databricks.sdk.service.compute.LibraryInstallStatus¶

The status of a library on a specific cluster.

FAILED = "FAILED"¶

INSTALLED = "INSTALLED"¶

INSTALLING = "INSTALLING"¶

PENDING = "PENDING"¶

RESOLVING = "RESOLVING"¶

RESTORED = "RESTORED"¶

SKIPPED = "SKIPPED"¶

UNINSTALL_ON_RESTART = "UNINSTALL_ON_RESTART"¶

class databricks.sdk.service.compute.ListAllClusterLibraryStatusesResponse¶

statuses: List[ClusterLibraryStatuses] | None = None¶: A list of cluster statuses.

as_dict() → dict¶: Serializes the ListAllClusterLibraryStatusesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListAllClusterLibraryStatusesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListAllClusterLibraryStatusesResponse¶: Deserializes the ListAllClusterLibraryStatusesResponse from a dictionary.

class databricks.sdk.service.compute.ListAvailableZonesResponse¶

default_zone: str | None = None¶: The availability zone if no zone_id is provided in the cluster creation request.

zones: List[str] | None = None¶: The list of available zones (e.g., [‘us-west-2c’, ‘us-east-2’]).

as_dict() → dict¶: Serializes the ListAvailableZonesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListAvailableZonesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListAvailableZonesResponse¶: Deserializes the ListAvailableZonesResponse from a dictionary.

class databricks.sdk.service.compute.ListClusterCompliancesResponse¶

clusters: List[ClusterCompliance] | None = None¶: A list of clusters and their policy compliance statuses.

next_page_token: str | None = None¶: This field represents the pagination token to retrieve the next page of results. If the value is “”, it means no further results for the request.

prev_page_token: str | None = None¶: This field represents the pagination token to retrieve the previous page of results. If the value is “”, it means no further results for the request.

as_dict() → dict¶: Serializes the ListClusterCompliancesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListClusterCompliancesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListClusterCompliancesResponse¶: Deserializes the ListClusterCompliancesResponse from a dictionary.

class databricks.sdk.service.compute.ListClustersFilterBy¶

cluster_sources: List[ClusterSource] | None = None¶: The source of cluster creation.

cluster_states: List[State] | None = None¶: The current state of the clusters.

is_pinned: bool | None = None¶: Whether the clusters are pinned or not.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

as_dict() → dict¶: Serializes the ListClustersFilterBy into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListClustersFilterBy into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListClustersFilterBy¶: Deserializes the ListClustersFilterBy from a dictionary.

class databricks.sdk.service.compute.ListClustersResponse¶

clusters: List[ClusterDetails] | None = None¶

next_page_token: str | None = None¶: This field represents the pagination token to retrieve the next page of results. If the value is “”, it means no further results for the request.

prev_page_token: str | None = None¶: This field represents the pagination token to retrieve the previous page of results. If the value is “”, it means no further results for the request.

as_dict() → dict¶: Serializes the ListClustersResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListClustersResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListClustersResponse¶: Deserializes the ListClustersResponse from a dictionary.

class databricks.sdk.service.compute.ListClustersSortBy¶

direction: ListClustersSortByDirection | None = None¶: The direction to sort by.

field: ListClustersSortByField | None = None¶: The sorting criteria. By default, clusters are sorted by 3 columns from highest to lowest precedence: cluster state, pinned or unpinned, then cluster name.

as_dict() → dict¶: Serializes the ListClustersSortBy into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListClustersSortBy into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListClustersSortBy¶: Deserializes the ListClustersSortBy from a dictionary.

class databricks.sdk.service.compute.ListClustersSortByDirection¶

ASC = "ASC"¶

DESC = "DESC"¶

class databricks.sdk.service.compute.ListClustersSortByField¶

CLUSTER_NAME = "CLUSTER_NAME"¶

DEFAULT = "DEFAULT"¶

class databricks.sdk.service.compute.ListGlobalInitScriptsResponse¶

scripts: List[GlobalInitScriptDetails] | None = None¶

as_dict() → dict¶: Serializes the ListGlobalInitScriptsResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListGlobalInitScriptsResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListGlobalInitScriptsResponse¶: Deserializes the ListGlobalInitScriptsResponse from a dictionary.

class databricks.sdk.service.compute.ListInstancePools¶

instance_pools: List[InstancePoolAndStats] | None = None¶

as_dict() → dict¶: Serializes the ListInstancePools into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListInstancePools into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListInstancePools¶: Deserializes the ListInstancePools from a dictionary.

class databricks.sdk.service.compute.ListInstanceProfilesResponse¶

instance_profiles: List[InstanceProfile] | None = None¶: A list of instance profiles that the user can access.

as_dict() → dict¶: Serializes the ListInstanceProfilesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListInstanceProfilesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListInstanceProfilesResponse¶: Deserializes the ListInstanceProfilesResponse from a dictionary.

class databricks.sdk.service.compute.ListNodeTypesResponse¶

node_types: List[NodeType] | None = None¶: The list of available Spark node types.

as_dict() → dict¶: Serializes the ListNodeTypesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListNodeTypesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListNodeTypesResponse¶: Deserializes the ListNodeTypesResponse from a dictionary.

class databricks.sdk.service.compute.ListPoliciesResponse¶

policies: List[Policy] | None = None¶: List of policies.

as_dict() → dict¶: Serializes the ListPoliciesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListPoliciesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListPoliciesResponse¶: Deserializes the ListPoliciesResponse from a dictionary.

class databricks.sdk.service.compute.ListPolicyFamiliesResponse¶

next_page_token: str | None = None¶: A token that can be used to get the next page of results. If not present, there are no more results to show.

policy_families: List[PolicyFamily] | None = None¶: List of policy families.

as_dict() → dict¶: Serializes the ListPolicyFamiliesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ListPolicyFamiliesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ListPolicyFamiliesResponse¶: Deserializes the ListPolicyFamiliesResponse from a dictionary.

class databricks.sdk.service.compute.ListSortColumn¶

POLICY_CREATION_TIME = "POLICY_CREATION_TIME"¶

POLICY_NAME = "POLICY_NAME"¶

class databricks.sdk.service.compute.ListSortOrder¶

ASC = "ASC"¶

DESC = "DESC"¶

class databricks.sdk.service.compute.LocalFileInfo¶

destination: str¶: local file destination, e.g. file:/my/local/file.sh

as_dict() → dict¶: Serializes the LocalFileInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the LocalFileInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → LocalFileInfo¶: Deserializes the LocalFileInfo from a dictionary.

class databricks.sdk.service.compute.LogAnalyticsInfo¶

log_analytics_primary_key: str | None = None¶

log_analytics_workspace_id: str | None = None¶

as_dict() → dict¶: Serializes the LogAnalyticsInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the LogAnalyticsInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → LogAnalyticsInfo¶: Deserializes the LogAnalyticsInfo from a dictionary.

class databricks.sdk.service.compute.LogSyncStatus¶

The log delivery status

last_attempted: int | None = None¶: The timestamp of last attempt. If the last attempt fails, last_exception will contain the exception in the last attempt.

last_exception: str | None = None¶: The exception thrown in the last attempt, it would be null (omitted in the response) if there is no exception in last attempted.

as_dict() → dict¶: Serializes the LogSyncStatus into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the LogSyncStatus into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → LogSyncStatus¶: Deserializes the LogSyncStatus from a dictionary.

class databricks.sdk.service.compute.MavenLibrary¶

coordinates: str¶: Gradle-style maven coordinates. For example: “org.jsoup:jsoup:1.7.2”.

exclusions: List[str] | None = None¶

List of dependences to exclude. For example: [“slf4j:slf4j”, “*:hadoop-client”].

Maven dependency exclusions: https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html.

repo: str | None = None¶: Maven repo to install the Maven package from. If omitted, both Maven Central Repository and Spark Packages are searched.

as_dict() → dict¶: Serializes the MavenLibrary into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the MavenLibrary into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → MavenLibrary¶: Deserializes the MavenLibrary from a dictionary.

class databricks.sdk.service.compute.NodeInstanceType¶

This structure embodies the machine type that hosts spark containers Note: this should be an internal data structure for now It is defined in proto in case we want to send it over the wire in the future (which is likely)

instance_type_id: str¶: Unique identifier across instance types

local_disk_size_gb: int | None = None¶: Size of the individual local disks attached to this instance (i.e. per local disk).

local_disks: int | None = None¶: Number of local disks that are present on this instance.

local_nvme_disk_size_gb: int | None = None¶: Size of the individual local nvme disks attached to this instance (i.e. per local disk).

local_nvme_disks: int | None = None¶: Number of local nvme disks that are present on this instance.

as_dict() → dict¶: Serializes the NodeInstanceType into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the NodeInstanceType into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → NodeInstanceType¶: Deserializes the NodeInstanceType from a dictionary.

class databricks.sdk.service.compute.NodeType¶

A description of a Spark node type including both the dimensions of the node and the instance type on which it will be hosted.

node_type_id: str¶: Unique identifier for this node type.

memory_mb: int¶: Memory (in MB) available for this node type.

num_cores: float¶: Number of CPU cores available for this node type. Note that this can be fractional, e.g., 2.5 cores, if the the number of cores on a machine instance is not divisible by the number of Spark nodes on that machine.

description: str¶: A string description associated with this node type, e.g., “r3.xlarge”.

instance_type_id: str¶: An identifier for the type of hardware that this node runs on, e.g., “r3.2xlarge” in AWS.

category: str¶: A descriptive category for this node type. Examples include “Memory Optimized” and “Compute Optimized”.

display_order: int | None = None¶: An optional hint at the display order of node types in the UI. Within a node type category, lowest numbers come first.

is_deprecated: bool | None = None¶: Whether the node type is deprecated. Non-deprecated node types offer greater performance.

is_encrypted_in_transit: bool | None = None¶: AWS specific, whether this instance supports encryption in transit, used for hipaa and pci workloads.

is_graviton: bool | None = None¶: Whether this is an Arm-based instance.

is_hidden: bool | None = None¶: Whether this node is hidden from presentation in the UI.

is_io_cache_enabled: bool | None = None¶: Whether this node comes with IO cache enabled by default.

node_info: CloudProviderNodeInfo | None = None¶: A collection of node type info reported by the cloud provider

node_instance_type: NodeInstanceType | None = None¶: The NodeInstanceType object corresponding to instance_type_id

num_gpus: int | None = None¶: Number of GPUs available for this node type.

photon_driver_capable: bool | None = None¶

photon_worker_capable: bool | None = None¶

support_cluster_tags: bool | None = None¶: Whether this node type support cluster tags.

support_ebs_volumes: bool | None = None¶: Whether this node type support EBS volumes. EBS volumes is disabled for node types that we could place multiple corresponding containers on the same hosting instance.

support_port_forwarding: bool | None = None¶: Whether this node type supports port forwarding.

as_dict() → dict¶: Serializes the NodeType into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the NodeType into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → NodeType¶: Deserializes the NodeType from a dictionary.

class databricks.sdk.service.compute.PendingInstanceError¶

Error message of a failed pending instances

instance_id: str | None = None¶

message: str | None = None¶

as_dict() → dict¶: Serializes the PendingInstanceError into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PendingInstanceError into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PendingInstanceError¶: Deserializes the PendingInstanceError from a dictionary.

class databricks.sdk.service.compute.PermanentDeleteCluster¶

cluster_id: str¶: The cluster to be deleted.

as_dict() → dict¶: Serializes the PermanentDeleteCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PermanentDeleteCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PermanentDeleteCluster¶: Deserializes the PermanentDeleteCluster from a dictionary.

class databricks.sdk.service.compute.PermanentDeleteClusterResponse¶

as_dict() → dict¶: Serializes the PermanentDeleteClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PermanentDeleteClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PermanentDeleteClusterResponse¶: Deserializes the PermanentDeleteClusterResponse from a dictionary.

class databricks.sdk.service.compute.PinCluster¶

cluster_id: str¶

as_dict() → dict¶: Serializes the PinCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PinCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PinCluster¶: Deserializes the PinCluster from a dictionary.

class databricks.sdk.service.compute.PinClusterResponse¶

as_dict() → dict¶: Serializes the PinClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PinClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PinClusterResponse¶: Deserializes the PinClusterResponse from a dictionary.

class databricks.sdk.service.compute.Policy¶

Describes a Cluster Policy entity.

created_at_timestamp: int | None = None¶: Creation time. The timestamp (in millisecond) when this Cluster Policy was created.

creator_user_name: str | None = None¶: Creator user name. The field won’t be included in the response if the user has already been deleted.

definition: str | None = None¶

Policy definition document expressed in [Databricks Cluster Policy Definition Language].

[Databricks Cluster Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

description: str | None = None¶: Additional human-readable description of the cluster policy.

is_default: bool | None = None¶: If true, policy is a default policy created and managed by Databricks. Default policies cannot be deleted, and their policy families cannot be changed.

libraries: List[Library] | None = None¶: A list of libraries to be installed on the next cluster restart that uses this policy. The maximum number of libraries is 500.

max_clusters_per_user: int | None = None¶: Max number of clusters per user that can be active using this policy. If not present, there is no max limit.

name: str | None = None¶: Cluster Policy name requested by the user. This has to be unique. Length must be between 1 and 100 characters.

policy_family_definition_overrides: str | None = None¶

Policy definition JSON document expressed in [Databricks Policy Definition Language]. The JSON document must be passed as a string and cannot be embedded in the requests.

You can use this to customize the policy definition inherited from the policy family. Policy rules specified here are merged into the inherited policy definition.

[Databricks Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

policy_family_id: str | None = None¶

ID of the policy family. The cluster policy’s policy definition inherits the policy family’s policy definition.

Cannot be used with definition. Use policy_family_definition_overrides instead to customize the policy definition.

policy_id: str | None = None¶: Canonical unique identifier for the Cluster Policy.

as_dict() → dict¶: Serializes the Policy into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Policy into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Policy¶: Deserializes the Policy from a dictionary.

class databricks.sdk.service.compute.PolicyFamily¶

definition: str | None = None¶

Policy definition document expressed in [Databricks Cluster Policy Definition Language].

[Databricks Cluster Policy Definition Language]: https://docs.databricks.com/administration-guide/clusters/policy-definition.html

description: str | None = None¶: Human-readable description of the purpose of the policy family.

name: str | None = None¶: Name of the policy family.

policy_family_id: str | None = None¶: Unique identifier for the policy family.

as_dict() → dict¶: Serializes the PolicyFamily into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PolicyFamily into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PolicyFamily¶: Deserializes the PolicyFamily from a dictionary.

class databricks.sdk.service.compute.PythonPyPiLibrary¶

package: str¶: The name of the pypi package to install. An optional exact version specification is also supported. Examples: “simplejson” and “simplejson==3.8.0”.

repo: str | None = None¶: The repository where the package can be found. If not specified, the default pip index is used.

as_dict() → dict¶: Serializes the PythonPyPiLibrary into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the PythonPyPiLibrary into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → PythonPyPiLibrary¶: Deserializes the PythonPyPiLibrary from a dictionary.

class databricks.sdk.service.compute.RCranLibrary¶

package: str¶: The name of the CRAN package to install.

repo: str | None = None¶: The repository where the package can be found. If not specified, the default CRAN repo is used.

as_dict() → dict¶: Serializes the RCranLibrary into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the RCranLibrary into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → RCranLibrary¶: Deserializes the RCranLibrary from a dictionary.

class databricks.sdk.service.compute.RemoveInstanceProfile¶

instance_profile_arn: str¶: The ARN of the instance profile to remove. This field is required.

as_dict() → dict¶: Serializes the RemoveInstanceProfile into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the RemoveInstanceProfile into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → RemoveInstanceProfile¶: Deserializes the RemoveInstanceProfile from a dictionary.

class databricks.sdk.service.compute.RemoveResponse¶

as_dict() → dict¶: Serializes the RemoveResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the RemoveResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → RemoveResponse¶: Deserializes the RemoveResponse from a dictionary.

class databricks.sdk.service.compute.ResizeCluster¶

cluster_id: str¶: The cluster to be resized.

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

as_dict() → dict¶: Serializes the ResizeCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ResizeCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ResizeCluster¶: Deserializes the ResizeCluster from a dictionary.

class databricks.sdk.service.compute.ResizeClusterResponse¶

as_dict() → dict¶: Serializes the ResizeClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the ResizeClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → ResizeClusterResponse¶: Deserializes the ResizeClusterResponse from a dictionary.

class databricks.sdk.service.compute.RestartCluster¶

cluster_id: str¶: The cluster to be started.

restart_user: str | None = None¶

as_dict() → dict¶: Serializes the RestartCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the RestartCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → RestartCluster¶: Deserializes the RestartCluster from a dictionary.

class databricks.sdk.service.compute.RestartClusterResponse¶

as_dict() → dict¶: Serializes the RestartClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the RestartClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → RestartClusterResponse¶: Deserializes the RestartClusterResponse from a dictionary.

class databricks.sdk.service.compute.ResultType¶

ERROR = "ERROR"¶

IMAGE = "IMAGE"¶

IMAGES = "IMAGES"¶

TABLE = "TABLE"¶

TEXT = "TEXT"¶

class databricks.sdk.service.compute.Results¶

cause: str | None = None¶: The cause of the error

data: Any | None = None¶

file_name: str | None = None¶: The image filename

file_names: List[str] | None = None¶

is_json_schema: bool | None = None¶: true if a JSON schema is returned instead of a string representation of the Hive type.

pos: int | None = None¶: internal field used by SDK

result_type: ResultType | None = None¶

schema: List[Dict[str, Any]] | None = None¶: The table schema

summary: str | None = None¶: The summary of the error

truncated: bool | None = None¶: true if partial results are returned.

as_dict() → dict¶: Serializes the Results into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the Results into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → Results¶: Deserializes the Results from a dictionary.

class databricks.sdk.service.compute.RuntimeEngine¶

NULL = "NULL"¶

PHOTON = "PHOTON"¶

STANDARD = "STANDARD"¶

class databricks.sdk.service.compute.S3StorageInfo¶

A storage location in Amazon S3

destination: str¶: S3 destination, e.g. s3://my-bucket/some-prefix Note that logs will be delivered using cluster iam role, please make sure you set cluster iam role and the role has write access to the destination. Please also note that you cannot use AWS keys to deliver logs.

canned_acl: str | None = None¶: (Optional) Set canned access control list for the logs, e.g. bucket-owner-full-control. If canned_cal is set, please make sure the cluster iam role has s3:PutObjectAcl permission on the destination bucket and prefix. The full list of possible canned acl can be found at http://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl. Please also note that by default only the object owner gets full controls. If you are using cross account role for writing data, you may want to set bucket-owner-full-control to make bucket owner able to read the logs.

enable_encryption: bool | None = None¶: (Optional) Flag to enable server side encryption, false by default.

encryption_type: str | None = None¶: (Optional) The encryption type, it could be sse-s3 or sse-kms. It will be used only when encryption is enabled and the default type is sse-s3.

endpoint: str | None = None¶: S3 endpoint, e.g. https://s3-us-west-2.amazonaws.com. Either region or endpoint needs to be set. If both are set, endpoint will be used.

kms_key: str | None = None¶: (Optional) Kms key which will be used if encryption is enabled and encryption type is set to sse-kms.

region: str | None = None¶: S3 region, e.g. us-west-2. Either region or endpoint needs to be set. If both are set, endpoint will be used.

as_dict() → dict¶: Serializes the S3StorageInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the S3StorageInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → S3StorageInfo¶: Deserializes the S3StorageInfo from a dictionary.

class databricks.sdk.service.compute.SparkNode¶

Describes a specific Spark driver or executor.

host_private_ip: str | None = None¶: The private IP address of the host instance.

instance_id: str | None = None¶: Globally unique identifier for the host instance from the cloud provider.

node_aws_attributes: SparkNodeAwsAttributes | None = None¶: Attributes specific to AWS for a Spark node.

node_id: str | None = None¶: Globally unique identifier for this node.

private_ip: str | None = None¶: Private IP address (typically a 10.x.x.x address) of the Spark node. Note that this is different from the private IP address of the host instance.

public_dns: str | None = None¶: Public DNS address of this node. This address can be used to access the Spark JDBC server on the driver node. To communicate with the JDBC server, traffic must be manually authorized by adding security group rules to the “worker-unmanaged” security group via the AWS console.

start_timestamp: int | None = None¶: The timestamp (in millisecond) when the Spark node is launched.

as_dict() → dict¶: Serializes the SparkNode into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the SparkNode into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → SparkNode¶: Deserializes the SparkNode from a dictionary.

class databricks.sdk.service.compute.SparkNodeAwsAttributes¶

Attributes specific to AWS for a Spark node.

is_spot: bool | None = None¶: Whether this node is on an Amazon spot instance.

as_dict() → dict¶: Serializes the SparkNodeAwsAttributes into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the SparkNodeAwsAttributes into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → SparkNodeAwsAttributes¶: Deserializes the SparkNodeAwsAttributes from a dictionary.

class databricks.sdk.service.compute.SparkVersion¶

key: str | None = None¶: Spark version key, for example “2.1.x-scala2.11”. This is the value which should be provided as the “spark_version” when creating a new cluster. Note that the exact Spark version may change over time for a “wildcard” version (i.e., “2.1.x-scala2.11” is a “wildcard” version) with minor bug fixes.

name: str | None = None¶: A descriptive name for this Spark version, for example “Spark 2.1”.

as_dict() → dict¶: Serializes the SparkVersion into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the SparkVersion into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → SparkVersion¶: Deserializes the SparkVersion from a dictionary.

class databricks.sdk.service.compute.StartCluster¶

cluster_id: str¶: The cluster to be started.

as_dict() → dict¶: Serializes the StartCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the StartCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → StartCluster¶: Deserializes the StartCluster from a dictionary.

class databricks.sdk.service.compute.StartClusterResponse¶

as_dict() → dict¶: Serializes the StartClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the StartClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → StartClusterResponse¶: Deserializes the StartClusterResponse from a dictionary.

class databricks.sdk.service.compute.State¶

The state of a Cluster. The current allowable state transitions are as follows: - PENDING -> RUNNING - PENDING -> TERMINATING - RUNNING -> RESIZING - RUNNING -> RESTARTING - RUNNING -> TERMINATING - RESTARTING -> RUNNING - RESTARTING -> TERMINATING - RESIZING -> RUNNING - RESIZING -> TERMINATING - TERMINATING -> TERMINATED

ERROR = "ERROR"¶

PENDING = "PENDING"¶

RESIZING = "RESIZING"¶

RESTARTING = "RESTARTING"¶

RUNNING = "RUNNING"¶

TERMINATED = "TERMINATED"¶

TERMINATING = "TERMINATING"¶

UNKNOWN = "UNKNOWN"¶

class databricks.sdk.service.compute.TerminationReason¶

code: TerminationReasonCode | None = None¶: status code indicating why the cluster was terminated

parameters: Dict[str, str] | None = None¶: list of parameters that provide additional information about why the cluster was terminated

type: TerminationReasonType | None = None¶: type of the termination

as_dict() → dict¶: Serializes the TerminationReason into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the TerminationReason into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → TerminationReason¶: Deserializes the TerminationReason from a dictionary.

class databricks.sdk.service.compute.TerminationReasonCode¶

The status code indicating why the cluster was terminated

ABUSE_DETECTED = "ABUSE_DETECTED"¶

ACCESS_TOKEN_FAILURE = "ACCESS_TOKEN_FAILURE"¶

ALLOCATION_TIMEOUT = "ALLOCATION_TIMEOUT"¶

ALLOCATION_TIMEOUT_NODE_DAEMON_NOT_READY = "ALLOCATION_TIMEOUT_NODE_DAEMON_NOT_READY"¶

ALLOCATION_TIMEOUT_NO_HEALTHY_AND_WARMED_UP_CLUSTERS = "ALLOCATION_TIMEOUT_NO_HEALTHY_AND_WARMED_UP_CLUSTERS"¶

ALLOCATION_TIMEOUT_NO_HEALTHY_CLUSTERS = "ALLOCATION_TIMEOUT_NO_HEALTHY_CLUSTERS"¶

ALLOCATION_TIMEOUT_NO_MATCHED_CLUSTERS = "ALLOCATION_TIMEOUT_NO_MATCHED_CLUSTERS"¶

ALLOCATION_TIMEOUT_NO_READY_CLUSTERS = "ALLOCATION_TIMEOUT_NO_READY_CLUSTERS"¶

ALLOCATION_TIMEOUT_NO_UNALLOCATED_CLUSTERS = "ALLOCATION_TIMEOUT_NO_UNALLOCATED_CLUSTERS"¶

ALLOCATION_TIMEOUT_NO_WARMED_UP_CLUSTERS = "ALLOCATION_TIMEOUT_NO_WARMED_UP_CLUSTERS"¶

ATTACH_PROJECT_FAILURE = "ATTACH_PROJECT_FAILURE"¶

AWS_AUTHORIZATION_FAILURE = "AWS_AUTHORIZATION_FAILURE"¶

AWS_INACCESSIBLE_KMS_KEY_FAILURE = "AWS_INACCESSIBLE_KMS_KEY_FAILURE"¶

AWS_INSTANCE_PROFILE_UPDATE_FAILURE = "AWS_INSTANCE_PROFILE_UPDATE_FAILURE"¶

AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE = "AWS_INSUFFICIENT_FREE_ADDRESSES_IN_SUBNET_FAILURE"¶

AWS_INSUFFICIENT_INSTANCE_CAPACITY_FAILURE = "AWS_INSUFFICIENT_INSTANCE_CAPACITY_FAILURE"¶

AWS_INVALID_KEY_PAIR = "AWS_INVALID_KEY_PAIR"¶

AWS_INVALID_KMS_KEY_STATE = "AWS_INVALID_KMS_KEY_STATE"¶

AWS_MAX_SPOT_INSTANCE_COUNT_EXCEEDED_FAILURE = "AWS_MAX_SPOT_INSTANCE_COUNT_EXCEEDED_FAILURE"¶

AWS_REQUEST_LIMIT_EXCEEDED = "AWS_REQUEST_LIMIT_EXCEEDED"¶

AWS_RESOURCE_QUOTA_EXCEEDED = "AWS_RESOURCE_QUOTA_EXCEEDED"¶

AWS_UNSUPPORTED_FAILURE = "AWS_UNSUPPORTED_FAILURE"¶

AZURE_BYOK_KEY_PERMISSION_FAILURE = "AZURE_BYOK_KEY_PERMISSION_FAILURE"¶

AZURE_EPHEMERAL_DISK_FAILURE = "AZURE_EPHEMERAL_DISK_FAILURE"¶

AZURE_INVALID_DEPLOYMENT_TEMPLATE = "AZURE_INVALID_DEPLOYMENT_TEMPLATE"¶

AZURE_OPERATION_NOT_ALLOWED_EXCEPTION = "AZURE_OPERATION_NOT_ALLOWED_EXCEPTION"¶

AZURE_PACKED_DEPLOYMENT_PARTIAL_FAILURE = "AZURE_PACKED_DEPLOYMENT_PARTIAL_FAILURE"¶

AZURE_QUOTA_EXCEEDED_EXCEPTION = "AZURE_QUOTA_EXCEEDED_EXCEPTION"¶

AZURE_RESOURCE_MANAGER_THROTTLING = "AZURE_RESOURCE_MANAGER_THROTTLING"¶

AZURE_RESOURCE_PROVIDER_THROTTLING = "AZURE_RESOURCE_PROVIDER_THROTTLING"¶

AZURE_UNEXPECTED_DEPLOYMENT_TEMPLATE_FAILURE = "AZURE_UNEXPECTED_DEPLOYMENT_TEMPLATE_FAILURE"¶

AZURE_VM_EXTENSION_FAILURE = "AZURE_VM_EXTENSION_FAILURE"¶

AZURE_VNET_CONFIGURATION_FAILURE = "AZURE_VNET_CONFIGURATION_FAILURE"¶

BOOTSTRAP_TIMEOUT = "BOOTSTRAP_TIMEOUT"¶

BOOTSTRAP_TIMEOUT_CLOUD_PROVIDER_EXCEPTION = "BOOTSTRAP_TIMEOUT_CLOUD_PROVIDER_EXCEPTION"¶

BOOTSTRAP_TIMEOUT_DUE_TO_MISCONFIG = "BOOTSTRAP_TIMEOUT_DUE_TO_MISCONFIG"¶

BUDGET_POLICY_LIMIT_ENFORCEMENT_ACTIVATED = "BUDGET_POLICY_LIMIT_ENFORCEMENT_ACTIVATED"¶

BUDGET_POLICY_RESOLUTION_FAILURE = "BUDGET_POLICY_RESOLUTION_FAILURE"¶

CLOUD_ACCOUNT_SETUP_FAILURE = "CLOUD_ACCOUNT_SETUP_FAILURE"¶

CLOUD_OPERATION_CANCELLED = "CLOUD_OPERATION_CANCELLED"¶

CLOUD_PROVIDER_DISK_SETUP_FAILURE = "CLOUD_PROVIDER_DISK_SETUP_FAILURE"¶

CLOUD_PROVIDER_INSTANCE_NOT_LAUNCHED = "CLOUD_PROVIDER_INSTANCE_NOT_LAUNCHED"¶

CLOUD_PROVIDER_LAUNCH_FAILURE = "CLOUD_PROVIDER_LAUNCH_FAILURE"¶

CLOUD_PROVIDER_LAUNCH_FAILURE_DUE_TO_MISCONFIG = "CLOUD_PROVIDER_LAUNCH_FAILURE_DUE_TO_MISCONFIG"¶

CLOUD_PROVIDER_RESOURCE_STOCKOUT = "CLOUD_PROVIDER_RESOURCE_STOCKOUT"¶

CLOUD_PROVIDER_RESOURCE_STOCKOUT_DUE_TO_MISCONFIG = "CLOUD_PROVIDER_RESOURCE_STOCKOUT_DUE_TO_MISCONFIG"¶

CLOUD_PROVIDER_SHUTDOWN = "CLOUD_PROVIDER_SHUTDOWN"¶

CLUSTER_OPERATION_THROTTLED = "CLUSTER_OPERATION_THROTTLED"¶

CLUSTER_OPERATION_TIMEOUT = "CLUSTER_OPERATION_TIMEOUT"¶

COMMUNICATION_LOST = "COMMUNICATION_LOST"¶

CONTAINER_LAUNCH_FAILURE = "CONTAINER_LAUNCH_FAILURE"¶

CONTROL_PLANE_REQUEST_FAILURE = "CONTROL_PLANE_REQUEST_FAILURE"¶

CONTROL_PLANE_REQUEST_FAILURE_DUE_TO_MISCONFIG = "CONTROL_PLANE_REQUEST_FAILURE_DUE_TO_MISCONFIG"¶

DATABASE_CONNECTION_FAILURE = "DATABASE_CONNECTION_FAILURE"¶

DATA_ACCESS_CONFIG_CHANGED = "DATA_ACCESS_CONFIG_CHANGED"¶

DBFS_COMPONENT_UNHEALTHY = "DBFS_COMPONENT_UNHEALTHY"¶

DISASTER_RECOVERY_REPLICATION = "DISASTER_RECOVERY_REPLICATION"¶

DNS_RESOLUTION_ERROR = "DNS_RESOLUTION_ERROR"¶

DOCKER_CONTAINER_CREATION_EXCEPTION = "DOCKER_CONTAINER_CREATION_EXCEPTION"¶

DOCKER_IMAGE_PULL_FAILURE = "DOCKER_IMAGE_PULL_FAILURE"¶

DOCKER_IMAGE_TOO_LARGE_FOR_INSTANCE_EXCEPTION = "DOCKER_IMAGE_TOO_LARGE_FOR_INSTANCE_EXCEPTION"¶

DOCKER_INVALID_OS_EXCEPTION = "DOCKER_INVALID_OS_EXCEPTION"¶

DRIVER_EVICTION = "DRIVER_EVICTION"¶

DRIVER_LAUNCH_TIMEOUT = "DRIVER_LAUNCH_TIMEOUT"¶

DRIVER_NODE_UNREACHABLE = "DRIVER_NODE_UNREACHABLE"¶

DRIVER_OUT_OF_DISK = "DRIVER_OUT_OF_DISK"¶

DRIVER_OUT_OF_MEMORY = "DRIVER_OUT_OF_MEMORY"¶

DRIVER_POD_CREATION_FAILURE = "DRIVER_POD_CREATION_FAILURE"¶

DRIVER_UNEXPECTED_FAILURE = "DRIVER_UNEXPECTED_FAILURE"¶

DRIVER_UNHEALTHY = "DRIVER_UNHEALTHY"¶

DRIVER_UNREACHABLE = "DRIVER_UNREACHABLE"¶

DRIVER_UNRESPONSIVE = "DRIVER_UNRESPONSIVE"¶

DYNAMIC_SPARK_CONF_SIZE_EXCEEDED = "DYNAMIC_SPARK_CONF_SIZE_EXCEEDED"¶

EOS_SPARK_IMAGE = "EOS_SPARK_IMAGE"¶

EXECUTION_COMPONENT_UNHEALTHY = "EXECUTION_COMPONENT_UNHEALTHY"¶

EXECUTOR_POD_UNSCHEDULED = "EXECUTOR_POD_UNSCHEDULED"¶

GCP_API_RATE_QUOTA_EXCEEDED = "GCP_API_RATE_QUOTA_EXCEEDED"¶

GCP_DENIED_BY_ORG_POLICY = "GCP_DENIED_BY_ORG_POLICY"¶

GCP_FORBIDDEN = "GCP_FORBIDDEN"¶

GCP_IAM_TIMEOUT = "GCP_IAM_TIMEOUT"¶

GCP_INACCESSIBLE_KMS_KEY_FAILURE = "GCP_INACCESSIBLE_KMS_KEY_FAILURE"¶

GCP_INSUFFICIENT_CAPACITY = "GCP_INSUFFICIENT_CAPACITY"¶

GCP_IP_SPACE_EXHAUSTED = "GCP_IP_SPACE_EXHAUSTED"¶

GCP_KMS_KEY_PERMISSION_DENIED = "GCP_KMS_KEY_PERMISSION_DENIED"¶

GCP_NOT_FOUND = "GCP_NOT_FOUND"¶

GCP_QUOTA_EXCEEDED = "GCP_QUOTA_EXCEEDED"¶

GCP_RESOURCE_QUOTA_EXCEEDED = "GCP_RESOURCE_QUOTA_EXCEEDED"¶

GCP_SERVICE_ACCOUNT_ACCESS_DENIED = "GCP_SERVICE_ACCOUNT_ACCESS_DENIED"¶

GCP_SERVICE_ACCOUNT_DELETED = "GCP_SERVICE_ACCOUNT_DELETED"¶

GCP_SERVICE_ACCOUNT_NOT_FOUND = "GCP_SERVICE_ACCOUNT_NOT_FOUND"¶

GCP_SUBNET_NOT_READY = "GCP_SUBNET_NOT_READY"¶

GCP_TRUSTED_IMAGE_PROJECTS_VIOLATED = "GCP_TRUSTED_IMAGE_PROJECTS_VIOLATED"¶

GKE_BASED_CLUSTER_TERMINATION = "GKE_BASED_CLUSTER_TERMINATION"¶

GLOBAL_INIT_SCRIPT_FAILURE = "GLOBAL_INIT_SCRIPT_FAILURE"¶

HIVE_METASTORE_PROVISIONING_FAILURE = "HIVE_METASTORE_PROVISIONING_FAILURE"¶

IMAGE_PULL_PERMISSION_DENIED = "IMAGE_PULL_PERMISSION_DENIED"¶

INACTIVITY = "INACTIVITY"¶

INIT_CONTAINER_NOT_FINISHED = "INIT_CONTAINER_NOT_FINISHED"¶

INIT_SCRIPT_FAILURE = "INIT_SCRIPT_FAILURE"¶

INSTANCE_POOL_CLUSTER_FAILURE = "INSTANCE_POOL_CLUSTER_FAILURE"¶

INSTANCE_POOL_MAX_CAPACITY_REACHED = "INSTANCE_POOL_MAX_CAPACITY_REACHED"¶

INSTANCE_POOL_NOT_FOUND = "INSTANCE_POOL_NOT_FOUND"¶

INSTANCE_UNREACHABLE = "INSTANCE_UNREACHABLE"¶

INSTANCE_UNREACHABLE_DUE_TO_MISCONFIG = "INSTANCE_UNREACHABLE_DUE_TO_MISCONFIG"¶

INTERNAL_CAPACITY_FAILURE = "INTERNAL_CAPACITY_FAILURE"¶

INTERNAL_ERROR = "INTERNAL_ERROR"¶

INVALID_ARGUMENT = "INVALID_ARGUMENT"¶

INVALID_AWS_PARAMETER = "INVALID_AWS_PARAMETER"¶

INVALID_INSTANCE_PLACEMENT_PROTOCOL = "INVALID_INSTANCE_PLACEMENT_PROTOCOL"¶

INVALID_SPARK_IMAGE = "INVALID_SPARK_IMAGE"¶

INVALID_WORKER_IMAGE_FAILURE = "INVALID_WORKER_IMAGE_FAILURE"¶

IN_PENALTY_BOX = "IN_PENALTY_BOX"¶

IP_EXHAUSTION_FAILURE = "IP_EXHAUSTION_FAILURE"¶

JOB_FINISHED = "JOB_FINISHED"¶

K8S_AUTOSCALING_FAILURE = "K8S_AUTOSCALING_FAILURE"¶

K8S_DBR_CLUSTER_LAUNCH_TIMEOUT = "K8S_DBR_CLUSTER_LAUNCH_TIMEOUT"¶

LAZY_ALLOCATION_TIMEOUT = "LAZY_ALLOCATION_TIMEOUT"¶

MAINTENANCE_MODE = "MAINTENANCE_MODE"¶

METASTORE_COMPONENT_UNHEALTHY = "METASTORE_COMPONENT_UNHEALTHY"¶

NEPHOS_RESOURCE_MANAGEMENT = "NEPHOS_RESOURCE_MANAGEMENT"¶

NETVISOR_SETUP_TIMEOUT = "NETVISOR_SETUP_TIMEOUT"¶

NETWORK_CHECK_CONTROL_PLANE_FAILURE = "NETWORK_CHECK_CONTROL_PLANE_FAILURE"¶

NETWORK_CHECK_DNS_SERVER_FAILURE = "NETWORK_CHECK_DNS_SERVER_FAILURE"¶

NETWORK_CHECK_METADATA_ENDPOINT_FAILURE = "NETWORK_CHECK_METADATA_ENDPOINT_FAILURE"¶

NETWORK_CHECK_MULTIPLE_COMPONENTS_FAILURE = "NETWORK_CHECK_MULTIPLE_COMPONENTS_FAILURE"¶

NETWORK_CHECK_NIC_FAILURE = "NETWORK_CHECK_NIC_FAILURE"¶

NETWORK_CHECK_STORAGE_FAILURE = "NETWORK_CHECK_STORAGE_FAILURE"¶

NETWORK_CONFIGURATION_FAILURE = "NETWORK_CONFIGURATION_FAILURE"¶

NFS_MOUNT_FAILURE = "NFS_MOUNT_FAILURE"¶

NO_MATCHED_K8S = "NO_MATCHED_K8S"¶

NO_MATCHED_K8S_TESTING_TAG = "NO_MATCHED_K8S_TESTING_TAG"¶

NPIP_TUNNEL_SETUP_FAILURE = "NPIP_TUNNEL_SETUP_FAILURE"¶

NPIP_TUNNEL_TOKEN_FAILURE = "NPIP_TUNNEL_TOKEN_FAILURE"¶

POD_ASSIGNMENT_FAILURE = "POD_ASSIGNMENT_FAILURE"¶

POD_SCHEDULING_FAILURE = "POD_SCHEDULING_FAILURE"¶

REQUEST_REJECTED = "REQUEST_REJECTED"¶

REQUEST_THROTTLED = "REQUEST_THROTTLED"¶

RESOURCE_USAGE_BLOCKED = "RESOURCE_USAGE_BLOCKED"¶

SECRET_CREATION_FAILURE = "SECRET_CREATION_FAILURE"¶

SECRET_PERMISSION_DENIED = "SECRET_PERMISSION_DENIED"¶

SECRET_RESOLUTION_ERROR = "SECRET_RESOLUTION_ERROR"¶

SECURITY_DAEMON_REGISTRATION_EXCEPTION = "SECURITY_DAEMON_REGISTRATION_EXCEPTION"¶

SELF_BOOTSTRAP_FAILURE = "SELF_BOOTSTRAP_FAILURE"¶

SERVERLESS_LONG_RUNNING_TERMINATED = "SERVERLESS_LONG_RUNNING_TERMINATED"¶

SKIPPED_SLOW_NODES = "SKIPPED_SLOW_NODES"¶

SLOW_IMAGE_DOWNLOAD = "SLOW_IMAGE_DOWNLOAD"¶

SPARK_ERROR = "SPARK_ERROR"¶

SPARK_IMAGE_DOWNLOAD_FAILURE = "SPARK_IMAGE_DOWNLOAD_FAILURE"¶

SPARK_IMAGE_DOWNLOAD_THROTTLED = "SPARK_IMAGE_DOWNLOAD_THROTTLED"¶

SPARK_IMAGE_NOT_FOUND = "SPARK_IMAGE_NOT_FOUND"¶

SPARK_STARTUP_FAILURE = "SPARK_STARTUP_FAILURE"¶

SPOT_INSTANCE_TERMINATION = "SPOT_INSTANCE_TERMINATION"¶

SSH_BOOTSTRAP_FAILURE = "SSH_BOOTSTRAP_FAILURE"¶

STORAGE_DOWNLOAD_FAILURE = "STORAGE_DOWNLOAD_FAILURE"¶

STORAGE_DOWNLOAD_FAILURE_DUE_TO_MISCONFIG = "STORAGE_DOWNLOAD_FAILURE_DUE_TO_MISCONFIG"¶

STORAGE_DOWNLOAD_FAILURE_SLOW = "STORAGE_DOWNLOAD_FAILURE_SLOW"¶

STORAGE_DOWNLOAD_FAILURE_THROTTLED = "STORAGE_DOWNLOAD_FAILURE_THROTTLED"¶

STS_CLIENT_SETUP_FAILURE = "STS_CLIENT_SETUP_FAILURE"¶

SUBNET_EXHAUSTED_FAILURE = "SUBNET_EXHAUSTED_FAILURE"¶

TEMPORARILY_UNAVAILABLE = "TEMPORARILY_UNAVAILABLE"¶

TRIAL_EXPIRED = "TRIAL_EXPIRED"¶

UNEXPECTED_LAUNCH_FAILURE = "UNEXPECTED_LAUNCH_FAILURE"¶

UNEXPECTED_POD_RECREATION = "UNEXPECTED_POD_RECREATION"¶

UNKNOWN = "UNKNOWN"¶

UNSUPPORTED_INSTANCE_TYPE = "UNSUPPORTED_INSTANCE_TYPE"¶

UPDATE_INSTANCE_PROFILE_FAILURE = "UPDATE_INSTANCE_PROFILE_FAILURE"¶

USER_INITIATED_VM_TERMINATION = "USER_INITIATED_VM_TERMINATION"¶

USER_REQUEST = "USER_REQUEST"¶

WORKER_SETUP_FAILURE = "WORKER_SETUP_FAILURE"¶

WORKSPACE_CANCELLED_ERROR = "WORKSPACE_CANCELLED_ERROR"¶

WORKSPACE_CONFIGURATION_ERROR = "WORKSPACE_CONFIGURATION_ERROR"¶

WORKSPACE_UPDATE = "WORKSPACE_UPDATE"¶

class databricks.sdk.service.compute.TerminationReasonType¶

type of the termination

CLIENT_ERROR = "CLIENT_ERROR"¶

CLOUD_FAILURE = "CLOUD_FAILURE"¶

SERVICE_FAULT = "SERVICE_FAULT"¶

SUCCESS = "SUCCESS"¶

class databricks.sdk.service.compute.UninstallLibraries¶

cluster_id: str¶: Unique identifier for the cluster on which to uninstall these libraries.

libraries: List[Library]¶: The libraries to uninstall.

as_dict() → dict¶: Serializes the UninstallLibraries into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UninstallLibraries into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UninstallLibraries¶: Deserializes the UninstallLibraries from a dictionary.

class databricks.sdk.service.compute.UninstallLibrariesResponse¶

as_dict() → dict¶: Serializes the UninstallLibrariesResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UninstallLibrariesResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UninstallLibrariesResponse¶: Deserializes the UninstallLibrariesResponse from a dictionary.

class databricks.sdk.service.compute.UnpinCluster¶

cluster_id: str¶

as_dict() → dict¶: Serializes the UnpinCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UnpinCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UnpinCluster¶: Deserializes the UnpinCluster from a dictionary.

class databricks.sdk.service.compute.UnpinClusterResponse¶

as_dict() → dict¶: Serializes the UnpinClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UnpinClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UnpinClusterResponse¶: Deserializes the UnpinClusterResponse from a dictionary.

class databricks.sdk.service.compute.UpdateCluster¶

cluster_id: str¶: ID of the cluster.

update_mask: str¶

Used to specify which cluster attributes and size fields to update. See https://google.aip.dev/161 for more details.

The field mask must be a single string, with multiple fields separated by commas (no spaces). The field path is relative to the resource object, using a dot (.) to navigate sub-fields (e.g., author.given_name). Specification of elements in sequence or map fields is not allowed, as only the entire collection field can be specified. Field names must exactly match the resource field names.

A field mask of * indicates full replacement. It’s recommended to always explicitly list the fields being updated and avoid using * wildcards, as it can lead to unintended results if the API changes in the future.

cluster: UpdateClusterResource | None = None¶: The cluster to be updated.

as_dict() → dict¶: Serializes the UpdateCluster into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UpdateCluster into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UpdateCluster¶: Deserializes the UpdateCluster from a dictionary.

class databricks.sdk.service.compute.UpdateClusterResource¶

autoscale: AutoScale | None = None¶: Parameters needed in order to automatically scale clusters up and down based on load. Note: autoscaling works best with DB runtime versions 3.0 or later.

autotermination_minutes: int | None = None¶: Automatically terminates the cluster after it is inactive for this time in minutes. If not set, this cluster will not be automatically terminated. If specified, the threshold must be between 10 and 10000 minutes. Users can also set this value to 0 to explicitly disable automatic termination.

aws_attributes: AwsAttributes | None = None¶: Attributes related to clusters running on Amazon Web Services. If not specified at cluster creation, a set of default values will be used.

azure_attributes: AzureAttributes | None = None¶: Attributes related to clusters running on Microsoft Azure. If not specified at cluster creation, a set of default values will be used.

cluster_log_conf: ClusterLogConf | None = None¶: The configuration for delivering spark logs to a long-term storage destination. Three kinds of destinations (DBFS, S3 and Unity Catalog volumes) are supported. Only one destination can be specified for one cluster. If the conf is given, the logs will be delivered to the destination every 5 mins. The destination of driver logs is $destination/$clusterId/driver, while the destination of executor logs is $destination/$clusterId/executor.

cluster_name: str | None = None¶: Cluster name requested by the user. This doesn’t have to be unique. If not specified at creation, the cluster name will be an empty string. For job clusters, the cluster name is automatically set based on the job and job run IDs.

custom_tags: Dict[str, str] | None = None¶

Additional tags for cluster resources. Databricks will tag all cluster resources (e.g., AWS instances and EBS volumes) with these tags in addition to default_tags. Notes:

Currently, Databricks allows at most 45 custom tags
Clusters can only reuse cloud resources if the resources’ tags are a subset of the cluster

tags

data_security_mode: DataSecurityMode | None = None¶

Data security mode decides what data governance model to use when accessing data from a cluster.

The following modes can only be used when kind = CLASSIC_PREVIEW. * DATA_SECURITY_MODE_AUTO: Databricks will choose the most appropriate access mode depending on your compute configuration. * DATA_SECURITY_MODE_STANDARD: Alias for USER_ISOLATION. * DATA_SECURITY_MODE_DEDICATED: Alias for SINGLE_USER.

The following modes can be used regardless of kind. * NONE: No security isolation for multiple users sharing the cluster. Data governance features are not available in this mode. * SINGLE_USER: A secure cluster that can only be exclusively used by a single user specified in single_user_name. Most programming languages, cluster features and data governance features are available in this mode. * USER_ISOLATION: A secure cluster that can be shared by multiple users. Cluster users are fully isolated so that they cannot see each other’s data and credentials. Most data governance features are supported in this mode. But programming languages and cluster features might be limited.

The following modes are deprecated starting with Databricks Runtime 15.0 and will be removed for future Databricks Runtime versions:

LEGACY_TABLE_ACL: This mode is for users migrating from legacy Table ACL clusters. *

LEGACY_PASSTHROUGH: This mode is for users migrating from legacy Passthrough on high concurrency clusters. * LEGACY_SINGLE_USER: This mode is for users migrating from legacy Passthrough on standard clusters. * LEGACY_SINGLE_USER_STANDARD: This mode provides a way that doesn’t have UC nor passthrough enabled.

docker_image: DockerImage | None = None¶: Custom docker image BYOC

driver_instance_pool_id: str | None = None¶: The optional ID of the instance pool for the driver of the cluster belongs. The pool cluster uses the instance pool with id (instance_pool_id) if the driver pool is not assigned.

driver_node_type_id: str | None = None¶

The node type of the Spark driver. Note that this field is optional; if unset, the driver node type will be set as the same value as node_type_id defined above.

This field, along with node_type_id, should not be set if virtual_cluster_size is set. If both driver_node_type_id, node_type_id, and virtual_cluster_size are specified, driver_node_type_id and node_type_id take precedence.

enable_elastic_disk: bool | None = None¶: Autoscaling Local Storage: when enabled, this cluster will dynamically acquire additional disk space when its Spark workers are running low on disk space. This feature requires specific AWS permissions to function correctly - refer to the User Guide for more details.

enable_local_disk_encryption: bool | None = None¶: Whether to enable LUKS on cluster VMs’ local disks

gcp_attributes: GcpAttributes | None = None¶: Attributes related to clusters running on Google Cloud Platform. If not specified at cluster creation, a set of default values will be used.

init_scripts: List[InitScriptInfo] | None = None¶: The configuration for storing init scripts. Any number of destinations can be specified. The scripts are executed sequentially in the order provided. If cluster_log_conf is specified, init script logs are sent to <destination>/<cluster-ID>/init_scripts.

instance_pool_id: str | None = None¶: The optional ID of the instance pool to which the cluster belongs.

is_single_node: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

When set to true, Databricks will automatically set single node related custom_tags, spark_conf, and num_workers

kind: Kind | None = None¶

The kind of compute described by this compute specification.

Depending on kind, different validations and default values will be applied.

Clusters with kind = CLASSIC_PREVIEW support the following fields, whereas clusters with no specified kind do not. * [is_single_node](/api/workspace/clusters/create#is_single_node) * [use_ml_runtime](/api/workspace/clusters/create#use_ml_runtime) * [data_security_mode](/api/workspace/clusters/create#data_security_mode) set to DATA_SECURITY_MODE_AUTO, DATA_SECURITY_MODE_DEDICATED, or DATA_SECURITY_MODE_STANDARD

By using the [simple form], your clusters are automatically using kind = CLASSIC_PREVIEW.

[simple form]: https://docs.databricks.com/compute/simple-form.html

node_type_id: str | None = None¶: This field encodes, through a single value, the resources available to each of the Spark nodes in this cluster. For example, the Spark nodes can be provisioned and optimized for memory or compute intensive workloads. A list of available node types can be retrieved by using the :method:clusters/listNodeTypes API call.

num_workers: int | None = None¶

Number of worker nodes that this cluster should have. A cluster has one Spark Driver and num_workers Executors for a total of num_workers + 1 Spark nodes.

Note: When reading the properties of a cluster, this field reflects the desired number of workers rather than the actual current number of workers. For instance, if a cluster is resized from 5 to 10 workers, this field will immediately be updated to reflect the target size of 10 workers, whereas the workers listed in spark_info will gradually increase from 5 to 10 as the new nodes are provisioned.

policy_id: str | None = None¶: The ID of the cluster policy used to create the cluster if applicable.

runtime_engine: RuntimeEngine | None = None¶

Determines the cluster’s runtime engine, either standard or Photon.

This field is not compatible with legacy spark_version values that contain -photon-. Remove -photon- from the spark_version and set runtime_engine to PHOTON.

If left unspecified, the runtime engine defaults to standard unless the spark_version contains -photon-, in which case Photon will be used.

single_user_name: str | None = None¶: Single user name if data_security_mode is SINGLE_USER

spark_conf: Dict[str, str] | None = None¶: An object containing a set of optional, user-specified Spark configuration key-value pairs. Users can also pass in a string of extra JVM options to the driver and the executors via spark.driver.extraJavaOptions and spark.executor.extraJavaOptions respectively.

spark_env_vars: Dict[str, str] | None = None¶

An object containing a set of optional, user-specified environment variable key-value pairs. Please note that key-value pair of the form (X,Y) will be exported as is (i.e., export X=’Y’) while launching the driver and workers.

In order to specify an additional set of SPARK_DAEMON_JAVA_OPTS, we recommend appending them to $SPARK_DAEMON_JAVA_OPTS as shown in the example below. This ensures that all default databricks managed environmental variables are included as well.

Example Spark environment variables: {“SPARK_WORKER_MEMORY”: “28000m”, “SPARK_LOCAL_DIRS”: “/local_disk0”} or {“SPARK_DAEMON_JAVA_OPTS”: “$SPARK_DAEMON_JAVA_OPTS -Dspark.shuffle.service.enabled=true”}

spark_version: str | None = None¶: The Spark version of the cluster, e.g. 3.3.x-scala2.11. A list of available Spark versions can be retrieved by using the :method:clusters/sparkVersions API call.

ssh_public_keys: List[str] | None = None¶: SSH public key contents that will be added to each Spark node in this cluster. The corresponding private keys can be used to login with the user name ubuntu on port 2200. Up to 10 keys can be specified.

use_ml_runtime: bool | None = None¶

This field can only be used when kind = CLASSIC_PREVIEW.

effective_spark_version is determined by spark_version (DBR release), this field use_ml_runtime, and whether node_type_id is gpu node or not.

workload_type: WorkloadType | None = None¶: Cluster Attributes showing for clusters workload types.

as_dict() → dict¶: Serializes the UpdateClusterResource into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UpdateClusterResource into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UpdateClusterResource¶: Deserializes the UpdateClusterResource from a dictionary.

class databricks.sdk.service.compute.UpdateClusterResponse¶

as_dict() → dict¶: Serializes the UpdateClusterResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UpdateClusterResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UpdateClusterResponse¶: Deserializes the UpdateClusterResponse from a dictionary.

class databricks.sdk.service.compute.UpdateResponse¶

as_dict() → dict¶: Serializes the UpdateResponse into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the UpdateResponse into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → UpdateResponse¶: Deserializes the UpdateResponse from a dictionary.

class databricks.sdk.service.compute.VolumesStorageInfo¶

A storage location back by UC Volumes.

destination: str¶: UC Volumes destination, e.g. /Volumes/catalog/schema/vol1/init-scripts/setup-datadog.sh or dbfs:/Volumes/catalog/schema/vol1/init-scripts/setup-datadog.sh

as_dict() → dict¶: Serializes the VolumesStorageInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the VolumesStorageInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → VolumesStorageInfo¶: Deserializes the VolumesStorageInfo from a dictionary.

class databricks.sdk.service.compute.WorkloadType¶

Cluster Attributes showing for clusters workload types.

clients: ClientsTypes¶: defined what type of clients can use the cluster. E.g. Notebooks, Jobs

as_dict() → dict¶: Serializes the WorkloadType into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the WorkloadType into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → WorkloadType¶: Deserializes the WorkloadType from a dictionary.

class databricks.sdk.service.compute.WorkspaceStorageInfo¶

A storage location in Workspace Filesystem (WSFS)

destination: str¶: wsfs destination, e.g. workspace:/cluster-init-scripts/setup-datadog.sh

as_dict() → dict¶: Serializes the WorkspaceStorageInfo into a dictionary suitable for use as a JSON request body.

as_shallow_dict() → dict¶: Serializes the WorkspaceStorageInfo into a shallow dictionary of its immediate attributes.

classmethod from_dict(d: Dict[str, Any]) → WorkspaceStorageInfo¶: Deserializes the WorkspaceStorageInfo from a dictionary.

Navigation

Related Topics

Compute¶