w.warehouses: SQL Warehouses

class databricks.sdk.service.sql.WarehousesAPI

A SQL warehouse is a compute resource that lets you run SQL commands on data objects within Databricks SQL. Compute resources are infrastructure resources that provide processing capabilities in the cloud.

create([, auto_stop_mins: Optional[int], channel: Optional[Channel], cluster_size: Optional[str], creator_name: Optional[str], enable_photon: Optional[bool], enable_serverless_compute: Optional[bool], instance_profile_arn: Optional[str], max_num_clusters: Optional[int], min_num_clusters: Optional[int], name: Optional[str], spot_instance_policy: Optional[SpotInstancePolicy], tags: Optional[EndpointTags], warehouse_type: Optional[CreateWarehouseRequestWarehouseType]]) → Wait[GetWarehouseResponse]

Usage:

import time

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import sql

w = WorkspaceClient()

created = w.warehouses.create(
    name=f"sdk-{time.time_ns()}",
    cluster_size="2X-Small",
    max_num_clusters=1,
    auto_stop_mins=10,
    tags=sql.EndpointTags(
        custom_tags=[sql.EndpointTagPair(key="Owner", value="eng-dev-ecosystem-team_at_databricks.com")]
    ),
).result()

# cleanup
w.warehouses.delete(id=created.id)

Creates a new SQL warehouse.

Parameters:

  • auto_stop_mins –

    int (optional) The amount of time in minutes that a SQL warehouse must be idle (i.e., no RUNNING queries) before it is automatically stopped.

    Supported values: - Must be == 0 or >= 10 mins - 0 indicates no autostop.

    Defaults to 120 mins

  • channel – Channel (optional) Channel Details

  • cluster_size –

    str (optional) Size of the clusters allocated for this warehouse. Increasing the size of a spark cluster allows you to run larger queries on it. If you want to increase the number of concurrent queries, please tune max_num_clusters.

    Supported values: - 2X-Small - X-Small - Small - Medium - Large - X-Large - 2X-Large - 3X-Large - 4X-Large - 5X-Large - Auto

  • creator_name – str (optional) warehouse creator name

  • enable_photon –

    bool (optional) Configures whether the warehouse should use Photon optimized clusters.

    Defaults to true.

  • enable_serverless_compute – bool (optional) Configures whether the warehouse should use serverless compute

  • instance_profile_arn – str (optional) Deprecated. Instance profile used to pass IAM role to the cluster

  • max_num_clusters –

    int (optional) Maximum number of clusters that the autoscaler will create to handle concurrent queries.

    Supported values: - Must be >= min_num_clusters - Must be <= 40.

    Defaults to min_clusters if unset.

  • min_num_clusters –

    int (optional) Minimum number of available clusters that will be maintained for this SQL warehouse. Increasing this will ensure that a larger number of clusters are always running and therefore may reduce the cold start time for new queries. This is similar to reserved vs. revocable cores in a resource manager.

    Supported values: - Must be > 0 - Must be <= min(max_num_clusters, 30)

    Defaults to 1

  • name –

    str (optional) Logical name for the cluster.

    Supported values: - Must be unique within an org. - Must be less than 100 characters.

  • spot_instance_policy – SpotInstancePolicy (optional) Configurations whether the endpoint should use spot instances.

  • tags –

    EndpointTags (optional) A set of key-value pairs that will be tagged on all resources (e.g., AWS instances and EBS volumes) associated with this SQL warehouse.

    Supported values: - Number of tags < 45.

  • warehouse_type – CreateWarehouseRequestWarehouseType (optional) Warehouse type: PRO or CLASSIC. If you want to use serverless compute, you must set to PRO and also set the field enable_serverless_compute to true.

Returns:

Long-running operation waiter for GetWarehouseResponse. See :method:wait_get_warehouse_running for more details.

create_and_wait([, auto_stop_mins: Optional[int], channel: Optional[Channel], cluster_size: Optional[str], creator_name: Optional[str], enable_photon: Optional[bool], enable_serverless_compute: Optional[bool], instance_profile_arn: Optional[str], max_num_clusters: Optional[int], min_num_clusters: Optional[int], name: Optional[str], spot_instance_policy: Optional[SpotInstancePolicy], tags: Optional[EndpointTags], warehouse_type: Optional[CreateWarehouseRequestWarehouseType], timeout: datetime.timedelta = 0:20:00]) → GetWarehouseResponse
create_default_warehouse_override(default_warehouse_override: DefaultWarehouseOverride, default_warehouse_override_id: str) → DefaultWarehouseOverride

Creates a new default warehouse override for a user. Users can create their own override. Admins can create overrides for any user.

Parameters:

  • default_warehouse_override – DefaultWarehouseOverride Required. The default warehouse override to create.

  • default_warehouse_override_id – str Required. The ID to use for the override, which will become the final component of the override’s resource name. Can be a numeric user ID or the literal string “me” for the current user.

Returns:

DefaultWarehouseOverride

delete(id: str)

Deletes a SQL warehouse.

Parameters:

id – str Required. Id of the SQL warehouse.

delete_default_warehouse_override(name: str)

Deletes the default warehouse override for a user. Users can delete their own override. Admins can delete overrides for any user. After deletion, the workspace default warehouse will be used.

Parameters:

name – str Required. The resource name of the default warehouse override to delete. Format: default-warehouse-overrides/{default_warehouse_override_id} The default_warehouse_override_id can be a numeric user ID or the literal string “me” for the current user.

edit(id: str [, auto_stop_mins: Optional[int], channel: Optional[Channel], cluster_size: Optional[str], creator_name: Optional[str], enable_photon: Optional[bool], enable_serverless_compute: Optional[bool], instance_profile_arn: Optional[str], max_num_clusters: Optional[int], min_num_clusters: Optional[int], name: Optional[str], spot_instance_policy: Optional[SpotInstancePolicy], tags: Optional[EndpointTags], warehouse_type: Optional[EditWarehouseRequestWarehouseType]]) → Wait[GetWarehouseResponse]

Usage:

import time

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import sql

w = WorkspaceClient()

created = w.warehouses.create(
    name=f"sdk-{time.time_ns()}",
    cluster_size="2X-Small",
    max_num_clusters=1,
    auto_stop_mins=10,
    tags=sql.EndpointTags(
        custom_tags=[sql.EndpointTagPair(key="Owner", value="eng-dev-ecosystem-team_at_databricks.com")]
    ),
).result()

_ = w.warehouses.edit(
    id=created.id,
    name=f"sdk-{time.time_ns()}",
    cluster_size="2X-Small",
    max_num_clusters=1,
    auto_stop_mins=10,
)

# cleanup
w.warehouses.delete(id=created.id)

Updates the configuration for a SQL warehouse.

Parameters:

  • id – str Required. Id of the warehouse to configure.

  • auto_stop_mins –

    int (optional) The amount of time in minutes that a SQL warehouse must be idle (i.e., no RUNNING queries) before it is automatically stopped.

    Supported values: - Must be == 0 or >= 10 mins - 0 indicates no autostop.

    Defaults to 120 mins

  • channel – Channel (optional) Channel Details

  • cluster_size –

    str (optional) Size of the clusters allocated for this warehouse. Increasing the size of a spark cluster allows you to run larger queries on it. If you want to increase the number of concurrent queries, please tune max_num_clusters.

    Supported values: - 2X-Small - X-Small - Small - Medium - Large - X-Large - 2X-Large - 3X-Large - 4X-Large - 5X-Large - Auto

  • creator_name – str (optional) warehouse creator name

  • enable_photon –

    bool (optional) Configures whether the warehouse should use Photon optimized clusters.

    Defaults to true.

  • enable_serverless_compute – bool (optional) Configures whether the warehouse should use serverless compute

  • instance_profile_arn – str (optional) Deprecated. Instance profile used to pass IAM role to the cluster

  • max_num_clusters –

    int (optional) Maximum number of clusters that the autoscaler will create to handle concurrent queries.

    Supported values: - Must be >= min_num_clusters - Must be <= 40.

    Defaults to min_clusters if unset.

  • min_num_clusters –

    int (optional) Minimum number of available clusters that will be maintained for this SQL warehouse. Increasing this will ensure that a larger number of clusters are always running and therefore may reduce the cold start time for new queries. This is similar to reserved vs. revocable cores in a resource manager.

    Supported values: - Must be > 0 - Must be <= min(max_num_clusters, 30)

    Defaults to 1

  • name –

    str (optional) Logical name for the cluster.

    Supported values: - Must be unique within an org. - Must be less than 100 characters.

  • spot_instance_policy – SpotInstancePolicy (optional) Configurations whether the endpoint should use spot instances.

  • tags –

    EndpointTags (optional) A set of key-value pairs that will be tagged on all resources (e.g., AWS instances and EBS volumes) associated with this SQL warehouse.

    Supported values: - Number of tags < 45.

  • warehouse_type – EditWarehouseRequestWarehouseType (optional) Warehouse type: PRO or CLASSIC. If you want to use serverless compute, you must set to PRO and also set the field enable_serverless_compute to true.

Returns:

Long-running operation waiter for GetWarehouseResponse. See :method:wait_get_warehouse_running for more details.

edit_and_wait(id: str [, auto_stop_mins: Optional[int], channel: Optional[Channel], cluster_size: Optional[str], creator_name: Optional[str], enable_photon: Optional[bool], enable_serverless_compute: Optional[bool], instance_profile_arn: Optional[str], max_num_clusters: Optional[int], min_num_clusters: Optional[int], name: Optional[str], spot_instance_policy: Optional[SpotInstancePolicy], tags: Optional[EndpointTags], warehouse_type: Optional[EditWarehouseRequestWarehouseType], timeout: datetime.timedelta = 0:20:00]) → GetWarehouseResponse
get(id: str) → GetWarehouseResponse

Usage:

import time

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import sql

w = WorkspaceClient()

created = w.warehouses.create(
    name=f"sdk-{time.time_ns()}",
    cluster_size="2X-Small",
    max_num_clusters=1,
    auto_stop_mins=10,
    tags=sql.EndpointTags(
        custom_tags=[sql.EndpointTagPair(key="Owner", value="eng-dev-ecosystem-team_at_databricks.com")]
    ),
).result()

wh = w.warehouses.get(id=created.id)

# cleanup
w.warehouses.delete(id=created.id)

Gets the information for a single SQL warehouse.

Parameters:

id – str Required. Id of the SQL warehouse.

Returns:

GetWarehouseResponse

get_default_warehouse_override(name: str) → DefaultWarehouseOverride

Returns the default warehouse override for a user. Users can fetch their own override. Admins can fetch overrides for any user. If no override exists, the UI will fallback to the workspace default warehouse.

Parameters:

name – str Required. The resource name of the default warehouse override to retrieve. Format: default-warehouse-overrides/{default_warehouse_override_id} The default_warehouse_override_id can be a numeric user ID or the literal string “me” for the current user.

Returns:

DefaultWarehouseOverride

get_permission_levels(warehouse_id: str) → GetWarehousePermissionLevelsResponse

Gets the permission levels that a user can have on an object.

Parameters:

warehouse_id – str The SQL warehouse for which to get or manage permissions.

Returns:

GetWarehousePermissionLevelsResponse

get_permissions(warehouse_id: str) → WarehousePermissions

Gets the permissions of a SQL warehouse. SQL warehouses can inherit permissions from their root object.

Parameters:

warehouse_id – str The SQL warehouse for which to get or manage permissions.

Returns:

WarehousePermissions

get_workspace_warehouse_config() → GetWorkspaceWarehouseConfigResponse

Gets the workspace level configuration that is shared by all SQL warehouses in a workspace.

Returns:

GetWorkspaceWarehouseConfigResponse

list([, page_size: Optional[int], page_token: Optional[str], run_as_user_id: Optional[int]]) → Iterator[EndpointInfo]

Usage:

from databricks.sdk import WorkspaceClient
from databricks.sdk.service import sql

w = WorkspaceClient()

all = w.warehouses.list(sql.ListWarehousesRequest())

Lists all SQL warehouses that a user has access to.

Parameters:

  • page_size – int (optional) The max number of warehouses to return.

  • page_token –

    str (optional) A page token, received from a previous ListWarehouses call. Provide this to retrieve the subsequent page; otherwise the first will be retrieved.

    When paginating, all other parameters provided to ListWarehouses must match the call that provided the page token.

  • run_as_user_id – int (optional) Deprecated: this field is ignored by the server. Service Principal which will be used to fetch the list of endpoints. If not specified, SQL Gateway will use the user from the session header.

Returns:

Iterator over EndpointInfo

list_default_warehouse_overrides([, page_size: Optional[int], page_token: Optional[str]]) → Iterator[DefaultWarehouseOverride]

Lists all default warehouse overrides in the workspace. Only workspace administrators can list all overrides.

Parameters:

  • page_size – int (optional) The maximum number of overrides to return. The service may return fewer than this value. If unspecified, at most 100 overrides will be returned. The maximum value is 1000; values above 1000 will be coerced to 1000.

  • page_token –

    str (optional) A page token, received from a previous ListDefaultWarehouseOverrides call. Provide this to retrieve the subsequent page.

    When paginating, all other parameters provided to ListDefaultWarehouseOverrides must match the call that provided the page token.

Returns:

Iterator over DefaultWarehouseOverride

set_permissions(warehouse_id: str [, access_control_list: Optional[List[WarehouseAccessControlRequest]]]) → WarehousePermissions

Sets permissions on an object, replacing existing permissions if they exist. Deletes all direct permissions if none are specified. Objects can inherit permissions from their root object.

Parameters:

  • warehouse_id – str The SQL warehouse for which to get or manage permissions.

  • access_control_list – List[WarehouseAccessControlRequest] (optional)

Returns:

WarehousePermissions

set_workspace_warehouse_config([, channel: Optional[Channel], config_param: Optional[RepeatedEndpointConfPairs], data_access_config: Optional[List[EndpointConfPair]], enable_serverless_compute: Optional[bool], enabled_warehouse_types: Optional[List[WarehouseTypePair]], global_param: Optional[RepeatedEndpointConfPairs], google_service_account: Optional[str], instance_profile_arn: Optional[str], security_policy: Optional[SetWorkspaceWarehouseConfigRequestSecurityPolicy], sql_configuration_parameters: Optional[RepeatedEndpointConfPairs]])

Sets the workspace level configuration that is shared by all SQL warehouses in a workspace.

Parameters:

  • channel – Channel (optional) Optional: Channel selection details

  • config_param – RepeatedEndpointConfPairs (optional) Deprecated: Use sql_configuration_parameters

  • data_access_config – List[EndpointConfPair] (optional) Spark confs for external hive metastore configuration JSON serialized size must be less than <= 512K

  • enable_serverless_compute – bool (optional) Deprecated: only setting this to true is allowed.

  • enabled_warehouse_types – List[WarehouseTypePair] (optional) List of Warehouse Types allowed in this workspace (limits allowed value of the type field in CreateWarehouse and EditWarehouse). Note: Some types cannot be disabled, they don’t need to be specified in SetWorkspaceWarehouseConfig. Note: Disabling a type may cause existing warehouses to be converted to another type. Used by frontend to save specific type availability in the warehouse create and edit form UI.

  • global_param – RepeatedEndpointConfPairs (optional) Deprecated: Use sql_configuration_parameters

  • google_service_account – str (optional) GCP only: Google Service Account used to pass to cluster to access Google Cloud Storage

  • instance_profile_arn – str (optional) AWS Only: The instance profile used to pass an IAM role to the SQL warehouses. This configuration is also applied to the workspace’s serverless compute for notebooks and jobs.

  • security_policy – SetWorkspaceWarehouseConfigRequestSecurityPolicy (optional) Security policy for warehouses

  • sql_configuration_parameters – RepeatedEndpointConfPairs (optional) SQL configuration parameters

start(id: str) → Wait[GetWarehouseResponse]

Starts a SQL warehouse.

Parameters:

id – str Required. Id of the SQL warehouse.

Returns:

Long-running operation waiter for GetWarehouseResponse. See :method:wait_get_warehouse_running for more details.

start_and_wait(id: str, timeout: datetime.timedelta = 0:20:00) → GetWarehouseResponse
stop(id: str) → Wait[GetWarehouseResponse]

Stops a SQL warehouse.

Parameters:

id – str Required. Id of the SQL warehouse.

Returns:

Long-running operation waiter for GetWarehouseResponse. See :method:wait_get_warehouse_stopped for more details.

stop_and_wait(id: str, timeout: datetime.timedelta = 0:20:00) → GetWarehouseResponse
update_default_warehouse_override(name: str, default_warehouse_override: DefaultWarehouseOverride, update_mask: FieldMask [, allow_missing: Optional[bool]]) → DefaultWarehouseOverride

Updates an existing default warehouse override for a user. Users can update their own override. Admins can update overrides for any user.

Parameters:

  • name – str The resource name of the default warehouse override. Format: default-warehouse-overrides/{default_warehouse_override_id}

  • default_warehouse_override – DefaultWarehouseOverride Required. The default warehouse override to update. The name field must be set in the format: default-warehouse-overrides/{default_warehouse_override_id} The default_warehouse_override_id can be a numeric user ID or the literal string “me” for the current user.

  • update_mask – FieldMask Required. Field mask specifying which fields to update. Only the fields specified in the mask will be updated. Use “*” to update all fields. When allow_missing is true, this field is ignored and all fields are applied.

  • allow_missing – bool (optional) If set to true, and the override is not found, a new override will be created. In this situation, update_mask is ignored and all fields are applied. Defaults to false.

Returns:

DefaultWarehouseOverride

update_permissions(warehouse_id: str [, access_control_list: Optional[List[WarehouseAccessControlRequest]]]) → WarehousePermissions

Updates the permissions on a SQL warehouse. SQL warehouses can inherit permissions from their root object.

Parameters:

  • warehouse_id – str The SQL warehouse for which to get or manage permissions.

  • access_control_list – List[WarehouseAccessControlRequest] (optional)

Returns:

WarehousePermissions

wait_get_warehouse_running(id: str, timeout: datetime.timedelta = 0:20:00, callback: Optional[Callable[[GetWarehouseResponse], None]]) → GetWarehouseResponse
wait_get_warehouse_stopped(id: str, timeout: datetime.timedelta = 0:20:00, callback: Optional[Callable[[GetWarehouseResponse], None]]) → GetWarehouseResponse