w.feature_engineering: Feature Engineering

class databricks.sdk.service.ml.FeatureEngineeringAPI

Feature Engineering provides APIs for managing features and materialized features.

backfill_features(feature_full_names: List[str], backfill_ranges: List[BackfillRange] [, budget_policy_id: Optional[str], request_id: Optional[str], tags: Optional[Dict[str, str]]]) → BackfillFeaturesOperation

Backfill features.

Parameters:

• feature_full_names – List[str] Full names of the features to backfill. All features must share the same interval between values and the same offset. The interval is the slide_duration of a sliding window or the window_duration of a tumbling window.

• backfill_ranges – List[BackfillRange] Time ranges of source data to backfill. Ranges must not overlap.

• budget_policy_id – str (optional) The budget policy ID, in UUID format, used to attribute the serverless compute cost of this backfill. If not specified, a default budget policy may be applied.

• request_id – str (optional) Idempotency token for the request.

• tags – Dict[str,str] (optional) Custom tags to associate with this backfill. They are applied to the backfill job and forwarded to the underlying compute as Databricks resource tags, so backfill cost can be attributed in the billing system tables. These tags apply only to the backfill compute; they are not applied to the Unity Catalog Feature resources themselves, whose tags are managed separately through the Unity Catalog tagging API. A maximum of 25 tags is supported; keys and values are subject to the same limitations as Databricks resource tags.

Returns:

Operation

batch_create_materialized_features(requests: List[CreateMaterializedFeatureRequest]) → BatchCreateMaterializedFeaturesResponse

Batch create materialized features.

Parameters:

requests – List[CreateMaterializedFeatureRequest] The requests to create materialized features.

Returns:

BatchCreateMaterializedFeaturesResponse

cancel_operation(name: str)

Cancel an operation.

Parameters:

name – str The name of the operation resource to be cancelled.

create_feature(feature: Feature) → Feature

Create a Feature.

Parameters:

feature – Feature Feature to create.

Returns:

Feature

create_feature_spec(feature_spec: FeatureSpec) → FeatureSpec

Create a feature spec.

Parameters:

feature_spec – FeatureSpec The feature spec to create.

Returns:

FeatureSpec

create_kafka_config(kafka_config: KafkaConfig) → KafkaConfig

Create a Kafka config. During PrPr, Kafka configs can be read and used when creating features under the entire metastore. Only the creator of the Kafka config can delete it.

Parameters:

kafka_config – KafkaConfig

Returns:

KafkaConfig

create_materialization_set(materialization_set: MaterializationSet [, request_id: Optional[str]]) → CreateMaterializationSetOperation

Create a materialization set.

Parameters:

• materialization_set – MaterializationSet The materialization set to create. Supply materialization_set_id to choose the identifier; leave it unset to have one assigned.

• request_id – str (optional) Idempotency token for the request. A second request carrying a token that has already been used returns the operation the first one created, whatever state it reached, and creates no second set. The token is not checked against the rest of the request, so reusing one for a different set returns the original operation rather than an error. Omit it and each request creates its own set.

Returns:

Operation

create_materialized_feature(materialized_feature: MaterializedFeature) → MaterializedFeature

Create a materialized feature.

Parameters:

materialized_feature – MaterializedFeature The materialized feature to create.

Returns:

MaterializedFeature

create_stream(stream: Stream) → Stream

Create a Stream, a governed UC entity representing an external streaming data source.

Parameters:

stream – Stream The Stream to create.

Returns:

Stream

delete_feature(full_name: str)

Delete a Feature.

Parameters:

full_name – str Name of the feature to delete.

delete_feature_spec(name: str)

Delete a feature spec.

Parameters:

name – str Three-part name (catalog.schema.feature_spec) of the feature spec to delete.

delete_kafka_config(name: str)

Delete a Kafka config. During PrPr, Kafka configs can be read and used when creating features under the entire metastore. Only the creator of the Kafka config can delete it.

Parameters:

name – str Name of the Kafka config to delete.

delete_materialization_set(materialization_set_id: str [, allow_missing: Optional[bool], request_id: Optional[str]]) → DeleteMaterializationSetOperation

Delete a materialization set and every materialized feature in it. The set is gone afterwards, so a later read returns NOT_FOUND and the set id is free to use again.

Parameters:

• materialization_set_id – str The ID of the materialization set to delete.

• allow_missing – bool (optional) Return success instead of NOT_FOUND when the set does not exist, so a repeated destroy is idempotent.

• request_id – str (optional) Idempotency token for the request. A second request carrying a token that has already been used returns the operation the first one created, whatever state it reached, and deletes nothing further. The token is not checked against the rest of the request. Omit it and each request deletes independently.

Returns:

Operation

delete_materialized_feature(materialized_feature_id: str)

Delete a materialized feature.

Parameters:

materialized_feature_id – str The ID of the materialized feature to delete.

delete_stream(name: str)

Delete a Stream by its full three-part name (catalog.schema.stream).

Parameters:

name – str Full three-part name (catalog.schema.stream) of the Stream to delete.

get_feature(full_name: str) → Feature

Get a Feature.

Parameters:

full_name – str Name of the feature to get.

Returns:

Feature

get_feature_spec(name: str) → FeatureSpec

Get a feature spec.

Parameters:

name – str Three-part name (catalog.schema.feature_spec) of the feature spec to get.

Returns:

FeatureSpec

get_kafka_config(name: str) → KafkaConfig

Get a Kafka config. During PrPr, Kafka configs can be read and used when creating features under the entire metastore. Only the creator of the Kafka config can delete it.

Parameters:

name – str Name of the Kafka config to get.

Returns:

KafkaConfig

get_materialization_set(materialization_set_id: str) → MaterializationSet

Get a materialization set and the materialized features in it. A set whose materialized feature members have all been deleted is returned with an empty member list.

Parameters:

materialization_set_id – str The ID of the materialization set.

Returns:

MaterializationSet

get_materialized_feature(materialized_feature_id: str) → MaterializedFeature

Get a materialized feature.

Parameters:

materialized_feature_id – str The ID of the materialized feature.

Returns:

MaterializedFeature

get_operation(name: str) → Operation

Get an operation.

Parameters:

name – str The name of the operation resource.

Returns:

Operation

get_stream(name: str) → Stream

Get a Stream by its full three-part name (catalog.schema.stream).

Parameters:

name – str Full three-part name (catalog.schema.stream) of the Stream to get.

Returns:

Stream

list_features(catalog_name: str, schema_name: str [, page_size: Optional[int], page_token: Optional[str]]) → Iterator[Feature]

List Features.

Parameters:

• catalog_name – str Name of parent catalog for features of interest.

• schema_name – str Name of parent schema relative to its parent catalog.

• page_size – int (optional) The maximum number of results to return.

• page_token – str (optional) Pagination token to go to the next page based on a previous query.

Returns:

Iterator over Feature

list_kafka_configs([, page_size: Optional[int], page_token: Optional[str]]) → Iterator[KafkaConfig]

List Kafka configs. During PrPr, Kafka configs can be read and used when creating features under the entire metastore. Only the creator of the Kafka config can delete it.

Parameters:

• page_size – int (optional) The maximum number of results to return.

• page_token – str (optional) Pagination token to go to the next page based on a previous query.

Returns:

Iterator over KafkaConfig

list_materialized_features([, feature_name: Optional[str], page_size: Optional[int], page_token: Optional[str]]) → Iterator[MaterializedFeature]

List materialized features.

Parameters:

• feature_name – str (optional) Filter by feature name. If specified, only materialized features materialized from this feature will be returned.

• page_size – int (optional) The maximum number of results to return. Defaults to 100 if not specified. Cannot be greater than 1000.

• page_token – str (optional) Pagination token to go to the next page based on a previous query.

Returns:

Iterator over MaterializedFeature

list_streams([, page_size: Optional[int], page_token: Optional[str], parent: Optional[str]]) → Iterator[Stream]

List Streams under a given catalog.schema parent.

Parameters:

• page_size – int (optional) The maximum number of results to return.

• page_token – str (optional) Pagination token to go to the next page based on a previous query.

• parent – str (optional) Two-part name (catalog.schema) of the parent under which to list Streams.

Returns:

Iterator over Stream

purge_feature_entities(features: List[str], entities_table: str [, budget_policy_id: Optional[str], delete_rows: Optional[bool], request_id: Optional[str], tags: Optional[Dict[str, str]]]) → PurgeFeatureEntitiesOperation

Purge materialized feature values for specified entities.

Parameters:

• features – List[str] Fully qualified names of the features to purge. At least one nonempty feature name is required. A request may contain at most 500 features; submit additional features in separate requests. Duplicate features are rejected.

• entities_table – str Fully qualified name of the Unity Catalog Delta table containing the entity keys to purge. The table may contain a subset of each feature’s entity-key columns. A partial key match deletes all feature rows matching the provided key values. Non-key columns are rejected; null key values are allowed.

• budget_policy_id – str (optional) The budget policy ID, in UUID format, used to attribute the serverless compute cost of this purge. If not specified, a default budget policy may be applied.

• delete_rows – bool (optional) If false feature values are set to null on purge but rows are not deleted. If true then the full rows containing entities are deleted. Defaults to false. When true, the request must include every feature which shares the same table_name property in their MaterializedFeature. Features from multiple tables may be included if all features materialized in each table are included.

• request_id – str (optional) Optional UUID4 idempotency token for the request.

• tags – Dict[str,str] (optional) Custom tags to associate with this purge. They are applied to the purge job and forwarded to the underlying compute as Databricks resource tags, so purge cost can be attributed in the billing system tables. These tags apply only to the purge compute; they are not applied to the Unity Catalog Feature resources themselves, whose tags are managed separately through the Unity Catalog tagging API. A maximum of 25 tags is supported; keys and values are subject to the same limitations as Databricks resource tags.

Returns:

Operation

trigger_materialization_set(materialization_set_id: str [, request_id: Optional[str]]) → TriggerMaterializationSetOperation

Requests materialization of all members of the set. The set must be nonempty and contain only batch materializations. Runs are allowed while automatic scheduling is paused. Requested runs may wait in a queue, subject to queue limits and expiry.

Returns a long-running operation that completes when every requested run reaches a terminal state. Success requires all runs and any required online synchronization to succeed. Individual outcomes are available in MaterializationSetTriggerProgress.

Use CancelOperation to request cancellation of this operation’s unfinished work. Cancellation is best effort and does not undo completed writes. Online synchronization already in progress may continue.

Runs use the features’ configured windows and settling delay.

Parameters:

• materialization_set_id – str The ID of the materialization set to trigger.

• request_id – str (optional) Idempotency token in UUID format. Reusing it returns the original operation without starting additional runs, even if the request differs or the operation has finished. If omitted, each request triggers independently.

Returns:

Operation

update_feature(full_name: str, feature: Feature, update_mask: str) → Feature

Update a Feature.

Parameters:

• full_name – str The full three-part name (catalog, schema, name) of the feature.

• feature – Feature Feature whose full_name identifies the target. Only description is mutable.

• update_mask – str Fields to update. The only supported path is description.

Returns:

Feature

update_kafka_config(name: str, kafka_config: KafkaConfig, update_mask: FieldMask) → KafkaConfig

Update a Kafka config. During PrPr, Kafka configs can be read and used when creating features under the entire metastore. Only the creator of the Kafka config can delete it.

Parameters:

• name – str Name that uniquely identifies this Kafka config within the metastore. This will be the identifier used from the Feature object to reference these configs for a feature. Can be distinct from topic name.

• kafka_config – KafkaConfig The Kafka config to update.

• update_mask – FieldMask The list of fields to update.

Returns:

KafkaConfig

update_materialization_set(materialization_set_id: str, materialization_set: MaterializationSet, update_mask: FieldMask [, request_id: Optional[str]]) → UpdateMaterializationSetOperation

Updates a materialization set. Supports changing pause settings and deleting materialized features from the set.

Changes are not atomic. Some updates or deletions may succeed even if the operation fails. Individual outcomes are available in MaterializationSetUpdateProgress.

Parameters:

• materialization_set_id – str Server-assigned unique identifier for the materialization set.

• materialization_set –

  MaterializationSet The desired state of the materialization set. materialization_set_id identifies the set.

  A request that changes pause settings must request the same paused value for all retained members with a cron schedule or table trigger.

• update_mask –

  FieldMask The fields to update, relative to the materialization set. Supported values are materialized_features and *.

  The materialized_features mask updates retained members and deletes omitted materialized features. The * mask selects all fields of the set and follows the same membership rules. Adding members is not supported.

  To change pause settings without changing membership, include every member of the set. An omitted paused is reset to false.

  Fields outside the mask are unchanged. Trigger types and settings other than paused must remain unchanged.

• request_id – str (optional) Idempotency token for the request. A second request carrying a token that has already been used returns the operation the first one created, whatever state it reached, and changes nothing further. The token is not checked against the rest of the request. Omit it and each request updates independently.

Returns:

Operation

update_materialized_feature(materialized_feature_id: str, materialized_feature: MaterializedFeature, update_mask: str) → MaterializedFeature

Updates a materialized feature.

Parameters:

• materialized_feature_id – str Server-assigned unique identifier for the materialized feature.

• materialized_feature – MaterializedFeature The materialized feature to update.

• update_mask – str The fields to update. Only pipeline_schedule_state is supported.

Returns:

MaterializedFeature

update_stream(name: str, stream: Stream, update_mask: FieldMask) → Stream

Update a Stream. Only fields listed in update_mask are mutated.

Parameters:

• name – str Full three-part (catalog.schema.stream) name of the stream.

• stream – Stream The Stream to update.

• update_mask – FieldMask The list of fields to update.

Returns:

Stream