Skip to content

Add default limit=1000 in search_data_assets_iterator #40

Closed
@helen-m-lin

Description

@helen-m-lin

Is your feature request related to a problem? Please describe.
Currently the search_data_assets_iterator will use the default limit (20) as the page size when iterating through results. This takes longer to iterate through large numbers of assets.

Describe the solution you'd like
If the user does not provide a limit for the search_data_assets_iterator, set the limit to 1000.

Describe alternatives you've considered
Leaving it as is does not block anything. Users can provide limit=1000 for faster results.

Additional context

def search_data_assets_iterator(self, search_params: DataAssetSearchParams) -> Iterator[DataAsset]:
params = search_params.to_dict()
while True:
response = self.search_data_assets(
search_params=DataAssetSearchParams(**params)
)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions