Skip to content

Conversation

lbk-fishtown
Copy link

@lbk-fishtown lbk-fishtown commented Sep 12, 2023

Fixes #1421

Exposes job timeout setting in Python job timeout config.
https://cloud.google.com/bigquery/docs/reference/rest/v2/Job#JobConfiguration.FIELDS.job_timeout_ms

# python3 test.py 5000
Client creating using default project: demo-project-xyz
Job config: {'query': {}, 'jobTimeoutMs': '5000'}
Trying to execute long (30s) running query: call `demo-project-xyz`.`dbt_lbondkennedy`.`sleep_seconds_approximate`(30);
An exception of type GoogleAPICallError occurred. Arguments:
('Job execution was cancelled: Job timed out after 5 sec',)

image


Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

@google-cla
Copy link

google-cla bot commented Sep 12, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@product-auto-label product-auto-label bot added size: s Pull request size is small. api: bigquery Issues related to the googleapis/python-bigquery API. labels Sep 12, 2023
@chalmerlowe
Copy link
Collaborator

@lbk-fishtown

Thanks for submitting this PR.
In terms of tests, a fairly straightforward test similar to this one ought to be a good place to start.

labels are an exposed property, much like the jobTimeoutMs property and are set in a similar way, so mirroring this test should work for our purposes.

https://github.com/googleapis/python-bigquery/blob/03194e0156ed9201cb36301967c5af117d7ef29c/tests/unit/model/test_model.py#L343C1-L348C42

I suspect that we might want to put your tests in the following file: python-bigquery/tests/unit/job/test_query.py

We may want to go further than just checking to see if we can set the jobTimeoutMs value, but for now, let's just get that functionality working while we come up with the list of cases to include as tests.

super(_JobConfig, self).__setattr__(name, value)

@property
def jobtimeout(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: normally we use snake_case to transform from rest API to python.

I'd prefer job_timeout_ms unless we change this to accept a datetime.timedelta, in which case job_timeout may make more sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW: We use default_table_expiration_ms here:

def default_table_expiration_ms(self):
so probably makes the most sense to stay consistent with that.

@chalmerlowe chalmerlowe added kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Oct 5, 2023
@yoshi-kokoro yoshi-kokoro removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels Oct 5, 2023
@chalmerlowe
Copy link
Collaborator

This PR was superceded by PR 1675.
Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. size: s Pull request size is small.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose jobTimeoutMs field in JobConfigurationQuery
4 participants