Skip to content

feat: set query label session property in bq session #4314

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

ncbkr
Copy link
Contributor

@ncbkr ncbkr commented May 5, 2025

we want to use the SQLMesh session properties to attach labels to the BigQuery jobs. Using these labels we want to enable ourselves to

  • map BigQuery jobs to SQLMesh models
  • map BigQuery jobs to owners/users of SQLMesh

In the model definition, this would like as follows

session_properties (
    query_label = [
      ('label1', 'value1'),
      ('label2', 'value2'),
      ('label3', 'value3')
    ]
  )

This is currently not possible in an easy way since all jobs are created by the same service account.

The screenshot below displays the information about a BigQuery job created by sqlmesh to backfill a model. In the last line you can see the label my_session_property_key: my_session_property_value. This comes from the session_properties of the sqlmesh model definition. According to the google docs for BigQuery, all jobs that are run within a session inherit the labels set at the beginning of the session using SET @@query_label = <my_labels>

image

@izeigerman
Copy link
Member

Thanks for your contribution! Can you please add some tests.

Copy link
Member

@izeigerman izeigerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really good! One last thing and this should be good to go. Thanks a lot for addressing the comments!

@ncbkr ncbkr force-pushed the feat/bq-session-property-query-label branch from a75d14c to af45f65 Compare May 22, 2025 12:37
Copy link
Contributor

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ncbkr! Left a few final comments as well.

Copy link
Contributor

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, well done! Also– thanks for your patience!

@izeigerman this is good to go from me, feel free to take another look.

@ncbkr
Copy link
Contributor Author

ncbkr commented May 23, 2025

Thank you as well for considering this change and providing feedback. This was very helpful. Happy to contribute to this great project

@georgesittas georgesittas requested a review from izeigerman May 26, 2025 12:15
Copy link
Member

@izeigerman izeigerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for addressing comments!

@izeigerman izeigerman merged commit 909548b into TobikoData:main May 26, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants