Skip to content

feat: read_gbq creates order deterministically without table copy #191

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Nov 11, 2023
Merged

feat: read_gbq creates order deterministically without table copy #191

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
0e16921
feat: read_gbq creates order deterministically without table copy
TrevorBergeron Nov 9, 2023
30f7cd1
add to_json_string ibis op as monkey patch
TrevorBergeron Nov 10, 2023
16e8c6e
explicitly enforce region in read_gbq in session
TrevorBergeron Nov 10, 2023
e863bfa
Merge remote-tracking branch 'github/main' into hashorder
TrevorBergeron Nov 10, 2023
53cee99
order some ml outputs, fix test failulres
TrevorBergeron Nov 10, 2023
e614482
more test, mypy fixes
TrevorBergeron Nov 10, 2023
945742f
revert removal of session table from read_pandas
TrevorBergeron Nov 10, 2023
d8e99f6
address pr comments
TrevorBergeron Nov 10, 2023
69b8e28
Update bigframes/session/__init__.py
TrevorBergeron Nov 10, 2023
aa504a8
Merge remote-tracking branch 'github/main' into hashorder
TrevorBergeron Nov 10, 2023
85a54f5
remove outdated comment
TrevorBergeron Nov 10, 2023
ef3dd3d
make _cached return self and fix golden sql ml tests
TrevorBergeron Nov 10, 2023
2865788
cache before creating time series model to avoid snapshot
TrevorBergeron Nov 10, 2023
ee1f3c5
remove doctest lines dependent on unspecified ordering
TrevorBergeron Nov 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove outdated comment
  • Loading branch information
@TrevorBergeron
TrevorBergeron committed Nov 10, 2023
commit 85a54f5cfba18eb9c5bace3bfe4d543b8faea16e
1 change: 0 additions & 1 deletion bigframes/session/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1186,7 +1186,6 @@ def _create_total_ordering(
# Used to disambiguate between identical rows (which will have identical hash)
random_value = ibis.random().name(ordering_rand_part)

# Might need to actually reselect here?
original_column_ids = table.columns
table_with_ordering = table.select(
itertools.chain(original_column_ids, [full_row_hash, random_value])
Expand Down