feat: add pd.get_dummies #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

milkshakeiii merged 28 commits into main from b297352026-get-dummies

Nov 1, 2023

Contributor

milkshakeiii commented Oct 26, 2023 •

edited

Loading

Emulate most aspects of the pandas get_dummies interface
Tests and doctest examples
Performance bottleneck is BigQuery column count in most cases.

Henry J Solberg added 2 commits

October 26, 2023 06:38


          feat: add pd.get_dummies

7a30750


          Merge branch 'main' of github.com:googleapis/python-bigquery-datafram…

1b3fae0

…es into b297352026-get-dummies

milkshakeiii requested review from a team as code owners

October 26, 2023 06:39

milkshakeiii requested a review from shobsi

October 26, 2023 06:39

product-auto-label bot added size: l api: bigquery labels

milkshakeiii requested a review from TrevorBergeron

October 26, 2023 06:40

Henry J Solberg and others added 2 commits

October 26, 2023 06:49


          remove unneeded prefix case

0aca97c


          Merge branch 'main' into b297352026-get-dummies

706fa35

shobsi reviewed

View reviewed changes

third_party/bigframes_vendored/pandas/core/reshape/encoding.py Outdated Show resolved Hide resolved

shobsi reviewed

View reviewed changes

tests/system/small/test_pandas.py Outdated Show resolved Hide resolved

shobsi reviewed

View reviewed changes

tests/system/small/test_pandas.py Outdated Show resolved Hide resolved

shobsi reviewed

View reviewed changes

tests/system/small/test_pandas.py Outdated Show resolved Hide resolved

shobsi reviewed

View reviewed changes

third_party/bigframes_vendored/pandas/core/reshape/encoding.py Outdated Show resolved Hide resolved

Henry J Solberg added 6 commits

October 27, 2023 16:51


          Merge branch 'main' of github.com:googleapis/python-bigquery-datafram…

cb2a8b1

…es into b297352026-get-dummies


          param/documentation fixes

0781c08


          be stricter about types in test

eec7822


          be stricter about types in series test

bdee75e


          remove unneeded comment

15b1aa3


          Merge branch 'b297352026-get-dummies' of github.com:googleapis/python…

ef57790

…-bigquery-dataframes into b297352026-get-dummies

milkshakeiii requested a review from shobsi

October 27, 2023 19:19


          adjust for type difference in pandas 1

0f7c38e

TrevorBergeron reviewed

View reviewed changes

third_party/bigframes_vendored/pandas/core/reshape/tile.py Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

Contributor Author

milkshakeiii commented Oct 30, 2023

Thanks for the review! Working on addressing these comments today.

Henry J Solberg added 5 commits

October 30, 2023 19:32


          add example code (tested)

b060475


          fix None columns and add test cases

ce5ea69


          variable names and _get_unique_values per-column

758bd6d


          account for pandas 1 behavior difference

dab3eba


          remove already_seen set

b2032e1

Henry J Solberg and others added 8 commits

October 30, 2023 22:21


          avoid unnecessary join/projection

1899a58


          fix column ordering edge case

1a71217


          adjust for picky examples checker

257531a


          example tweak

87b358e


          make part of the example comments

979eb39


          use ellipsis in doctest comment

5b3dc18


          add <BLANKLINES> to doctest string

aa7a0a3


          Merge branch 'main' into b297352026-get-dummies

9db8707

milkshakeiii requested a review from TrevorBergeron

October 31, 2023 04:09

TrevorBergeron approved these changes

View reviewed changes

Contributor

TrevorBergeron left a comment

Logic looks good, just a few suggestions on cutting down the method length a bit

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Outdated Show resolved Hide resolved

bigframes/pandas/__init__.py Show resolved Hide resolved

Henry J Solberg added 4 commits

November 1, 2023 17:09


          extract parameter standardization

f178012


          extract submethods

cc4aa4c


          Merge branch 'main' of github.com:googleapis/python-bigquery-datafram…

6bc36a5

…es into b297352026-get-dummies


          Merge branch 'b297352026-get-dummies' of github.com:googleapis/python…

3fcdd5f

…-bigquery-dataframes into b297352026-get-dummies

milkshakeiii merged commit d8baad5 into main

milkshakeiii deleted the b297352026-get-dummies branch

November 1, 2023 20:41

release-please bot mentioned this pull request

chore(main): release 0.12.0 #151

Merged

ashleyxuu pushed a commit that referenced this pull request


          feat: add pd.get_dummies (#149)

* feat: add pd.get_dummies

* remove unneeded prefix case

* param/documentation fixes

* be stricter about types in test

* be stricter about types in series test

* remove unneeded comment

* adjust for type difference in pandas 1

* add example code (tested)

* fix None columns and add test cases

* variable names and _get_unique_values per-column

* account for pandas 1 behavior difference

* remove already_seen set

* avoid unnecessary join/projection

* fix column ordering edge case

* adjust for picky examples checker

* example tweak

* make part of the example comments

* use ellipsis in doctest comment

* add <BLANKLINES> to doctest string

* extract parameter standardization

* extract submethods

---------

Co-authored-by: Henry J Solberg <[email protected]>

ashleyxuu pushed a commit that referenced this pull request


          feat: add pd.get_dummies (#149)

32e03cc

* feat: add pd.get_dummies

* remove unneeded prefix case

* param/documentation fixes

* be stricter about types in test

* be stricter about types in series test

* remove unneeded comment

* adjust for type difference in pandas 1

* add example code (tested)

* fix None columns and add test cases

* variable names and _get_unique_values per-column

* account for pandas 1 behavior difference

* remove already_seen set

* avoid unnecessary join/projection

* fix column ordering edge case

* adjust for picky examples checker

* example tweak

* make part of the example comments

* use ellipsis in doctest comment

* add <BLANKLINES> to doctest string

* extract parameter standardization

* extract submethods

---------

Co-authored-by: Henry J Solberg <[email protected]>

blunderbuss-gcf bot assigned TrevorBergeron

milkshakeiii unassigned TrevorBergeron

blunderbuss-gcf bot assigned junyazhang

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigquery size: l