Commits

Branch selector

 master

User selector

ueshin

Datepicker

All time

Commit History

Commits on Aug 15, 2023

• [SPARK-44749][PYTHON][FOLLOWUP][TESTS] Add more tests for named arguments in Python UDTF

  Show description for f7002fb
  ueshin

  committed
  f7002fb

  Copy full SHA for f7002fb

Commits on Aug 14, 2023

• [SPARK-44749][SQL][PYTHON] Support named arguments in Python UDTF

  Show description for d462956
  ueshin

  committed
  d462956

  Copy full SHA for d462956

Commits on Aug 7, 2023

• [SPARK-44561][PYTHON] Fix AssertionError when converting UDTF output to a complex type

  Show description for f1a161c
  ueshin

  committed
  f1a161c

  Copy full SHA for f1a161c

Commits on Aug 4, 2023

• [SPARK-43967][SQL][PYTHON] Add memory limits for Python UDTF analyzer

  Show description for bd9dd38
  

  ueshin

  authored and

  zhengruifeng

  committed
  bd9dd38

  Copy full SHA for bd9dd38

Commits on Aug 3, 2023

• [SPARK-44643][SQL][PYTHON] Fix Row.__repr__ for the case the field is empty Row

  Show description for 8a84718
  

  ueshin

  authored and

  HyukjinKwon

  committed
  8a84718

  Copy full SHA for 8a84718

Commits on Aug 1, 2023

• [SPARK-44614][PYTHON][CONNECT] Add missing packages in setup.py

  Show description for a011d16
  ueshin

  committed
  a011d16

  Copy full SHA for a011d16

Commits on Jul 27, 2023

• [SPARK-44533][PYTHON] Add support for accumulator, broadcast, and Spark files in Python UDTF's analyze

  Show description for 8647b24
  ueshin

  committed
  8647b24

  Copy full SHA for 8647b24

• [SPARK-44479][PYTHON] Fix ArrowStreamPandasUDFSerializer to accept no-column pandas DataFrame

  Show description for 02e36dd
  ueshin

  committed
  02e36dd

  Copy full SHA for 02e36dd

Commits on Jul 25, 2023

• [SPARK-43964][PYTHON][TESTS][FOLLOWUP] Skip a test using pandas when pandas is not available

  Show description for b930f7a
  

  ueshin

  authored and

  HyukjinKwon

  committed
  b930f7a

  Copy full SHA for b930f7a

Commits on Jul 22, 2023

• [SPARK-44380][PYTHON][FOLLOWUP] Set __doc__ for analyze static method when Arrow is enabled

  Show description for 784b942
  

  ueshin

  authored and

  HyukjinKwon

  committed
  784b942

  Copy full SHA for 784b942

Commits on Jul 20, 2023

• [SPARK-44380][SQL][PYTHON] Support for Python UDTF to analyze in Python

  Show description for 964f755
  

  ueshin

  authored and

  HyukjinKwon

  committed
  964f755

  Copy full SHA for 964f755

Commits on Jul 3, 2023

• [SPARK-44200][SQL] Support TABLE argument parser rule for TableValuedFunction

  Show description for 8390b03
  

  ueshin

  authored and

  HyukjinKwon

  committed
  8390b03

  Copy full SHA for 8390b03

• [SPARK-44249][SQL][PYTHON] Refactor PythonUDTFRunner to send its return type separately

  Show description for 1fbb38e
  

  ueshin

  authored and

  HyukjinKwon

  committed
  1fbb38e

  Copy full SHA for 1fbb38e

Commits on Jun 29, 2023

• [SPARK-44150][PYTHON][FOLLOW-UP] Fix ArrowStreamPandasSerializer to set arguments properly

  Show description for 414bc75
  

  ueshin

  authored and

  xinrong-meng

  committed
  414bc75

  Copy full SHA for 414bc75

Commits on May 31, 2023

• [SPARK-41971][CONNECT][PYTHON][FOLLOWUP] Fix to_pandas to support the older Spark

  Show description for 04125eb
  

  ueshin

  authored and

  HyukjinKwon

  committed
  04125eb

  Copy full SHA for 04125eb

Commits on May 28, 2023

• [SPARK-43817][SPARK-43702][PYTHON] Support UserDefinedType in createDataFrame from pandas DataFrame and toPandas

  Show description for 916b0d3
  

  ueshin

  authored and

  zhengruifeng

  committed
  916b0d3

  Copy full SHA for 916b0d3

Commits on May 25, 2023

• [SPARK-43545][SQL][PYTHON] Support nested timestamp type

  Show description for 46949e6
  

  ueshin

  authored and

  zhengruifeng

  committed
  46949e6

  Copy full SHA for 46949e6

Commits on May 24, 2023

• [SPARK-43759][SQL][PYTHON] Expose TimestampNTZType in pyspark.sql.types

  Show description for 1f04bb6
  

  ueshin

  authored and

  zhengruifeng

  committed
  1f04bb6

  Copy full SHA for 1f04bb6

Commits on May 17, 2023

• [SPARK-43531][CONNECT][PYTHON][TESTS] Enable more parity tests for Pandas UDFs

  Show description for 5b8721b
  

  ueshin

  authored and

  zhengruifeng

  committed
  5b8721b

  Copy full SHA for 5b8721b

• [SPARK-43528][SQL][PYTHON] Support duplicated field names in createDataFrame with pandas DataFrame

  Show description for 71bac15
  

  ueshin

  authored and

  HyukjinKwon

  committed
  71bac15

  Copy full SHA for 71bac15

Commits on May 16, 2023

• [SPARK-43473][PYTHON] Support struct type in createDataFrame from pandas DataFrame

  Show description for 6221995
  

  ueshin

  authored and

  HyukjinKwon

  committed
  6221995

  Copy full SHA for 6221995

Commits on May 15, 2023

• [SPARK-41971][PYTHON][FOLLOWUP] Fix toPandas to support empty columns

  Show description for 1d7edea
  

  ueshin

  authored and

  HyukjinKwon

  committed
  1d7edea

  Copy full SHA for 1d7edea

Commits on May 12, 2023

• [SPARK-42945][CONNECT][FOLLOWUP] Disable JVM stack trace by default

  Show description for f05b658
  

  ueshin

  authored and

  dongjoon-hyun

  committed
  f05b658

  Copy full SHA for f05b658

Commits on May 5, 2023

• [SPARK-43363][SQL][PYTHON] Make to call `astype` to the category type only when the arrow type is not provided

  Show description for 7b5d7af
  

  ueshin

  authored and

  HyukjinKwon

  committed
  7b5d7af

  Copy full SHA for 7b5d7af

Commits on May 4, 2023

• [SPARK-41971][SQL][PYTHON] Add a config for pandas conversion how to handle struct types

  Show description for 305aa4a
  

  ueshin

  authored and

  xinrong-meng

  committed
  305aa4a

  Copy full SHA for 305aa4a

Commits on May 1, 2023

• [SPARK-43323][SQL][PYTHON] Fix DataFrame.toPandas with Arrow enabled to handle exceptions properly

  Show description for afa7f3d
  

  ueshin

  authored and

  HyukjinKwon

  committed
  afa7f3d

  Copy full SHA for afa7f3d

Commits on Apr 21, 2023

• [SPARK-43055][CONNECT][PYTHON][FOLLOWUP] Fix deduplicate field names and refactor

  Show description for 50d9f94
  

  ueshin

  authored and

  HyukjinKwon

  committed
  50d9f94

  Copy full SHA for 50d9f94

Commits on Apr 19, 2023

• [SPARK-43146][CONNECT][PYTHON] Implement eager evaluation for __repr__ and _repr_html_

  Show description for 780aeec
  

  ueshin

  authored and

  HyukjinKwon

  committed
  780aeec

  Copy full SHA for 780aeec

Commits on Apr 18, 2023

• [SPARK-43153][CONNECT] Skip Spark execution when the dataframe is local

  Show description for 66392c4
  

  ueshin

  authored and

  zhengruifeng

  committed
  66392c4

  Copy full SHA for 66392c4

• [SPARK-42984][CONNECT][PYTHON][TESTS] Enable test_createDataFrame_with_single_data_type

  Show description for 61e8c5b
  

  ueshin

  authored and

  zhengruifeng

  committed
  61e8c5b

  Copy full SHA for 61e8c5b

Commits on Apr 13, 2023

• [SPARK-42982][CONNECT][PYTHON] Fix createDataFrame to respect the given schema ddl

  Show description for f66611d
  

  ueshin

  authored and

  HyukjinKwon

  committed
  f66611d

  Copy full SHA for f66611d

• [SPARK-43115][CONNECT][PS][TESTS] Split pyspark-pandas-connect from pyspark-connect module

  Show description for 69abf14
  

  ueshin

  authored and

  zhengruifeng

  committed
  69abf14

  Copy full SHA for 69abf14

Commits on Apr 12, 2023

• [SPARK-42985][CONNECT][PYTHON] Fix createDataFrame to respect the SQL configs

  Show description for 7e2c6c7
  

  ueshin

  authored and

  HyukjinKwon

  committed
  7e2c6c7

  Copy full SHA for 7e2c6c7

• [SPARK-43055][CONNECT][PYTHON] Support duplicated nested field names

  Show description for 631ee67
  

  ueshin

  authored and

  HyukjinKwon

  committed
  631ee67

  Copy full SHA for 631ee67

Commits on Apr 8, 2023

• [SPARK-43062][INFRA][PYTHON][TESTS] Add options to lint-python to run each test separately

  Show description for 193deed
  

  ueshin

  authored and

  HyukjinKwon

  committed
  193deed

  Copy full SHA for 193deed

Pagination

Previous

Next