22 Pull requests opened by 17 people

• [WIP] Fix inconsistencies and refactor primitive types in parser

  #51335 opened Jul 1, 2025

• [WIP][SQL][TESTS] Disable stable column aliases in tests if assumed

  #51337 opened Jul 1, 2025

• [SS][SPARK-52637] Fix version ID mismatch issue for RocksDB compaction leading to incorrect file mapping

  #51340 opened Jul 1, 2025

• [SPARK-52638][SQL] Allow preserving Hive-style column order to be configurable

  #51342 opened Jul 1, 2025

• [SPARK-52640][SDP] Propagate Python Source Code Location

  #51344 opened Jul 1, 2025

• [SPARK-52409][SDP] Only use PipelineRunEventBuffer in tests

  #51352 opened Jul 2, 2025

• [SPARK-52669][PySpark]Improvement PySpark choose pythonExec in cluster yarn client mode

  #51357 opened Jul 3, 2025

• [SPARK-52673][CONNECT][CLIENT] Add grpc RetryInfo handling to Spark Connect retry policies

  #51363 opened Jul 3, 2025

• [WIP][SPARK-52646][PS] Avoid CAST_INVALID_INPUT of `__eq__` in ANSI mode

  #51370 opened Jul 4, 2025

• [SPARK-52686][SQL] `Union` should be resolved only if there are no duplicates

  #51376 opened Jul 4, 2025

• [WIP] [SPARK-52689][SQL] Send DML Metrics to V2Write

  #51377 opened Jul 4, 2025

• [SPARK-52659][SQL]Misleading modulo error message in ansi mode

  #51378 opened Jul 5, 2025

• [SPARK-52545][SQL][DOCS] Update string literal docs for quote escaping rules

  #51379 opened Jul 5, 2025

• [SPARK-52617][SQL]Cast TIME to/from TIMESTAMP_NTZ

  #51381 opened Jul 5, 2025

• [SPARK-52691][BUILD] Upgrade ORC to 2.1.3

  #51382 opened Jul 5, 2025

• [SPARK-52693][SQL] Support +/- ANSI day-time intervals to/from TIME

  #51383 opened Jul 6, 2025

• [SPARK-52696][SQL] Strip `__is_duplicate` metadata after analysis

  #51389 opened Jul 7, 2025

• Pull out nondeterminstic

  #51391 opened Jul 7, 2025

• approx_top_k_combine

  #51393 opened Jul 7, 2025

• [SPARK-52701][PS] Fix float32 type widening in `mod` with bool under ANSI

  #51394 opened Jul 7, 2025

• [SPARK-52699][SQL] Support aggregating TIME type in interpreted mode

  #51395 opened Jul 8, 2025

• [IN PROGRESS] Support getting pod state using Informers/Listers

  #51396 opened Jul 8, 2025

25 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-52187][SQL] Introduce Join pushdown for DSv2

  #50921 commented on Jul 7, 2025 • 35 new comments

• [SPARK-47547] BloomFilter fpp degradation

  #50933 commented on Jul 7, 2025 • 18 new comments

• [SPARK-52582][SQL] Improve the memory usage of XML parser

  #51287 commented on Jul 8, 2025 • 14 new comments

• [SPARK-52575][SQL] Introduce contextIndependentFoldable attribute for Expressions

  #51282 commented on Jul 3, 2025 • 12 new comments

• [SPARK-52495][SQL] Allow including partition columns in the single variant column

  #51206 commented on Jul 7, 2025 • 11 new comments

• [SPARK-48359][SQL] Built-in functions for Zstd compression and decompression

  #46672 commented on Jul 7, 2025 • 7 new comments

• [SDP] [SPARK-52576] Drop/recreate on full refresh and MV update

  #51280 commented on Jul 7, 2025 • 3 new comments

• [DRAFT][PYTHON] Improve Python UDF Arrow Serializer Performance

  #51225 commented on Jul 7, 2025 • 1 new comment

• [MINOR][DOCS] Updated the docstring of DataStreamWriter.foreach() method

  #51316 commented on Jul 7, 2025 • 0 new comments

• [SPARK-52588][SQL] Approx_top_k: accumulate and estimate

  #51308 commented on Jul 7, 2025 • 0 new comments

• [SPARK-52598][DOCS] Reorganize Spark Connect programming guide

  #51305 commented on Jul 4, 2025 • 0 new comments

• [SPARK-52580][PS] Avoid CAST_INVALID_INPUT of `replace` in ANSI mode

  #51297 commented on Jul 2, 2025 • 0 new comments

• [WIP][PYTHON] Arrow UDF for aggregation

  #51292 commented on Jul 7, 2025 • 0 new comments

• [CORE] Let LocalSparkContext clear active context in beforeAll

  #51284 commented on Jul 7, 2025 • 0 new comments

• [SPARK-52563][PS] Fix var naming bug in _assert_pandas_almost_equal

  #51253 commented on Jul 8, 2025 • 0 new comments

• [SPARK-51035][BUILD] Upgrade Janino to 3.1.12

  #51239 commented on Jul 1, 2025 • 0 new comments

• [SPARK-52544][SQL] Allow configuring Json datasource string length limit through SQLConf

  #51235 commented on Jul 7, 2025 • 0 new comments

• [WIP][SPARK-51224][BUILD] Test Maven 4

  #51230 commented on Jul 3, 2025 • 0 new comments

• [SPARK-52444][SQL][CONNECT] Add support for Variant/Char/Varchar Literal

  #51215 commented on Jul 1, 2025 • 0 new comments

• [SPARK-51168][BUILD] Test Hadoop 3.4.2

  #51127 commented on Jul 4, 2025 • 0 new comments

• [WIP][SPARK-52011][SQL] Reduce HDFS NameNode RPC on vectorized Parquet reader

  #50765 commented on Jul 4, 2025 • 0 new comments

• [SPARK-51359][CORE][SQL] Set INT64 as the default timestamp type for Parquet files

  #50215 commented on Jul 7, 2025 • 0 new comments

• [SPARK-49547][SQL][PYTHON] Add iterator of `RecordBatch` API to `applyInArrow`

  #49005 commented on Jul 4, 2025 • 0 new comments

• [SPARK-22876][YARN] Respect YARN AM failure validity interval

  #42570 commented on Jul 3, 2025 • 0 new comments

• [WIP][SPARK-24815] [CORE] Trigger Interval based DRA for Structured Streaming

  #42352 commented on Jul 8, 2025 • 0 new comments