29 Pull requests opened by 26 people

• [SPARK-52449][CONNECT][PYTHON][SQL] Make datatypes for Expression.Literal.Map/Array optional

  #51473 opened Jul 14, 2025

• [SPARK-52790][CORE] Introduce new grid testing method in SparkFunSuite

  #51474 opened Jul 14, 2025

• [SPARK-52791][PS] Fix error when inferring a UDT with a null first element

  #51475 opened Jul 14, 2025

• [SPARK-52787][SS] Reorganize streaming execution dir around runtime and checkpoint areas

  #51477 opened Jul 14, 2025

• [SPARK-52794][PYTHON] Performance improvement of arrow-optimized python UDTF with LocalDataToArrowConversion check

  #51479 opened Jul 14, 2025

• [SPARK-52709][SQL] Fix parsing of STRUCT<>

  #51480 opened Jul 14, 2025

• [SPARK-52795] [SS] Include runId in state store logs

  #51484 opened Jul 14, 2025

• [SPARK-52511][SDP] Support dry-run mode in spark-pipelines command

  #51489 opened Jul 15, 2025

• Test maven 3.9.11

  #51491 opened Jul 15, 2025

• [SPARK-52800]Remove several deprecated commons-lang3 methods

  #51495 opened Jul 15, 2025

• [SPARK-52803][BUILD] Update `aws-java-sdk-sts` to 1.12.720 in `kinesis-asl` module

  #51498 opened Jul 15, 2025

• [DO NOT REVIEW] temp

  #51501 opened Jul 15, 2025

• [SPARK-52807][SDP] Proto changes to support analysis inside Declarative Pipelines query functions

  #51502 opened Jul 15, 2025

• [SPARK-52798] [SQL] Add function approx_top_k_combine

  #51505 opened Jul 15, 2025

• [MINOR][SQL][Test] Support V2 write schema evolution in InMemoryTable

  #51506 opened Jul 16, 2025

• [SPARK- 52810][SDP][SQL] Spark Pipelines CLI Selection Options

  #51507 opened Jul 16, 2025

• [SPARK-52817][SQL] Fix Like Expression performance

  #51510 opened Jul 16, 2025

• [SPARK-52820][SQL] Capture full plans in golden files

  #51512 opened Jul 16, 2025

• single pass

  #51513 opened Jul 16, 2025

• wip [SPARK-52779][CONNECT][PYTHON] Support TimeType literal in Connect

  #51515 opened Jul 16, 2025

• [SPARK-52813][CONNECT] Allow DAGs in Spark Connect

  #51516 opened Jul 16, 2025

• [SPARK-52824][SS] CheckpointFileManager error classification

  #51518 opened Jul 16, 2025

• [SPARK-52823][SQL] Support DSv2 Join pushdown for Oracle connector

  #51519 opened Jul 16, 2025

• [SS][SPARK-52637] Fix version ID mismatch issue for RocksDB compaction leading to incorrect file mapping

  #51520 opened Jul 16, 2025

• [SPARK-52828][SQL] Make hashing for collated strings collation agnostic

  #51521 opened Jul 16, 2025

• [SPARK-52830][K8S] Support `spark.kubernetes.(driver|executor).pod.excludedFeatureSteps`

  #51522 opened Jul 16, 2025

• [SPARK-52792][CORE][FOLLOWUP] Remove redundant condition check

  #51524 opened Jul 16, 2025

• [SPARK-52641][SQL] Primary key columns should not be nullable

  #51525 opened Jul 16, 2025

• [SPARK-52833] Fix VariantBuilder.appendFloat

  #51526 opened Jul 16, 2025

27 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-47547] BloomFilter fpp degradation

  #50933 commented on Jul 15, 2025 • 30 new comments

• [SPARK-52689][SQL] Send DML Metrics to V2Write

  #51377 commented on Jul 15, 2025 • 10 new comments

• [SPARK-52737][CORE] Pushdown predicate and number of apps to FsHistoryProvider when listing applications

  #51428 commented on Jul 16, 2025 • 8 new comments

• [SPARK-52617][SQL] Cast TIME to/from TIMESTAMP_NTZ

  #51381 commented on Jul 16, 2025 • 8 new comments

• [SPARK-52620][SQL] Support casting of TIME to DECIMAL

  #51455 commented on Jul 16, 2025 • 7 new comments

• [SPARK-52575][SQL] Introduce contextIndependentFoldable attribute for Expressions

  #51282 commented on Jul 16, 2025 • 6 new comments

• [SPARK-52729][SQL] Add GENERAL_TABLE v2 table capacity

  #51419 commented on Jul 15, 2025 • 4 new comments

• [SPARK-52592][PS] Support creating a ps.Series from a ps.Series

  #51300 commented on Jul 14, 2025 • 4 new comments

• [WIP][SQL] Incapsulate type operations

  #51467 commented on Jul 16, 2025 • 3 new comments

• [SPARK-52772][SQL] Inconsistent table attribute handling during updates

  #51466 commented on Jul 15, 2025 • 2 new comments

• [SPARK-42746][SQL][FIXUP] Fix optimizer failure for SortOrder in the LISTAGG function

  #51117 commented on Jul 16, 2025 • 2 new comments

• [DRAFT] Parameter markers in DDL.

  #51410 commented on Jul 16, 2025 • 2 new comments

• [SPARK-52767][SQL] Optimize the performance of maxRows in join

  #51451 commented on Jul 14, 2025 • 2 new comments

• [SPARK-52214][PYTHON] Arrow UDF for aggregation

  #51292 commented on Jul 16, 2025 • 1 new comment

• [SPARK-52752][CORE] Check if tasks on executor are finished when killing executor

  #51437 commented on Jul 16, 2025 • 1 new comment

• [SPARK-51741][SQL][TESTS] Remove --ONLY_IF spark for subquery tests: PR#1

  #50532 commented on Jul 17, 2025 • 0 new comments

• [SPARK-52777][SQL] Enable shuffle cleanup mode configuration in Spark SQL

  #51458 commented on Jul 14, 2025 • 0 new comments

• [SPARK-51728][SQL] Add SELECT EXCEPT Support

  #50536 commented on Jul 15, 2025 • 0 new comments

• [SPARK-52769][SQL] InjectRuntimeFilter should take into account join type and hints

  #51453 commented on Jul 14, 2025 • 0 new comments

• [SPARK-52741][SQL] RemoveFiles ShuffleCleanup mode doesnt work with non-adaptive execution

  #51432 commented on Jul 17, 2025 • 0 new comments

• [SPARK-51554][SQL] Add the time_trunc() function for TIME datatype

  #50607 commented on Jul 15, 2025 • 0 new comments

• [SPARK-52171] [SS] StateDataSource join implementation for state v3

  #51004 commented on Jul 16, 2025 • 0 new comments

• [SPARK-52751][PYTHON][CONNECT] Don't eagerly validate column name in `dataframe['col_name']`

  #51400 commented on Jul 14, 2025 • 0 new comments

• [SPARK-52444][SQL][CONNECT] Add support for Variant/Char/Varchar Literal

  #51215 commented on Jul 16, 2025 • 0 new comments

• [SPARK-52598][DOCS] Reorganize Spark Connect programming guide

  #51305 commented on Jul 15, 2025 • 0 new comments

• [DRAFT][PYTHON] Improve Python UDF Arrow Serializer Performance

  #51225 commented on Jul 16, 2025 • 0 new comments

• [SPARK-52407][SQL] Add support for Theta Sketch

  #51298 commented on Jul 16, 2025 • 0 new comments