Merge pull request #543 from databrickslabs/releases/v_0.4.1

Release for 0.4.1

Author: Milos Colic

CHANGELOG.md

@@ -1,3 +1,17 @@
+## v0.4.1 [DBR 13.3 LTS]
+- Fixed python bindings for MosaicAnalyzer functions.
+- Added tiller functions, ST_AsGeoJSONTile and ST_AsMVTTile, for creating GeoJSON and MVT tiles as aggregations of geometries.
+- Added filter and convolve functions for raster data.
+- Raster tile schema changed to be <tile:struct<index_id:bigint, tile:binary, metadata:map<string, string>>.
+- Raster tile metadata will contain driver, parentPath and path.
+- Raster tile metadata will contain warnings and errors in case of failures.
+- All raster functions ensure rasters are TILED and not STRIPED when appropriate.
+- GDAL cache memory has been decreased to 512MB to reduce memory usage and competition with Spark.
+- Add RST_MakeTiles that allows for different raster creations.
+- Rasters can now be passed as file pointers using checkpoint location.
+- Added logic to handle zarr format for raster data.
+- Added RST_SeparateBands to separate bands from a raster for NetCDF and Zarr formats.
+
 ## v0.4.0 [DBR 13.3 LTS]
 - First release for DBR 13.3 LTS which is Ubuntu Jammy and Spark 3.4.1. Not backwards compatible, meaning it will not run on prior DBRs; requires either a Photon DBR or a ML Runtime (__Standard, non-Photon DBR no longer allowed__).
 - New `setup_fuse_install` function to meet various requirements arising with Unity Catalog + Shared Access clusters; removed the scala equivalent function, making artifact setup and install python-first for scala and Spark SQL. 

R/sparkR-mosaic/sparkrMosaic/DESCRIPTION

@@ -1,6 +1,6 @@
 Package: sparkrMosaic
 Title: SparkR bindings for Databricks Mosaic
-Version: 0.4.0
+Version: 0.4.1
 Authors@R:
     person("Robert", "Whiffin", , "[email protected]", role = c("aut", "cre")
            )

R/sparklyr-mosaic/sparklyrMosaic/DESCRIPTION

@@ -1,6 +1,6 @@
 Package: sparklyrMosaic
 Title: sparklyr bindings for Databricks Mosaic
-Version: 0.4.0
+Version: 0.4.1
 Authors@R:
     person("Robert", "Whiffin", , "[email protected]", role = c("aut", "cre")
            )

R/sparklyr-mosaic/tests.R

@@ -9,7 +9,7 @@ library(sparklyr.nested)
 spark_home <- Sys.getenv("SPARK_HOME")
 spark_home_set(spark_home)
 
-install.packages("sparklyrMosaic_0.4.0.tar.gz", repos = NULL)
+install.packages("sparklyrMosaic_0.4.1.tar.gz", repos = NULL)
 library(sparklyrMosaic)
 
 # find the mosaic jar in staging

docs/source/conf.py

@@ -22,7 +22,7 @@
 author = 'Stuart Lynn, Milos Colic, Erni Durdevic, Robert Whiffin, Timo Roest'
 
 # The full version, including alpha/beta/rc tags
-release = "v0.4.0"
+release = "v0.4.1"
 
 
 # -- General configuration ---------------------------------------------------

pom.xml

@@ -278,7 +278,7 @@
                 <scala.version>2.12.10</scala.version>
                 <scala.compat.version>2.12</scala.compat.version>
                 <spark.version>3.4.0</spark.version>
-                <mosaic.version>0.4.0</mosaic.version>
+                <mosaic.version>0.4.1</mosaic.version>
             </properties>
         </profile>
     </profiles>

python/mosaic/__init__.py

@@ -4,4 +4,4 @@
 from .models import SpatialKNN
 from .readers import read
 
-__version__ = "0.4.0"
+__version__ = "0.4.1"

src/main/scala/com/databricks/labs/mosaic/functions/MosaicContext.scala

@@ -1043,7 +1043,7 @@ class MosaicContext(indexSystem: IndexSystem, geometryAPI: GeometryAPI) extends
 object MosaicContext extends Logging {
 
     var _tmpDir: String = ""
-    val mosaicVersion: String = "0.4.0"
+    val mosaicVersion: String = "0.4.1"
 
     private var instance: Option[MosaicContext] = None