SparkConnectPlugin and SparkConnectService

Author: jaceklaskowski

docs/server/SparkConnectPlugin.md

@@ -1,8 +1,10 @@
 # SparkConnectPlugin
 
-`SparkConnectPlugin` is a Spark driver plugin (a `SparkPlugin` ([Apache Spark]({{ book.spark_core }}/plugins/SparkPlugin)) with the [driver-side component](#driverPlugin) only).
+`SparkConnectPlugin` is a Spark driver plugin that [starts a SparkConnectService](SparkConnectService.md#start) upon initialization.
 
-`SparkConnectPlugin` is the main entry point for Spark Connect in Apache Spark applications.
+`SparkConnectPlugin` is a `SparkPlugin` ([Spark Core]({{ book.spark_core }}/plugins/SparkPlugin)) with the [driver-side component](#driverPlugin) only.
+
+`SparkConnectPlugin` can be installed into Spark applications using `spark.plugins` ([Spark Core]({{ book.spark_core }}/configuration-properties/#spark.plugins)) configuration property (e.g., [Spark Connect Shell](../spark-connect-shell.md)).
 
 ## Driver-Side Component { #driverPlugin }
 
@@ -12,7 +14,7 @@
     driverPlugin(): DriverPlugin
     ```
 
-    `driverPlugin` is part of the `SparkPlugin` ([Apache Spark]({{ book.spark_core }}/plugins/SparkPlugin/#driverPlugin)) abstraction.
+    `driverPlugin` is part of the `SparkPlugin` ([Spark Core]({{ book.spark_core }}/plugins/SparkPlugin/#driverPlugin)) abstraction.
 
 `driverPlugin` creates a new `DriverPlugin` (Apache Spark) that does the following:
 

docs/server/SparkConnectService.md

@@ -7,7 +7,7 @@
 * Apache Spark applications (as a [Spark driver plugin](SparkConnectPlugin.md#driverPlugin))
 * On command line as a [SparkConnectServer](SparkConnectServer.md) standalone application
 
-Right after `SparkConnectService` has been [started](#start), [a `SparkListenerConnectServiceStarted` event is posted](#postSparkConnectServiceStarted) with all the network connectivity information.
+Right after having been [started](#start), `SparkConnectService` [posts a SparkListenerConnectServiceStarted event](#postSparkConnectServiceStarted) with all the network connectivity information.
 
 ## Creating Instance
 
@@ -61,12 +61,6 @@ Configuration Property | Default Value
 
 `startGRPCService` [builds the server](#server) and starts it.
 
-If successful, prints out the following INFO message to the logs:
-
-```text
-Successfully started service [serviceName] on port [port].
-```
-
 ### createListenerAndUI { #createListenerAndUI }
 
 ```scala
@@ -139,3 +133,16 @@ postSparkConnectServiceStarted(): Unit
     `releaseSession` is generated by the gRPC proto compiler from `spark/connect/base.proto`.
 
 `releaseSession` creates a [SparkConnectReleaseSessionHandler](SparkConnectReleaseSessionHandler.md) to [handle](SparkConnectReleaseSessionHandler.md#handle) the `ReleaseSessionRequest` request.
+
+## Logging
+
+Enable `ALL` logging level for `org.apache.spark.sql.connect.service.SparkConnectService` logger to see what happens inside.
+
+Add the following line to `conf/log4j2.properties`:
+
+```text
+logger.SparkConnectService.name = org.apache.spark.sql.connect.service.SparkConnectService
+logger.SparkConnectService.level = all
+```
+
+Refer to [Logging](../logging.md).