Network Configuration

❗️
This is a legacy Apache Ignite documentation
The new documentation is hosted here: https://ignite.apache.org/docs/latest/

TcpCommunicationSpi
Configuration

TcpCommunicationSpi

CommunicationSpi provides basic plumbing to send and receive grid messages and is utilized for all distributed grid operations, such as task execution, monitoring data exchange, distributed event querying, and others. Ignite provides TcpCommunicationSpi as the default implementation of CommunicationSpi, that uses TCP/IP to communicate with other nodes.

To enable communication with other nodes, TcpCommunicationSpi adds TcpCommunicationSpi.ATTR_ADDRS and TcpCommunicationSpi.ATTR_PORT local node attributes. At start up, this SPI tries to start listening to the local port specified by the TcpCommunicationSpi.setLocalPort(int) method. If the local port is occupied, then the SPI will automatically increment the port number until it can successfully bind for listening. The TcpCommunicationSpi.setLocalPortRange(int) configuration parameter controls the maximum number of ports that the SPI will try before it fails.

📘
Local Port Range
Port range comes in very handy when starting multiple grid nodes on the same machine or even in the same VM. In this case, all nodes can be brought up without a single change in the configuration.

📘
IPv4 vs IPv6
Ignite tries to support IPv4 and IPv6 but this can sometimes lead to issues where the cluster becomes detached. A possible solution — unless you require IPv6 — is to restrict Ignite to IPv4 via the -Djava.net.preferIPv4Stack=true JVM parameter.

Configuration

The following configuration parameters can be optionally configured on TcpCommunicationSpi:

Setter Method	Description	Default
setLocalAddress(String)	Sets local host address for socket binding.	Any available local IP address.
setLocalPort(int)	Sets local port for socket binding.	47100
setLocalPortRange(int)	Controls maximum number of local ports tried if all previously tried ports are occupied.	100
setTcpNoDelay(boolean)	Sets value for the `TCP_NODELAY` socket option. Each socket accepted or created will use the provided value. This should be set to true (default) for reducing request/response time during communication over TCP protocol. In most cases we do not recommend changing this option.	true
setConnectTimeout(long)	Sets connect timeout used when establishing connection with remote nodes.	1000
setIdleConnectionTimeout(long)	Sets maximum idle connection timeout upon which a connection to client will be closed.	30000
setBufferSizeRatio(double)	Sets the buffer size ratio for this SPI. As messages are sent, the buffer size is adjusted using this ratio.	0.8 or `IGNITE_COMMUNICATION_BUF_RESIZE_RATIO` system property value, if set.
setMinimumBufferedMessageCount(int)	Sets the minimum number of messages for this SPI, that are buffered prior to sending.	512 or `IGNITE_MIN_BUFFERED_COMMUNICATION_MSG_CNT` system property value, if set.
setUsePairedConnections(boolean)	Sets a flag indicating whether dual socket connection between nodes should be enforced. If set to true, two separate connections will be established between communicating nodes: one for outgoing messages, and one for incoming. When set to false, a single TCP connection will be used for both directions. This flag is useful on some operating systems, when messages take too long to get delivered.	false
setConnectionBufferSize(int)	This parameter is used only when `setAsyncSend(boolean)` is set to false. Sets connection buffer size for synchronous connections. Increase buffer size if using synchronous send and sending large amount of small sized messages. However, most of the time this should be set to 0 (default).	0
setSelectorsCount(int)	Sets the count of selectors to be used in TCP server.	Default count of selectors equals to the expression result - Math.min(4, Runtime.getRuntime() .availableProcessors())
setConnectionBufferFlushFrequency(long)	This parameter is used only when `setAsyncSend(boolean)` is set to false. Sets connection buffer flush frequency in milliseconds. This parameter makes sense only for synchronous send when connection buffer size is not 0. Buffer will be flushed once within specified period if there is not enough messages to make it flush automatically.	100
setDirectBuffer(boolean)	Switches between using NIO direct and NIO heap allocation buffers. Although direct buffers perform better, in some cases (especially on Windows) they may cause JVM crashes. If that happens in your environment, set this property to false.	true
setDirectSendBuffer(boolean)	Switches between using NIO direct and NIO heap allocation buffers usage for message sending in asynchronous mode.	false
setAsyncSend(boolean)	Switches between synchronous and asynchronous message sending. This should be set to true (default) if grid nodes send large amount of data over the network from multiple threads, however this may be environment and application specific and we recommend benchmarking the application in both modes.	true
setSharedMemoryPort(int)	Sets the port which will be used by `IpcSharedMemoryServerEndpoint`. Nodes started on the same host will communicate over IPC shared memory (only for Linux and MacOS hosts). Set this to -1 to disable IPC shared memory communication.	48100
setSocketReceiveBuffer(int)	Sets receive buffer size for sockets created or accepted by this SPI. If not provided, the default is 0 which leaves the buffer unchanged after socket creation (i.e. uses Operating System default value).	0
setSocketSendBuffer(int)	Sets send buffer size for sockets created or accepted by this SPI. If not provided, the default is 0 which leaves the buffer unchanged after socket creation (i.e. uses Operating System default value).	0

Example

<bean class="org.apache.ignite.configuration.IgniteConfiguration">
  ...
  <property name="communicationSpi">
    <bean class="org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi">
      <!-- Override local port. -->
      <property name="localPort" value="4321"/>
    </bean>
  </property>
  ...
</bean>

TcpCommunicationSpi commSpi = new TcpCommunicationSpi();
 
// Override local port.
commSpi.setLocalPort(4321);
 
IgniteConfiguration cfg = new IgniteConfiguration();
 
// Override default communication SPI.
cfg.setCommunicationSpi(commSpi);
 
// Start grid.
Ignition.start(cfg);