Sentry blew up memory usage

[piotrblasiak]

Integration

sentry-spring-boot-starter-jakarta

Java Version

24

Version

8.14.0

Steps to Reproduce

With default settings and only DSN set, it seems like tracing was enabled and completely blew up RAM usage from 1 to 50 GB per instance. Both versions 7.14.0 and 8.14.0.

Setting sentry.traces-sample-rate=0 fixed the problem, but shouldn´t tracing be disabled by default (+ why would I ever enable it if it uses this much memory?).

Expected Result

RAM usage should be < 1 GB.

Actual Result

50 GB.

[adinauer]

Thanks for opening this issue @piotrblasiak, we'll take a look and report back here.

[adinauer]

@piotrblasiak I just tested using our sample, as well as a fresh project created via start.spring.io and both behave normally, i.e. memory usage stays within a couple hundred megs. After a couple manual GCs it returns to ~30 megs.

What does your setup look like?

• What Sentry dependencies are you adding?
• Are you using our OpenTelemetry integration?
• What does your Spring setup look like?
• What does your application do? Database interaction, etc.

Can you please provide a sample application that reproduces the behaviour so we can help fix?

[piotrblasiak]

I´m afraid I´m not able to provide a sample application as I´m super busy with other things, but I´ll show you my pom.xml, application.properties, logging config and where I´m using sentry in my code which hopefully should give you some clues.

application.properties

spring.aop.auto=false

server.port=8080
server.compression.enabled=true
server.error.whitelabel.enabled=false

# Graceful shutdown
server.shutdown=graceful
spring.lifecycle.timeout-per-shutdown-phase=60s

management.server.port=8081
management.endpoints.enabled-by-default=false
management.endpoint.health.enabled=true
management.endpoint.health.probes.enabled=true
management.endpoint.health.show-details=never
management.health.livenessState.enabled=true
management.health.readinessState.enabled=true
management.endpoint.prometheus.enabled=true
management.endpoints.web.exposure.include=health,prometheus

spring.web.resources.add-mappings=false

spring.servlet.multipart.max-file-size=100MB
spring.servlet.multipart.max-request-size=100MB

spring.jpa.open-in-view=false
spring.jpa.properties.hibernate.default_batch_fetch_size=1000
spring.jpa.properties.hibernate.jdbc.batch_size=1000
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true
spring.jpa.properties.hibernate.jdbc.batch_versioned_data=true
spring.jpa.properties.hibernate.connection.provider_disables_autocommit=true
spring.jpa.properties.hibernate.connection.release_mode=on_close
spring.jpa.properties.hibernate.fail_on_pagination_over_collection_fetch=true
spring.jpa.properties.hibernate.query.in_clause_parameter_padding=true
spring.jpa.properties.hibernate.jdbc.time_zone=UTC
#spring.jpa.hibernate.ddl-auto=update

spring.data.redis.host=${REDIS_HOST}
spring.data.redis.port=${REDIS_PORT:6379}

spring.rabbitmq.addresses=${RABBITMQ_URI}

spring.quartz.auto-startup=${QUARTZ_AUTO_STARTUP}
spring.quartz.job-store-type=jdbc
#spring.quartz.jdbc.initialize-schema=ALWAYS
spring.quartz.properties.org.quartz.jobStore.isClustered=true
spring.quartz.properties.org.quartz.scheduler.instanceId=AUTO
spring.quartz.properties.org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.PostgreSQLDelegate

spring.main.bannerMode=off
spring.main.allow-circular-references=true

spring.threads.virtual.enabled=true

sentry.traces-sample-rate=0 # Otherwise this is enabled(?) and blows up RAM usage

pom.xml (relevant parts only)

<?xml version="1.0" encoding="UTF-8" ?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>3.5.0</version>
		<relativePath/><!-- lookup parent from repository -->
	</parent>

	<properties>

		<java.version>24</java.version>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
		<hibernate.version>6.6.13.Final</hibernate.version>
	</properties>

	<dependencyManagement>
		<dependencies>

			<dependency>
				<groupId>software.amazon.awssdk</groupId>
				<artifactId>bom</artifactId>
				<version>2.31.61</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>

		</dependencies>
	</dependencyManagement>

	<repositories>

		<repository>
			<id>jitpack.io</id>
			<url>https://jitpack.io</url>
		</repository>

	</repositories>

	<dependencies>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-starter-tomcat</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-jetty</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-websocket</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-security</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-amqp</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-jpa</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-data-redis</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.integration</groupId>
			<artifactId>spring-integration-aws</artifactId>
			<version>3.0.9</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-quartz</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web-services</artifactId>
			<exclusions>
				<exclusion>
					<groupId>org.springframework.boot</groupId>
					<artifactId>spring-boot-starter-tomcat</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.springframework.amqp</groupId>
			<artifactId>spring-rabbit-test</artifactId>
			<version>3.2.1</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.retry</groupId>
			<artifactId>spring-retry</artifactId>
		</dependency>

		<dependency>
			<groupId>com.graphql-java</groupId>
			<artifactId>graphql-java</artifactId>
			<version>22.0</version>
		</dependency>

		<dependency>
			<groupId>org.hibernate</groupId>
			<artifactId>hibernate-core</artifactId>
			<version>${hibernate.version}</version>
		</dependency>
		<dependency>
			<groupId>org.hibernate</groupId>
			<artifactId>hibernate-jpamodelgen</artifactId>
			<version>${hibernate.version}</version>
		</dependency>

		<!-- Needed by hibernate -->
		<dependency>
			<groupId>jakarta.data</groupId>
			<artifactId>jakarta.data-api</artifactId>
			<version>1.0.1</version>
		</dependency>

		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>jcl-over-slf4j</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>org.slf4j</groupId>
			<artifactId>jul-to-slf4j</artifactId>
		</dependency>
		<dependency>
			<groupId>io.sentry</groupId>
			<artifactId>sentry-spring-boot-starter-jakarta</artifactId>
			<version>8.14.0</version>
		</dependency>
		<dependency>
			<groupId>io.sentry</groupId>
			<artifactId>sentry-logback</artifactId>
			<version>8.14.0</version>
		</dependency>

		<dependency>
			<groupId>io.micrometer</groupId>
			<artifactId>micrometer-registry-prometheus</artifactId>
			<version>1.15.1</version>
		</dependency>

	</dependencies>

	<build>

		<plugins>

			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
				<configuration>
					<image>
						<name>...</name>
						<imagePlatform>linux/arm64</imagePlatform>
						<buildpacks>
							<buildpack>docker.io/paketobuildpacks/java</buildpack>
							<buildpack>docker.io/paketobuildpacks/opentelemetry</buildpack>
						</buildpacks>
						<env>
							<BP_JVM_VERSION>${java.version}</BP_JVM_VERSION>
							<BP_OPENTELEMETRY_ENABLED>true</BP_OPENTELEMETRY_ENABLED>
						</env>
					</image>
					<!-- JDWP for local debugging -->
					<jvmArguments>
						-XX:+UseZGC -XX:SoftMaxHeapSize=2G -XX:ZUncommitDelay=30 -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005
					</jvmArguments>
				</configuration>
			</plugin>

		</plugins>

	</build>

</project>

Custom event sending:

try {

				final var event = new SentryEvent(throwable);

				final var authentication = SecurityContextHolder.getContext().getAuthentication();
				if (authentication != null) {
					final var user = new User();
					user.setUsername((String) authentication.getPrincipal());
					event.setUser(user);
				}

				if (throwable instanceof LogValuesException contextedException) event.setExtras(contextedException.getLogValues());

				final var mdcContextMap = MDC.getCopyOfContextMap();
				if (mdcContextMap != null) {
					for (final var entry : mdcContextMap.entrySet()) event.setExtra(entry.getKey(), entry.getValue());
				}
				
				reference = Sentry.captureEvent(event).toString().replaceAll("-", "");

				exceptionCache.put(throwable, reference);
			} catch(final Exception e) {

				LOGGER.error("Error sending custom sentry event.", e);
				LOGGER.error(throwable.getMessage(), throwable);
			}

BeforeSendCallback:

@Bean
	public SentryOptions.BeforeSendCallback getSentryBeforeSendCallback() {
		return (event, hint) -> {
			final var request = event.getRequest();
			if (request != null) {
				// Remove potentially sensitive data
				request.setHeaders(null);
				request.setQueryString(null);
				request.setCookies(null);
			}
			return event;
		};
	}

[piotrblasiak]

Oh and the logging config:

logback.xml

<configuration>

	<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
		<resetJUL>true</resetJUL>
	</contextListener>

	<appender name="Console" class="ch.qos.logback.core.ConsoleAppender">
		<encoder>
			<pattern>%d{yyyy-MM-dd HH:mm:ss} @%level: %logger{5} %msg%n</pattern>
		</encoder>
	</appender>

	<appender name="Sentry" class="io.sentry.logback.SentryAppender" />

	<logger name="com.google.i18n.phonenumbers.PhoneNumberUtil" level="ERROR" />
	<logger name="org.springframework.web.client.RestTemplate" level="ERROR" />
	<logger name="org.apache.fop.apps.FOUserAgent" level="ERROR" />
	<!--<logger name="org.hibernate" level="DEBUG" />-->
	<logger name="org.springframework.amqp.rabbit.listener.BlockingQueueConsumer" level="ERROR" />

	<root level="INFO">
		<appender-ref ref="Console" />
		<appender-ref ref="Sentry" />
	</root>

	<turboFilter class="com.attendium.infrastructure.logging.DuplicateFilter" />

</configuration>

logging.properties
handlers = org.slf4j.bridge.SLF4JBridgeHandler

[adinauer]

Thanks we can try again with your details, but I can't say when we'll get to it yet.