creating many records in a single query

[killJoy-03]

I have an API that does multiple tasks in multiple threads. (In a single transaction)

Suppose in a task, it is creating 6000 records in a single query
Format is like
MATCH (a:A)
/// 6000 subqueries for creating

NOTE : And we are consuming all the executed queries even though we do not need them, for immediate commit.

Now, for the above query, it throws Out of Memory Exception. some heap was out of memory.

How do you tackle this.
I solution I can do is run in batches.

Any thing else?

[AndyHeap-NeoTech]

Hi KillJoy.

Is the out of memory exception being thrown by the server or in the driver? Your initial suggestion of batching is the approach I would take to tackle this.

One other observation, if you are using driver sessions then you need to be aware that they are not thread safe. They are designed to be light weight and short lived without being shared across threads. The driver itself is thread safe.

[killJoy-03]

@AndyHeap-NeoTech
I guess the exception is thrown by driver.

Regarding thread safe ....I am not executing parallel creation (instead wait for one batch to complete to start another).

I've tried using UNWIND
I am sending all the neessary data as List<List>
then using UNWIND to iterate the loop and create records ///// this is superfast and no exceptions are thrown.