JavaScript heap out of memory errors after upgrading 1.7.1 -> 1.8.0

[jpalo]

Category

• Question
• Typo
• Bug
• Additional article idea

Expected or Desired Behavior

No memory errors when building and working with the code in SP Online workbench.

Observed Behavior

Upgraded from 1.7.1 to 1.8.0 and started seeing this in my project. Occurs every 3-4th time I modify file & save, which triggers build. As you can see, when the issue occurs on the copy-assets step, webpack step takes long (1 min in this case, also had it occur just now with 26 secs runtime). When it doesn't fail, webpack step only takes 6-11 secs.

[15:21:39] [tslint] tslint version: 5.12.1
[15:21:39] Starting subtask 'tsc'...
[15:21:39] [tsc] typescript version: 3.3.4000
[15:21:39] Finished subtask 'copy-static-assets' after 179 ms
[15:21:46] Finished subtask 'tsc' after 7.75 s
[15:21:46] Finished subtask 'tslint' after 7.76 s
[15:21:46] Starting subtask 'post-copy'...
[15:21:46] Finished subtask 'post-copy' after 562 μs
[15:21:46] Starting subtask 'collectLocalizedResources'...
[15:21:46] Finished subtask 'collectLocalizedResources' after 1.17 ms
[15:21:46] Starting subtask 'configure-webpack'...
[15:21:46] Finished subtask 'configure-webpack' after 1.15 ms
[15:21:46] Starting subtask 'webpack'...
[15:22:49] Finished subtask 'webpack' after 1.03 min
[15:22:49] Starting subtask 'configure-webpack-external-bundling'...
[15:22:49] Finished subtask 'configure-webpack-external-bundling' after 222 ms
[15:22:49] Starting subtask 'copy-assets'...

<--- Last few GCs --->

[11816:00000191464683D0]   629760 ms: Mark-sweep 1009.4 (1672.0) -> 1009.4 (1671.0) MB, 217.7 / 0.0 ms  allocation failure GC in old space requested
[11816:00000191464683D0]   630004 ms: Mark-sweep 1009.4 (1671.0) -> 1009.4 (1639.5) MB, 243.7 / 0.0 ms  last resort GC in old space requested
[11816:00000191464683D0]   630212 ms: Mark-sweep 1009.4 (1639.5) -> 1009.4 (1639.5) MB, 207.9 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0000019448425879 <JSObject>
    1: stringSlice(aka stringSlice) [buffer.js:~555] [pc=000000511DFE3C0C](this=000000ABBE1822D1 <undefined>,buf=0000007CD33D5AB9 <Uint8Array map = 000003C65C2C3A71>,encoding=0000019448435221 <String[4]: utf8>,start=0,end=3488682)
    2: toString [buffer.js:~609] [pc=000000511F3114C8](this=0000007CD33D5AB9 <Uint8Array map = 000003C65C2C3A71>,encoding=0000019448435221 <String[4]: utf8>,start=00...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewRawTwoByteString
 5: v8::internal::Factory::NewStringFromUtf8
 6: v8::String::NewFromUtf8
 7: v8::internal::wasm::AsmType::Extern
 8: std::vector<v8::CpuProfileDeoptFrame,std::allocator<v8::CpuProfileDeoptFrame> >::vector<v8::CpuProfileDeoptFrame,std::allocator<v8::CpuProfileDeoptFrame> >
 9: 000000511E32C466

Package.json

{
  "name": "project-folders",
  "version": "3.1.1",
  "private": true,
  "engines": {
    "node": ">=0.10.0"
  },
  "dependencies": {
    "@microsoft/sp-core-library": "1.8.0",
    "@microsoft/sp-http": "1.8.0",
    "@microsoft/sp-lodash-subset": "1.8.0",
    "@microsoft/sp-property-pane": "1.8.0",
    "@microsoft/sp-webpart-base": "1.8.0",
    "@pnp/common": "~1.2.7",
    "@pnp/logging": "~1.2.7",
    "@pnp/odata": "~1.2.7",
    "@pnp/sp": "~1.2.7",
    "@types/es6-collections": "^0.5.31",
    "@types/es6-promise": "0.0.33",
    "@types/microsoft-ajax": "^0.0.33",
    "@types/react": "16.4.2",
    "@types/react-dom": "16.0.5",
    "@types/sharepoint": "^2016.1.2",
    "@types/webpack-env": "1.13.1",
    "core-js": "^2.5.4",
    "office-ui-fabric-react": "^6.156.0",
    "react": "16.7.0",
    "react-dom": "16.7.0"
  },
  "resolutions": {
    "@types/react": "16.4.2"
  },
  "devDependencies": {
    "@microsoft/sp-build-web": "1.8.0",
    "@microsoft/sp-module-interfaces": "1.8.0",
    "@microsoft/sp-tslint-rules": "1.8.0",
    "@microsoft/sp-webpart-workbench": "1.8.0",
    "@microsoft/rush-stack-compiler-3.3": "0.1.7",
    "@types/chai": "3.4.34",
    "@types/mocha": "2.2.38",
    "ajv": "~5.2.2",
    "gulp": "~3.9.1",
    "webpack-bundle-analyzer": "^3.0.3"
  },
  "scripts": {
    "build": "gulp bundle",
    "clean": "gulp clean",
    "test": "gulp test"
  }
}

tslint.json

{
  "extends": "@microsoft/sp-tslint-rules/base-tslint.json",
  "rules": {
    "class-name": false,
    "export-name": false,
    "forin": false,
    "label-position": false,
    "member-access": true,
    "no-arg": false,
    "no-console": false,
    "no-construct": false,
    "no-duplicate-variable": true,
    "no-eval": false,
    "no-function-expression": true,
    "no-internal-module": true,
    "no-shadowed-variable": true,
    "no-switch-case-fall-through": true,
    "no-unnecessary-semicolons": true,
    "no-unused-expression": true,
    "no-use-before-declare": true,
    "no-with-statement": true,
    "semicolon": true,
    "trailing-comma": false,
    "typedef": false,
    "typedef-whitespace": false,
    "use-named-parameter": true,
    "variable-name": false,
    "whitespace": false
  }
}

tsconfig.json

{
  "extends": "./node_modules/@microsoft/rush-stack-compiler-3.3/includes/tsconfig-web.json", 
  "compilerOptions": {
    "target": "es5",
    "forceConsistentCasingInFileNames": true,
    "module": "esnext",
    "moduleResolution": "node",
    "jsx": "react",
    "declaration": true,
    "sourceMap": true,
    "experimentalDecorators": true,
    "skipLibCheck": true,
    "outDir": "lib",
    "strictNullChecks": false,
    "inlineSources": false,   
    "typeRoots": [
      "./node_modules/@types",
      "./node_modules/@microsoft"
    ],
    "types": [
      "webpack-env",
      "microsoft-ajax",
      "sharepoint"
    ],
    "lib": [
      "es6",
      "dom",
      "es5",
      "es2015.collection"
    ]
  },
  "include": [
    "src/**/*.ts"
  ],
  "exclude": [
    "node_modules",
    "lib"
  ]
}

Steps to Reproduce

1. Modify any file in project, save file
2. Auto build fires
3. Every 3-4th time, gives the error

Project contains 1 React-based web part.

I understand this is a bit vague, apologies, but I don't have much idea what to try, or what kind of addition information you would really need to be able to comment on the issue.

[msft-github-bot]

Thank you for reporting this issue. We will be triaging your incoming issue as soon as possible.

[seriewe]

We have the same problem.

<--- Last few GCs --->
[14092:0000027560E69E70]   417467 ms: Mark-sweep 1339.7 (1468.8) -> 1339.6 (1469.3) MB, 1244.5 / 0.0 ms  allocation failure GC in old space requested
[14092:0000027560E69E70]   418437 ms: Mark-sweep 1339.6 (1469.3) -> 1339.6 (1426.3) MB, 969.9 / 0.0 ms  last resort GC in old space requested
[14092:0000027560E69E70]   419427 ms: Mark-sweep 1339.6 (1426.3) -> 1339.6 (1426.3) MB, 989.2 / 0.0 ms  last resort GC in old space requested
<--- JS stacktrace --->

==== JS stack trace =========================================
Security context: 00000200413A57C1 <JSObject>
    1: /* anonymous */ [C:\Repos\[MYSOLUTIONNAME]\node_modules\source-map\lib\source-node.js:~342] [pc=000003641B16BB55](this=0000008B8008C211 <JSGlobal Object>,chunk=000003CC049E9C69 <String[1]: }>,original=000000A52B208441 <Object map = 0000002368E4C971>)
    2: SourceNode_walk [C:\Repos\[MYSOLUTIONNAME]\node_modules\source-map\lib\source-node.js:~221] [pc=000003641B16537E](t...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewUninitializedFixedArray
 5: v8::internal::WasmDebugInfo::SetupForTesting
 6: v8::internal::interpreter::BytecodeArrayRandomIterator::UpdateOffsetFromIndex
 7: 000003641A2843C1

Our Setup/Config:
We have approx. 33 WebParts and 5 extensions.

@microsoft/rush-stack-compiler-3.3
"office-ui-fabric-react": "^6.159.1"
...

What we've tried out so far:

1. Changing the typescript rush-stack-compilers to a lower version does not change anything about the behavior.
2. After a downgrade "office-ui-fabric-react": "5.131.0" we did not get the error.
3. If we reduce the number of WebParts we bundle, the error does not occur anymore.
4. observing the memory with memwatch provides this values:

» full     trend     current         min         max                                                                                                                                                         
» 1            0    92.24 MB   122.19 MB   122.19 MB 
peak during copy-assets
» 29           2     1.16 GB   122.19 MB     1.49 GB
after first reload
» 32         2.2   767.34 MB   122.19 MB     1.49 GB
[14:35:26] Starting subtask 'webpack'...
» 232          0   798.53 MB   122.19 MB     1.49 GB
» 233          0   825.90 MB   122.19 MB     1.49 GB
» 237        0.2   958.69 MB   122.19 MB     1.49 GB
CRASH
<--- Last few GCs --->

Also we see that the subtask webpack takes more than 1min, and our "rebuild" after a change on running gulp serve takes over 2 Minutes.

What is the recommendation for projects with many WebParts, should we move them into several bundles or own projects?

Hope this information helps and it can be improved soon

[jpalo]

It doesn't appear to be directly related to number of web parts, or size of the bundle as I only have 1 web part. Considering my webpack subtask runtime (~8 secs in our case), I'd say my project is still very much smaller compared to yours - yet same issue is occurring.

What would be interesting to know is if this is just issue occurring during development time, so is it safe to deploy the solution.

[justdevelopment]

We have the same issue, but it fails on the webpack step. When downgrading to v5 it bundles fine but we'd like to use the latest. Seems like maybe there's a loop somewhere just eating memory? There are more issues with the fabric for react package in relation to SPFx, but upgrading typescript seems to solve those (ie #3622). Could it be related to #3618 (comment), where pat miller says internally they are building with typescript 2?

[OliverZeiser]

I am seeing the same issue. It seems to be related to using office-ui-fabric-react v6. But why v6 and SPFx don't work together...I can't figure that out

[justdevelopment]

Fun fact, if you increase the max memory for node it will bundle/serve eventually.

Using this to set the memory to a max of around 4gb, it works for me:
gulp bundle --max_old_space_size=4000
Using the flag same for gulp serve will allow me to debug:
gulp serve --nobrowser --max_old_space_size=4000

For our project it seems to take around 3.5 gb of memory to complete it's tasks. Does make me wonder why it's using to much memory for the new fabric...?

[jpalo]

It peaks at around 3.2 GB in my case at the webpack stage, also CPU usage is constant 30%. Have now waited for few minutes it to complete.

And turns out it even crashes with 4000 space_size, but not nearly as often (this is the first time in 2 days).

[12:59:47] Starting subtask 'webpack'...

<--- Last few GCs --->

[20100:000001E25EC267F0] 14003990 ms: Mark-sweep 2740.0 (4863.3) -> 2740.0 (4863.3) MB, 358.3 / 0.0 ms  allocation failure scavenge might not succeed
[20100:000001E25EC267F0] 14004339 ms: Mark-sweep 2740.0 (4863.3) -> 2740.0 (4847.3) MB, 348.1 / 0.0 ms  last resort GC in old space requested
[20100:000001E25EC267F0] 14004686 ms: Mark-sweep 2740.0 (4847.3) -> 2740.0 (4847.3) MB, 347.0 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 00000208F9C258B9 <JSObject>
    1: DoJoin(aka DoJoin) [native array.js:~94] [pc=0000023FABDA738C](this=000002A3B15022D1 <undefined>,o=000002EF2C15D421 <JSArray[808]>,p=808,D=000002A3B1502371 <true>,z=00000208F9C7CDE9 <String[1]:  >,y=000002A3B15023E1 <false>)
    2: Join(aka Join) [native array.js:~119] [pc=0000023FAB34949E](this=000002A3B15022D1 <undefined>,o=000002EF2C15D421 <JSArray[808]>,p=808,z=00000208F9C7CDE9 <Str...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewRawTwoByteString
 5: v8::internal::AsmJsScanner::IsNumberStart
 6: 0000023FAB2043C1

[GuthrieS]

We are also having a similar problem. The solution has been upgraded from SPFx 1.7.0 to version 1.8.0 however, in our case this error appears when it is performing the tslint subtask. This occurs with gulp build, serve or bundle:

[14:24:33] Starting 'build'...
[14:24:33] Starting subtask 'configure-sp-build-rig'...
[14:24:33] Finished subtask 'configure-sp-build-rig' after 5.19 ms
[14:24:33] Starting subtask 'pre-copy'...
[14:24:33] Finished subtask 'pre-copy' after 64 ms
[14:24:33] Starting subtask 'copy-static-assets'...
[14:24:33] Starting subtask 'sass'...
[14:24:33] Finished subtask 'copy-static-assets' after 175 ms
[14:24:33] Finished subtask 'sass' after 275 ms
[14:24:33] Starting subtask 'tslint'...
[14:24:34] [tslint] tslint version: 5.11.0
[14:24:34] Starting subtask 'tsc'...
[14:24:34] [tsc] typescript version: 2.7.2
[14:24:38] Finished subtask 'tsc' after 4.3 s
[14:24:44] Error - [tslint] FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
[14:24:44] Error - [tslint] 1: node_module_register
 2: v8::internal::FatalProcessOutOfMemory
 3: v8::internal::FatalProcessOutOfMemory
 4: v8::internal::Factory::NewUninitializedFixedArray
 5: v8::internal::WasmDebugInfo::SetupForTesting
 6: v8::internal::interpreter::BytecodeArrayRandomIterator::UpdateOffsetFromIndex
 7: 000000BFDEA043C1
[14:24:44] Finished subtask 'tslint' after 12 s
[14:24:44] Starting subtask 'post-copy'...
[14:24:44] Finished subtask 'post-copy' after 782 μs
[14:24:44] Finished 'build' after 12 s
[14:24:45] ==================[ Finished ]==================

By removing a lot of the rules from the tslint.json file the problem goes away however we know this isn't the solution and the underlying problem is elsewhere. I can confirm similarities as well with the memory issues being described by @jpalo

[andrewconnell]

This smells like a common issue due to large projects, such as SPFx+Fabric React v6 (which is now supported with SPFx v1.8). As suggested above, try building with the --max_old_space_size flag & increasing the amount of available memory? For instance:

gulp bundle --max_old_space_size=8192 --ship

[vman]

Yup we have this issue for one of larger projects as well when upgrading to 1.8 and TypeScript 3.3.4000

Does this have something to do with multiple versions of TypeScript being present in the build setup now? And hence being loaded in memory during bundling? I have noticed the build system still uses different TS versions for different internal packages and and top of that we can use another TS version in our webparts.

(The disk size of node_modules has also increased from ~450mb to ~650mb due to the multiple TS versions)

[OliverZeiser]

@andrewconnell it does help increasing the memory but it seems to be some sort of leak. If you are running gulp serve and make changes on your code it will build over and over until it reaches the memory limit. Besides the fact that the build time goes up by about 400% every time. So this is definitley not a solution, but can help to at least build the project once. Adding @patmill and @VesaJuvonen since this seems to be an issue for many of us. I am sure they can add the right people to get a fix/solution for that.