Tune parser for query with lots of nested fields

[rmosolgo]

I saw that @DmitryTsepelev gave a really good talk about graphql-ruby's internals, and it included some detailed benchmarks. I decided to see if I could improve them 😈

They're here: https://gist.github.com/DmitryTsepelev/36e290cf64b4ec0b18294d0a57fb26ff

Specifically, I looked at the parsing benchmark. I was able to reduce the memory by about half, but the time is about the same.

Before

~/code/graphql-ruby $ ruby benchmark.rb
                                   user     system      total        real
8 fields, 0 nested levels (   122 bytes):   0.000366   0.000017   0.000383 (  0.000382)
8 fields, 2 nested levels ( 10706 bytes):   0.022603   0.000751   0.023354 (  0.024115)
8 fields, 4 nested levels (853970 bytes):   0.948247   0.013408   0.961655 (  0.974112)
Total allocated: 77147304 bytes (664744 objects)
Total retained:  0 bytes (0 objects)

allocated memory by gem
-----------------------------------
  54076800  graphql-ruby/lib
  15392936  other
   7677568  racc

After

~/code/graphql-ruby $ ruby benchmark.rb
                                   user     system      total        real
8 fields, 0 nested levels (   122 bytes):   0.000298   0.000012   0.000310 (  0.000305)
8 fields, 2 nested levels ( 10706 bytes):   0.013025   0.000106   0.013131 (  0.013165)
8 fields, 4 nested levels (853970 bytes):   0.887415   0.012889   0.900304 (  0.907571)

Total allocated: 32733272 bytes (393241 objects)
Total retained:  0 bytes (0 objects)

allocated memory by gem
-----------------------------------
  13520544  other
  11535160  graphql-ruby/lib
   7677568  racc

The optimizations were:

• Use positional args instead of keyword args for Token.new, to avoid allocating a Hash (😢 I wish that wasn't necessary!)
• Hardcode token values for constant tokens (punctuation), then skip the allocate-then-pack for determining the token value
• Don't allocate a hash in Field#initialize_node
• Don't call .line_and_column which allocates a needless array (hilariously, this is a carryover from the very first parser, written in parslet. Great gem.)
• Instead of allocating a new next_token, use the same array over and over. This one seems kind of risky -- there's no guarantee that racc isn't mutating the array in its own way!

[DmitryTsepelev]

Wow, 2x memory reduce is a huge win!

[rmosolgo]

I don't know how much it really matters in a production setting, but I enjoyed the hunt :P Thanks again for sharing the benchmarks. It looks like it was a great talk too, even though all I could do was read the slides!

I also have a suggestion for improving the runtime benchmark:

  class GraphqlSchema < GraphQL::Schema
    query QueryType
+   use GraphQL::Execution::Interpreter
+   use GraphQL::Analysis::AST
  end
# And change the warmup: 

-   GraphqlSchema.graphql_definition 
+   GraphqlSchema.execute(query, variables: { id: "1" })

Total allocated: 69.65 kB (704 objects)
Total retained:  0 B (0 objects)

allocated memory by gem
-----------------------------------
  53.78 kB  graphql-ruby/lib
   7.19 kB  other
   5.81 kB  ostruct
   2.33 kB  racc
  544.00 B  forwardable

About a 40% improvement! I'm hoping to make Interpreter the default in 1.12.

[DmitryTsepelev]

It looks like it was a great talk too, even though all I could do was read the slides!

Thank you soo much! I plan to give it at least once in English somewhere 🙂I've updated the benchmark and got almost the same performance boost, will re-run it again as soon as this PR is merged and released