Request to expose internal methods

[deammer]

Hello, it would be amazing to expose the following as exports:

• normalizeTokens()
• useTokenize()

I'm currently facing a situation where I want to insert elements between lines of code and I'd like to do it outside of the <Highlight> component, but I still want the tokenization improvements performed by normalizeTokens().

Does this make sense?

[carloskelly13]

@deammer We've published a new patch version at 2.3.1 which should expose these functions.

[deammer]

Thank you @carloskelly13 and team! Love that you took the time to add documentation, too.

[johnrengelman]

Is there an example of how to use the output of useTokenize? It says you can use this function and pass the tokens to <Highlight /> but I don't see how to do that as it still requires passing the code property and then providing the rendering functioning.