micromark-util-subtokenize results in high cpu costs and memory allocations

[starpit]

It looks like the subcontent() function calls slice and splice. On larger documents, we have found that this results in 22% of cpu ticks being spent in array splicing. Would it be possible to change the Events[] into a data structure more suitable for splicing?

[wooorm]

I am not sure the overhead of a linked list or similar will be that much faster.
Yes, a lot of time is spent merging things together; it might be needed to spend time on something similar to it.

You are welcome to test it out while working on a PR though.
It will mean a giant breaking change of everything though. Every package, every extension, deeply depends on simple arrays.

Perhaps there is an alternative. I have recently noted that a following approach can help speed it up a lot:

• Split events off from the end if an array, from the end backwards to the start, pushing each existing slice into an array, also pushing new slices to that array
• Append each slice to the array