[Feature] bulk load optimization

[luohaha]

Feature request

Is your feature request related to a problem? Please describe.

When performing large data ingestions, memory limitations during the ingestion process can result in the final Rowset containing a large number of segment files. This negatively impacts query performance and increases resource consumption (CPU and memory) during subsequent compaction operations.

Describe the solution you'd like

To address this, we propose a new large data ingestion strategy that spills data during the ingestion process and assembles it into final segment files in the final stage of the ingestion. This approach reduces the number of segment files in the Rowset.

• Part 1 - Block manager [Feature] bulk load optimization (Part 1 - block manager) #53921
• Part 2 - Spill mem table sink
• Part 3 - Block merger