Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question]: layout mode law, size chunks missconception #4017

Open
devMls opened this issue Dec 12, 2024 · 1 comment
Open

[Question]: layout mode law, size chunks missconception #4017

devMls opened this issue Dec 12, 2024 · 1 comment
Labels
question Further information is requested

Comments

@devMls
Copy link
Contributor

devMls commented Dec 12, 2024

Describe your problem

I have read several articles arround chunking methods for law documents. In addition I have visit different specoaliced embbeding modelos

Articles always speak arround long embbedings and embbedings models has more a more context capacity.

Muy expiments goed in the same direction , longs chunks improve the results

But when I use ragflow law layout, It always create small chunks about a pararagth.

What is ny blind?

@devMls devMls added the question Further information is requested label Dec 12, 2024
@KevinHuSh
Copy link
Collaborator

If there's not much chunks say, 1K number of chunks, probably, the bigger the chunk is the better the answer is.

If there's much chunks say, 10K number of chunks, much more chunks will be retrieved, it's realy hard to distinguish which chunk is more relevant to quesition that we need to sent to LLM, if we have long chunks.

@JinHai-CN JinHai-CN changed the title [Question]: layout mode law, sze chunks missconception [Question]: layout mode law, size chunks missconception Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants