Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new blosc2_schunk_decompress_chunk_ctx() #147

Open
FrancescAlted opened this issue Sep 10, 2020 · 4 comments
Open

Create new blosc2_schunk_decompress_chunk_ctx() #147

FrancescAlted opened this issue Sep 10, 2020 · 4 comments
Labels

Comments

@FrancescAlted
Copy link
Member

The existing blosc2_schunk_decompress_chunk() function uses the internal context associated with the superchunk for decompressing the specified chunk. However, reusing the same context is intrinsically thread unsafe.

It would be nice to add another blosc2_schunk_decompress_chunk_ctx() function where the user can pass a different context per thread, so that it can be thread safe.

@cf-natali
Copy link
Contributor

Just to be clear, this means that it's not possible to use blosc concurrently from multiple threads.
I'm not talking about the threads managed by blosc itself, but by user threads.

For example if one creates a b2nd_array_t there is no way to concurrently perform read-only operations on it (e.g. slicing without holding a lock?

@FrancescAlted
Copy link
Member Author

Well, there are two different kinds of functions. The ones that create a new b2nd_array_t (e.g. b2nd_get_slice), they accept a context as param, and should be thread-safe. However, the ones that do not create a new array (e.g. b2nd_get_slice_cbuffer) they should be not. So in case you want thread-safety, you can use the former.

@cf-natali
Copy link
Contributor

cf-natali commented Mar 1, 2023

Hm, reading the code I can't convince myself that event b2nd_get_slice is thread-safe:
Under the hood it calls b2nd_get_slice_cbuffer with the source array, which in turn calls get_set_slice, which in turns calls blosc2_compress_ctx on the source array, with the source context.

Generally I think that any function which uses the source array context is not thread-safe, and should actually use a user-provided context?

@FrancescAlted
Copy link
Member Author

Ah, that maybe. Would you be up to send a PR with a _ctx counterpart of the functions you need?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants