Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable connection reuse in libcurl #80

Closed

Conversation

copperlight
Copy link
Collaborator

@copperlight copperlight commented Oct 19, 2023

Originally, in the HttpClient::perform() method, a new CurlHandle was created
upon every invocation, leading to subsequent calls to curl_easy_init() and
curl_easy_cleanup(), as the variable went in and out of scope while it executed
on the asio::thread_pool. This meant that there was no opportunity to reuse
connections from the underlying connection pool, because it was always cleaned up.

By marking the CurlHandle with the thread_local storage duration, it ensures
that the same one remains in use for each thread, thus providing the means to
reuse the established connection pool.

Fixes #51.

@copperlight copperlight force-pushed the connection-reuse branch 2 times, most recently from 13c45a9 to 802819e Compare October 19, 2023 21:48
Originally, in the `HttpClient::perform()` method, a new `CurlHandle` was created
upon every invocation, leading to subsequent calls to `curl_easy_init()` and
`curl_easy_cleanup()`, as the variable went in and out of scope while it executed
on the `asio::thread_pool`. This meant that there was no opportunity to reuse
connections from the underlying connection pool, because it was always cleaned up.

By marking the `CurlHandle` with the `thread_local` storage duration, it ensures
that the same one remains in use for each thread, thus providing the means to
reuse the established connection pool.

Fixes Netflix-Skunkworks#51.
@copperlight
Copy link
Collaborator Author

To implement connection reuse properly, we need to use the multi interface, which is more involved:

https://curl.se/libcurl/c/libcurl-multi.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-Use Connections When Publishing to the Aggregator
1 participant