Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up atlas downloads #407

Open
adamltyson opened this issue Sep 9, 2024 · 4 comments
Open

Speed up atlas downloads #407

adamltyson opened this issue Sep 9, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@adamltyson
Copy link
Member

Downloading the atlases can take a long time. In some areas it can seemingly take hours. We should investigate alternatives to improve this, e.g.:

@adamltyson
Copy link
Member Author

adamltyson commented Sep 30, 2024

Just to note, there are lots of places we could store the data, but with various caveats.

  • Some data sharing repositories - most are not suitable for this type of data, e.g. because the data needs to meet certain standards, and pass a curation process.
  • AWS/cloudflare etc - potentially unpredictable costs
  • EBRAINS - there's an object store (in addition to the curated knowledge base)

@NicoKiaru
Copy link

Hi! Just to report the speed in Switzerland / Lausanne: it can go as slow at 1 MB/s

image

@alessandrofelder
Copy link
Member

Thanks for reporting @NicoKiaru - hopefully we can improve on this in the near future.

@NicoKiaru
Copy link

Sure, I don't want to make you feel that this post is to put pressure - it's just to be informative and report typical DL speed. I mean, waiting 15 minutes once is not a big problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants