Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect pMapIterable behaviour #76

Open
dmitri-gb opened this issue Aug 11, 2024 · 0 comments · May be fixed by #77 or #74
Open

Incorrect pMapIterable behaviour #76

dmitri-gb opened this issue Aug 11, 2024 · 0 comments · May be fixed by #77 or #74

Comments

@dmitri-gb
Copy link

dmitri-gb commented Aug 11, 2024

pMapIterable doesn't adhere to the concurrency limit when the source is an async iterable and backpressure > concurrency.

Here's an example program that reproduces the issue:

import { setTimeout } from 'timers/promises';
import { pMapIterable } from 'p-map';

async function* source() {
  yield 1;
  yield 2;
  yield 3;
}

const target = pMapIterable(source(), async n => {
  console.log(`Running with ${n}...`);
  await setTimeout(1000);
  console.log(`Finished running with ${n}`);
}, {
  concurrency: 1,
  backpressure: 2
});

for await (const _ of target) { }

Current output:

Running with 1...
Finished running with 1
Running with 2...
Running with 3...
Finished running with 2
Finished running with 3

Expected output:

Running with 1...
Finished running with 1
Running with 2...
Finished running with 2
Running with 3...
Finished running with 3

As you can see, the mapper function has two instances running in parallel even though concurrency is set to 1.

FYI @Richienb

dmitri-gb added a commit to dmitri-gb/p-map that referenced this issue Aug 11, 2024
@dmitri-gb dmitri-gb linked a pull request Aug 11, 2024 that will close this issue
tgfisher4 added a commit to tgfisher4/p-map that referenced this issue Aug 25, 2024
…ddress case of: infinite concurrency + async iterable producing >1 element

2. use `!isSyncIterator` as shortcut for `isPromiseLike(next)` (`next` is promise iff iterator is async)
3. add `trySpawn` to the `returnValue === pMapSkip && preserveOrder && (promise mapping next input iterable element is pending` branch
4. add tests for changes (1) and (3)
5. tests `rangeAround` helper
6. extra `pMapSkip` tests
7. test for sindresorhus#76
tgfisher4 added a commit to tgfisher4/p-map that referenced this issue Aug 25, 2024
… case of: infinite concurrency + async iterable producing >1 element

2. use `!isSyncIterator` as shortcut for `isPromiseLike(next)` (`next` is promise iff iterator is async)
3. add `trySpawn` to the `returnValue === pMapSkip && preserveOrder && (promise mapping next input iterable element is pending` branch
4. add tests for changes (1) and (3)
5. tests `rangeAround` helper
6. extra `pMapSkip` tests
7. test for sindresorhus#76
@tgfisher4 tgfisher4 linked a pull request Aug 25, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant