[Suggestion] Metapackage as solution to current issues with opencv as dependency #987
Comments
### What I think it should fix most of - #6901 In short, the opencv packages are an aboslute dumpster fire. This is a good summary of relevant discussions on the matter: - opencv/opencv-python#987 In short: `opencv-python` and `opencv-contrib-python` are two distinct, conflicting packages. But `pypi` doesn't have a way of specifying package conflicts (same as our `rerun` vs `rerun-sdk` issue). If you install both, one will shadow the other, non-determinically causing some features to appear to be missing. Conda fortunately somewhat side-steps this by only providing one. Unfortunately, the pixi pypi-conda-mapping only maps `opencv-python` -> `conda: py-opencv` but doesn't handle the mapping for `opencv-contrib-python`. To solve this we now: force `opencv` to be installed via conda for both `base` and `examples`. This means any package that depends on `opencv-python` will pick up the conda version (which actually includes contrib). We still have both direct and indirect (via mediapipe) packages that depend on `opencv-contrib-python` that lead to some dangerous package-shadowing. But at least they're mostly the same package at this point. (I added a comment to the tracking issue with with pix as well: prefix-dev/pixi#771 (comment)) Also I hit some weird issues with `numpy` since `rerun-sdk` has a dependency on `numpy<2`. Certain permutations of packages seem to trick the solver into thinking the constraints (when pulled from pypy) are non-satisfiable. By pulling the numpy dep into conda dep list everything seems happy. ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6946?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6946?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! * [x] If have noted any breaking changes to the log API in `CHANGELOG.md` and the migration guide - [PR Build Summary](https://build.rerun.io/pr/6946) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`.
|
This would be really helpful. As when using Python package managers, i.e. Poetry, the package manager will resolve the correct versions of all dependencies recursively. So therefore when my package has a dependency on i.e. the headless opencv, but I want the normal version, as I want to visualize stuff. I get both. So I have to manually fix the situation by removing and installing packages. While when this was done with extras instead of different packages, it would just resolve the different opencv dependencies to one package with the required extras, so no conflicts. I think this is only be possible as the base version would be headless and Gui is an extra. As a library would want to use the headless version. But then the application, which is depending on that library, would want to use the Gui. Which would require the headless extra to not be added. Which is still causing conflicts. (Preferably the opencv-python package is adapted to this new way of distributing instead of creating a new metapackage.) |
Expected behaviour
Depending on opencv-python has always been an issue, depending on whether libraries used
opencv-python,opencv-python-headless,opencv-contrib-pythonoropencv-contrib-python-headless. I do know that opencv isn't designed as a plugin system, which is why it had to be done that way. But as a result, usually special workarounds like detecting insidesetup.pywhat version of opencv-python is installed, or needing the user to manually uninstall then reinstall the version of opencv they want, is needed. This makes dependency management when depending on opencv complicated.My suggestion is what if there could be an
opencv-metapackage instead, where which version is pulled in depends on extra flags. The mapping could be like:opencv-meta->opencv-pythonopencv-meta[headless]->opencv-python-headlessopencv-meta[contrib]->opencv-contrib-pythonopencv-meta[contrib,headless]->opencv-contrib-python-headlessThis can be on top of the existing four packages to avoid breaking existing libraries. Given sufficient adoption of depending on
opencv-metarather than the different versions of opencv, it should hopefully resolve the dependency issues one day.That said, I might not be thinking deep enough and there is still something seriously wrong about the above approach.
Issue submission checklist
opencv-pythonThe text was updated successfully, but these errors were encountered: