[io-managers-deps] Properly handle deps mapping in multi_asset decorator #26466
+66
−11
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
This was referenced Dec 13, 2024
dpeng817
changed the title
dpeng817
force-pushed
the
dpeng817/allow_deps_and_ins_to_coexist
branch
from
b87b89d to
bc2d254
Compare
dpeng817
force-pushed
the
dpeng817/properly_handle_deps_mapping_multi_asset
branch
from
2c70e99 to
a528136
Compare
| for key in asset_in_keys - set(upstream_deps.keys()) | ||
| ] | ||
| raise DagsterInvalidDefinitionError( | ||
| f"Asset inputs {input_names_not_in_deps} specified as asset inputs, but are not specified as dependencies on any constituent asset specs." |
There was a problem hiding this comment.
same comment as https://github.com/dagster-io/dagster/pull/26465/files#r1884541448 - i think hitting this exception could be pretty confusing (especially for a new user dealing w these concepts for the first time). Giving the specific class names would be a big improvement
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary & Motivation
Properly handle deps in the multi_asset decorator. Previously, the behavior when various combinations of multi_asset deps-related args were smashed together was not really under test.
Now that build_and_validate_named_ins can handle overlaps in deps and ins, we can simplify the logic in this part of the decorator significantly. Still leaving a hefty comment about why we disallow unmapped inputs _outside_of build_and_validate_named_ins.
How I Tested These Changes
I added a bunch of new test cases for all the possible interactions.