Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CustomClass cannot be used in dagster.yaml file #26342

Open
danielkroeders opened this issue Dec 8, 2024 · 0 comments
Open

CustomClass cannot be used in dagster.yaml file #26342

danielkroeders opened this issue Dec 8, 2024 · 0 comments
Labels
type: bug Something isn't working

Comments

@danielkroeders
Copy link

What's the issue?

I have a subclass defined like this to store data in MS SQL Server. I could use the python upgrade function to create the tables and I can use the custom_module.py file in another python file, but when I try to use the module and class in the dagster.yaml file to set the run_storage to the custom class and configure the connection_string in the dagster file.

Both the dagster.yaml file and the custom_module.py file are in the same directory.

class SqlServerRunStorage(SqlRunStorage):
def init(self, connection_string: str):
self.connection_string = connection_string
# Create the SQLAlchemy engine
self._engine = create_engine(connection_string)

@contextmanager
def connect(self):
    """Provide a connection to the SQL Server database."""
    conn = pyodbc.connect(self.connection_string)
    try:
        yield conn
    finally:
        conn.close()

def upgrade(self):
    # Create tables without indexes
    RunStorageSqlMetadata.create_all(self._engine)

Yaml file:
run_storage:
module: custom_module
class: SqlServerRunStorage
config:
conncetion_string: <connection_string>

What did you expect to happen?

I expected that the run storage could be set with the subclass based on the SqlRunStorage base class. But I keep getting the error that the module could not be found.

How to reproduce?

See provided code and use dagster dev to launch.

Dagster version

dagster, version 1.9.3

Deployment type

Local

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
By submitting this issue, you agree to follow Dagster's Code of Conduct.

@danielkroeders danielkroeders added the type: bug Something isn't working label Dec 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant