You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following sample code with BashOperator works but PythonOperator does not works.
When I call python_task (i.e, remove #), It fails as follows.
I am ruuning on airflow 1.10.12 with helm installation.
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
from airflow.operators.python_operator import PythonOperator
from airflow.operators.bash_operator import BashOperator
from datetime import datetime
def my_func():
print('Hello from my_func')
with DAG('python_dag', description='Python DAG',start_date=datetime(2018, 11, 1), catchup=False) as dag:
dummy_task = DummyOperator(task_id='dummy_task', retries=3)
## following operator works fine
bash_task = BashOperator( task_id='print_date', bash_command='date', dag=dag)
dummy_task >> bash_task
## following operator does not work
#python_task = PythonOperator(task_id='python_task', python_callable=my_func)
#dummy_task >> python_task
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 37, in <module>
args.func(args)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/utils/cli.py", line 81, in wrapper
return f(*args, **kwargs)
File "/home/airflow/.local/lib/python3.6/site-packages/airflow/bin/cli.py", line 618, in run
dag_pickle = session.query(DagPickle).filter(DagPickle.id == args.pickle).first()
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3429, in first
ret = list(self[0:1])
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3203, in __getitem__
return list(res)
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 100, in instances
cursor.close()
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 80, in instances
rows = [proc(row) for row in fetch]
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
rows = [proc(row) for row in fetch]
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 588, in _instance
populators,
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 725, in _populate_full
dict_[key] = getter(row)
File "/home/airflow/.local/lib/python3.6/site-packages/sqlalchemy/sql/sqltypes.py", line 1723, in process
return loads(value)
File "/home/airflow/.local/lib/python3.6/site-packages/dill/_dill.py", line 283, in loads
return load(file, ignore, **kwds)
File "/home/airflow/.local/lib/python3.6/site-packages/dill/_dill.py", line 278, in load
return Unpickler(file, ignore=ignore, **kwds).load()
File "/home/airflow/.local/lib/python3.6/site-packages/dill/_dill.py", line 481, in load
obj = StockUnpickler.load(self)
File "/home/airflow/.local/lib/python3.6/site-packages/dill/_dill.py", line 471, in find_class
return StockUnpickler.find_class(self, module, name)
ModuleNotFoundError: No module named 'unusual_prefix_a05a97b4444c348d2eecd28622afa81f72b1f848_pt'
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Following sample code with BashOperator works but PythonOperator does not works.
When I call python_task (i.e, remove #), It fails as follows.
I am ruuning on airflow 1.10.12 with helm installation.
Beta Was this translation helpful? Give feedback.
All reactions