如何通过apache气流运行shell脚本答案

【问题标题】：How to run a shell script through apache airflow如何通过apache气流运行shell脚本
【发布时间】：2023-03-10 23:24:01
【问题描述】：

我正在尝试通过气流运行 shell 脚本，当我在本地执行 shell 脚本时它可以工作。

这是我的 Dag 代码：

    dag = DAG(dag_id='Phase1_dag_v1', default_args=args, schedule_interval=None)

with dag:
    test= BashOperator(
        task_id='test',
        bash_command='bash -i /home/lnxuser/airflow/files/test_ns/env.sh'
    )
    test

这是气流的错误日志：

[2021-02-04 19:21:08,127] {taskinstance.py:1396} ERROR - bash -i /home/lnxuser/airflow/files/pam_ns/env.sh
Traceback (most recent call last):
  File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1086, in _run_raw_task
    self._prepare_and_execute_task_with_callbacks(context, task)
  File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1224, in _prepare_and_execute_task_with_callbacks
    self.render_templates(context=context)
  File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py", line 1684, in render_templates
    self.task.render_template_fields(context)
  File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 857, in render_template_fields
    self._do_render_template_fields(self, self.template_fields, context, jinja_env, set())
  File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 870, in _do_render_template_fields
    rendered_content = self.render_template(content, context, jinja_env, seen_oids)
  File "/home/lnxuser/.local/lib/python3.8/site-packages/airflow/models/baseoperator.py", line 905, in render_template
    return jinja_env.get_template(content).render(**context)
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 830, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/usr/lib/python3/dist-packages/jinja2/environment.py", line 804, in _load_template
    template = self.loader.load(self, name, globals)
  File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 113, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "/usr/lib/python3/dist-packages/jinja2/loaders.py", line 187, in get_source
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: bash -i /home/lnxuser/airflow/files/pam_ns/env.sh
[2021-02-04 19:21:08,169] {taskinstance.py:1433} INFO - Marking task as FAILED. dag_id=Phase1_dag_v1, task_id=PAM, execution_date=20210204T192105, start_date=20210204T192107, end_date=20210204T192108
[2021-02-04 19:21:08,382] {local_task_job.py:118} INFO - Task exited with return code 1

我有点新手，所以不太确定出了什么问题，我可能以完全错误的方式使用气流。

有人知道这里发生了什么吗？

【问题讨论】：

标签： bash airflow

【解决方案1】：

您需要在脚本名称后添加空格，因为您没有使用模板并且脚本不在 dag 文件的相对路径中。

bash_command='bash -i /home/lnxuser/airflow/files/test_ns/env.sh '

需要空间时的解释可以在docs中找到

【讨论】：