Taskfile + Monorepos

[antdking]

I'll be able to share some concrete examples of what I'm doing at a later point.

Firstly, am I insane for using Taskfile in this way?

Secondly, I think these features could be massively beneficial when looking at using Taskfile for more complex situations. Thoughts?

I'm using Taskfile to manage a shared interface between projects in a multi-language monorepo (JS + Python for now).

There are 3 features that will go a long way for our usecase

Context Managers

Reduce boilerplate around pre-task operations + traps; as well as a safe way to have dynamic environment variables.

Currently, we can easily define a pre-test task through use of dependencies, however we have to call the cleanup task from within the main task using defer.
Context Managers would be great here for keeping boilerplate out your tasks.

Proposed way of defining + using context managers

contexts:
  cypress-env:
    desc: Provision a Test environment on the server ready for e2e tests
    open:
      - ./setup-cypress.py --open --state "${STATE_FILE}"
    close:
      - ./setup-cypress.py --teardown --state "${STATE_FILE}"
    env:
      STATE_FILE:
        sh: echo "$(mktemp  --suffix=.json)"
tasks:
  e2e:
    ctx: [cypress-env]
    cmds:
      - ./cypress.sh

  manual-entry:
    cmds:
      - ctx.open: cypress-env
      - ./cypress.sh
      - ctx.close: cypress-env

In the manual example, failing to have ctx.close should report an error when you go to parse the tasks (before anything has executed).

For reusable context managers, referencing by ID should be supported:

manual-entry:
  - ctx.open: cypress-env
    id: first-open
  - ctx.open: cypress-env
    id: second-open
  #- ctx.close: cypress-env  # Error, ID 'cypress-env' not found
  #- ctx.close: first-open  # Error, Can't close, 'second-open' is still open
  - ctx.close: second-open
  - ctx.close: first-open

Possible equivalent for how to do it currently.

# equivilant now based on intent, it likely won't work as intended due to env/var scope complications.
tasks:
  e2e:
    cmds:
      - task: cypress-env-open
        vars: { STATE_FILE: "${CYPRESS_STEP_FILE}" }
      - ./cypress.sh
      - task: cypress-env-close
        vars: { STATE_FILE: "${CYPRESS_STEP_FILE}" **}**
    env:
      CYPRESS_STEP_FILE:
        sh: echo "$(mktemp  --suffix=.json)"
  
  cypress-env-open:
    cmds:
      - ./setup-cypress.py --open --state '{{.STATE_FILE}}'
    vars:
      STATE_FILE:
  cypress-env-close:
    cmds:
      - ./setup-cypress.py --teardown --state '{{.STATE_FILE}}'
    vars:
      STATE_FILE:

Artifacts

For concrete dependencies without knowledge of who will be responsible.

Tasks can produce "things", in the same way that Make or Bazel can.
It'd be good to be able to depend on the outputs of a task that can be passed around.

For further flexibility, it'd be good to declare that an artifact will exist, and have a task plug it further down (Dependency Injection).

The reasoning here is one project shouldn't care about anything within another project, only that it gets the input it wants.

Brief example to outline what I'm getting at

## ./projects/client/Taskfile.yml
tasks:
  build:
    deps:
      - artifact: local-report.css
    cmds:
      - cp '{{.artifacts.local-report.css}}' src/resources/local-report/local-report.css

## ./projects/report-builder/Taskfile.yml
global:
  artifacts:
    local-report.css:
      - from: tasks.build.artifacts.local-report.css
tasks:
  build:
    artifacts:
      local-report.css: ./dist/local-report/local-report.css

The same principals could be applied for task injection in a more flexible way than the current "use Vars in Includes".

Wildcard includes

Allow discoving Taskfiles, and including them by name.
This will allow the root taskfile to be less rigid; as well as provide a nice way to do task/artifact injection.

include:
  *:
    - ./projects/*

equiv-include:
  client: ./projects/client

---
# ./project/client/Taskfile.yml

name: client
tasks:
  ...:

The above becomes problematic when trying to decide the dir scope of the include.

What are your thoughts here?
How well would the above fit into v4? Aware there's a lot of effort going into how Environment + Variable scopes function.

[andreynering]

Hi @antdking,

I'm sorry for taking so long to respond. We just happen to be busy and the number of issues if higher than ever.

1. I don't see anything like the proposed Context Manager being added, honestly.
2. For Artifacts, the idea is interesting, but I believe that you can already kind of have something like this by declaring these globs on a global variable, and then concatenating it to the sources and generates of the tasks you want. If you still want your proposal to be considered in the future, please open a dedicated issue for it.
3. Wildcard includes is an interesting idea, but I feel that having a list of includes as today is probably fine enough. As in the last topic, if you feel strong about it, open a dedicated issue, otherwise your proposal will be forgotten here.