Skip to content

Commit

Permalink
Merge pull request #54 from nf-core/dev
Browse files Browse the repository at this point in the history
PR for first release
  • Loading branch information
kbestak authored Feb 12, 2024
2 parents e4ee2c7 + 14841ae commit 7605a53
Show file tree
Hide file tree
Showing 112 changed files with 4,469 additions and 1,372 deletions.
9 changes: 5 additions & 4 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"name": "nfcore",
"image": "nfcore/gitpod:latest",
"remoteUser": "gitpod",
"runArgs": ["--privileged"],

// Configure tool-specific properties.
"customizations": {
Expand All @@ -17,11 +18,11 @@
"python.linting.flake8Path": "/opt/conda/bin/flake8",
"python.linting.pycodestylePath": "/opt/conda/bin/pycodestyle",
"python.linting.pydocstylePath": "/opt/conda/bin/pydocstyle",
"python.linting.pylintPath": "/opt/conda/bin/pylint"
"python.linting.pylintPath": "/opt/conda/bin/pylint",
},

// Add the IDs of extensions you want installed when the container is created.
"extensions": ["ms-python.python", "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack"]
}
}
"extensions": ["ms-python.python", "ms-python.vscode-pylance", "nf-core.nf-core-extensionpack"],
},
},
}
8 changes: 6 additions & 2 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ Please use the pre-filled template to save time.
However, don't be put off by this template - other more general issues and suggestions are welcome!
Contributions to the code are even more welcome ;)

> If you need help using or modifying nf-core/molkart then the best place to ask is on the nf-core Slack [#molkart](https://nfcore.slack.com/channels/molkart) channel ([join our Slack here](https://nf-co.re/join/slack)).
:::info
If you need help using or modifying nf-core/molkart then the best place to ask is on the nf-core Slack [#molkart](https://nfcore.slack.com/channels/molkart) channel ([join our Slack here](https://nf-co.re/join/slack)).
:::

## Contribution workflow

Expand All @@ -25,6 +27,9 @@ If you're not used to this workflow with git, you can start with some [docs from

## Tests

You can optionally test your changes by running the pipeline locally. Then it is recommended to use the `debug` profile to
receive warnings about process selectors and other debug info. Example: `nextflow run . -profile debug,test,docker --outdir <OUTDIR>`.

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.

Expand Down Expand Up @@ -116,4 +121,3 @@ To get started:
Devcontainer specs:

- [DevContainer config](.devcontainer/devcontainer.json)
- [Dockerfile](.devcontainer/Dockerfile)
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ body:
attributes:
label: System information
description: |
* Nextflow version _(eg. 22.10.1)_
* Nextflow version _(eg. 23.04.0)_
* Hardware _(eg. HPC, Desktop, Cloud)_
* Executor _(eg. slurm, local, awsbatch)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
Expand Down
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/molk
- [ ] If necessary, also make a PR on the nf-core/molkart _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
Expand Down
14 changes: 8 additions & 6 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,21 +14,23 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Launch workflow via tower
uses: seqeralabs/action-tower-launch@v1
# TODO nf-core: You can customise AWS full pipeline tests as required
# Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters
uses: seqeralabs/action-tower-launch@v2
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
revision: ${{ github.sha }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/molkart/work-${{ github.sha }}
parameters: |
{
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/molkart/results-${{ github.sha }}"
}
profiles: test_full,aws_tower
profiles: test_full

- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
path: |
tower_action_*.log
tower_action_*.json
10 changes: 7 additions & 3 deletions .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,18 +12,22 @@ jobs:
steps:
# Launch workflow using Tower CLI tool action
- name: Launch workflow via tower
uses: seqeralabs/action-tower-launch@v1
uses: seqeralabs/action-tower-launch@v2
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
revision: ${{ github.sha }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/molkart/work-${{ github.sha }}
parameters: |
{
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/molkart/results-test-${{ github.sha }}"
}
profiles: test,aws_tower
profiles: test

- uses: actions/upload-artifact@v3
with:
name: Tower debug log file
path: tower_action_*.log
path: |
tower_action_*.log
tower_action_*.json
87 changes: 76 additions & 11 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,43 +1,108 @@
name: nf-core CI
# This workflow runs the pipeline with the minimal test dataset to check that it completes without any syntax errors
name: nf-core CI
on:
push:
branches:
- dev
- "dev"
pull_request:
branches:
- "dev"
- "master"
release:
types: [published]

env:
NXF_ANSI_LOG: false
NFTEST_VER: "0.8.2"

concurrency:
group: "${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}"
cancel-in-progress: true

jobs:
list:
name: List Tests
if: "${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/molkart') }}"
outputs:
# Expose matched filters as job 'modules' output variable
tests: ${{ steps.list.outputs.tests }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Setup Nextflow
uses: nf-core/setup-nextflow@v1

- name: Install nf-test
if: steps.cache-software.outputs.cache-hit != 'true'
run: |
wget -qO- https://code.askimed.com/install/nf-test | bash
sudo mv nf-test /usr/local/bin/
- name: List nf-test
id: list
run: |
echo "tests=$(nf-test list --silent --format=json)" >> "$GITHUB_OUTPUT"
test:
name: Run pipeline with test data
name: ${{ matrix.tags }} (${{ matrix.profile }}-${{ matrix.NXF_VER }})
# Only run on push if this is the nf-core dev branch (merged PRs)
if: "${{ github.event_name != 'push' || (github.event_name == 'push' && github.repository == 'nf-core/molkart') }}"
if: needs.list.outputs.tests != '[]'
needs: [list]
runs-on: ubuntu-latest
strategy:
matrix:
NXF_VER:
- "22.10.1"
- "23.04.0"
- "latest-everything"
profile: ["docker"] #TODO add "singularity"
test: ["${{ fromJson(needs.list.outputs.tests) }}"]

steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Cache Nextflow installation
id: cache-software
uses: actions/cache@v3
with:
path: |
/usr/local/bin/nf-test
/home/runner/.nf-test/nf-test.jar
key: molkart-${{ runner.os }}-${{ matrix.NXF_VER }}

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1
with:
version: "${{ matrix.NXF_VER }}"

- name: Run pipeline with test data
# TODO nf-core: You can customise CI pipeline run tests as required
# For example: adding multiple test runs with different parameters
# Remember that you can parallelise this by using strategy.matrix
- name: Install nf-test
if: steps.cache-software.outputs.cache-hit != 'true'
run: |
wget -qO- https://code.askimed.com/install/nf-test | bash
sudo mv nf-test /usr/local/bin/
- name: Set up Singularity
if: matrix.profile == 'singularity'
uses: eWaterCycle/setup-singularity@v5
with:
singularity-version: 3.7.1

- name: Run nf-test
run: |
nf-test test \
--profile="test,${{ matrix.profile }}" \
${{ matrix.test }} \
--junitxml=${{ matrix.test }}-${{ matrix.profile }}.xml
- name: Publish Test Report
uses: mikepenz/action-junit-report@v3
if: success() || failure() # run this step even if previous step failed
with:
path: ${{ matrix.test }}-${{ matrix.profile }}.xml

- name: Output log on failure
if: failure()
run: |
nextflow run ${GITHUB_WORKSPACE} -profile test,docker --outdir ./results
sudo apt install bat > /dev/null
batcat --decorations=always --color=always ${{ github.workspace }}/.nf-test/tests/*/output/pipeline_info/software_versions.yml
4 changes: 2 additions & 2 deletions .github/workflows/fix-linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-latest
steps:
# Use the @nf-core-bot token to check out so we can push later
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
token: ${{ secrets.nf_core_bot_auth_token }}

Expand All @@ -24,7 +24,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.nf_core_bot_auth_token }}

- uses: actions/setup-node@v3
- uses: actions/setup-node@v4

- name: Install Prettier
run: npm install -g prettier @prettier/plugin-php
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ jobs:
EditorConfig:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- uses: actions/setup-node@v3
- uses: actions/setup-node@v4

- name: Install editorconfig-checker
run: npm install -g editorconfig-checker
Expand All @@ -27,9 +27,9 @@ jobs:
Prettier:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- uses: actions/setup-node@v3
- uses: actions/setup-node@v4

- name: Install Prettier
run: npm install -g prettier
Expand All @@ -40,7 +40,7 @@ jobs:
PythonBlack:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Check code lints with Black
uses: psf/black@stable
Expand Down Expand Up @@ -71,14 +71,14 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out pipeline code
uses: actions/checkout@v3
uses: actions/checkout@v4

- name: Install Nextflow
uses: nf-core/setup-nextflow@v1

- uses: actions/setup-python@v4
with:
python-version: "3.8"
python-version: "3.11"
architecture: "x64"

- name: Install dependencies
Expand Down
68 changes: 68 additions & 0 deletions .github/workflows/release-announcements.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: release-announcements
# Automatic release toot and tweet anouncements
on:
release:
types: [published]
workflow_dispatch:

jobs:
toot:
runs-on: ubuntu-latest
steps:
- uses: rzr/fediverse-action@master
with:
access-token: ${{ secrets.MASTODON_ACCESS_TOKEN }}
host: "mstdn.science" # custom host if not "mastodon.social" (default)
# GitHub event payload
# https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads#release
message: |
Pipeline release! ${{ github.repository }} v${{ github.event.release.tag_name }} - ${{ github.event.release.name }}!
Please see the changelog: ${{ github.event.release.html_url }}
send-tweet:
runs-on: ubuntu-latest

steps:
- uses: actions/setup-python@v4
with:
python-version: "3.10"
- name: Install dependencies
run: pip install tweepy==4.14.0
- name: Send tweet
shell: python
run: |
import os
import tweepy
client = tweepy.Client(
access_token=os.getenv("TWITTER_ACCESS_TOKEN"),
access_token_secret=os.getenv("TWITTER_ACCESS_TOKEN_SECRET"),
consumer_key=os.getenv("TWITTER_CONSUMER_KEY"),
consumer_secret=os.getenv("TWITTER_CONSUMER_SECRET"),
)
tweet = os.getenv("TWEET")
client.create_tweet(text=tweet)
env:
TWEET: |
Pipeline release! ${{ github.repository }} v${{ github.event.release.tag_name }} - ${{ github.event.release.name }}!
Please see the changelog: ${{ github.event.release.html_url }}
TWITTER_CONSUMER_KEY: ${{ secrets.TWITTER_CONSUMER_KEY }}
TWITTER_CONSUMER_SECRET: ${{ secrets.TWITTER_CONSUMER_SECRET }}
TWITTER_ACCESS_TOKEN: ${{ secrets.TWITTER_ACCESS_TOKEN }}
TWITTER_ACCESS_TOKEN_SECRET: ${{ secrets.TWITTER_ACCESS_TOKEN_SECRET }}

bsky-post:
runs-on: ubuntu-latest
steps:
- uses: zentered/[email protected]
with:
post: |
Pipeline release! ${{ github.repository }} v${{ github.event.release.tag_name }} - ${{ github.event.release.name }}!
Please see the changelog: ${{ github.event.release.html_url }}
env:
BSKY_IDENTIFIER: ${{ secrets.BSKY_IDENTIFIER }}
BSKY_PASSWORD: ${{ secrets.BSKY_PASSWORD }}
#
9 changes: 8 additions & 1 deletion .gitpod.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
image: nfcore/gitpod:latest

tasks:
- name: Update Nextflow and setup pre-commit
command: |
pre-commit install --install-hooks
nextflow self-update
- name: unset JAVA_TOOL_OPTIONS
command: |
unset JAVA_TOOL_OPTIONS
vscode:
extensions: # based on nf-core.nf-core-extensionpack
- codezombiech.gitignore # Language support for .gitignore files
Expand Down
Loading

0 comments on commit 7605a53

Please sign in to comment.