Merge pull request #11 from lochhh/fix-external-urls

Check and update external URLs

.github/workflows/docs_build_and_deploy.yml

@@ -9,7 +9,6 @@ on:
   push:
     branches:
       - main
-      - new-theme # temporarily add branch to test new theme deployment
     tags:
       - '*'
   pull_request:
@@ -61,10 +60,9 @@ jobs:
         shell: bash
         run: python3 -m pip install -r ./requirements.txt
 
-      # temporarily disabled until we fix broken links
-      # - name: Check links
-      #  shell: bash
-      #  run: make linkcheck
+      - name: Check external links
+        shell: bash
+        run: make linkcheck
 
       # needs to have sphinx.ext.githubpages in conf.py extensions list
       - name: Building documentation

readme.md

@@ -2,7 +2,7 @@
 
 This repo contains the source for the currently WIP version of Project Aeon's online docs. 
 
-The docs are built via Sphinx, and hosted via GitHub Pages at: sainsburywellcomecentre.github.io/aeon_docs/. `src/` is the Sphinx source directory, and the site is built and deployed from the `gh-pages` branch. This is handled by a GitHub actions workflow (`.github/workflows/docs_build_and_deploy.yml`). The build job is triggered on each PR, ensuring that the documentation build is not broken by new changes. The deployment job is only triggered whenever a tag is pushed to the main branch.
+The docs are built via [Sphinx](https://www.sphinx-doc.org/en/master/), and hosted via GitHub Pages at [sainsburywellcomecentre.github.io/aeon_docs/](https://sainsburywellcomecentre.github.io/aeon_docs/). `src/` is the Sphinx source directory, and the site is built and deployed from the `gh-pages` branch. This is handled by a GitHub actions workflow (`.github/workflows/docs_build_and_deploy.yml`). The build job is triggered on each PR, ensuring that the documentation build is not broken by new changes. The deployment job is only triggered whenever a tag is pushed to the main branch.
 
 
 ## Building the documentation locally
@@ -28,13 +28,48 @@ Finally, build the documentation:
 ```bash
 make html
 ```
-You can view the local build by opening ``docs/html/index.html`` in a browser.
+You can view the local build by opening `docs/html/index.html` in a browser.
 
 To apply new changes to the documentation, remove all automatically generated files and folders, and rebuild:
 ```bash
 make clean html
 ```
 
+To check that external URLs are correctly resolved, run:
+```bash
+make linkcheck
+```
+
+If the linkcheck step incorrectly marks URLs with valid anchors as broken, 
+you can skip checking the anchors in specific URLs by adding them to 
+`linkcheck_anchors_ignore_for_url` in `src/conf.py`, e.g.: 
+```python
+# linkcheck will skip verifying that anchors exist when checking
+# these URLs
+linkcheck_anchors_ignore_for_url = [
+    "https://example.com",
+]
+```
+
+To skip linkcheck for specific URLs, add them to
+`linkcheck_ignore` in `src/conf.py`, e.g.:
+```python
+# linkcheck will skip checking these URLs entirely
+linkcheck_ignore = [
+    "https://github.com/org/private_repository",
+]
+```
+
+To suppress warnings for expected redirects, add them to 
+`linkcheck_allowed_redirects` in `src/conf.py`, e.g.:
+```python
+# linkcheck will treat redirections from these source URI:canonical URI
+# mappings as "working".
+linkcheck_allowed_redirects = {
+    r"https://zenodo\.org/doi/.*": r"https://zenodo\.org/records/.*",
+}
+```
+
 ## Project Aeon Organization Overview
 
 ProjectAeon is a collaborative effort to perform behavioral neuroscience experiments where the behavior and neural activity of freely moving animals engaging in a complex task are continuously recorded. This project is contributed to by researchers and support staff at UCL's SWC, Neurogears, and Datajoint.
@@ -67,7 +102,7 @@ To be granted these credentials, please send a single email to all contact parti
 Project Aeon's main library for interfacing with acquired data. Contains Python modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines. This is the main user repository.
 
 > [!NOTE]
-> All experiment data is acquired and/or triggered and/or synced by [Harp devices](https://www.cf-hw.org/harp). Code in the 'aeon_acquisition' and 'aeon_mecha' repos makes use of the [Harp protocol](https://github.com/harp-tech/protocol) during data acquisition, raw data file writing, and raw data file reading. In the 'harp-tech/protocol' Github repo, you can find documentation on [Harp device operation and common registers](https://github.com/harp-tech/protocol/blob/master/Device%201.0%201.4%2020200901.pdf), the [Harp binary protocol](https://github.com/harp-tech/protocol/blob/master/Binary%20Protocol%201.0%201.1%2020180223.pdf), and [Harp clock synchronization](https://github.com/harp-tech/protocol/blob/master/Synchronization%20Clock%201.0%201.0%2020200712.pdf).
+> All experiment data is acquired and/or triggered and/or synced by [Harp devices](https://www.cf-hw.org/harp). Code in the 'aeon_acquisition' and 'aeon_mecha' repos makes use of the [Harp protocol](https://harp-tech.org/articles/about.html) during data acquisition, raw data file writing, and raw data file reading. See also the documentation on [Harp device operation and common registers](https://harp-tech.org/protocol/Device.html), the [Harp binary protocol](https://harp-tech.org/protocol/BinaryProtocol-8bit.html), and [Harp clock synchronization](https://harp-tech.org/protocol/SynchronizationClock.html).
 
 ### [aeon_experiments](https://github.com/SainsburyWellcomeCentre/aeon_experiments)
 

src/conf.py

@@ -28,8 +28,8 @@
 # -- Project information -----------------------------------------------------
 
 project = "aeon_docs"
-copyright = f"2022–{date.today().year}, Jai Bhagat, Goncalo Lopes, Chang Huan Lo"
 author = "Jai Bhagat, Goncalo Lopes, Chang Huan Lo"
+copyright = f"2022–{date.today().year}, {author}"
 organisation = "Sainsbury Wellcome Centre"
 
 
@@ -48,7 +48,6 @@ def get_current_release_tag():
 
 # The full version, including alpha/beta/rc tags
 release = get_current_release_tag()
-# release = "0.1.0"
 
 # GitHub repo URL
 github_url = f"https://github.com/{organisation.replace(' ', '')}/{project}"
@@ -125,3 +124,26 @@ def get_current_release_tag():
 html_css_files = [
     "https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.1.1/css/all.min.css"
 ]
+
+# linkcheck will skip checking these URLs entirely
+linkcheck_ignore = [
+    "https://github.com/SainsburyWellcomeCentre/aeon_experiments",  # This is currently a private repository
+]
+
+# linkcheck will treat redirections from these source URI:canonical URI
+# mappings as "working".
+linkcheck_allowed_redirects = {
+    r"https://doi\.org/10\.5281/zenodo\..*": r"https://zenodo\.org/records/.*",
+    r"https://zenodo\.org/doi/.*": r"https://zenodo\.org/records/.*",
+}
+
+myst_url_schemes = {
+    "http": None,
+    "https": None,
+    "ftp": None,
+    "mailto": None,
+    "aeon-mecha-github": "https://github.com/SainsburyWellcomeCentre/aeon_mecha/{{path}}",
+    "semver": "https://semver.org/",
+    "harp-tech": "https://harp-tech.org/{{path}}#{{fragment}}",
+    "python-pep": "https://peps.python.org/pep-{{path}}",
+}

src/dev_practices/code_of_conduct.md

@@ -43,4 +43,4 @@ Project maintainers who do not follow or enforce the Code of Conduct in good fai
 
 ## Attribution
 
-This Code of Conduct is adapted from the Contributor Covenant, version 1.4, available at http://contributor-covenant.org/version/1/4.
+This Code of Conduct is adapted from the [Contributor Covenant, version 1.4](https://www.contributor-covenant.org/version/1/4/).

src/dev_practices/index.md

@@ -6,7 +6,7 @@ Our SDLC roughly follows the [iterative model](https://www.tutorialspoint.com/sd
 
 ## Versioning
 
-We version all the following, according to [SemVer](http://semver.org/) numbering: 
+We version all the following, according to [SemVer](semver:) numbering: 
 
 - Experiments, by name, including full hardware specs (arena, I/O devices, 
   acquisition computer, etc.)
@@ -27,10 +27,10 @@ We use Github Actions to run CI. We run unit tests on Github Virtual Machines on
 
 ## Contributing
 
-Each repository roughly follows the [github flow](https://guides.github.com/introduction/flow/) (which is adapted from the more general 
+Each repository roughly follows the [github flow](https://docs.github.com/en/get-started/quickstart/github-flow) (which is adapted from the more general 
 [gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)). 
 
-In brief, each of our repos has 'main' and 'prod' branches. Feature and bug fix branches are branched off of 'main', with Pull Requests sent back into 'main'. 'main' contains the full commit history of the project, up to the latest stable commit. Upon merges into 'main', a squash merge is performed into 'prod', such that 'prod' contains an abbreviated commit history of the project, with commits pertaining only to Pull Request merges. 'prod' thus serves as a "production" branch to allow for easier readability of project history and easier reversion of uncaught bug commits. At certain agreed upon timepoints we create "stable releases" (available in the "releases" section of the repository) which serve as a snapshot of the code at the time, version numbered according to [SemVer](http://semver.org/).
+In brief, each of our repos has 'main' and 'prod' branches. Feature and bug fix branches are branched off of 'main', with Pull Requests sent back into 'main'. 'main' contains the full commit history of the project, up to the latest stable commit. Upon merges into 'main', a squash merge is performed into 'prod', such that 'prod' contains an abbreviated commit history of the project, with commits pertaining only to Pull Request merges. 'prod' thus serves as a "production" branch to allow for easier readability of project history and easier reversion of uncaught bug commits. At certain agreed upon timepoints we create "stable releases" (available in the "releases" section of the repository) which serve as a snapshot of the code at the time, version numbered according to [SemVer](semver:).
 
 When contributing to any repository, the change to be made should first be discussed in a Github Discussion or a Github Issue. Thereafter, contributors should create a new branch (branched off of 'main') that contains the changes/additions they wish to make, and then create a pull request for merging this branch into 'main'.
 
@@ -40,7 +40,7 @@ All pull requests will be reviewed by the [project maintainers](target-project-m
 
 2) As necessary, ensure `changelog`, `readme`, config and doc files are updated.
 
-3) When a branch is ready to be merged back into 'main', always make sure to first pull 'main' locally, then rebase the feature branch onto 'main' (cleaning up any merge conflicts as necessary), before merging the PR. The squash merge into 'prod' can be handled via CI. E.g., see [here](https://github.com/SainsburyWellcomeCentre/aeon_mecha/tree/main/.github/workflows/squash_merge_to_prod.yml)
+3) When a branch is ready to be merged back into 'main', always make sure to first pull 'main' locally, then rebase the feature branch onto 'main' (cleaning up any merge conflicts as necessary), before merging the PR. The squash merge into 'prod' can be handled via CI. E.g., see [here](aeon-mecha-github:blob/main/.github/workflows/squash_merge_to_prod.yml)
 
 ## Style and Documentation Guidelines
 

src/dev_practices/style_and_doc_guidelines.md

@@ -3,7 +3,7 @@
 
 The purpose of this document is to establish Project Aeon code style and documentation guidelines.
 
-We generally adhere to [pep8](https://www.python.org/dev/peps/pep-0008/), [pep257](https://www.python.org/dev/peps/pep-0257/), and [google's style guide](https://google.github.io/styleguide/pyguide.html) for code and docstring style guidelines. We run black, ruff, and pyright to format our Python code, the config settings for which can be found in a repository's `pyproject.toml`.
+We generally adhere to [pep8](python-pep:0008), [pep257](python-pep:0257), and [google's style guide](https://google.github.io/styleguide/pyguide.html) for code and docstring style guidelines. We run black, ruff, and pyright to format our Python code, the config settings for which can be found in a repository's `pyproject.toml`.
 
 ## General Guidelines
 

src/repositories/index.md

@@ -46,5 +46,5 @@ Source code for pellet delivery via feeders used in Aeon experiments
 ::::
 
 :::{note}
-All experiment data is acquired and/or triggered and/or synced by [Harp devices](https://www.cf-hw.org/harp). Code in the 'aeon_acquisition' and 'aeon_mecha' repos makes use of the [Harp protocol](https://github.com/harp-tech/protocol) during data acquisition, raw data file writing, and raw data file reading. In the 'harp-tech/protocol' Github repo, you can find documentation on [Harp device operation and common registers](https://github.com/harp-tech/protocol/blob/master/Device%201.0%201.4%2020200901.pdf), the [Harp binary protocol](https://github.com/harp-tech/protocol/blob/master/Binary%20Protocol%201.0%201.1%2020180223.pdf), and [Harp clock synchronization](https://github.com/harp-tech/protocol/blob/master/Synchronization%20Clock%201.0%201.0%2020200712.pdf).
+All experiment data is acquired and/or triggered and/or synced by [Harp devices](https://www.cf-hw.org/harp). Code in the 'aeon_acquisition' and 'aeon_mecha' repos makes use of the [Harp protocol](harp-tech:articles/about) during data acquisition, raw data file writing, and raw data file reading. See also the documentation on [Harp device operation and common registers](harp-tech:protocol/Device), the [Harp binary protocol](harp-tech:protocol/BinaryProtocol-8bit), and [Harp clock synchronization](harp-tech:protocol/SynchronizationClock).
 :::