Skip to content

Commit

Permalink
Deprecate blog (#9)
Browse files Browse the repository at this point in the history
* Update submodules and point to the latest commits

* Remove refs to aeon_blog

* Move dev practices to new file

* Update admonitions
  • Loading branch information
lochhh authored Dec 7, 2023
1 parent 8a16681 commit e029440
Show file tree
Hide file tree
Showing 19 changed files with 514 additions and 184 deletions.
1 change: 0 additions & 1 deletion .github/workflows/docs_build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ jobs:
git submodule sync
git submodule update --init aeon_acquisition
git submodule update --init aeon_analysis
git submodule update --init aeon_blog
git submodule update --init aeon_mecha
- name: Setup Python
Expand Down
2 changes: 1 addition & 1 deletion aeon_experiments
2 changes: 1 addition & 1 deletion aeon_mecha
62 changes: 5 additions & 57 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ This repo contains the source for the currently WIP version of Project Aeon's on

The docs are built via Sphinx, and hosted via GitHub Pages at: sainsburywellcomecentre.github.io/aeon_docs/. `src/` is the Sphinx source directory, and the site is built and deployed from the `gh-pages` branch. This is handled by a GitHub actions workflow (`.github/workflows/docs_build_and_deploy.yml`). The build job is triggered on each PR, ensuring that the documentation build is not broken by new changes. The deployment job is only triggered whenever a tag is pushed to the main branch.

All organizational details will eventually be available on the docs, but for now some useful info can be found in the sections below:

## Building the documentation locally

Expand Down Expand Up @@ -57,7 +56,8 @@ To be granted these credentials, please send a single email to all contact parti

## Repositories

You must be an SWC Github 'aeon' project member to view some of these repositories.
> [!IMPORTANT]
> You must be an SWC Github 'aeon' project member to view some of these repositories.
### [aeon_mecha](https://github.com/SainsburyWellcomeCentre/aeon_mecha)

Expand All @@ -66,7 +66,8 @@ You must be an SWC Github 'aeon' project member to view some of these repositori

Project Aeon's main library for interfacing with acquired data. Contains Python modules for raw data file io, data querying, data processing, data qc, database ingestion, and building computational data pipelines. This is the main user repository.

*Note*: All experiment data is acquired and/or triggered and/or synced by [Harp devices](https://www.cf-hw.org/harp). Code in the 'aeon_acquisition' and 'aeon_mecha' repos makes use of the [Harp protocol](https://github.com/harp-tech/protocol) during data acquisition, raw data file writing, and raw data file reading. In the 'harp-tech/protocol' Github repo, you can find documentation on [Harp device operation and common registers](https://github.com/harp-tech/protocol/blob/master/Device%201.0%201.4%2020200901.pdf), the [Harp binary protocol](https://github.com/harp-tech/protocol/blob/master/Binary%20Protocol%201.0%201.1%2020180223.pdf), and [Harp clock synchronization](https://github.com/harp-tech/protocol/blob/master/Synchronization%20Clock%201.0%201.0%2020200712.pdf).
> [!NOTE]
> All experiment data is acquired and/or triggered and/or synced by [Harp devices](https://www.cf-hw.org/harp). Code in the 'aeon_acquisition' and 'aeon_mecha' repos makes use of the [Harp protocol](https://github.com/harp-tech/protocol) during data acquisition, raw data file writing, and raw data file reading. In the 'harp-tech/protocol' Github repo, you can find documentation on [Harp device operation and common registers](https://github.com/harp-tech/protocol/blob/master/Device%201.0%201.4%2020200901.pdf), the [Harp binary protocol](https://github.com/harp-tech/protocol/blob/master/Binary%20Protocol%201.0%201.1%2020180223.pdf), and [Harp clock synchronization](https://github.com/harp-tech/protocol/blob/master/Synchronization%20Clock%201.0%201.0%2020200712.pdf).
### [aeon_experiments](https://github.com/SainsburyWellcomeCentre/aeon_experiments)

Expand All @@ -92,59 +93,6 @@ Contains low-level source code for pellet delivery via feeders used in Aeon expe

Contains source code for the Aeon docs site, built via Sphinx. Built docs at: https://sainsburywellcomecentre.github.io/aeon_docs/

## Dev Practices

### Software Development Life Cycle (SDLC)

Our SDLC roughly follows the [iterative model](https://www.tutorialspoint.com/sdlc/sdlc_iterative_model.htm).

### Versioning

We version all the following, according to [SemVer](http://semver.org/) numbering:

- Experiments, by name, including full hardware specs (arena, I/O devices,
acquisition computer, etc.)
- Repositories, including code related to:
- Bonsai experiment workflows
- Quality Control protocols (for raw and preprocessed data)
- Data processing algorithms
- aeon-db Database
- aeon-db tables

### Issue Tracking

We prioritize and track dev progress using Github Discussions and Github Issues in Github Projects. Issues and Discussions should ideally be created in the specific repository appropriate for the Issue/Discussion; all experiment and general Issues and Discussions should be created in 'aeon_experiments'.

### Continuous Integration (CI)

We use Github Actions to run CI. We run unit tests on Github Virtual Machines on Windows, MacOS, and Ubuntu. We run integration tests on the SWC HPC. Workflows of the CI jobs we run can be found in each repo's respective `.github/workflows/` directory.

### Contributing

Each repository roughly follows the [github flow](https://guides.github.com/introduction/flow/) (which is adapted from the more general
[gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)).

In brief, each of our repos has 'main' and 'prod' branches. Feature and bug fix branches are branched off of 'main', with Pull Requests sent back into 'main'. 'main' contains the full commit history of the project, up to the latest stable commit. Upon merges into 'main', a squash merge is performed into 'prod', such that 'prod' contains an abbreviated commit history of the project, with commits pertaining only to Pull Request merges. 'prod' thus serves as a "production" branch to allow for easier readability of project history and easier reversion of uncaught bug commits. At certain agreed upon timepoints we create "stable releases" (available in the "releases" section of the repository) which serve as a snapshot of the code at the time, version numbered according to [SemVer](http://semver.org/).

When contributing to any repository, the change to be made should first be discussed in a Github Discussion or a Github Issue. Thereafter, contributors should create a new branch (branched off of 'main') that contains the changes/additions they wish to make, and then create a pull request for merging this branch into 'main'.

All pull requests will be reviewed by the [project maintainers](#project-maintainers). Minimally, maintainers should follow the below steps when reviewing pull requests:

1) Ensure new code adheres to the [style and documentation guidelines](#style-and-documentation-guidelines), is covered by a test, and passes a build test. These can all be checked via CI.

2) As necessary, ensure `changelog`, `readme`, config and doc files are updated.

3) When a branch is ready to be merged back into 'main', always make sure to first pull 'main' locally, then rebase the feature branch onto 'main' (cleaning up any merge conflicts as necessary), before merging the PR. The squash merge into 'prod' can be handled via CI. E.g., see [here](https://github.com/SainsburyWellcomeCentre/aeon_mecha/tree/main/.github/workflows/squash_merge_to_prod.yml)

### Style and Documentation Guidelines

Please see our [style and documentation guidelines](https://github.com/ProjectAeon/blog/blob/main/style_and_doc_guidelines.md).

We also believe in the [readme manifesto](http://thinkinghard.com/blog/TheREADMEManifesto.html), which says that `readme` files should provide at least a general description that covers _all_ of a project's files, and that one `readme` per subdirectory is generally good practice.

### Code of Conduct

Please see our [code of conduct](https://github.com/ProjectAeon/blog/blob/main/code_of_conduct.md).

## Project Maintainers

Expand All @@ -160,6 +108,6 @@ If you use this software, please cite it as below:

Sainsbury Wellcome Centre Foraging Behaviour Working Group. (2023). Aeon: An open-source platform to study the neural basis of ethological behaviours over naturalistic timescales, https://doi.org/10.5281/zenodo.8413142

[![DOI](https://zenodo.org/badge/485512362.svg)](https://zenodo.org/badge/latestdoi/485512362)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.8413142.svg)](https://zenodo.org/doi/10.5281/zenodo.8411157)


2 changes: 1 addition & 1 deletion src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
os.path.abspath("../aeon_analysis/aeon_analysis"),
os.path.abspath("../aeon_acquisition"),
os.path.abspath("../aeon_experiments"),
os.path.abspath("../aeon_blog"),
]
)

Expand Down Expand Up @@ -57,6 +56,7 @@
myst_enable_extensions = [
"colon_fence",
"linkify",
"deflist",
]

# Automatically add anchors to markdown headings
Expand Down
111 changes: 111 additions & 0 deletions src/data_contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
(target-data-schema)=
# Data Schema

**Version**: 0.2.0-draft

## Devices

### VideoController

* harp data: `<name>_address.bin [HARP]`
| name | address | payload | attributes | description |
| ---------- | ------- | --------- | ---------- | ----------- |
| `pwm_enable` | `39` | `uint16` | bitmask | specifies which PWM has been enabled |
| `pwm1_freq` | `50` | `float32` | frequency | the frequency of PWM1 (max: 1000Hz) |
| `pwm1_dutycycle` | `51` | `float32` | dutycycle | the duty cycle of PWM1 (max: 100%) |
| `pwm1_mode` | `55` | `uint8` | mode | should always be 0 (infinite) |
| `pwm1_trig` | `56` | `uint8` | start_trigger | should always be 0 (software trigger) |
| `pwm1_conf_event` | `57` | `uint8` | rise_event | should always be 1 (enabled) |
| `pwm2_freq` | `58` | `float32` | frequency | the frequency of PWM1 (max: 1000Hz) |
| `pwm2_dutycycle` | `59` | `float32` | dutycycle | the duty cycle of PWM1 (max: 100%) |
| `pwm2_mode` | `63` | `uint8` | mode | should always be 0 (infinite) |
| `pwm2_trig` | `64` | `uint8` | start_trigger | should always be 0 (software trigger) |
| `pwm2_conf_event` | `65` | `uint8` | rise_event | should always be 1 (enabled) |
| `pwm_start` | `66` | `uint8` | bitmask | specifies which PWM has started |
| `pwm_stop` | `67` | `uint8` | bitmask | specifies which PWM has stopped |
| `pwm_rise_event` | `68` | `uint8` | bitmask | specifies which PWM has been triggered |


### VideoSource

* video frames: `<name>.avi [FMP4 codec]`
* video metadata: `<name>.csv [CSV]`
| time | hw_counter | hw_timestamp |
| --------- | ---------- | ------------ |
| `float64` | `int64` | `int64` |

* (optional) harp data: `<name>_address.bin [HARP]`
| name | address | payload | attributes | description |
| ---------- | ------- | --------- | ---------- | ----------- |
| `position` | `200` | `float32` | [x, y, angle, major, minor, area, id] | one entry for each animal, for each frame |
| `region` | `201` | `uint8` | area code | one entry for each animal, for each frame |

* area code map:
| name | code |
| -------- | ---- |
| none | 0 |
| nest | 1 |
| corridor | 2 |
| arena | 3 |
| patch1 | 4 |
| patch2 | 5 |


### AudioSource

* audio buffers: `<name>.wav [PCM 16-bit mono]`
* audio metadata: `MISSING`

### PatchController

* patch state: `<name>_State.csv [CSV]`
| time | threshold | d1 | delta |
| --------- | ---------- | ----------| --------- |
| `float64` | `float64` | `float64` | `float64` |

* harp data: `<name>_address.bin [HARP]`
| name | address | payload | attributes | description |
| ---------- | ------- | --------- | ---------- | ----------- |
| `beam_break` | `32` | `uint8` | bitmask | the current state of the beam break |
| `delivery_set` | `35` | `uint8` | bitmask | a pellet delivery has been requested |
| `delivery_clear` | `36` | `uint8` | bitmask | the pellet delivery pin has been cleared |
| `expansion_board` | `87` | `uint8` | expansion | the type of board in the expansion port |
| `encoder_read` | `90` | `uint16` | [angle, intensity] | the current state of the magnetic encoder |
| `encoder_mode` | `91` | `uint8` | mode | should always be 4 (500Hz) |
| `dispenser_state` | `200` | `float32` | value | the estimated number of pellets in the dispenser |
| `delivery_manual` | `201` | `uint8` | event | a manual pellet delivery has been requested |
| `missed_pellet` | `202` | `uint8` | event | a pellet has been missed |
| `delivery_retry` | `203` | `uint8` | bitmask | a pellet delivery command has been retried |

### WeightScale

* harp data: `<name>_address.bin [HARP]`
| name | address | payload | attributes | description |
| ---------- | ------- | --------- | ---------- | ----------- |
| `weight_raw` | `200` | `float32` | [value, stable] | the raw weight samples (g) |
| `weight_tare` | `201` | `uint8` | event | a tare command has been sent |
| `weight_filtered` | `202` | `float32` | [value, stable] | filtered weight samples (`stable` is slope of regression) |
| `weight_baseline` | `203` | `uint8` | event | a baseline has been taken |
| `weight_subject` | `204` | `float32` | [value, stable] | the stable baselined weight of a subject in the nest (g) |

### ExperimentalMetadata

* environment state: `<name>_EnvironmentState.csv [CSV]`
| time | type |
| --------- | -------------------------- |
| `float64` | `Maintenance \ Experiment` |

* subject state: `<name>_SubjectState.csv [CSV]`
| time | id | weight | event |
| --------- | ---------- | ----------| ----- |
| `float64` | `str` | `float64` | `str` |

* subject visits: `<name>_SubjectVisits.csv [CSV]`
| time | id | event | area |
| --------- | ---------- | ----- | ----- |
| `float64` | `str` | `Enter \ Exit` | `str` |

* message log: `<name>_MessageLog.csv [CSV]`
| time | priority | type | message |
| --------- | ---------- | ----------| --------- |
| `float64` | `Notification \ Alert` | `str` | `TSV str` |
6 changes: 0 additions & 6 deletions src/data_contract.rst

This file was deleted.

Loading

0 comments on commit e029440

Please sign in to comment.