Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add operator for resolving the epoch data path #134

Merged
merged 2 commits into from
Aug 17, 2023
Merged

Add operator for resolving the epoch data path #134

merged 2 commits into from
Aug 17, 2023

Conversation

glopesdev
Copy link
Contributor

@glopesdev glopesdev commented Aug 16, 2023

This PR introduces a new operator for resolving the epoch data path used by RepositoryLogController. The expected behavior is reproduced in the following flowchart.

flowchart LR
    A{Is Repo\nClean?} --> |no|B
    B(Detached\nHead?) --> |no|C
    B --> |yes|D
    C[fallback/branch-sha1]
    D[fallback/detached-sha1]
    A --> |yes|E
    E{Is Commit\nTagged?} --> |no|B
    E --> |yes|F
    F[data/tagname]
Loading

The tag name is assumed to end with a full semver string from which all characters after the last dot are removed. Changes to the tag version assume that what we are versioning is the data schema of the experiment:

  • Changing the format of an existing stream or data file > change the major version
  • Adding a new stream to the existing schema > change the minor version (note: schemas targeting older minor versions can still open this data)
  • Removing a stream from an experiment:
    • one perspective > dataset is broken
    • other perspective > data is just empty for that stream
    • finally we could make this into a schema setting (i.e. certain streams are always required)
  • Changing anything that doesn't affect the schema or data formats > change the patch version

Fixes SainsburyWellcomeCentre/aeon_experiments#291

@glopesdev glopesdev added the feature New planned feature label Aug 16, 2023
@glopesdev glopesdev merged commit 0a5d34c into main Aug 17, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New planned feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prefer generating dataset folder name from tag version name
1 participant