Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear all CI checks on all platforms #402

Merged
merged 38 commits into from
Sep 20, 2024
Merged

Clear all CI checks on all platforms #402

merged 38 commits into from
Sep 20, 2024

Conversation

glopesdev
Copy link
Contributor

@glopesdev glopesdev commented Sep 6, 2024

We have failing CI checks for a long time now purely because of ruff linting and pyright warnings. This PR clears all these warnings, either by addressing the suggestions, or ignoring the warnings where reasonable, e.g. docstrings on magic methods, etc.

The current PR is taking us from ~250 errors to zero errors. The final errors were grouped into the below 4 types and have all been resolved:

  • aeon\io\reader.py:338 and 341: PLR2004 Magic value used in comparison, consider replacing 0.3 with a constant variable
  • aeon\dj_pipeline\utils\streams_maker.py:129:9: PLW2901 for loop variable col overwritten by assignment target
  • aeon\dj_pipeline\populate\worker.py:49:5: F841 Local variable experiment_names is assigned to but never used
  • multiple C408 Unnecessary dict call (rewrite as a literal) in aeon\dj_pipeline\analysis\block_analysis.py

To reproduce the required checks:
run pre-commit install and pre-commit run --all-files
or see Workflow run

In the process we have also now enabled resolving all checks for all platforms, including resolving an outstanding issue for datajoint macOS wheels (see #406)

Fixes #411
Fixes #412
Fixes #344
Fixes #249

@coderabbitai: ignore

Summary by CodeRabbit

  • Refactor: Improved code readability and maintainability across multiple modules by addressing linting warnings, updating docstrings, and refactoring some parts of the code.
  • Style: Made minor formatting adjustments for better consistency and readability in various files.
  • Test: Simplified test assertions and improved test functions related to pipeline instantiation and experiment creation.
  • Chore: Updated CI workflow and pre-commit hooks configurations for better code quality checks.
  • Documentation: Enhanced function docstrings for clarity and consistency across the project.

@glopesdev glopesdev added the help wanted Extra attention is needed label Sep 6, 2024
Copy link

github-actions bot commented Sep 6, 2024

Image description CodeRabbit

Walkthrough

The proposed changes primarily focus on improving code readability, maintainability, and consistency. This is achieved through addressing linting warnings, updating docstrings, refactoring some parts of the code, and making minor adjustments in variable assignments and dictionary usage. Some changes also involve function signatures and import statements that may impact module resolution and external interfaces. The CI workflow and pre-commit hooks configurations have been updated as well.

Changes

Files Summary
aeon/analysis/block_plotting.py, aeon/analysis/movies.py, aeon/analysis/plotting.py, aeon/dj_pipeline/tracking.py, aeon/io/device.py, aeon/analysis/utils.py, aeon/dj_pipeline/init.py, aeon/dj_pipeline/acquisition.py, aeon/dj_pipeline/utils/load_metadata.py, aeon/io/api.py, aeon/io/reader.py, aeon/dj_pipeline/scripts/clone_and_freeze_exp01.py, aeon/dj_pipeline/scripts/update_timestamps_longblob.py, aeon/util.py, aeon/dj_pipeline/create_experiments/insert_experiment_directory.ipynb, aeon/dj_pipeline/docs/notebooks/diagram.ipynb, aeon/dj_pipeline/docs/notebooks/social_experiments_block_analysis.ipynb, docs/examples/dj_example_octagon1_experiment.ipynb, aeon/dj_pipeline/populate/worker.py, tests/conftest.py Improved code readability and maintainability by addressing linting warnings, updating docstrings, refactoring some parts of the code, and making minor adjustments in variable assignments and dictionary usage.
aeon/dj_pipeline/qc.py, aeon/io/video.py, aeon/qc/video.py, aeon/dj_pipeline/scripts/clone_and_freeze_exp02.py, aeon/schema/core.py, aeon/schema/dataset.py, aeon/schema/foraging.py, aeon/schema/schemas.py, aeon/schema/social_02.py, aeon/schema/streams.py Changes in function signatures and import statements that may impact module resolution and external interfaces.
tests/dj_pipeline/test_acquisition.py, tests/dj_pipeline/test_pipeline_instantiation.py, tests/dj_pipeline/test_tracking.py, tests/io/test_api.py, aeon/init.py, aeon/dj_pipeline/utils/load_metadata.py, tests/dj_pipeline/test_tracking.py, tests/io/test_api.py Modifications in test functions to improve code readability and consistency.
.github/workflows/build_env_run_tests.yml, .pre-commit-config.yaml Updated the CI workflow and pre-commit hooks configurations.
aeon/dj_pipeline/webapps/sciviz/apk_requirements.txt Minor change related to adding a newline at the end of the file in the git bash script.
aeon/dj_pipeline/analysis/block_analysis.py Optimized code by addressing linting warnings, improving variable assignments, and enhancing dictionary usage.
aeon/dj_pipeline/populate/worker.py Removed a function and added a new one with minor adjustments in the code.
aeon/dj_pipeline/utils/streams_maker.py Updated variable name in a loop within a function.
aeon/io/reader.py Improved code readability and maintainability by addressing linting warnings, updating docstrings, using constant variables, and refactoring some parts of the code.
tests/dj_pipeline/test_pipeline_instantiation.py Changes in test functions related to pipeline instantiation and experiment creation.
tests/dj_pipeline/test_qc.py Removed extraneous whitespace in a test function.
tests/io/test_api.py Modified test assertions related to data indexing and timestamps, introducing a constant variable for downsampling period.

Uplevel your code reviews with CodeRabbit Pro

CodeRabbit Pro

If you like this project, please support us by purchasing the Pro version. The Pro version has advanced context, superior noise reduction and several proprietary improvements compared to the open source version. Moreover, CodeRabbit Pro is free for open source projects.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image description CodeRabbit

Commits Files that changed from the base of the PR and between 0b52944 and 0927a9a commits.
Files selected (35)
  • aeon/analysis/block_plotting.py (2)
  • aeon/analysis/movies.py (4)
  • aeon/analysis/plotting.py (4)
  • aeon/analysis/utils.py (3)
  • aeon/dj_pipeline/init.py (1)
  • aeon/dj_pipeline/acquisition.py (1)
  • aeon/dj_pipeline/create_experiments/insert_experiment_directory.ipynb (2)
  • aeon/dj_pipeline/docs/notebooks/diagram.ipynb (3)
  • aeon/dj_pipeline/docs/notebooks/social_experiments_block_analysis.ipynb (2)
  • aeon/dj_pipeline/populate/worker.py (2)
  • aeon/dj_pipeline/qc.py (1)
  • aeon/dj_pipeline/scripts/clone_and_freeze_exp01.py (1)
  • aeon/dj_pipeline/scripts/clone_and_freeze_exp02.py (2)
  • aeon/dj_pipeline/scripts/update_timestamps_longblob.py (1)
  • aeon/dj_pipeline/tracking.py (1)
  • aeon/dj_pipeline/utils/load_metadata.py (3)
  • aeon/io/api.py (2)
  • aeon/io/device.py (3)
  • aeon/io/reader.py (3)
  • aeon/io/video.py (1)
  • aeon/qc/video.py (1)
  • aeon/schema/core.py (1)
  • aeon/schema/dataset.py (1)
  • aeon/schema/foraging.py (1)
  • aeon/schema/schemas.py (1)
  • aeon/schema/social_02.py (3)
  • aeon/schema/streams.py (2)
  • aeon/util.py (1)
  • docs/examples/dj_example_octagon1_experiment.ipynb (8)
  • docs/examples/understanding_aeon_data_architecture.ipynb (20)
  • tests/conftest.py (3)
  • tests/dj_pipeline/test_acquisition.py (1)
  • tests/dj_pipeline/test_pipeline_instantiation.py (1)
  • tests/dj_pipeline/test_tracking.py (2)
  • tests/io/test_api.py (1)
Files ignored due to filter (1)
  • pyproject.toml
Files not summarized due to errors (1)
  • docs/examples/understanding_aeon_data_architecture.ipynb (diff tokens exceeds limit)
Files skipped from review due to trivial changes (19)
  • aeon/analysis/block_plotting.py
  • aeon/analysis/utils.py
  • aeon/dj_pipeline/init.py
  • aeon/dj_pipeline/acquisition.py
  • aeon/dj_pipeline/create_experiments/insert_experiment_directory.ipynb
  • aeon/dj_pipeline/docs/notebooks/diagram.ipynb
  • aeon/dj_pipeline/docs/notebooks/social_experiments_block_analysis.ipynb
  • aeon/dj_pipeline/scripts/update_timestamps_longblob.py
  • aeon/dj_pipeline/utils/load_metadata.py
  • aeon/io/api.py
  • aeon/io/reader.py
  • aeon/qc/video.py
  • aeon/schema/core.py
  • aeon/schema/social_02.py
  • docs/examples/dj_example_octagon1_experiment.ipynb
  • tests/conftest.py
  • tests/dj_pipeline/test_acquisition.py
  • tests/dj_pipeline/test_tracking.py
  • tests/io/test_api.py
Review comments generated (40)
  • Review: 17
  • LGTM: 23

Tips

Chat with Image description CodeRabbit Bot (@coderabbitai)

  • Reply on review comments left by this bot to ask follow-up questions. A review comment is a comment on a diff or a file.
  • Invite the bot into a review comment chain by tagging @coderabbitai in a reply.

Code suggestions

  • The bot may make code suggestions, but please review them carefully before committing since the line number ranges may be misaligned.
  • You can edit the comment made by the bot and manually tweak the suggestion if it is slightly off.

Pausing incremental reviews

  • Add @coderabbitai: ignore anywhere in the PR description to pause further reviews from the bot.

aeon/dj_pipeline/tracking.py Show resolved Hide resolved
aeon/dj_pipeline/populate/worker.py Show resolved Hide resolved
aeon/dj_pipeline/qc.py Show resolved Hide resolved
aeon/io/video.py Show resolved Hide resolved
aeon/schema/schemas.py Show resolved Hide resolved
aeon/analysis/movies.py Show resolved Hide resolved
aeon/util.py Show resolved Hide resolved
aeon/schema/streams.py Show resolved Hide resolved
aeon/schema/streams.py Show resolved Hide resolved
@ttngu207
Copy link
Contributor

@glopesdev Thanks
I'll go back and apply the fixes suggested here to clear as much of the Ruff rules as possible

@lochhh
Copy link
Contributor

lochhh commented Sep 12, 2024

I opened a separate PR #409 that adds on to the current PR and reduces the ruff errors down to 9 - could use some input from @ttngu207 as most of those are in dj_pipeline. The remaining 16 pyright errors can be addressed in a separate PR.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image description CodeRabbit

Commits Files that changed from the base of the PR and between 9c9a88b and df20e9f commits.
Files selected (7)
  • aeon/dj_pipeline/analysis/block_analysis.py (7)
  • aeon/dj_pipeline/populate/worker.py (2)
  • aeon/dj_pipeline/utils/streams_maker.py (1)
  • aeon/io/reader.py (10)
  • tests/dj_pipeline/test_pipeline_instantiation.py (1)
  • tests/dj_pipeline/test_qc.py (1)
  • tests/io/test_api.py (2)
Files skipped from review due to trivial changes (2)
  • aeon/dj_pipeline/utils/streams_maker.py
  • tests/dj_pipeline/test_pipeline_instantiation.py
Review comments generated (22)
  • Review: 11
  • LGTM: 11

Tips

Chat with Image description CodeRabbit Bot (@coderabbitai)

  • Reply on review comments left by this bot to ask follow-up questions. A review comment is a comment on a diff or a file.
  • Invite the bot into a review comment chain by tagging @coderabbitai in a reply.

Code suggestions

  • The bot may make code suggestions, but please review them carefully before committing since the line number ranges may be misaligned.
  • You can edit the comment made by the bot and manually tweak the suggestion if it is slightly off.

Pausing incremental reviews

  • Add @coderabbitai: ignore anywhere in the PR description to pause further reviews from the bot.

aeon/dj_pipeline/populate/worker.py Show resolved Hide resolved
aeon/io/reader.py Show resolved Hide resolved
aeon/io/reader.py Show resolved Hide resolved
aeon/io/reader.py Show resolved Hide resolved
aeon/io/reader.py Show resolved Hide resolved
aeon/dj_pipeline/analysis/block_analysis.py Show resolved Hide resolved
aeon/dj_pipeline/analysis/block_analysis.py Show resolved Hide resolved
aeon/dj_pipeline/analysis/block_analysis.py Show resolved Hide resolved
aeon/dj_pipeline/analysis/block_analysis.py Show resolved Hide resolved
aeon/dj_pipeline/analysis/block_analysis.py Show resolved Hide resolved
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image description CodeRabbit

Commits Files that changed from the base of the PR and between df20e9f and c904feb commits.
Files selected (1)
  • .github/workflows/build_env_run_tests.yml (1)
Review comments generated (3)
  • Review: 2
  • LGTM: 1

Tips

Chat with Image description CodeRabbit Bot (@coderabbitai)

  • Reply on review comments left by this bot to ask follow-up questions. A review comment is a comment on a diff or a file.
  • Invite the bot into a review comment chain by tagging @coderabbitai in a reply.

Code suggestions

  • The bot may make code suggestions, but please review them carefully before committing since the line number ranges may be misaligned.
  • You can edit the comment made by the bot and manually tweak the suggestion if it is slightly off.

Pausing incremental reviews

  • Add @coderabbitai: ignore anywhere in the PR description to pause further reviews from the bot.

.github/workflows/build_env_run_tests.yml Outdated Show resolved Hide resolved
.github/workflows/build_env_run_tests.yml Outdated Show resolved Hide resolved
@glopesdev glopesdev changed the title Clear all ruff linting rules Clear all ruff linting and pyright rules Sep 19, 2024
Copy link

codecov bot commented Sep 19, 2024

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

Copy link
Contributor

@lochhh lochhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@glopesdev glopesdev changed the title Clear all ruff linting and pyright rules Clear all CI checks on all platforms Sep 20, 2024
@glopesdev glopesdev added feature New planned feature fix Pull request that fixes an issue and removed help wanted Extra attention is needed labels Sep 20, 2024
* Persist venv across job steps
* Update codecov-action version
* Remove `env_config`
@glopesdev glopesdev merged commit a889dba into main Sep 20, 2024
6 checks passed
@glopesdev glopesdev deleted the gl-ruff-check branch September 20, 2024 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New planned feature fix Pull request that fixes an issue
Projects
None yet
4 participants