Skip to content

Commit

Permalink
Release/v3.0.0 (#31)
Browse files Browse the repository at this point in the history
  • Loading branch information
fmind authored Dec 14, 2024
1 parent 6baa645 commit f16d240
Show file tree
Hide file tree
Showing 64 changed files with 8,287 additions and 8,109 deletions.
12 changes: 7 additions & 5 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,11 @@ description: Setup for project workflows
runs:
using: composite
steps:
- run: pipx install invoke poetry
shell: bash
- uses: actions/setup-python@v5
- name: Install uv
uses: astral-sh/setup-uv@v4
with:
python-version: 3.12
cache: poetry
enable-cache: true
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version-file: .python-version
10 changes: 7 additions & 3 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Check
on:
pull_request:
branches:
- main
- '*'
concurrency:
cancel-in-progress: true
group: ${{ github.workflow }}-${{ github.ref }}
Expand All @@ -12,5 +12,9 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- run: poetry install --with checks
- run: poetry run invoke checks
- run: uv sync --group=checks
- run: uv run invoke checks.format
- run: uv run invoke checks.type
- run: uv run invoke checks.code
- run: uv run invoke checks.security
- run: uv run invoke checks.coverage
8 changes: 4 additions & 4 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- run: poetry install --with docs
- run: poetry run invoke docs
- run: uv sync --group=docs
- run: uv run invoke docs
- uses: JamesIves/github-pages-deploy-action@v4
with:
folder: docs/
Expand All @@ -28,8 +28,8 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/setup
- run: poetry install --with dev
- run: poetry run invoke packages
- run: uv sync --only-dev
- run: uv run invoke packages
- uses: docker/login-action@v3
with:
registry: ghcr.io
Expand Down
12 changes: 4 additions & 8 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ default_language_version:
python: python3.12
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
rev: v5.0.0
hooks:
- id: check-added-large-files
- id: check-case-conflict
Expand All @@ -16,18 +16,14 @@ repos:
- id: end-of-file-fixer
- id: mixed-line-ending
- id: trailing-whitespace
- repo: https://github.com/python-poetry/poetry
rev: 1.8.3
hooks:
- id: poetry-check
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.5.0
rev: v0.8.1
hooks:
- id: ruff
- id: ruff-format
- repo: https://github.com/commitizen-tools/commitizen
rev: v3.27.0
rev: v4.0.0
hooks:
- id: commitizen
- id: commitizen-branch
stages: [push]
stages: [pre-push]
20 changes: 20 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,23 @@
## v3.0.0 (2024-12-14)

### Feat

- **mlflow**: bump to 2.19.0
- **manager**: switch from poetry to uv (#22)
- **manager**: switch from poetry to uv

### Fix

- **tasks**: merge conflict
- **tasks**: fix mlflow.serve task attribute
- **github-actions**: run check on all PR

### Refactor

- **release**: prepare before release
- **actions**: split check tasks into subtask for easier debugging (#29)
- **code**: improve abstractions (#28)

## v2.0.0 (2024-07-28)

### Feat
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# https://docs.docker.com/engine/reference/builder/

FROM python:3.12
FROM ghcr.io/astral-sh/uv:python3.12-bookworm
COPY dist/*.whl .
RUN pip install *.whl
RUN uv pip install --system *.whl
CMD ["bikes", "--help"]
2 changes: 0 additions & 2 deletions LICENCE.txt → LICENSE.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
Copyright 2024 Médéric HURIER (Fmind)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
Expand Down
51 changes: 26 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ The package leverages several [tools](#tools) and [tips](#tips) to make your MLO
You can use this package as part of your MLOps toolkit or platform (e.g., Model Registry, Experiment Tracking, Realtime Inference, ...).

**Related Resources**:
- **[LLMOps Coding Package (Example)](https://github.com/callmesora/llmops-python-package/)**: Example with best practices and tools to support your LLMOps projects.
- **[MLOps Coding Course (Learning)](https://github.com/MLOps-Courses/mlops-coding-course)**: Learn how to create, develop, and maintain a state-of-the-art MLOps code base.
- **[Cookiecutter MLOps Package (Template)](https://github.com/fmind/cookiecutter-mlops-package)**: Start building and deploying Python packages and Docker images for MLOps tasks.

Expand Down Expand Up @@ -69,7 +70,7 @@ You can use this package as part of your MLOps toolkit or platform (e.g., Model
- [Package](#package)
- [Evolution: Changelog](#evolution-changelog)
- [Format: Wheel](#format-wheel)
- [Manager: Poetry](#manager-poetry)
- [Manager: uv](#manager-uv)
- [Runtime: Docker](#runtime-docker)
- [Programming](#programming)
- [Language: Python](#language-python)
Expand Down Expand Up @@ -119,7 +120,7 @@ This section details the requirements, actions, and next steps to kickstart your
## Prerequisites

- [Python>=3.12](https://www.python.org/downloads/): to benefit from [the latest features and performance improvements](https://docs.python.org/3/whatsnew/3.12.html)
- [Poetry>=1.8.2](https://python-poetry.org/): to initialize the project [virtual environment](https://docs.python.org/3/library/venv.html) and its dependencies
- [uv>=0.5.5](https://docs.astral.sh/uv/): to initialize the project [virtual environment](https://docs.python.org/3/library/venv.html) and its dependencies

## Installation

Expand All @@ -130,10 +131,10 @@ $ git clone [email protected]:fmind/mlops-python-package
# with https
$ git clone https://github.com/fmind/mlops-python-package
```
2. [Run the project installation with poetry](https://python-poetry.org/docs/)
2. [Run the project installation with uv](https://docs.astral.sh/uv/)
```bash
$ cd mlops-python-package/
$ poetry install
$ uv sync
```
3. Adapt the code base to your desire

Expand Down Expand Up @@ -171,26 +172,26 @@ This config file instructs the program to start a `TrainingJob` with 2 parameter

You can find all the parameters of your program in the `src/[package]/jobs/*.py` files.

You can also print the full schema supported by this package using `poetry run bikes --schema`.
You can also print the full schema supported by this package using `uv run bikes --schema`.

## Execution

The project code can be executed with poetry during your development:
The project code can be executed with uv during your development:

```bash
$ poetry run [package] confs/tuning.yaml
$ poetry run [package] confs/training.yaml
$ poetry run [package] confs/promotion.yaml
$ poetry run [package] confs/inference.yaml
$ poetry run [package] confs/evaluations.yaml
$ poetry run [package] confs/explanations.yaml
$ uv run [package] confs/tuning.yaml
$ uv run [package] confs/training.yaml
$ uv run [package] confs/promotion.yaml
$ uv run [package] confs/inference.yaml
$ uv run [package] confs/evaluations.yaml
$ uv run [package] confs/explanations.yaml
```

In production, you can build, ship, and run the project as a Python package:

```bash
poetry build
poetry publish # optional
uv build
uv publish # optional
python -m pip install [package]
[package] confs/inference.yaml
```
Expand All @@ -211,7 +212,7 @@ with job as runner:
- You can pass several config files in the command-line to merge them from left to right
- You can define common configurations shared between jobs (e.g., model params)
- The right job task will be selected automatically thanks to [Pydantic Discriminated Unions](https://docs.pydantic.dev/latest/concepts/unions/#discriminated-unions)
- This is a great way to run any job supported by the application (training, tuning, ....
- This is a great way to run any job supported by the application (training, tuning, ...)

## Automation

Expand All @@ -233,7 +234,6 @@ $ inv --list
- **checks.code** - Check the codes with ruff.
- **checks.coverage** - Check the coverage with coverage.
- **checks.format** - Check the formats with ruff.
- **checks.poetry** - Check poetry config files.
- **checks.security** - Check the security with bandit.
- **checks.test** - Check the tests with pytest.
- **checks.type** - Check the types with mypy.
Expand All @@ -247,15 +247,15 @@ $ inv --list
- **cleans.mlruns** - Clean the mlruns folder.
- **cleans.mypy** - Clean the mypy tool.
- **cleans.outputs** - Clean the outputs folder.
- **cleans.poetry** - Clean poetry lock file.
- **cleans.pytest** - Clean the pytest tool.
- **cleans.projects** - Run all projects tasks.
- **cleans.pytest** - Clean the pytest tool.
- **cleans.python** - Clean python caches and bytecodes.
- **cleans.requirements** - Clean the project requirements file.
- **cleans.reset** - Run all tools, folders, and sources tasks.
- **cleans.ruff** - Clean the ruff tool.
- **cleans.sources** - Run all sources tasks.
- **cleans.tools** - Run all tools tasks.
- **cleans.uv** - Clean uv lock file.
- **cleans.venv** - Clean the venv folder.
- **commits.all (commits)** - Run all commit tasks.
- **commits.bump** - Bump the version of the package.
Expand All @@ -272,8 +272,8 @@ $ inv --list
- **formats.imports** - Format python imports with ruff.
- **formats.sources** - Format python sources with ruff.
- **installs.all (installs)** - Run all install tasks.
- **installs.poetry** - Install poetry packages.
- **installs.pre-commit** - Install pre-commit hooks on git.
- **installs.uv** - Install uv packages.
- **mlflow.all (mlflow)** - Run all mlflow tasks.
- **mlflow.doctor** - Run mlflow doctor to diagnose issues.
- **mlflow.serve** - Start mlflow server with the given host, port, and backend uri.
Expand Down Expand Up @@ -684,17 +684,18 @@ Define and build modern Python package.
- [Source](https://docs.python.org/3/distutils/sourcedist.html): older format, less powerful
- [Conda](https://conda.io/projects/conda/en/latest/user-guide/install/index.html): slow and hard to manage

### Manager: [Poetry](https://python-poetry.org/)
### Manager: [uv](https://docs.astral.sh/uv/)

- **Motivations**:
- Define and build Python package
- Most popular solution by GitHub stars
- Fast and compliant package manager
- Pack every metadata in a single static file
- **Limitations**:
- Cannot add dependencies beyond Python (e.g., CUDA)
- i.e., use Docker container for this use case
- **Alternatives**:
- [Setuptools](https://docs.python.org/3/distutils/setupscript.html): dynamic file is slower and more risky
- [Poetry](https://python-poetry.org/): previous solution of this package
- Pdm, Hatch, PipEnv: https://xkcd.com/1987/

### Runtime: [Docker](https://www.docker.com/resources/what-container/)
Expand Down Expand Up @@ -937,10 +938,10 @@ Using Python package for your AI/ML project has the following benefits:
- Install Python package as a library (e.g., like pandas)
- Expose script entry points to run a CLI or a GUI

To build a Python package with Poetry, you simply have to type in a terminal:
To build a Python package with uv, you simply have to type in a terminal:
```bash
# for all poetry project
poetry build
# for all uv project
uv build
# for this project only
inv packages
```
Expand Down Expand Up @@ -1044,7 +1045,7 @@ Semantic Versioning (SemVer) provides a simple schema to communicate code change
- *Minor* (Y): minor release with new features (i.e., provide new capabilities)
- *Patch* (Z): patch release to fix bugs (i.e., correct wrong behavior)

Poetry and this package leverage Semantic Versioning to let developers control the speed of adoption for new releases.
Uv and this package leverage Semantic Versioning to let developers control the speed of adoption for new releases.

## [Testing Tricks](https://en.wikipedia.org/wiki/Software_testing)

Expand Down
Binary file modified data/inputs_test.parquet
Binary file not shown.
Binary file modified data/inputs_train.parquet
Binary file not shown.
Binary file modified data/targets_test.parquet
Binary file not shown.
Binary file modified data/targets_train.parquet
Binary file not shown.
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

services:
mlflow:
image: ghcr.io/mlflow/mlflow:v2.14.3
image: ghcr.io/mlflow/mlflow:v2.19.0
ports:
- 5000:5000
environment:
Expand Down
Loading

0 comments on commit f16d240

Please sign in to comment.