14 Commits

Author SHA1 Message Date
dependabot[bot]
4f8165fe31 build(deps): bump the python-dependencies group with 4 updates
Bumps the python-dependencies group with 4 updates: [requests](https://github.com/psf/requests), [pandas](https://github.com/pandas-dev/pandas), [pytest](https://github.com/pytest-dev/pytest) and [mkdocs-material](https://github.com/squidfunk/mkdocs-material).


Updates `requests` from 2.32.5 to 2.33.0
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.5...v2.33.0)

Updates `pandas` from 2.3.3 to 3.0.1
- [Release notes](https://github.com/pandas-dev/pandas/releases)
- [Commits](https://github.com/pandas-dev/pandas/compare/v2.3.3...v3.0.1)

Updates `pytest` from 8.4.2 to 9.0.2
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/8.4.2...9.0.2)

Updates `mkdocs-material` from 9.6.23 to 9.7.6
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.6.23...9.7.6)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.33.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
- dependency-name: pandas
  dependency-version: 3.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: python-dependencies
- dependency-name: pytest
  dependency-version: 9.0.2
  dependency-type: direct:development
  update-type: version-update:semver-major
  dependency-group: python-dependencies
- dependency-name: mkdocs-material
  dependency-version: 9.7.6
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: python-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-27 21:12:54 +00:00
Loïc GUEZO
416cfcbf8b Add Python package ecosystem to Dependabot config
Configure Dependabot for Python package updates.
2026-03-27 22:11:53 +01:00
32c5310e37 fix: mettre à jour les tests pytest 2026-03-27 22:06:36 +01:00
9dfc7457a0 fix(scraper.py): retirer commentaire code et print 2026-03-27 22:06:06 +01:00
f5d5703e49 fix(scraper): recherche _getproduitslist actualisé
Suite à une refont de l'UI et du backend, la structure de données JSON envoyé par la page web a été simplifié.

Ancienne structure:

- `"props"->"pageProps"->"initialReduxState"->"categ"->"content->"produits"`

Nouvelle structure:

- `"props"->"pageProps"->"produits"`
2026-03-27 21:47:06 +01:00
888defb6b6 retravail: changement documentation et suppression version anglaise 2026-03-09 19:10:06 +01:00
734e3898e9 ajout: commencement écriture README.md 2026-03-09 14:35:57 +01:00
4bb3112dd0 ajout: creation de fichier out.csv main et ajout README 2026-03-09 14:16:05 +01:00
54e4b7860b ajout: 3e jalon pdf 2026-03-06 22:54:10 +01:00
b865a59aba ajout: cleaning.md et modification scraper.md 2026-03-06 21:56:51 +01:00
Loïc GUEZO
fde1f36148 Enhance workflow with Python setup and docs build
Added Python setup and documentation build steps to workflow.
2026-03-06 21:41:34 +01:00
6fbb36ea37 Merge branch 'jalon2-loic' of https://github.com/guezoloic/millesima_projetS6 2026-03-06 21:36:48 +01:00
Loïc GUEZO
bcacd7a915 Merge pull request #11 from guezoloic/jalon2-loic
Jalon2 loic
2026-03-06 21:09:52 +01:00
Loïc GUEZO
d182e08f9b Merge pull request #10 from guezoloic/jalon2-loic
Jalon2 loic
2026-03-04 12:52:47 +01:00
13 changed files with 201 additions and 34 deletions

18
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,18 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "pip"
directory: "/"
schedule:
interval: "weekly"
day: "saturday"
open-pull-requests-limit: 5
groups:
python-dependencies:
patterns:
- "*"

58
.github/workflows/static.yml vendored Normal file
View File

@@ -0,0 +1,58 @@
# Simple workflow for deploying static content to GitHub Pages
name: Deploy static content to Pages
on:
# Runs on pushes targeting the default branch
push:
branches: ["main"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: '3.10'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
# Installe le projet en mode éditable avec les extras de doc
pip install -e ".[doc]"
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Build Documentation
run: mkdocs build
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
# Upload entire repository
path: './site'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

View File

@@ -1 +1,37 @@
# millesima_projetS6
# Millesima AI Engine 🍷
> A **University of Paris-Est Créteil (UPEC)** Semester 6 project.
## Documentation
- 🇫🇷 [Version Française](https://guezoloic.github.io/millesima-ai-engine)
> note: only french version enabled for now.
---
## Installation
> Make sure you have **Python 3.10+** installed.
1. **Clone the repository:**
```bash
git clone https://github.com/votre-pseudo/millesima-ai-engine.git
cd millesima-ai-engine
```
2. **Set up a virtual environment:**
```bash
python3 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
```
3. **Install dependencies:**
```bash
pip install -e .
```
## Usage
### 1. Data Extraction (Scraping)
To fetch the latest wine data from Millesima:
```bash
python3 src/scraper.py
```
> Note: that will take some time to fetch all data depending on the catalog size.

17
docs/cleaning.md Normal file
View File

@@ -0,0 +1,17 @@
# Cleaning
## Sommaire
[TOC]
---
## Classe `Cleaning`
::: src.cleaning.Cleaning
options:
heading_level: 3
members:
- __init__
- getVins
- drop_empty_appellation
- fill_missing_scores
- encode_appellation

View File

@@ -1 +1,16 @@
# Millesima
# Millesima
Lobjectif de ce projet est détudier, en utilisant des méthodes dapprentissage automatique, limpact de différents critères (notes des critiques, appelation) sur le prix dun vin. Pour ce faire, on sappuiera sur le site Millesima (https://www.millesima.fr/), qui a lavantage de ne pas posséder de protection contre les bots. Par respect pour lhébergeur du site, on veillera à limiter au maximum le nombre de requêtes. En particulier, on sassurera davoir un code fonctionnel avant de scraper lintégralité du site, pour éviter les répétitions.
## projet
<div style="text-align: center;">
<object
data="/millesima-ai-engine/projet.pdf"
type="application/pdf"
width="100%"
height="1000px"
>
<p>Votre navigateur ne peut pas afficher ce PDF.
<a href="/millesima-ai-engine/projet.pdf">Cliquez ici pour le télécharger.</a></p>
</object>
</div>

Binary file not shown.

View File

@@ -1,3 +1,31 @@
# Scraper
::: scraper.Scraper
## Sommaire
[TOC]
---
## Classe `Scraper`
::: scraper.Scraper
options:
members:
- __init__
- getvins
- getjsondata
- getresponse
- getsoup
heading_level: 4
## Classe `_ScraperData`
::: scraper._ScraperData
options:
members:
- __init__
- getdata
- appellation
- parker
- robinson
- suckling
- prix
- informations
heading_level: 4

View File

@@ -1,4 +0,0 @@
# _ScraperData
::: scraper._ScraperData

View File

@@ -1,4 +1,5 @@
site_name: "Projet Millesima S6"
site_url: "https://github.guezoloic.com/millesima-ai-engine/"
theme:
name: "material"
@@ -7,6 +8,11 @@ plugins:
- search
- mkdocstrings
extra:
generator: false
copyright: "Loïc GUEZO & Chahrazad DAHMANI UPEC S6 2026"
markdown_extensions:
- admonition
- pymdownx.details

View File

@@ -2,15 +2,19 @@
name = "projet-millesima-s6"
version = "0.1.0"
dependencies = [
"requests==2.32.5",
"requests==2.33.0",
"beautifulsoup4==4.14.3",
"pandas==2.3.3",
"pandas==3.0.1",
"tqdm==4.67.3",
]
[tool.pytest.ini_options]
pythonpath = "src"
testpaths = ["tests"]
[project.optional-dependencies]
test = ["pytest==8.4.2", "requests-mock==1.12.1", "flake8==7.3.0"]
doc = ["mkdocs<2.0.0", "mkdocs-material==9.6.23", "mkdocstrings[python]"]
test = ["pytest==9.0.2", "requests-mock==1.12.1", "flake8==7.3.0"]
doc = ["mkdocs<2.0.0", "mkdocs-material==9.7.6", "mkdocstrings[python]"]
[build-system]
requires = ["setuptools", "wheel"]

View File

@@ -99,7 +99,11 @@ def main() -> None:
filename = argv[1]
cleaning: Cleaning = Cleaning(filename)
_ = cleaning.drop_empty_appellation().fill_missing_scores().encode_appellation()
cleaning.drop_empty_appellation() \
.fill_missing_scores() \
.encode_appellation() \
.getVins() \
.to_csv("clean.csv", index=False)
if __name__ == "__main__":

View File

@@ -377,9 +377,6 @@ class Scraper:
try:
data: dict[str, object] = self.getjsondata(subdir).getdata()
for element in ["initialReduxState", "categ", "content"]:
data = cast(dict[str, object], data.get(element))
products: list[dict[str, Any]] = cast(
list[dict[str, Any]], data.get("products")
)

View File

@@ -185,17 +185,11 @@ def mock_site():
{dumps({
"props": {
"pageProps": {
"initialReduxState": {
"categ": {
"content": {
"products": [
{"seoKeyword": "/nino-negri-5-stelle-sfursat-2022.html",},
{"seoKeyword": "/poubelle",},
{"seoKeyword": "/",}
]
}
}
}
"products": [
{"seoKeyword": "/nino-negri-5-stelle-sfursat-2022.html",},
{"seoKeyword": "/poubelle",},
{"seoKeyword": "/",}
]
}
}
}
@@ -213,14 +207,8 @@ def mock_site():
{dumps({
"props": {
"pageProps": {
"initialReduxState": {
"categ": {
"content": {
"products": [
]
}
}
}
"products": [
]
}
}
}