add iapp_itouchvision_com, chilteren inherits from iapp_itouchvision_com (#2522)

* Update chiltern_gov_uk.py * add iapp_itouchvision_com, chilteren inherits from iapp_itouchvision_com based on the work of @craigsblackie in #2521 - added iapp_itouchvision_com - chilteren now deprecated and inherits from iapp_itouchvision_com - GUI configuration get automatically migrated - added requirement pycryptodome to handle aes en/de-cryption --------- Co-authored-by: craigsblackie <craig@craigsblackie.com> Co-authored-by: 5ila5 <5ila5@users.noreply.github.com>
2026-03-21 04:06:03 +01:00 · 2024-08-21 16:45:22 +02:00
parent ae58843ee7
commit aa794752e4
13 changed files with 272 additions and 87 deletions
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -47,6 +47,7 @@ repos:
        args:
          - --multi-line=3
          - --trailing-comma
+          - --profile=black
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
--- a/README.md
+++ b/README.md
@@ -1421,7 +1421,7 @@ If your service provider is not listed, feel free to open a [source request issu
 - [Broadland District Council](/doc/source/south_norfolk_and_broadland_gov_uk.md) / area.southnorfolkandbroadland.gov.uk
 - [Bromsgrove City Council](/doc/source/bromsgrove_gov_uk.md) / bromsgrove.gov.uk
 - [Broxtowe Borough Council](/doc/source/broxtowe_gov_uk.md) / broxtowe.gov.uk
- [Buckinghamshire Waste Collection - Former Chiltern, South Bucks or Wycombe areas](/doc/source/chiltern_gov_uk.md) / chiltern.gov.uk
+- [Buckinghamshire: Former (Chiltern, South Bucks, Wycombe)](/doc/source/iapp_itouchvision_com.md) / buckinghamshire.gov.uk
 - [Burnley Council](/doc/source/burnley_gov_uk.md) / burnley.gov.uk
 - [Bury Council](/doc/source/bury_gov_uk.md) / bury.gov.uk
 - [Cambridge City Council](/doc/source/cambridge_gov_uk.md) / cambridge.gov.uk
@@ -1444,6 +1444,7 @@ If your service provider is not listed, feel free to open a [source request issu
 - [Croydon Council](/doc/source/croydon_gov_uk.md) / croydon.gov.uk
 - [Darlington Borough Council](/doc/source/darlington_gov_uk.md) / darlington.gov.uk
 - [Denbighshire County Council](/doc/source/denbighshire_gov_uk.md) / denbighshire.gov.uk
+- [Deprecated: Buckinghamshire](/doc/source/chiltern_gov_uk.md) / chiltern.gov.uk
 - [Derby City Council](/doc/source/derby_gov_uk.md) / derby.gov.uk
 - [Dudley Metropolitan Borough Council](/doc/source/dudley_gov_uk.md) / dudley.gov.uk
 - [Durham County Council](/doc/source/durham_gov_uk.md) / durham.gov.uk
@@ -1484,6 +1485,7 @@ If your service provider is not listed, feel free to open a [source request issu
 - [Hull City Council](/doc/source/hull_gov_uk.md) / hull.gov.uk
 - [Huntingdonshire District Council](/doc/source/huntingdonshire_gov_uk.md) / huntingdonshire.gov.uk
 - [iTouchVision](/doc/source/iweb_itouchvision_com.md) / iweb.itouchvision.com
+- [Itouchvision Source using the encrypted API](/doc/source/iapp_itouchvision_com.md) / itouchvision.com
 - [Joint Waste Solutions](/doc/source/jointwastesolutions_org.md) / jointwastesolutions.org
 - [Kirklees Council](/doc/source/kirklees_gov_uk.md) / kirklees.gov.uk
 - [Lancaster City Council](/doc/source/lancaster_gov_uk.md) / lancaster.gov.uk
@@ -1517,6 +1519,7 @@ If your service provider is not listed, feel free to open a [source request issu
 - [Newark & Sherwood District Council](/doc/source/newark_sherwooddc_gov_uk.md) / newark-sherwooddc.gov.uk
 - [Newcastle City Council](/doc/source/newcastle_gov_uk.md) / community.newcastle.gov.uk
 - [Newcastle Under Lyme Borough Council](/doc/source/newcastle_staffs_gov_uk.md) / newcastle-staffs.gov.uk
+- [Newport City Council](/doc/source/iapp_itouchvision_com.md) / newport.gov.uk
 - [Newport City Council](/doc/source/iweb_itouchvision_com.md) / newport.gov.uk/
 - [North Ayrshire Council](/doc/source/north_ayrshire_gov_uk.md) / north-ayrshire.gov.uk
 - [North Herts Council](/doc/source/northherts_gov_uk.md) / north-herts.gov.uk
--- a/custom_components/waste_collection_schedule/config_flow.py
+++ b/custom_components/waste_collection_schedule/config_flow.py
@@ -62,6 +62,7 @@ from .const import (
    CONF_SOURCE_NAME,
    CONF_TYPE,
    CONF_USE_DEDICATED_CALENDAR,
+    CONFIG_MINOR_VERSION,
    CONFIG_VERSION,
    DOMAIN,
 )
@@ -279,6 +280,7 @@ class WasteCollectionConfigFlow(ConfigFlow, domain=DOMAIN):  # type: ignore[call
    """Config flow."""

    VERSION = CONFIG_VERSION
+    MINOR_VERSION = CONFIG_MINOR_VERSION
    _country: str | None = None
    _source: str | None = None

--- a/custom_components/waste_collection_schedule/const.py
+++ b/custom_components/waste_collection_schedule/const.py
@@ -8,6 +8,7 @@ DOMAIN: Final = "waste_collection_schedule"
 UPDATE_SENSORS_SIGNAL: Final = "wcs_update_sensors_signal"

 CONFIG_VERSION: Final = 2
+CONFIG_MINOR_VERSION: Final = 2

 # Config var names
 CONF_SOURCES: Final = "sources"
--- a/custom_components/waste_collection_schedule/init_ui.py
+++ b/custom_components/waste_collection_schedule/init_ui.py
@@ -18,7 +18,7 @@ _LOGGER = logging.getLogger(__name__)
 PLATFORMS = ["calendar", "sensor"]


-async def async_setup_entry(hass: HomeAssistant, entry) -> bool:
+async def async_setup_entry(hass: HomeAssistant, entry: ConfigEntry) -> bool:
    """Set up component from a config entry, entry contains data from config entry database."""
    options = entry.options
    _LOGGER.debug(
@@ -82,20 +82,23 @@ async def async_setup_entry(hass: HomeAssistant, entry) -> bool:
    return True


-async def async_update_listener(hass, entry):
+async def async_update_listener(hass: HomeAssistant, entry: ConfigEntry):
    # Reload this instance
    await hass.config_entries.async_reload(entry.entry_id)

    return True


-async def async_unload_entry(hass, entry):
+async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry):
    """Unload a config entry."""
    return await hass.config_entries.async_unload_platforms(entry, PLATFORMS)


-async def async_migrate_entry(hass, config_entry: ConfigEntry) -> bool:
+async def async_migrate_entry(hass: HomeAssistant, config_entry: ConfigEntry) -> bool:
    """Migrate old entry."""
+    _LOGGER.debug("Migrating from version %s", config_entry.version)
+    _LOGGER.debug("minor version %s", config_entry.minor_version)
+
    # Version number has gone backwards
    if const.CONFIG_VERSION < config_entry.version:
        _LOGGER.error(
@@ -109,13 +112,27 @@ async def async_migrate_entry(hass, config_entry: ConfigEntry) -> bool:
        new_data = {**config_entry.data}

        if config_entry.version < 2 and const.CONFIG_VERSION >= 2:
+            # Migrate from wychavon_gov_uk to roundlookup_uk
            if new_data.get("name", "") == "wychavon_gov_uk":
                _LOGGER.debug("Migrating from wychavon_gov_uk to roundlookup_uk")
                new_data["name"] = "roundlookup_uk"
                new_data["args"]["council"] = "Wychavon"

+        # Implicitly migrate from any version <= 2.1 to 2.2 (or higher)
+        if config_entry.version < 2 or (
+            config_entry.version == 2 and config_entry.minor_version < 2
+        ):
+            # Migrate from chiltern_gov_uk to iapp_itouchvision_com
+            if new_data.get("name", "") == "chiltern_gov_uk":
+                _LOGGER.debug("Migrating from chiltern_gov_uk to iapp_itouchvision_com")
+                new_data["name"] = "iapp_itouchvision_com"
+                new_data["args"]["municipality"] = "BUCKINGHAMSHIRE"
+
        hass.config_entries.async_update_entry(
-            config_entry, data=new_data, version=const.CONFIG_VERSION
+            config_entry,
+            data=new_data,
+            version=const.CONFIG_VERSION,
+            minor_version=const.CONFIG_MINOR_VERSION,
        )

        _LOGGER.debug("Migration to version %s successful", config_entry.version)
--- a/custom_components/waste_collection_schedule/manifest.json
+++ b/custom_components/waste_collection_schedule/manifest.json
@@ -7,6 +7,6 @@
  "documentation": "https://github.com/mampfes/hacs_waste_collection_schedule#readme",
  "integration_type": "hub",
  "iot_class": "cloud_polling",
-  "requirements": ["icalendar", "recurring_ical_events", "icalevents", "beautifulsoup4", "lxml"],
+  "requirements": ["icalendar", "recurring_ical_events", "icalevents", "beautifulsoup4", "lxml", "pycryptodome"],
  "version": "2.1.0"
 }
--- a/custom_components/waste_collection_schedule/sources.json
+++ b/custom_components/waste_collection_schedule/sources.json
@@ -7697,9 +7697,11 @@
      "default_params": {}
    },
    {
-      "title": "Buckinghamshire Waste Collection - Former Chiltern, South Bucks or Wycombe areas",
-      "module": "chiltern_gov_uk",
-      "default_params": {}
+      "title": "Buckinghamshire: Former (Chiltern, South Bucks, Wycombe)",
+      "module": "iapp_itouchvision_com",
+      "default_params": {
+        "municipality": "BUCKINGHAMSHIRE"
+      }
    },
    {
      "title": "Burnley Council",
@@ -7811,6 +7813,11 @@
      "module": "denbighshire_gov_uk",
      "default_params": {}
    },
+    {
+      "title": "Deprecated: Buckinghamshire",
+      "module": "chiltern_gov_uk",
+      "default_params": {}
+    },
    {
      "title": "Derby City Council",
      "module": "derby_gov_uk",
@@ -8013,6 +8020,11 @@
      "module": "iweb_itouchvision_com",
      "default_params": {}
    },
+    {
+      "title": "Itouchvision Source using the encrypted API",
+      "module": "iapp_itouchvision_com",
+      "default_params": {}
+    },
    {
      "title": "Joint Waste Solutions",
      "module": "jointwastesolutions_org",
@@ -8188,6 +8200,13 @@
      "module": "newcastle_staffs_gov_uk",
      "default_params": {}
    },
+    {
+      "title": "Newport City Council",
+      "module": "iapp_itouchvision_com",
+      "default_params": {
+        "municipality": "NEWPORT"
+      }
+    },
    {
      "title": "Newport City Council",
      "module": "iweb_itouchvision_com",
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/chiltern_gov_uk.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/chiltern_gov_uk.py
@@ -1,75 +1,20 @@
-import re
-import base64
-import json
-import requests
-from datetime import datetime
-from bs4 import BeautifulSoup
-from waste_collection_schedule import Collection
+import logging

-TITLE = "Buckinghamshire Waste Collection - Former Chiltern, South Bucks or Wycombe areas"
-DESCRIPTION = "Source for chiltern.gov.uk services for parts of Buckinghamshire"
+from waste_collection_schedule.source.iapp_itouchvision_com import (
+    Source as iapp_iTouchVision,
+)
+
+_LOGGER = logging.getLogger(__name__)
+TITLE = "Deprecated: Buckinghamshire"
+DESCRIPTION = "Deprecated: use the iapp_iTouchVision instead."
 URL = "https://chiltern.gov.uk"

 TEST_CASES = {
-    "Test_001": {"uprn": "200000811701"},
-    "Test_002": {"uprn": "100080550517"},
-    "Test_003": {"uprn": "100081091932"},
    "Test_004": {"uprn": 10094593823},
 }

-ICON_MAP = {
-    "Domestic Refuse Collection": "mdi:trash-can",
-    "Domestic Food Collection": "mdi:food",
-    "Domestic Garden Collection": "mdi:leaf",
-    "Domestic Paper/Card Collection": "mdi:newspaper",
-    "Domestic Mixed Dry Recycling Collection": "mdi:glass-fragile",
-    "Communal Paper/Card Collection": "mdi:newspaper",
-    "Communal Mixed Dry Recycling Collection": "mdi:glass-fragile",
-    "Communal Refuse Collection": "mdi:trash-can",
-}

-class Source:
+class Source(iapp_iTouchVision):
    def __init__(self, uprn):
-        self._uprn = uprn
-
-    def fetch(self):
-        session = requests.Session()
-        # Start a session
-        r = session.get("https://chiltern.gov.uk/collection-dates")
-        r.raise_for_status()
-        soup = BeautifulSoup(r.text, features="html.parser")
-
-        # Extract form submission url
-        form = soup.find("form", attrs={"id": "COPYOFECHOCOLLECTIONDATES_FORM"})
-        form_url = form["action"]
-
-        # Submit form
-        form_data = {
-            "COPYOFECHOCOLLECTIONDATES_FORMACTION_NEXT": "COPYOFECHOCOLLECTIONDATES_ADDRESSSELECTION_NAV1",
-            "COPYOFECHOCOLLECTIONDATES_ADDRESSSELECTION_SELECTEDADDRESS": "dummy serverside validaiton only",
-            "COPYOFECHOCOLLECTIONDATES_ADDRESSSELECTION_UPRN": self._uprn,
-        }
-        r = session.post(form_url, data=form_data)
-        r.raise_for_status()
-
-        # Extract collection dates
-        pattern = r'var COPYOFECHOCOLLECTIONDATES_PAGE1_DATES2Data = JSON.parse\(helper\.utilDecode\(\'([^\']+)\'\)\);'
-        match = re.search (pattern,r.text)
-        if match:
-           decoded_jsonstr = base64.b64decode(match.group(1)).decode('utf-8')
-  
-        servicedata = json.loads(decoded_jsonstr)
-
-        entries = []
-        
-        #Loop through services and append to Collection 
-        for service in servicedata['services']:
-            entries.append(
-                Collection(
-                    date=datetime.strptime(service['nextDate'], '%d/%m/%Y').date(),
-                    t=service['serviceName'],
-                    icon=ICON_MAP.get(service['serviceName']),
-                )
-            )
-
-        return entries
+        super().__init__(uprn, "BUCKINGHAMSHIRE")
+        _LOGGER.warning("This source is deprecated. Use iapp_iTouchVision instead.")
--- a/custom_components/waste_collection_schedule/waste_collection_schedule/source/iapp_itouchvision_com.py
+++ b/custom_components/waste_collection_schedule/waste_collection_schedule/source/iapp_itouchvision_com.py
@@ -0,0 +1,157 @@
+import binascii
+import json
+from datetime import datetime
+from typing import Literal, TypedDict
+
+import requests
+from Crypto.Cipher import AES
+from Crypto.Util.Padding import pad, unpad
+from waste_collection_schedule import Collection
+
+
+class Municipality(TypedDict):
+    PAYLOAD: dict[str, str | int]
+    API_URL: str
+    title: str
+    url: str
+
+
+TITLE = "Itouchvision Source using the encrypted API"
+DESCRIPTION = "Source for Itouchvision Source using the encrypted API."
+URL = "https://www.itouchvision.com/"
+TEST_CASES = {
+    "chiltern: 100080550517": {"uprn": 100080550517, "municipality": "BUCKINGHAMSHIRE"},
+    "newport: 100080550517": {"uprn": "10090955364", "municipality": "NEWPORT"},
+}
+COUNTRY = "uk"
+
+
+ICON_MAP = {
+    "Food waste": "mdi:food",
+    "General waste": "mdi:trash-can",
+    "Mixed recycling": "mdi:recycle",
+    "Paper and cardboard": "mdi:newspaper",
+    "Textiles/Batteries/Electricals": "mdi:battery",
+    "GARDEN WASTE": "mdi:flower",
+    "HOUSEHOLD WASTE": "mdi:trash-can",
+    "RECYCLING": "mdi:recycle",
+}
+
+# Global variables for encryption key and IV
+KEY = binascii.unhexlify(
+    "F57E76482EE3DC3336495DEDEEF3962671B054FE353E815145E29C5689F72FEC"
+)
+IV = binascii.unhexlify("2CBF4FC35C69B82362D393A4F0B9971A")
+
+
+# Encryption function
+def encrypt_aes(plaintext: str) -> str:
+    data = plaintext.encode("utf-8")
+    padded_data = pad(data, AES.block_size)
+    cipher = AES.new(KEY, AES.MODE_CBC, IV)
+    ciphertext = cipher.encrypt(padded_data)
+    return binascii.hexlify(ciphertext).decode("utf-8")
+
+
+# Decryption function
+def decrypt_aes(ciphertext_hex: str) -> str:
+    ciphertext = binascii.unhexlify(ciphertext_hex)
+    cipher = AES.new(KEY, AES.MODE_CBC, IV)
+    decrypted_data = cipher.decrypt(ciphertext)
+    plaintext = unpad(decrypted_data, AES.block_size).decode("utf-8")
+    return plaintext
+
+
+MUNICIPALITIES: dict[str, Municipality] = {
+    "BUCKINGHAMSHIRE": {
+        "PAYLOAD": {
+            "P_CLIENT_ID": 152,
+            "P_COUNCIL_ID": 34505,
+        },
+        "API_URL": "https://itouchvision.app/portal/itouchvision/kmbd/collectionDay",
+        "title": "Buckinghamshire: Former (Chiltern, South Bucks, Wycombe)",
+        "url": "https://www.buckinghamshire.gov.uk/",
+    },
+    "NEWPORT": {
+        "PAYLOAD": {
+            "P_CLIENT_ID": 130,
+            "P_COUNCIL_ID": 260,
+        },
+        "API_URL": "https://iweb.itouchvision.com/portal/itouchvision/kmbd/collectionDay",
+        "title": "Newport City Council",
+        "url": "https://www.newport.gov.uk/",
+    },
+}
+
+MUNICIPALITY_LITERALS = Literal["BUCKINGHAMSHIRE", "NEWPORT"]
+
+EXTRA_INFO = [
+    {
+        "title": m["title"],
+        "url": m["url"],
+        "country": COUNTRY,
+        "default_params": {"municipality": key},
+    }
+    for key, m in MUNICIPALITIES.items()
+]
+
+
+class Source:
+    def __init__(self, uprn: str | int, municipality: MUNICIPALITY_LITERALS):
+        self._uprn: str | int = uprn
+        if not municipality.upper() in MUNICIPALITIES:
+            raise ValueError(f"Unknown municipality: {municipality}")
+        self._payload = MUNICIPALITIES[municipality.upper()]["PAYLOAD"]
+        self._api_url = MUNICIPALITIES[municipality.upper()]["API_URL"]
+
+    def fetch(self) -> list[Collection]:
+        session = requests.Session()
+
+        # Prepare the data to be encrypted
+        payload: dict[str, str | int] = {
+            "P_UPRN": self._uprn,
+            **self._payload,
+            "P_LANG_CODE": "EN",
+        }
+
+        # Encrypt the payload
+        encrypted_payload = encrypt_aes(json.dumps(payload))
+
+        # Send the request with the encrypted data
+        response = session.post(
+            self._api_url,
+            data=encrypted_payload,
+            headers={
+                "Content-Type": "application/json; charset=UTF-8",
+                "Accept": "*/*",
+            },
+        )
+        response.raise_for_status()
+
+        # Decrypt the response
+        decrypted_response = decrypt_aes(response.text)
+
+        # Parse the JSON response
+        servicedata = json.loads(decrypted_response)
+
+        # Process the collection dates
+        entries = []
+        for service in servicedata["collectionDay"]:
+            collection_dates = [
+                datetime.strptime(service["collectionDay"], "%d-%m-%Y").date()
+            ]
+            try:
+                collection_dates.append(
+                    datetime.strptime(service["followingDay"], "%d-%m-%Y").date()
+                )
+            except Exception:
+                pass
+            bin_type = service["binType"]
+            for collection_date in collection_dates:
+                entries.append(
+                    Collection(
+                        date=collection_date, t=bin_type, icon=ICON_MAP.get(bin_type)
+                    )
+                )
+
+        return entries
--- a/doc/source/chiltern_gov_uk.md
+++ b/doc/source/chiltern_gov_uk.md
@@ -1,13 +1,14 @@
-# Chiltern Area - Buckinghamshire Council
+# DEPRECATED Chiltern Area - Buckinghamshire Council
+
+This integration is deprecated and may be removed in a future release. Please use the [Itouchvision Source using the encrypted API](/doc/source/iapp_itouchvision_com.md) instead.

-Support for schedules provided by former Chiltern, SouthBucks or Wycombe area,  Council](https://chiltern.gov.uk/collection-dates) that covers High Wycombe.

 ## Configuration via configuration.yaml

 ```yaml
 waste_collection_schedule:
    sources:
-    - name: chiltern_gov_uk
+    - name: chiltern_gov_uk # DEPRECATED USE iapp_itouchvision_com INSTEAD with mun
      args:
        uprn: UPRN
 ```
@@ -22,11 +23,7 @@ waste_collection_schedule:
 ```yaml
 waste_collection_schedule:
    sources:
-    - name: chiltern_gov_uk
+    - name: chiltern_gov_uk # DEPRECATED USE iapp_itouchvision_com INSTEAD
      args:
        uprn: 200000811701
 ```
-
-## How to get the source argument
-
-Search for your address on the [FindMyAddress service](https://www.findmyaddress.co.uk/) which displays the UPRN in the result.
--- a/doc/source/iapp_itouchvision_com.md
+++ b/doc/source/iapp_itouchvision_com.md
@@ -0,0 +1,42 @@
+# Itouchvision Source using the encrypted API
+
+Support for schedules provided by [Itouchvision Source using the encrypted API](https://www.itouchvision.com/), serving multiple, UK.
+
+## Configuration via configuration.yaml
+
+```yaml
+waste_collection_schedule:
+    sources:
+    - name: iapp_itouchvision_com
+      args:
+        uprn: "UPRN"
+        municipality: "MUNICIPALITY"
+```
+
+### Configuration Variables
+
+**uprn**  
+*(String | Integer) (required)*
+
+**municipality**  
+*(String) (required)*  
+supported are:
+
+- BUCKINGHAMSHIRE
+- CHILTERN
+
+## Example
+
+```yaml
+waste_collection_schedule:
+    sources:
+    - name: iapp_itouchvision_com
+      args:
+        uprn: "100080550517"
+        municipality: "BUCKINGHAMSHIRE"
+        
+```
+
+## How to get the source argument
+
+An easy way to discover your Unique Property Reference Number (UPRN) is by going to <https://www.findmyaddress.co.uk/> and entering in your address details.
--- a/info.md
+++ b/info.md
--- a/requirements.txt
+++ b/requirements.txt
@@ -10,3 +10,4 @@ requests>=2.31.0
 urllib3>=2.0.7
 jinja2>=3.1.2
 lxml>=4.9.4
+pycryptodome>=3.20.0