Add support for Borough of Broxbourne Council (UK), broxbourne.gov.uk. (#2054)

* creating copy of oxford for broxbourne collection days form

* Partially working Broxbourne implementation, broken where not all services available

* Working Broxbourne implementation

* Adding comments that explain Broxbourne's table structure

* Adding markdown for Broxbourne

* Added README and info.md updates, fixed handling of the current year for Broxbourne API

* Adding an integer UPRN test case per recent comments on Birmingham PR

* remove leap year error + reformatting

---------

Co-authored-by: Simon Leigh <simon.leigh@ditto.live>
Co-authored-by: 5ila5 <5ila5@users.noreply.github.com>
Co-authored-by: 5ila5 <38183212+5ila5@users.noreply.github.com>

README.md

@@ -1216,6 +1216,7 @@ Waste collection schedules in the following formats and countries are supported.
 - [Blackburn with Darwen Borough Council](/doc/source/blackburn_gov_uk.md) / blackburn.gov.uk
 - [Blackpool Council](/doc/source/blackpool_gov_uk.md) / blackpool.gov.uk
 - [Borough Council of King's Lynn & West Norfolk](/doc/source/west_norfolk_gov_uk.md) / west-norfolk.gov.uk
+- [Borough of Broxbourne Council](/doc/source/broxbourne_gov_uk.md) / broxbourne.gov.uk
 - [Bracknell Forest Council](/doc/source/bracknell_forest_gov_uk.md) / selfservice.mybfc.bracknell-forest.gov.uk
 - [Bradford Metropolitan District Council](/doc/source/bradford_gov_uk.md) / bradford.gov.uk
 - [Braintree District Council](/doc/source/braintree_gov_uk.md) / braintree.gov.uk

custom_components/waste_collection_schedule/waste_collection_schedule/source/broxbourne_gov_uk.py

@@ -0,0 +1,107 @@
+import datetime
+import logging
+
+import requests
+from bs4 import BeautifulSoup
+from waste_collection_schedule import Collection  # type: ignore[attr-defined]
+
+TITLE = "Borough of Broxbourne Council"
+DESCRIPTION = "Source for broxbourne.gov.uk services for Broxbourne, UK."
+URL = "https://www.broxbourne.gov.uk"
+TEST_CASES = {
+    "Old School Cottage (Domestic Waste Only)": {
+        "uprn": "148040092",
+        "postcode": "EN10 7PX",
+    },
+    "11 Park Road (All Services)": {"uprn": "148028240", "postcode": "EN11 8PU"},
+    "11 Pulham Avenue (All Services)": {"uprn": 148024643, "postcode": "EN10 7TA"},
+}
+
+API_URLS = {
+    "get_session": "https://www.broxbourne.gov.uk/bin-collection-date",
+    "collection": "https://www.broxbourne.gov.uk/xfp/form/205",
+}
+
+LOGGER = logging.getLogger(__name__)
+
+ICON_MAP = {
+    "Domestic": "mdi:trash-can",
+    "Recycling": "mdi:recycle",
+    "Green Waste": "mdi:leaf",
+    "Food": "mdi:food-apple",
+}
+
+
+class Source:
+    def __init__(self, uprn: str, postcode: str):
+        self._uprn = uprn
+        self._postcode = postcode
+
+    def fetch(self):
+        entries: list[Collection] = []
+        session = requests.Session()
+
+        token_response = session.get(API_URLS["get_session"])
+        soup = BeautifulSoup(token_response.text, "html.parser")
+        token = soup.find("input", {"name": "__token"}).attrs["value"]
+        if not token:
+            raise ValueError(
+                "Could not parse CSRF Token from initial response. Won't be able to proceed."
+            )
+
+        form_data = {
+            "__token": token,
+            "page": "490",
+            "locale": "en_GB",
+            "qacf7e570cf99fae4cb3a2e14d5a75fd0d6561058_0_0": self._postcode,
+            "qacf7e570cf99fae4cb3a2e14d5a75fd0d6561058_1_0": self._uprn,
+            "next": "Next",
+        }
+
+        collection_response = session.post(API_URLS["collection"], data=form_data)
+
+        collection_soup = BeautifulSoup(collection_response.text, "html.parser")
+        tr = collection_soup.findAll("tr")
+
+        # The council API returns no year for the collections
+        # and so it needs to be calculated to format the date correctly
+
+        today = datetime.date.today()
+        year = today.year
+
+        for item in tr[1:]:  # Ignore table header row
+            td = item.findAll("td")
+            waste_type = td[1].text.rstrip()
+
+            # We need to replace characters due to encoding in form
+            collection_date_text = (
+                td[0].text.split(" ")[0].replace("\xa0", " ") + " " + str(year)
+            )
+
+            try:
+                # Broxbourne give an empty date field where there is no collection
+                collection_date = datetime.datetime.strptime(
+                    collection_date_text, "%a %d %B %Y"
+                ).date()
+
+            except ValueError as e:
+                LOGGER.warning(
+                    f"No date found for wastetype: {waste_type}. The date field in the table is empty or corrupted. Failed with error: {e}"
+                )
+                continue
+
+            # Calculate the year. As we only get collections a week in advance we can assume the current
+            # year unless the month is January in December where it will be next year
+
+            if (collection_date.month == 1) and (today.month == 12):
+                collection_date = collection_date.replace(year=year + 1)
+
+            entries.append(
+                Collection(
+                    date=collection_date,
+                    t=waste_type,
+                    icon=ICON_MAP.get(waste_type),
+                )
+            )
+
+        return entries

doc/source/broxbourne_gov_uk.md

@@ -0,0 +1,61 @@
+# Borough of Broxbourne Council
+
+Support for schedules provided by [Borough of Broxbourne Council](https://www.broxbourne.gov.uk/), in the UK.
+
+## Configuration via configuration.yaml
+
+```yaml
+waste_collection_schedule:
+    sources:
+    - name: broxbourne_gov_uk
+      args:
+        uprn: UNIQUE_PROPERTY_REFERENCE_NUMBER
+        postcode: POSTCODE
+```
+
+### Configuration Variables
+
+**uprn**  
+*(string)*
+
+The "Unique Property Reference Number" for your address. You can find it by searching for your address at <https://www.findmyaddress.co.uk/>.
+
+**postcode**  
+*(string)*
+
+The Post Code for your address. This needs to match the postcode corresponding to your UPRN.
+
+## Example
+
+```yaml
+waste_collection_schedule:
+    sources:
+    - name: broxbourne_gov_uk
+      args:
+        uprn: 148028240
+        postcode: EN11 8PU
+```
+
+## Returned Collections
+
+This source will return the next collection date for each container type serviced at your address.
+If you don't subscribe to a garden waste bin, we don't return data for it.
+
+## Returned collection types
+
+### Domestic
+
+Black bin for general waste
+
+### Recycling
+
+Black recycling box for mixed recycling
+
+### Green Waste
+
+Green Bin for garden waste.
+If you don't pay for a garden waste bin, it won't be included.
+
+### Food
+
+Green or Brown Caddy for food waste.