Feature: Integrate MPPTs of internal battery providers with existing solar charger view

[vaterlangen]

Successor of #1824

Internal data handling heavily reworked to address issues mentioned in #1824 (comment) .

Additional to the new integration, the maximum data age for solarcharger and battery webview has been made adjustable via API.

ToDo

• Code cleanup
• Arch review by @schlimmchen

Description

This PR implements a generic interface to integrate MPPT data reported by integrated battery providers (currently, only Zendure is supported) with the existing solar charger view. The user has to manually enable the new provider in the solar charger settings.

Updated settings page:

Updated status Page

The implementation should be re-usable for other battery integrations and should also work with multiple battery providers.
@AndreasBoehm / @schlimmchen Please have a look if this implementation fits your architectural concept and give your comments. ;-)

This change set in this version is running on my own setup for about one week now.

Last comments from old PR

@schlimmchen commented on Apr 16

So... The idea is to allow to funnel MPPT-related data from a battery provider implementation up and to the side to a new solar charger provider which can interpret that data and present it through the solar charger interface. If that's right, I get it. It makes sense, as smart buffer batteries have an MPPT integrated -- that's the whole point of them being smart buffer batteries.

I feel there is a naming issue for sure: This new solar charger provider cannot have "smart buffer" in its name. That suggests that the battery's MPPT data will magically appear even though there is no connection whatsoever to the smart buffer. That is a problem because the feature to support "smart buffer batteries" in the DPL is generic to all smart buffer batteries, as no interface to the battery is required to implement that feature. However, this solar charger provider only works for smart buffer batteries that are actually supported through the battery interface. That mix-up has to be made clear.

Scrolling through the changeset feels a bit rough around the edges. Especially the shoveling of data using setters and getters is... off-putting. The introduction of the smartbufferbattery provider class makes sense, though.

This change hinges on the fact that there will be other smart buffer batteries, who OpenDTU-OnBattery can interface in the future, which will also spit out MPPT data. That is... a wild bet. Until then, this is a Zendure-special-treatment, or isn't it?

Also, please make me understand why the MQTT solar charger provider does not work here? I thought that all Zendure data is available through MQTT anyways? And if it isn't in a format that the current MQTT solar charger provider can process (I remember that not all Zendure data is just "there" but has to be retrieved and processed?), why can't the Zendure battery provider publish the MPPT data in the respective format for the solar charger MQTT provider to consume?

@vaterlangen commented on Apr 19

So... The idea is to allow to funnel MPPT-related data from a battery provider implementation up and to the side to a new solar charger provider which can interpret that data and present it through the solar charger interface. If that's right, I get it. It makes sense, as smart buffer batteries have an MPPT integrated -- that's the whole point of them being smart buffer batteries.

Correct. On a long term perspective, this data could also be used by the SmartBufferBattery DPL in order to pass-through excess power (MPPT input when battery is full) from supported SmartBufferBatteries (atm: Zendure) and feed it into the grid instead of wasting it.

I feel there is a naming issue for sure: This new solar charger provider cannot have "smart buffer" in its name. That suggests that the battery's MPPT data will magically appear even though there is no connection whatsoever to the smart buffer. That is a problem because the feature to support "smart buffer batteries" in the DPL is generic to all smart buffer batteries, as no interface to the battery is required to implement that feature. However, this solar charger provider only works for smart buffer batteries that are actually supported through the battery interface. That mix-up has to be made clear.

Good point - will think about better naming for this. Already ideas from your side?

Scrolling through the changeset feels a bit rough around the edges. Especially the shoveling of data using setters and getters is... off-putting. The introduction of the smartbufferbattery provider class makes sense, though.

Ok, I will have a look at it again and try to optimize/simplify things. But - as both sides may be re-initilized at any time (configuration change) - I don't think it will work without fetching the Stats on every call.

This change hinges on the fact that there will be other smart buffer batteries, who OpenDTU-OnBattery can interface in the future, which will also spit out MPPT data. That is... a wild bet. Until then, this is a Zendure-special-treatment, or isn't it?

I did not like the idea of implementing stuff dedicated to single implementations. So I tried to keep it re-useable for future implementations, too. But you are right - currently it is only used by the Zendure integration.

Also, please make me understand why the MQTT solar charger provider does not work here? I thought that all Zendure data is available through MQTT anyways? And if it isn't in a format that the current MQTT solar charger provider can process (I remember that not all Zendure data is just "there" but has to be retrieved and processed?), why can't the Zendure battery provider publish the MPPT data in the respective format for the solar charger MQTT provider to consume?

In generell this would work, yes (at least, when I would pre-process the data within the battery provider and publish them there). But as far as I see in the code, the MQTT solar charger provider only supports a single MPPT. Zendure has two integrated MPPTs and I would like to make them visible. An other drawback would be the additional configuration/complexity exposed to the user, who would have to setup the topics correctly.

[github-actions]

Build Artifacts

Firmware built from this pull request's code:

• opendtu-onbattery-generic_esp32_8mb.zip
• opendtu-onbattery-generic_esp32_4mb_no_ota.zip
• opendtu-onbattery-generic_esp32s3.zip
• opendtu-onbattery-generic_esp32s3_usb.zip

Notice

• These artifacts are ZIP files containing the factory update binary as well as the OTA update binary.
  Extract the binaries from the ZIP files first. Do not use the ZIP files themselves to perform an update.
• These links point to artifacts of the latest successful build run.
• The linked artifacts were built from f7a9075.