I have set up the openems on a test and is using it to monitor a random single phase meter. Currently, I have a aws server that is running influxdb to capture the data stream. My question is regarding internet stability. If I have weak connectivity, I seem to lose data. Does the influxdb service, built into openems, have some sort of a queueing service to buffer some data locally, if the internet connection is lost? Or how would you suggest me solving this issue?
are you using the local Timedata.InfluxDB on the OpenEMS Edge to connecto the AWS InfluxDB? In such a setup I would recommend running an OpenEMS Backend on the AWS server, because the Edge to Backend connection is better suited for unstable connections.
We are internally working on a local cache and resending feature, built using a local RRD4j timeseries database. Unfortunately InfluxDB showd some implementation problems when using this backfilling/resending together with continuous aggregates. That’s why we currently stopped this development and switch to TimescaleDB first (see this thread: New integration: TimescaleDB)
I will do as you have suggested. When do you think will the new local RRD4j database be available for implementation in this regard? Would be great to know the data is backed up locally so that no data is lost.
the local RRD4j database is already available and can simply be activated. It stores the data locally for about one month in a round-robin database to make sure, that a local SD card or eMMC does not wear out.
The logic to resend data in case of internet connection problems to the Backend will be added to the Backend-Api-Controller.
May I know where (in which file/directory) does RRD4j store the data? Or how can I retrieve the data? My goal is to somehow retrieve the data in some (preferably CSV) format so I can use it. I tried using InfluxDB, but the Edge app shows some error in the log after I enabled InfluxDB.
Just some more questions on the backfilling feature that we have discussed above:
I see TimescaleDB is already available in the dev branch. Is the backfilling feature available on the latest dev version of TimescaleDB and backend? I would love to test it.
the resending feature is not directly related to the timeseries database on the backend (InfluxDB or TimescaleDB). The functionality will have to be implemented in the Controller.Api.Backend.
The reason why we prioritized TimescaleDB in our development was, that InfluxDB has problems using backfilled data (i.e. data that has been sent via the ‘resending feature’) in its Continuous Aggregates. But switching to TimescaleDB brought its own problems at the moment…
We have an internal development branch with a first implementation of the resending feature. I’ll see if we can at least update it to latest version and maybe develop it publically on Github.
Having a version to test would be great. In the meanwhile I am thinking of implementing the telegraph workaround that another user have mentioned. This will limit the backend functionality, but I am holding thumbs that it should do the trick. For now at least.
unfortunately, the internal development is not yet usable. Anyhow, I published the current (not-compiling) draft as a PR to join forces and avoid duplicated development. It’s really only a first draft, so it’s likely that every line in this current implementation will change eventually…