Online data logging/plotting site

Forums › Gadgets, robotics, home automation, electronics (including wearables) › Online data logging/plotting site

neb

11294 posts

Uber Geek
+1 received by user: 10018

Trusted

Lifetime subscriber

#269725 4-Apr-2020 18:31

I've just set up a power monitor to log voltage, frequency, power factor, etc over modbus, and I'd like to upload and graph it somewhere long-term. There are a bunch of online graphing sites, but they're mostly just Excel for the web. Some time ago I saw a site that allows posting of data via HTTP for long-term storage and viewing but I can't remember what it was, can anyone recommend anything? Data volume is low, three values every few minutes, so any kind of free account should do it, I'd just like to view trends over time and compare what it was like last month vs. this month. Eventually I'll also hook up IFTTT to alarm on the voltage going out of range, which was the motivation for doing this in the first place, but that's for another day.

richms

29098 posts

Uber Geek
+1 received by user: 10208

Trusted

Lifetime subscriber

#2454571 4-Apr-2020 18:33

adafruit.io has some good charting options, and can use mqtt which is more use for this sort of thing than http is.

Richard rich.ms

neb

11294 posts

Uber Geek
+1 received by user: 10018

Trusted

Lifetime subscriber

#2454574 4-Apr-2020 18:36

Ah, it was probably that. HTTP is easier (lazier) than MQTT, I can just script a POST from the modbus data without having to go through another protocol layer.

richms

29098 posts

Uber Geek
+1 received by user: 10208

Trusted

Lifetime subscriber

#2454579 4-Apr-2020 18:40

It might support it, I have just used example sketches of theirs in arduino and changed it a little when I was doing things with it. Not played with it in a few years after I found that the probelms with my weather station were all just because 433MHz is a trashfire of a band for anything to operate on.

Richard rich.ms

davidcole

6099 posts

Uber Geek
+1 received by user: 1465

Trusted

#2454620 4-Apr-2020 19:55

Influx does have a cloud database if you don’t want to host one yourself.

Previously known as psycik

Home Assistant: Gigabyte AMD A8 Brix, Home Assistant with Aeotech ZWave Controller, Raspberry PI, Wemos D1 Mini, Zwave, Shelly Humidity and Temperature sensors
Media:Chromecast v2, ATV4 4k, ATV4, HDHomeRun Dual
Server Host Plex Server 3x3TB, 4x4TB using MergerFS, Samsung 850 evo 512 GB SSD, Proxmox Server with 1xW10, 2xUbuntu 22.04 LTS, Backblaze Backups, usenetprime.com fastmail.com Sharesies Trakt.TV Sharesight

nzkc

1634 posts

Uber Geek
+1 received by user: 1041

#2454626 4-Apr-2020 20:06

davidcole: Influx does have a cloud database if you don’t want to host one yourself.

As does Grafana now too. I've not used either cloud products. I do use them both at home and log locally.

I have used Datadog in the past with a free account. Don't know if they still offer that though. I was logging data from a BME280 sensor to it from a Raspberry Pi.

frankv

5705 posts

Uber Geek
+1 received by user: 3666

Lifetime subscriber

#2454685 4-Apr-2020 22:54

Sounds like the sort of thing to do via AWS or some other cloud provider, but I'm more familiar with AWS. AWS has a fairly generous free tier, allowing you to put your data into a MySQL database up to 20GB for free. Then you could have static parts of your website (again free) stored in an S3 bucket, and a Lambda function to do the database work.

HP

Shop now for HP laptops and other devices (affiliate link).

TwoSeven

1712 posts

Uber Geek
+1 received by user: 304

Subscriber

#2454688 4-Apr-2020 23:04

You could just use an Azure IoT event hub to pick up the events and push the data into PowerBI via stream analytics.

Software Engineer
(the practice of real science, engineering and management)
A.I. (Automation rebranded)
Gender Neutral
(a person who believes in equality and who does not believe in/use stereotypes. Examples such as gender, binary, nonbinary, male/female etc.)

...they/their/them...

hio77

'That VDSL Cat'
13036 posts

Uber Geek
+1 received by user: 3896

ID Verified

Trusted

Lizard Networks

Subscriber

#2454698 5-Apr-2020 00:48

frankv: Sounds like the sort of thing to do via AWS or some other cloud provider, but I'm more familiar with AWS. AWS has a fairly generous free tier, allowing you to put your data into a MySQL database up to 20GB for free. Then you could have static parts of your website (again free) stored in an S3 bucket, and a Lambda function to do the database work.

until your 12 month period ends ;)

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

davidcole

6099 posts

Uber Geek
+1 received by user: 1465

Trusted

#2454737 5-Apr-2020 08:07

If you've just got one sensor, you could try thingspeak.

That was the first thing I ever published to before getting into openhab/influxdb froma esp8266

frankv

5705 posts

Uber Geek
+1 received by user: 3666

Lifetime subscriber

#2454765 5-Apr-2020 09:05

hio77:
frankv: Sounds like the sort of thing to do via AWS or some other cloud provider, but I'm more familiar with AWS. AWS has a fairly generous free tier, allowing you to put your data into a MySQL database up to 20GB for free. Then you could have static parts of your website (again free) stored in an S3 bucket, and a Lambda function to do the database work.
until your 12 month period ends ;)

No, AWs free tier is forever. I think it's Azure that has the 12 month limit?

And the post above about Azure reminded me that AWS has the same kind of software stack... you can use MQTT to connect to AWS IoT endpoints to analytics.

TwoSeven

1712 posts

Uber Geek
+1 received by user: 304

Subscriber

#2455148 5-Apr-2020 15:05

frankv: quote hio77:

frankv: Sounds like the sort of thing to do via AWS or some other cloud provider, but I'm more familiar with AWS. AWS has a fairly generous free tier, allowing you to put your data into a MySQL database up to 20GB for free. Then you could have static parts of your website (again free) stored in an S3 bucket, and a Lambda function to do the database work.
until your 12 month period ends ;)

/quote
No, AWs free tier is forever. I think it's Azure that has the 12 month limit?

And the post above about Azure reminded me that AWS has the same kind of software stack... you can use MQTT to connect to AWS IoT endpoints to analytics.

As I understand things AWS has an always free tier, and a 12 month free for some products that sit on top. I thing the AWS IoT Core has a free tier option, although I have never used it. Perhaps there is someone that could give some good guidance on it.

With Azure you can sign up for free and get 200 hours. You can then switch to pay as you go - however, pretty much all of the services have a free version of some kind and for those that don’t the basic charges are minimal.

The Azure IoT hub standard tier is free for up to 8000 messages per day (a message is 512 bytes in free tier, 4kb otherwise).

One can use VS Code to connect to the IoT hub (runs on linux and windows). One can receive messages from devices and send messages to them.

Stream analytics is a realtime processing engine, its job is to do something with the message. It can take input from event hubs, allow one to manipulate it and produce an output say to machine learning, azure functions and database or PowerBI for display (or all of them at once)

it is worth while having a read through some of the many tutorials online.

...they/their/them...

Samsung

Shop now on Samsung phones, tablets, TVs and more (affiliate link).

nzkc

1634 posts

Uber Geek
+1 received by user: 1041

#2455153 5-Apr-2020 15:11

Re AWS Free tier...

TLDR: Some is 12months, some is perpetual (on limits - eg Lambda functions).

Full details: https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc

hio77

'That VDSL Cat'
13036 posts

Uber Geek
+1 received by user: 3896

ID Verified

Trusted

Lizard Networks

Subscriber

#2455162 5-Apr-2020 15:25

nzkc:

Re AWS Free tier...

TLDR: Some is 12months, some is perpetual (on limits - eg Lambda functions).

Full details: https://aws.amazon.com/free/?all-free-tier.sort-by=item.additionalFields.SortRank&all-free-tier.sort-order=asc

Yep. such a confusing stack...

Personally i use a MariaDB instance configured with Replication on sitting at a few locations.

Not that it's critical or anything but dockers are easy to accidentally delete and data is always nice to look back over...

Far more efficient than my ELK stack i used for everything else.

#include <std_disclaimer>

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.

neb

11294 posts

Uber Geek
+1 received by user: 10018

Trusted

Lifetime subscriber

#2455169 5-Apr-2020 15:37

davidcole:
If you've just got one sensor, you could try thingspeak.

That was the first thing I ever published to before getting into openhab/influxdb froma esp8266

Ah, it was Thingspeak that I'd seen before but couldn't remember the name of. Currently playing with Adafruit, but I'll give that a go as well, thanks!

As a general response to the DIY suggestions, I was just looking for a minimal-effort solution, and in particular one with good existing tool support, which both Adafruit and Thingspeak have. Thingspeak at least also has Android/iOS monitoring apps so I can check it via my phone.

neb

11294 posts

Uber Geek
+1 received by user: 10018

Trusted

Lifetime subscriber

#2455360 5-Apr-2020 20:38

Well, that was really easy, it's up now. The first data points are a bit wonky, I was just testing it out manually, now it's a high-tech "while sleep 59 ; do wget ... done" which is good enough. Be interesting to see what it turns into long-term. Now to see if they have an existing tool to email an alert if it drops to 190V like it did for the neighbours... there's a ThingTweet app but no ThingEmail app that I can find.