Suggestions to fix odd Linux Raspberry Pi 4 problem - system up ssh down only some services running

Forums › Linux › Suggestions to fix odd Linux Raspberry Pi 4 problem - system up ssh down only some services running

1 | 2 | 3

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3199358 24-Feb-2024 07:50

Tinkerisk:

I wouldn't leave such existential network functions to a Raspberry Pi without redundancy - but I've already said that. 😉

What would you use? The Pi's are reliable, in this case I had bad luck with a disk. I'll buy a better disk - this one.

I don't want to leave my main PCs on 24/7, and lower power consumption is good.

Tinkerisk

4227 posts

Uber Geek

#3199468 24-Feb-2024 13:03

timmmay:

Tinkerisk:

I wouldn't leave such existential network functions to a Raspberry Pi without redundancy - but I've already said that. 😉

What would you use? The Pi's are reliable, in this case I had bad luck with a disk. I'll buy a better disk - this one.

I don't want to leave my main PCs on 24/7, and lower power consumption is good.

If you absolutely want to stick with the Raspberry Pi, possibly Docker Swarm or a Kubernetes cluster with 3-4 Pis, so that one instance takes over if the other fails. Recently I also saw Proxmox running on several Pi 5s and rumour has it that the Pi5 will be officially supported.

- NET: FTTH, OPNsense, 10G backbone, GWN APs, ipPBX
- SRV: 12 RU HA server cluster, 0.1 PB storage on premise
- IoT: thread, zigbee, tasmota, BidCoS, LoRa, WX suite, IR
- 3D: two 3D printers, 3D scanner, CNC router, laser cutter

richms

28187 posts

Uber Geek

Trusted

Lifetime subscriber

#3199469 24-Feb-2024 13:12

Sata SSDs are a dead end technology. I am amazed that someone would bother making a case to take them when the only use case for them being made is replacements for old obsolete laptops from a very very limited timeframe that are probably now all ewaste anyway.

I have my home assistant constant work in progress running in a little NES shaped case that takes the SSD in a nintendo cartridge shaped housing, so I can quickly pull it out and plug it into a PC when it needs the constant re imaging etc because I dared to follow an out of date howto do something and murdered my installation.

Richard rich.ms

Tinkerisk

4227 posts

Uber Geek

#3199478 24-Feb-2024 13:27

richms:

Sata SSDs are a dead end technology. I am amazed that someone would bother making a case to take them when the only use case for them being made is replacements for old obsolete laptops from a very very limited timeframe that are probably now all ewaste anyway.

I have my home assistant constant work in progress running in a little NES shaped case that takes the SSD in a nintendo cartridge shaped housing, so I can quickly pull it out and plug it into a PC when it needs the constant re imaging etc because I dared to follow an out of date howto do something and murdered my installation.

I also have a retro NES case for the RPi4 lying around somewhere.

Home Assistant is a special case of learning pretty quickly what a configuration update is. But if DHCP and DNS are affected, you're pretty quickly in nirvana (and that's also one reason why I would never "co-virtualise" a router in a large, powerful machine for reasons of economy, although this is undoubtedly technically easy to do).

@timmmay

The essence of constant availability is constant availability and this must be ensured through redundancy. After all, there is a good reason why I was surrounded with triple redundancy in aviation, because you can't just pull over to the side of the road in the event of a failure. A simple light switch is more reliable than a non-redundant router or home assistant installation and once you're in the dark because ... you know.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3199491 24-Feb-2024 14:00

I have a couple of Pi4's, maybe I'll do a cluster one day. For now I can revert back to the router DHCP very easily, the home assistant automations aren't doing anything essential, just visibility and a single control point and useful things.

The m.2 SATA SSD is a weird beast, hard to find and more expensive. I didn't look at the fine details before I ordered the case. The Pi5 seems like a good advance in this area, but the Pi4's work fine so no need to bother with a Pi5 yet. I'll probably stick with this case / weird SSD until it dies then upgrade in a few years.

PBTech accepted the SSD return, I'll order the transcend soon and hope that lasts a bit longer.

Tinkerisk

4227 posts

Uber Geek

#3199507 24-Feb-2024 14:11

timmmay:

I have a couple of Pi4's, maybe I'll do a cluster one day. For now I can revert back to the router DHCP very easily, the home assistant automations aren't doing anything essential, just visibility and a single control point and useful things.

The m.2 SATA SSD is a weird beast, hard to find and more expensive. I didn't look at the fine details before I ordered the case. The Pi5 seems like a good advance in this area, but the Pi4's work fine so no need to bother with a Pi5 yet. I'll probably stick with this case / weird SSD until it dies then upgrade in a few years.

PBTech accepted the SSD return, I'll order the transcend soon and hope that lasts a bit longer.

Don't take it personally and it's really meant with a smile, but you really are a special case: „we would have liked to do it, but we didn't dare to do it.“ 🤭

You can adapt your problems to existing hardware/software, but wouldn't it make more sense to adapt the hardware/software to the existing and potential future problems with foresight? I mean, if only because of the flexibility? However. 😉

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3199512 24-Feb-2024 14:23

The downside of setting up a cluster is I won't have the automation or services while I get it all working. A single Pi is reliable enough, given the backups available in the router. I'll keep the SD card of the second RPi when I get the main one with the M.2 SSD working, that way if i get another failure I can switch back fairly quickly and easily.

If it was running anything critical I'd do a cluster, or host it somewhere like AWS over a VPN.

Quic Broadband

Move to New Zealand's best fibre broadband service (affiliate link). Free setup code: R587125ERQ6VE. Note that to use Quic Broadband you must be comfortable with configuring your own router.

roobarb

653 posts

Ultimate Geek

Trusted

#3200364 27-Feb-2024 04:06

With Raspbian it is easy to run with a read-only filesystem, that can help against power loss in that the OS should still be intact.

You can have a 2nd drive or partition for data that changes, eg home directories etc.

sshd can fail to come up if it is told to listen to an address that is not available.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3200556 27-Feb-2024 16:00

roobarb:

With Raspbian it is easy to run with a read-only filesystem, that can help against power loss in that the OS should still be intact.

You can have a 2nd drive or partition for data that changes, eg home directories etc.

sshd can fail to come up if it is told to listen to an address that is not available.

Interesting option. How practical is a read only file system? I have PostgreSQL database, Home Assistant, Pi Hole, Nginx, and a couple of other things running and storing data in various folders on the disk, plus of course syslog and such.

It might be simpler to get a basic USB UPS like this one, which can be be had for about $60. 8800mah should last a Pi long enough to survive most outages, or shut it down.

nzkc

1572 posts

Uber Geek

#3200592 27-Feb-2024 18:53

Another option for you to consider, especially with regards to your PostgreSQL database, is to use Oracle's cloud. Reason I suggest it is they have a very generous free tier for compute VMs. Ive got 4 (2 x x86 and 2 x Ampere) running costing me $0/mth forever! The Ampere tier is interesting cause you get 4 x CPU and 24 GB of RAM to cut up how you like. I did 2 VMs as it happens.

You can then set up a VPN to it, or if you have a static IP some inbound rules from that IP address (or range if IPv6).

Probably wont be suitable for all your use cases. But the price is right ;-)

I use one of the Ampere VMs for a Minecraft server that my son wanted.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3200607 27-Feb-2024 20:30

Thanks for the suggestion @nzkc. I have a couple of Oracle free tier VMs, they work fine, but Oracle shut them down once a month because they think I'm not using them even though I am. I could upgrade to a paid subscription and then they would keep running on the free tier, but I don't want to give Oracle my credit card. The Oracle free tier is extremely generous. If you know Azure or AWS it's simple enough to do most things, but it's clearly nowhere near as developed.

I'd rather keep this workload running locally. PostgreSQL just keeps a week of Home Assistant history, temperatures and such, then it is purged. I could run everything on the Pi on Oracle, via a VPN, but I quite like having a little R.Pi here to tinker with.

nzkc

1572 posts

Uber Geek

#3200650 28-Feb-2024 00:27

Interesting... I have not hit that. I guess its because I update them regularly using Ansible. It'll look like I'm logging into them weekly (at least).

Agree Oracle Cloud is not as polished in many areas. I've used Terraform to stand them up so its just as easy as AWS or Azure ;-)

michaelmurfy

meow
13249 posts

Uber Geek

Moderator

ID Verified

Trusted

Lifetime subscriber

#3200662 28-Feb-2024 01:56

@timmmay I would actually recommend checking out smaller 1L PC's like the Lenovo Tiny or a 1L Dell Optiplex (like the 9020) and just simply run Proxmox on one. I've got a HA cluster of them and they're very low power and really capable plus can be had for cheap (I paid on average $150 each for mine).

Michael Murphy | https://murfy.nz
Referral Links: Quic Broadband (use R122101E7CV7Q for free setup)

Are you happy with what you get from Geekzone? Please consider supporting us by subscribing.
Opinions are my own and not the views of my employer.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3200687 28-Feb-2024 08:58

Given this is a long running service, I probably will move to something a bit more integrated at some point in the future, probably when the Pi4s are no longer doing the job.

outdoorsnz

674 posts

Ultimate Geek

#3200744 28-Feb-2024 10:20

timmmay:

I have a couple of Pi4's, maybe I'll do a cluster one day. For now I can revert back to the router DHCP very easily, the home assistant automations aren't doing anything essential, just visibility and a single control point and useful things.

The m.2 SATA SSD is a weird beast, hard to find and more expensive. I didn't look at the fine details before I ordered the case. The Pi5 seems like a good advance in this area, but the Pi4's work fine so no need to bother with a Pi5 yet. I'll probably stick with this case / weird SSD until it dies then upgrade in a few years.

PBTech accepted the SSD return, I'll order the transcend soon and hope that lasts a bit longer.

My Adata external Sata SSD that I'm using with Pi4 running OMV nas, has to run in quirks mode to get around driver / incompatibility issues. Basically the drive ran extremely slow with out the following setting in cmdline.txt

usb-storage.quirks=125f:a88a:u

More details. But unlikely your issue.

https://forums.raspberrypi.com/viewtopic.php?t=245931

1 | 2 | 3