Suggestions to fix odd Linux Raspberry Pi 4 problem - system up ssh down only some services running

Forums › Linux › Suggestions to fix odd Linux Raspberry Pi 4 problem - system up ssh down only some services running

1 | 2 | 3

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3200746 28-Feb-2024 10:33

Interesting. I'll look into that in more detail this evening.

The Adata must have developed a fault. I put it into the PC, tried to delete the partitions, it hang and timed out - it was fine when it was new. The replacement Transcend works fine in the same enclosure.

The Argon 40 case is quite convenient, but keeps everything at around 40 - 42 degrees idle. I think next time I get hardware for this it will be something better designed for cooling and running 24/7.

boosacnoodle

963 posts

Ultimate Geek

#3200758 28-Feb-2024 11:02

My process for Pi's till now has been:

Convert everything to Docker Compose files
Wipe SSD and install latest Raspberry Pi OS
Install Docker
Create Docker Compose for management that includes Portainer, watchdog and auto-heal images
Install everything through Docker Compose

Then when something goes poof, you just install the Compose again.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3200763 28-Feb-2024 11:28

Yeah I have everything done with docker-compose, but HA / PiHole have plenty of dependent files. There's some basic OS setup and tweaking, backups setup, I have it in a text file so I can run through it pretty easily. Then I restore from the old pi to the new pi, tweak the config, and start. It didn't take all that long for me to set up the latest pi, but it took a couple of days for DHCP to issue new IPs to different devices so HA could find them, the router isn't configured quite the same.

Tinkerisk

4227 posts

Uber Geek

#3201018 28-Feb-2024 19:39

timmmay:

The Adata must have developed a fault. I put it into the PC, tried to delete the partitions, it hang and timed out - it was fine when it was new. The replacement Transcend works fine in the same enclosure.

What is underestimated by many is the minimal TBW that the cheaper SSDs allow to die prematurely through write operations of the virtualisation environment. Home Assistant also logs quite a lot.

- NET: FTTH, OPNsense, 10G backbone, GWN APs, ipPBX
- SRV: 12 RU HA server cluster, 0.1 PB storage on premise
- IoT: thread, zigbee, tasmota, BidCoS, LoRa, WX suite, IR
- 3D: two 3D printers, 3D scanner, CNC router, laser cutter

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3201024 28-Feb-2024 19:49

Tinkerisk:

What is underestimated by many is the minimal TBW that the cheaper SSDs allow to die prematurely through write operations of the virtualisation environment. Home Assistant also logs quite a lot.

My new SSD is rated for just under a petabyte written. My last SSD was rated for 1000TBW, but probably didn't even make it to 1TB.

I've turned HA logs way down, it barely logs anything. It does write to the database quite a bit, I've tuned that to reduce it storing data I don't want to keep, and tweaked the PostgreSQL settings a bit as well. Logs aren't much.

Windows 11 though, it logs a lot of data even when pretty much idle. Chrome logs heaps when it's idle, even more when you're actively using it.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3201033 28-Feb-2024 20:08

outdoorsnz:

My Adata external Sata SSD that I'm using with Pi4 running OMV nas, has to run in quirks mode to get around driver / incompatibility issues. Basically the drive ran extremely slow with out the following setting in cmdline.txt

usb-storage.quirks=125f:a88a:u

More details. But unlikely your issue.

https://forums.raspberrypi.com/viewtopic.php?t=245931

Before Quirks

hdparm -tT --direct /dev/sda2

/dev/sda2:
Timing O_DIRECT cached reads: 328 MB in 2.01 seconds = 163.16 MB/sec
Timing O_DIRECT disk reads: 880 MB in 3.00 seconds = 292.94 MB/sec

After Quirks

/dev/sda2:
Timing O_DIRECT cached reads: 304 MB in 2.01 seconds = 151.20 MB/sec
Timing O_DIRECT disk reads: 760 MB in 3.00 seconds = 253.20 MB/sec

@outdoorsnz quirks actually made it slower! The disk is meant to do up to 560GB/sec read and write, but through a USB adapter a bit over half that isn't bad.

outdoorsnz

674 posts

Ultimate Geek

#3201568 29-Feb-2024 13:16

You'd only need quirks mode if drive was running at a crawl speed as in my case.

My Pi 4 8gb desktop has input/output faults. If I slightly bump USB connection I have to pull the power cable. Tried other cables. But now same on HDMI and power cables which causes low voltage message. I have to turn off and re-plug in HDMI / power to fix. Annoying. Do you think this would apply in your situation (cable / connection faults)?

I'm a diehard pi fan, so will soon get a Pi5 soon.

Lenovo

Shop now for Lenovo laptops and other devices (affiliate link).

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3201694 29-Feb-2024 16:57

outdoorsnz:

My Pi 4 8gb desktop has input/output faults. If I slightly bump USB connection I have to pull the power cable. Tried other cables. But now same on HDMI and power cables which causes low voltage message. I have to turn off and re-plug in HDMI / power to fix. Annoying. Do you think this would apply in your situation (cable / connection faults)?

I don't have any connection problems with either of my Pi4's. I did have to unplug then plug in the network cable while I was setting one pi up, but that was a DHCP issue. I confirmed the issue was a faulty SSD, which manifested during a reboot.

timmmay

20580 posts

Uber Geek

Trusted

Lifetime subscriber

#3233110 21-May-2024 15:52

I think I worked out why the SSD failed - excessive heat. I checked the SMART data for the new SSD and it was 85 degrees C. The ADATA is spec'd for operation up to 70c, so it did pretty well lasting as long as it did. The Transcend is rated for that temperature, but it will have affected its lifespan.

I tried a few things to reduce the temperature

85c - m.2 inside case, no fan
71c - m.2 inside case, constant 50% fan
66c - m.2 with case open, using an extra USB cable
64c - m.2 with case open with m.2 label removed
45c - m.2 case open, passive cooler fitted (Jeyi Battleship, from memory)

Currently I have the two parts of the case separate (SSD enclosure part, PI part) with a USB cable between them, the Pi part down so the heat rises. I'll probably need to put the SSD with its USB adapter into a new case and put the base back on the Pi so it's physically protected. That's ugly and annoying.

1 | 2 | 3