Waikato DHB Conficker Outbreak

Forums › IT Pro and developers › Waikato DHB Conficker Outbreak

skaffen

14 posts

Geek

#54572 18-Dec-2009 11:33

Thought I'd start a discussion on the Waikato DHB getting hammered by Conflicker yesterday.

I hear from an inside source that they didn't use WSUS as they thought the risk of patches breaking apps was too great. Man, that's a hard way to learn that lesson. The MS patch has only been out since Oct 2008.

The spokesperson is talking in the press about installing a better password regime, although they have a good one already. It can't be very good if conflicker is able to guess passwords that meet the existing standard.

http://www.sophos.com/blogs/gc/g/2009/01/16/passwords-conficker-worm/

Someone deserves an arse-kicking over this!

1 | 2

BDFL - Memuneh
80646 posts

Uber Geek
+1 received by user: 41029

Administrator

ID Verified

Trusted

Geekzone

Lifetime subscriber

#283701 18-Dec-2009 11:36

And someone on my Twitter said they work hard there. I am sorry, but not deploying patches and not enforcing policies is not hard enough in my books.

Kick them hard.

Referral links: Quic Broadband (free setup code: R587125ERQ6VE) | Samsung | AliExpress | Wise | Sharesies

Support Geekzone by subscribing (browse ads-free), or making a one-off or recurring donation through PressPatron.

browned

636 posts

Ultimate Geek
+1 received by user: 36

#283706 18-Dec-2009 11:50

Something a bit more powerful for patching might be a good idea. but in saying that WSUS does allow for some patch removal. Landesk do a great patching tool.

Situation
1. Ok rollout patches after testing as much as you can.
2. Oh no some people are reporting app crashes and issues.
3. Check issues on PC, remove patches confirm which patch causes the issue.
4. Back to Landesk/Wsus....force patch removal from client PC's with this app installed.

If you are going WSUS then just approve critical patches that allow removal. My main issues with WSUS are having to download Itanium patches, I would also like to be able to decline an old service pack and have all patches for that service pack declined as well. A lot of wasted bandwidth with WSUS.

They will be working hard now. Last time I was involved in a virus outbreak it was 2004 and we didn't get much sleep in 72 hours.

I would expect some kicking to be involved....I would also not be surprised if some external companies got chatting with the DHB about supplying management services instead of using inhouse staff and kit.

Home Server: AMD Threadripper 1950X, 64GB, 56TB HDD, Define R6 Case, 10GbE, ESXi 6.7, UNRAID, NextPVR, Emby Server, Plex Server.
Lounge Media Center: NVIDIA Shield TV 16GB: Kodi18 with Titan MOD, Emby.
Kids Media Center: NVIDIA Shield TV 16GB: Kodi18 with Titan MOD, Emby.
Main PC: Ryzen 7 2700, 16GB RAM, RX 570, 2 x 24"

skaffen

14 posts

Geek

#283752 18-Dec-2009 13:39

Yeah, WSUS is a bit inflexible. But not using a free tool to distribute a year old patch when the malware targeting that vulnerability is so prevalent in the wild, that's insane.

We're only using WSUS at this stage, and pushing patches immediately (with the exception of SQL Server updates) has only caused one minor problem (SAP client not playing well with IE7 after an update) in the last three years. We pretty much followed the procedure you described, rolled back with WSUS, waited for the fix, re-rolled two days later. I think the risk of zero day malware is far greater than the risk of breaking apps.

I can imagine the pain the WDHB techies are going through at the moment. The only fix is to shut all Windows devices off, boot each in safe mode and clean and patch. On the brighter side, Conflicker is a good way to find unsecured network shares!

I wonder how much accountability we'll see out of this - who's head will roll, if anyone's...

We're investigating Lumension Patchlink at the moment, it'll do all patching - Java, Adobe, etc, not just Windows.

Ragnor

8279 posts

Uber Geek
+1 received by user: 585

Trusted

#283757 18-Dec-2009 13:45

Rolling out all critical security patches and fire fighting individual application problems is much lower risk that not patching.

Basically they are retarded!

amanzi

Amanzi
1354 posts

Uber Geek
+1 received by user: 331

ID Verified

Trusted

Lifetime subscriber

#283793 18-Dec-2009 15:06

They aren't the only organisation that doesn't roll out patches due to fear of breaking applications. I know of several others too... (I'm not saying that it's right, but I'm not surprised by it.)

jaymz

1136 posts

Uber Geek
+1 received by user: 76

#283867 18-Dec-2009 19:04

amanzi: They aren't the only organisation that doesn't roll out patches due to fear of breaking applications. I know of several others too... (I'm not saying that it's right, but I'm not surprised by it.)

Agreed, If they have been bitten before with patches causing issues with software then I can completely understand them being a little gun-shy about patches. But that being said, they should have employed a patch rollout plan that involved imaging one workstation, apply the patch(es), then test.

Just a question though, is it the conficker virus that spread earlier in the year? I have seen 3-4 different spellings of it so far:

Stuff.co.nz = conficka
Here = conflicker
NZhearld = conflicka

Which one is it? or is it a variant of the conficker virus (but i was sure they were a,b,e variants only)

What this has done is bring security back into the spotlight again.

Dyson

Shop now for Dyson appliances (affiliate link).

richms

29098 posts

Uber Geek
+1 received by user: 10207

Trusted

Lifetime subscriber

#283885 18-Dec-2009 20:08

I lot of custom software is plainly retarded. I know of a place that was still using an unpactched really old IE because they needed the username:password@site logins to work for some braindead half-assed client that used that to authenticate to an external server, and that was only last year that they were still using it.

Alarmingly high number of IE6 clients from corporate IPs hit a friends website too.

Richard rich.ms

skaffen

14 posts

Geek

#283894 18-Dec-2009 20:21

I can understand that breaking apps might make you test thoroughly before deploying, but there's no excuse for not having rolled out a patch that's over a year old and addresses a critical vulnerability that a prolific worm exploits.

My understanding is this is conflicker/conficker e

Bung

6733 posts

Uber Geek
+1 received by user: 2926

Subscriber

#283919 18-Dec-2009 23:07

skaffen: I can understand that breaking apps might make you test thoroughly before deploying, but there's no excuse for not having rolled out a patch that's over a year old and addresses a critical vulnerability that a prolific worm exploits.

Do you know that was the case?

There was a spokesman on Nat Radio Morning Report who said that contrary to some claims they were fully patched. The DHB had thought that they had a similar attack previously and other DHB's were also having trouble. They were also working on segmenting their network to try to restrict this type of attack. He would be brave to say that if the patches weren't in place.

skaffen

14 posts

Geek

#283924 18-Dec-2009 23:59

I heard it from someone who's worked onsite there. Can't say any more for obvious reasons...

I can't see how conflicker could have cleaned them out if they had kb958644 in place, unless they had really poor password policies, and I heard another spokesperson saying they had good password enforcement. Has to be one or the other.

exportgoldman

1202 posts

Uber Geek
+1 received by user: 3

Trusted

#283925 19-Dec-2009 00:05

This is probably the patch they were missing

http://www.microsoft.com/technet/security/bulletin/MS08-067.mspx Came out October 2008

This is where Microsoft is ahead of the security game, people complain about Windows Patches, but the reason there are so many is because each patch only fixes a few of the same type of vulnerabilities and doesn't add or change existing features or change how the product works.

I have not found any applications which vendors now actively reccomend against patching.

On the Apple platform it's another matter, they bundle 100 odd security updates into a big ball with no documentation and a whole lot of new features and throw it out the door.

We use Kaseya to manage our clients patches, and when bringing clients onboard they usually sit at 60-80 missing patches which takes a while to get down to less than 5.

There really is no excuse to not patch nowdays, patches are all uninstallable and there have not been problems with patches for a very long time.

Shame on the geeks!

People still don't understand that Antivirus is the salvalation. If your not patching Antivirus is not going to save you. It's like hiring a security guard for your mansion and leaving all the doors and windows open. The security guard will stop people AFTER they are in your longue, but why not just close all the doors and Windows.

Tyler - Parnell Geek - iPhone 3G - Lenovo X301 - Kaseya - Great Western Steak House, these are some of my favourite things.

Apple TV

Stream your favourite shows now on Apple TV (affiliate link).

exportgoldman

1202 posts

Uber Geek
+1 received by user: 3

Trusted

#283926 19-Dec-2009 00:10

jaymz:
amanzi: They aren't the only organisation that doesn't roll out patches due to fear of breaking applications. I know of several others too... (I'm not saying that it's right, but I'm not surprised by it.)

Agreed, If they have been bitten before with patches causing issues with software then I can completely understand them being a little gun-shy about patches. But that being said, they should have employed a patch rollout plan that involved imaging one workstation, apply the patch(es), then test.
\

Or pick one or two computers from each department, flag them at the helpdesk for beta testing to flag up any problems and roll out the patches to them a week before. If nothing breaks progressively roll out the patches in groups over the month.

This isn't rocket science kids. Thats they scary part.

Tyler - Parnell Geek - iPhone 3G - Lenovo X301 - Kaseya - Great Western Steak House, these are some of my favourite things.

mikeymike76

14 posts

Geek
+1 received by user: 1

#284350 21-Dec-2009 12:47

yeah a lot of the problems lie with trying to get trust from those deptpartments though.
Inside of the DHB if that computer crashes at the wrong time (how do you know when the right time to push out the patch is?), then potentially people die, and they don't want to risk that.

Additionally some applications are only installed on one computer, so you can't just test it on one computer for a week then role it out.

In other words, give these guys a bit of a break, it might not be there fault it could be the departments themselves not letting the patches being apllied.

amanzi

Amanzi
1354 posts

Uber Geek
+1 received by user: 331

ID Verified

Trusted

Lifetime subscriber

#284354 21-Dec-2009 12:57

mikeymike76: Inside of the DHB if that computer crashes at the wrong time (how do you know when the right time to push out the patch is?), then potentially people die, and they don't want to risk that.

That's not a good reason, as worse can happen if an unpatched network of computers gets hit by a virus. If the risks of a patch crashing a pc is that someone may die, that doesn't mean the patch shouldn't be pushed out, it just means that more testing has to be done prior to the patch being rolled out. (plus a good backout plan in case it all goes pear-shaped.)

exportgoldman

1202 posts

Uber Geek
+1 received by user: 3

Trusted

#284358 21-Dec-2009 13:02

mikeymike76: yeah a lot of the problems lie with trying to get trust from those deptpartments though.
Inside of the DHB if that computer crashes at the wrong time (how do you know when the right time to push out the patch is?), then potentially people die, and they don't want to risk that.

Additionally some applications are only installed on one computer, so you can't just test it on one computer for a week then role it out.

In other words, give these guys a bit of a break, it might not be there fault it could be the departments themselves not letting the patches being apllied.

How's that trust thing working out now?

My advise is to not let non-technical people make technical decisions which if what you describe is true is what is happening here. They often choose wrong and taking hospitals offline for a a day.

I'm suprised to be honest that there are PC's that run life critical software on one PC only, I would have throught that with 3000 machines you would put it on a few machines in a department for fault tollerance.

As to when to roll out patches, I would check with the department heads, but I assume Friday night would be out, check when department patient load is lowest. Roll out in batches and have test machines in each department that the department heads know about that are first to be patched and they know may break. Stop patching if these machines break and leave a good gap of time inbetween for a weekly cycle to happen (e.g. Payroll, Friday night peak load etc.)

The assumption is that it's better to have one patch a year break 15 machines and have to swap to other machines in the department for those tasks than to go down hard on all 3000 machines for 24+ hours.

Oh, and for all life critical tasks, there should be paper based backup systems. Fire and Police have them, ambo's and hospitals should as well. The police and Fire had to resort to them 6 months ago when comms went down.

Tyler - Parnell Geek - iPhone 3G - Lenovo X301 - Kaseya - Great Western Steak House, these are some of my favourite things.

1 | 2