Geekzone: technology news, blogs, forums
Guest
Welcome Guest.
You haven't logged in yet. If you don't have an account you can register now.
Filter this topic showing only the reply marked as answer View this topic in a long page with up to 500 replies per page Create new topic
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | ... | 21
3631 posts

Uber Geek
+1 received by user: 753


  Reply # 1123480 7-Sep-2014 08:03
Send private message

Do you have some basis for that comment?

176 posts

Master Geek
+1 received by user: 9


  Reply # 1123481 7-Sep-2014 08:13
One person supports this post
Send private message

quickymart: Do you have some basis for that comment?


Well currently Spark's DNS is severely hampering Spark's customers' ability to use their internet connection, and nothing since the issue arose as given me any indication that they know what they're doing.

For example, the "help" and "status" info is basically useless.

It would be probably better if they had a domain (hosted by somebody other than Spark) where they just had a message "Something's wrong and we're trying to fix it. Meanwhile, here are two ways to use alternative DNS, depending on your platform" (examples with screenshots, etc.) and an up-to-date status dashboard.

So the "basis" for my comment is (a) Spark is NZ's ISP and they still can't mitigate issues like this despite huge resources and (b) Sparks previous SMTP and email issues have shown clearly that they are second or third-rate as a company and basically are still coasting on the inertia of them being a huge, government-owned monopoly (which is good from a commercial sense, but not for customers).

Also, I have personal experience (and evidence) of a totally broken attitude towards customers, that persists to this day in Spark.

Your experiences?

See for example: https://twitter.com/BenRoss_AKL/status/508346736352325632

26469 posts

Uber Geek
+1 received by user: 6025

Moderator
Trusted
Biddle Corp
Lifetime subscriber

  Reply # 1123483 7-Sep-2014 08:23
3 people support this post
Send private message

pohutukawa:
The answer to all of this is that if Spark are NZ's largest ISP then they are also making the most money and should be providing for occurrences like this.


Such a comment shows a complete lack of understanding of DNS amplification and the fact it is the scariest thing to hit an ISP.

Exactly how should they be "providing for an occurrence like this" and how should such an attack be prevented? You've clearly got the answers so could be very rick this week.

At the end of the day there are going to be a lot of people trashing Spark and thinking they're experts on the matter. The scary thing is there is absolutely nothing stopping this exact same thing from happening to Spark again next week, or happening to any other ISP in NZ.

I predict a day in the not too distant future where the global internet goes down for a period of time. It's not a matter of if this will happen, but when this will happen.





1586 posts

Uber Geek
+1 received by user: 258

Subscriber

  Reply # 1123497 7-Sep-2014 09:06
Send private message

Does anyone know why this is only affecting Spark and not other ISPs in NZ?  From all the information around its a global attack thats basically lead to a DOS on their network, but I am wondering why nobody else seems to be affected like Spark?

12418 posts

Uber Geek
+1 received by user: 5822

Trusted

  Reply # 1123499 7-Sep-2014 09:07
Send private message

Lessons....

Don't look for porn.
Educate before you click.
Don't click.




Mike
Retired IT Manager. 
The views stated in my posts are my personal views and not that of any other organisation.

 

 It's our only home, lets clean it up then...

 

Take My Advice, Pull Down Your Pants And Slide On The Ice!

 

 


dwl

362 posts

Ultimate Geek
+1 received by user: 42


  Reply # 1123513 7-Sep-2014 09:37
3 people support this post
Send private message

I am sure hindsight and reviews will provide answers (maybe not that public) as to why Spark seemed more affected than others.  I totally agree with those posting about this being a complex issue and in many cases it can just be luck how resilient a service is to specific attacks.  I don't understand the details but I have heard other scary stories and vulnerabilities that may exist in those other networks that have not yet been exploited.  

I think we need to think of those within Spark who have had a very stressful 24+ hours.  From my location Smokeping DNS requests I started last night seemed to have found much better performance, noting that last night maybe only 1 in 5 requests were getting through (5 second timeout) in the case where a result was shown - other times no answer at all:



Ultimately customers have the right to complain and chose their provider.  What this forum hopefully shows is these are complex systems.  Workarounds like officially suggesting Google is a brave move.  Earlier this year such DNS requests may have resulted in pulling CDN lots of content from offshore rather than the local CDN and you might imagine the impact in international bandwidth (assuming the local caches don't deliver).  Google now seems to better understand Spark addressing but even now it might still point offshore for some local cached content or to a Spark CDN cluster at the other end of the country.  

Spark now has a lot of inappropriate DNS entries that will continue to work but not necessarily deliver the best outcome for the customer or their network.  I am quite happy to cut them some slack for this type of issue.  Sorry guys but I'm glad I wasn't on call for Spark starting from Friday.

10 posts

Wannabe Geek
+1 received by user: 9


  Reply # 1123521 7-Sep-2014 10:09
5 people support this post
Send private message

itxtme: Does anyone know why this is only affecting Spark and not other ISPs in NZ?  From all the information around its a global attack thats basically lead to a DOS on their network, but I am wondering why nobody else seems to be affected like Spark?


My guess is that Spark didn't have adequate filtering on their DNS servers, and other ISPs do.

With a DNS amplification attack, someone sends a DNS request to a DNS server as a UDP packet with a fake return IP address, this fake IP points back at the server you are trying to attack. This is so you can basically multiply the amount of traffic you are sending at the attacking server, because a DNS response is a lot bigger than a DNS request. Telecom says the attacks are going overseas, so the fake IP addresses will be for overseas sites. What should normally happen on a private DNS server is that any packets with IP addresses that are not controlled by them should be filtered out and disregarded. As a private DNS should only be sending responses to its users, not to addresses overseas.
This filtering should happen at a very early stage, and not even let the packet hit the actual DNS server. But the tricky part is that because DNS uses UDP it s very very hard to work out where the traffic actually came from, so you basically just have to absorb it on the server. Which means that if you were getting a really really large amount it might overload your filter, which could stop it working for everyone. Spark is saying that they have a lot of new fiber customers who have a massive amount of upstream bandwidth, this means these customers could send an enormous amount of requests compared to someone on regular broadband. So this leads me to think that for Spark the amount of requests they were getting overloaded their filtering system, where as for other ISPs it didn't. So basically it is in part Sparks fault for not having the capacity to deal with these kind of problems, which as been amplified by the increase in viruses from people taking advantage of nude celebrity leak.  Or Spark didn't have filtering at all. So last night they basically increased the capacity, or added filtering, to a level where they can handle requests again.


810 posts

Ultimate Geek
+1 received by user: 191
Inactive user


  Reply # 1123536 7-Sep-2014 10:34
Send private message

dwl: ... Workarounds like officially suggesting Google is a brave move.

Except they aren't, service status still suggests to use the dns servers that are the ones actually having issues...

303 posts

Ultimate Geek
+1 received by user: 56

Subscriber

  Reply # 1123556 7-Sep-2014 10:41
Send private message

As a not-very-geeky person, I wondered:  Most people's DNS server addresses are set on Auto (I've just put mine back to that from Google), but is this "auto" set by the modem or by Spark servers? If the latter, then why could Spark not set everyone's machines to use Google or the business DNS server addresses? In the interim, at least?  Just asking...




gml


755 posts

Ultimate Geek
+1 received by user: 224


  Reply # 1123562 7-Sep-2014 10:46
2 people support this post
Send private message

They are set by Spark, but if they changed them, the malicious traffic would then be directed at these other DNS servers.  I'm assuming - dangerous I know - that the business DNS server wouldn't have fared any better.  Would have been interesting to see Google's response though

'That VDSL Cat'
7920 posts

Uber Geek
+1 received by user: 1619

Trusted
Spark
Subscriber

  Reply # 1123566 7-Sep-2014 10:53
Send private message

GregV: They are set by Spark, but if they changed them, the malicious traffic would then be directed at these other DNS servers.  I'm assuming - dangerous I know - that the business DNS server wouldn't have fared any better.  Would have been interesting to see Google's response though


Exactly this.

Ild say Google are hit on a daily basis, and probably have some form of a filter designed specially to identify and block dns attacks.

OnlyJoe:
 which as been amplified by the increase in viruses from people taking advantage of nude celebrity leak.  


what a novel distribution method that would be!




#include <std_disclaimer>

 

Any comments made are personal opinion and do not reflect directly on the position my current or past employers may have.


810 posts

Ultimate Geek
+1 received by user: 191
Inactive user


  Reply # 1123570 7-Sep-2014 11:17
Send private message

gujal: They also have a set of Business customer DNS Servers. These are quite nice to use rather than Google's DNS Servers which are in US and have to go over Southern Cross cable for something simple as DNS lookup
ns1.netgate.net.nz 202.37.245.17
ns2.netgate.net.nz 202.37.245.20

Stopped working for me about 11am, although I can still ping them. Back to OpenDNS then...

310 posts

Ultimate Geek
+1 received by user: 21


  Reply # 1123571 7-Sep-2014 11:23
Send private message

Kirdog:  Very slow and laggy ingress play on Spark mobile data today and yesterday evening. :(
Just out of Curiosity: Why Voda customers are not affected by that? ( http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=11320100 < some very strange excuse detected here, as my opinion) 

Also someone mentioned that its "paid attack", it can be/why not...


Yes I agree, I smell a rat.
Why only Spark customers?




Now on 2talk Network and it's better.



26469 posts

Uber Geek
+1 received by user: 6025

Moderator
Trusted
Biddle Corp
Lifetime subscriber

  Reply # 1123573 7-Sep-2014 11:25
One person supports this post
Send private message

techmeister:
Kirdog:  Very slow and laggy ingress play on Spark mobile data today and yesterday evening. :(
Just out of Curiosity: Why Voda customers are not affected by that? ( http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=11320100 < some very strange excuse detected here, as my opinion) 

Also someone mentioned that its "paid attack", it can be/why not...


Yes I agree, I smell a rat.
Why only Spark customers?


How about the sample fact they have over 50% of NZers as broadband customers, hence they're more than twice the size of the next biggest network? This makes something on this scale so much harder to contain.





7 posts

Wannabe Geek
+1 received by user: 1


  Reply # 1123576 7-Sep-2014 11:32
One person supports this post
Send private message

Something doesn't quite make sense to me.  If it's a DNS amplification attack on an overseas target by Spark's customers, using Spark's recursive resolvers (the ones that fell over), that implies that packets sent by customers with seriously spoofed source IP addresses (not even within Spark's address range) can reach the DNS servers.  I don't have an easy way to test it at the moment, but I thought most ISPs blocked source IP address spoofing pretty close to the customer?

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | ... | 21
Filter this topic showing only the reply marked as answer View this topic in a long page with up to 500 replies per page Create new topic

Twitter »

Follow us to receive Twitter updates when new discussions are posted in our forums:



Follow us to receive Twitter updates when news items and blogs are posted in our frontpage:



Follow us to receive Twitter updates when tech item prices are listed in our price comparison site:





News »

N4L helping TAKA Trust bridge the digital divide for Lower Hutt students
Posted 18-Jun-2018 13:08


Winners Announced for 2018 CIO Awards
Posted 18-Jun-2018 13:03


Logitech Rally sets new standard for USB-connected video conference cameras
Posted 18-Jun-2018 09:27


Russell Stanners steps down as Vodafone NZ CEO
Posted 12-Jun-2018 09:13


Intergen recognised as 2018 Microsoft Country Partner of the Year for New Zealand
Posted 12-Jun-2018 08:00


Finalists Announced For Microsoft NZ Partner Awards
Posted 6-Jun-2018 15:12


Vocus Group and Vodafone announce joint venture to accelerate fibre innovation
Posted 5-Jun-2018 10:52


Kogan.com to launch Kogan Mobile in New Zealand
Posted 4-Jun-2018 14:34


Enable doubles fibre broadband speeds for its most popular wholesale service in Christchurch
Posted 2-Jun-2018 20:07


All or Nothing: New Zealand All Blacks arrives on Amazon Prime Video
Posted 2-Jun-2018 16:21


Innovation Grant, High Tech Awards and new USA office for Kiwi tech company SwipedOn
Posted 1-Jun-2018 20:54


Commerce Commission warns Apple for misleading consumers about their rights
Posted 30-May-2018 13:15


IBM leads Call for Code to use cloud, data, AI, blockchain for natural disaster relief
Posted 25-May-2018 14:12


New FUJIFILM X-T100 aims to do better job than smartphones
Posted 24-May-2018 20:17


Stuff takes 100% ownership of Stuff Fibre
Posted 24-May-2018 19:41



Geekzone Live »

Try automatic live updates from Geekzone directly in your browser, without refreshing the page, with Geekzone Live now.



Are you subscribed to our RSS feed? You can download the latest headlines and summaries from our stories directly to your computer or smartphone by using a feed reader.

Alternatively, you can receive a daily email with Geekzone updates.