Have Consumer Magazine lost it?

By Steve Biddle, in , posted: 24-Dec-2009 12:43

I've been a Consumer magazine reader on and off again for many years now. What I've noticed over the past few years however has been a complete change in direction of the organisation, a change for the worse. Articles have been poorly researched and written and it seems that as an organisation they're more focussed on getting their name out there in the media than they are on their core business of delivering unbiased, factual reviews of consumer goods and services.

This morning Consumer released a press release detailing the results of their yearly member survey of ISP's in New Zealand. This survey, taken by Consumer members ranks a list of the best and worst ISP's in New Zealand. This survey is available in the January/February version of Consumer and online on their website if you are a subscriber.

This survey does not rely on any technical testing, plan comparisions or ask the end user if they really know what they're talking about. It doesn't ask if they have master filter, have a shitty modem that will only sync at ADSL speeds on an ADSL2+ line or whether there internet is "slow" because they're downloading torrents on a plan that's clearly sold as being traffic managed 24 hours per day. The survey is also statistically flawed because it does not represent a true sample - it is a voluntary response that is also flawed because small ISP's with fewer responses can easily have skewed ratings. 

After browsing through this what really caught my attention was the section on mobile data

Mobile broadband

Nearly a quarter of survey respondents used their mobile phone for internet access.

New entrant 2degrees was the best for speed and connection reliability while Vodafone rated the worst on disconnections and dropouts, slower-than-expected speeds, and unexpected charges for excess usage.

I'm glad I wasn't eating Weetbix at the time because I would have choked on them. This claim by Consumer would have to be one of the most extravagant I've read anywhere for a long time.

Why? Because there is absolutely no way it can be true. I'd love to see Consumer magazine show some data that backs up this very bold claim that is presumably based solely on random end user comments rather than any actual performance testing.

2degrees have their own GSM mobile network in Auckland, Wellington, Christchurch and Queenstown. Their network also has EDGE deployed on top, this gives a maximum theoretical data speed of approximately 240kbps. If you're a 2degrees customer and are within these geographic boundaries you'll connect to the 2degrees network. Outside these areas your phone will connect to the Vodafone's GSM network and you will roam on this using GPRS that delivers a maximum speed of approximately 48kbps.

Both Vodafone and Telecom have nationwide WCDMA 3G networks. Both of these networks will deliver up to 7.2Mbps at present, both networks also have trial HSPA+ sites delivering real life speeds upwards of 16Mbps. Telecom's CDMA network is still in use by a large majority of their customers but is in effect is obsolete and typically no new connections are being made. Likewise Vodafone have more 3G customers on their network than GSM and the majority of handsets sold these days support 3G so for this comparision I'm solely comparing the two WCDMA networks.

I've spent the past couple of weeks doing some pretty extensive testing of both Vodafone and Telecom's XT network around the lower North Island and can conclusively say that without a doubt that there is no way the EDGE network deployed by 2degrees can come anywhere close to the speeds delivered on the Vodafone or Telecom XT networks. I did encounter some retainability issues on the XT network which now appear to be resolved but overall both networks perform extremely well. Across the XT network it's not uncommon to obtain speeds upwards of 6Mbps and with average speeds in the 2Mbps - 4Mbps region. On Vodafone speeds in the 1.5Mbps - 3Mbps range are fairly typical and with one exception near a crowded mall I've never received average speeds of under 1Mbps on either Vodafone or Telecom's XT network.

I've also succesfully tested a 1.3Mbps video stream to a laptop while mobile on the XT network and saw exceptionally low packet loss, even during handovers. Retainability and connectability (the ability to establish and hold a data connection) are very good on both networks. If that's not delivering a significantly better speed and connection reliability experience than the EDGE network deployed by 2dgrees then I don't know what is. 2degrees don't have a "bad" network, it's actually very good and in my testing there are no issues and speeds close to 200kbps are easily achievable. It's just simply no match for Vodafone or Telecom's XT network because it's older GSM technology that is simply inferior to the WCDMA technology that Vodafone and Telecom XT use. 2degrees will be launching WCDMA 3G services aross their network in early 2010.

In terms of charging a casual user on Vodafone or Telecom's XT network will pay more than a 2degrees customer for data "overuse" charges - but the statement itself is contentious. 2degrees do not offer any data plans, all data is priced at 50c per MB. On Vodafone or Telecom a casual user will pay $1 per day for up to 10MB of data usage, after that data is charged at $1 per MB but very few customers actually pay this. If you're a regular data user you'll have a data plan (which 2degrees don't offer) offering you data for 10c per MB or less.

So the challenge is out there for you Consumer - exactly what basis do you base these bold claims on and do you have any statistics to back them up?

Other related posts:
CCTV exposed. Why understanding network security is so important.
Anker make some of the best USB chargers and powerbanks available. Now you can get their products shipped directly to New Zealand
United Airlines pulls out of New Zealand for Southern Hemisphere Winter – AKL/SFO becomes seasonal.

Comment by n4, on 24-Dec-2009 13:36

I think its all about expectations. VF and XT advertise 'up to 7.2Mbps' but rarely deliver, automatically setting themselves up to fail in terms of expectations. 2degrees does not pretend to be offering the latest and greatest data speeds, but in point of fact for many 2G users coming over from VF GPRS (and probably Telecom CDMA) the EDGE speeds on 2degrees are a pleasant surprise. Unlike typical geekzoners, most consumers are not running around doing speed tests with HSDPA handsets. Especially 2degrees consumers. The same logic would apply to data bundles and charges.

Comment by nate, on 24-Dec-2009 14:37

They should've also mentioned how expensive data is on 2Degrees.  It's all nice saying it's supposedly quick when no-one can afford to pay for it.

Comment by Ragnor, on 25-Dec-2009 01:44

Their ISP survery has always been dubious, it's a fundamentally flawed concept.

They should get in touch with the guy(s) behind http://bgrade.bage.net.nz/results_summary.php

Comment by insane, on 25-Dec-2009 17:29

The worse part of these surveys is that average Joe takes the results for granted and then acts on them, changing ISPs or telling friends and living in ignorance of the truth. I believe they have a responsibility to provide objective results as customers of consumer.org are paying for this. I'm appalled that these writer still believes the ISPs are responsible for individual customer connection line/disconnection issue. *sigh* @Ragnor, totally agree those results from that lite are more 'real-world' than asking consumers for their perception.

Comment by vexxxboy, on 26-Dec-2009 22:09

how can you take them seriously when they dont even mention Snap in there isp ratings, from winning isp of the year to not been worth a mention, great research Consumer.

Comment by ajw, on 28-Dec-2009 17:00

Perhaps UMTS for 2 degrees next year. As noted in this recent job ad looks like they have the money to rollout a nationwide network pronto.

National Site Design Manager Full autonomy Exciting Start-up Telco!! Awesome opportunity, excellent environment! As the National Site Design Manager you will set up the engineering and design standards for the project from scratch, responsible for provisioning, employing & managing of direct reports & functions of radio planning, property/leasing, town planning & site build project managers. This position will be to direct the Civil & Structural Engineering support for the national roll out of 400+ sites per year min and be ready to scale up.  You will set delivery targets & progress, establish management systems and process that will ensure your team delivers.You will produce details of specs and outline designs, recommending mods in the light of prototype test results. A very hands on, dedicated, professional senior manger position!! To qualify for this amazing opportunity you willhave held a senior management position in a similar roleHave 5 yrs minimum experience working on telecommunications related projectsEngineering DegreeExperience managing a team to deliver a high work load through-put under time & cost pressuresHave Commercial business acumen with clear & concise communications skillsYou will be organized and disciplined with the ability to set targets, focus and budget resource to meet delivery targets Don't hesitate, apply online now and send your CV and cover letter  To discuss this opportunity, please contact Phone
Email: Please click the 'Apply Now' button below.  

Comment by cyril7, on 19-Jan-2010 14:21

Over the years we used to get Consumer Mag I was forever appalled at the garbage they seemed to reliably pour out on subjects that I had a close knowledge of, this includes most things IT, Broadcast recepetion and equipment, and AV matters. Infact I cannot remember one report relating to these subjects that I did not have a "choke on the weetbix" experience, hence I no longer subscribe and take any report they release on any subject with suspicsion.


sbiddle's profile

Steve Biddle
New Zealand

I'm an engineer who loves building solutions to solve problems.

I also love sharing my views and analysis of the tech world on this blog, along with the odd story about aviation and the travel industry.

My interests and skillset include:

*VoIP (Voice over IP). I work with various brands of hardware and PBX's on a daily basis
  -Asterisk (incl PiaF, FreePBX, Elastix)

  -xDSL deployments

*Structured cabling
  -Home/office cabling
  -Phone & Data

*Computer networking
  -Mikrotik hardware
  -WAN/LAN solutions

*Wireless solutions
  -Motel/Hotel hotspot deployments
  -Outdoor wireless deployments, both small and large scale
  -Temporary wireless deployments
*CCTV solutions
  -Analogue and IP

I'm an #avgeek who loves to travel the world (preferably in seat 1A) and stay in nice hotels.

+My views do no represent my employer. I'm sure they'll be happy to give their own if you ask them.

You can contact me here or by email at stevenbiddle@gmail.com