pppoe connection issues - Timeout waiting for PADO packets

Forums › New Zealand Broadband › pppoe connection issues - Timeout waiting for PADO packets

gareth41

742 posts

Ultimate Geek

#112834 22-Dec-2012 21:45

Im with KC Internet, they are using pppoe on the ONT. It works fine no problems at all plugged into my pc's lan port with a pppoe broadband connection setup on win7, it connects everytime without fail. However my tp-link tl-wr1043nd takes a long time to connect with lots of timeouts (Timeout waiting for PADO packets). It never used to have this problem, just started in last few days when I noticed the internet was down and rebooted router. I tried another linksys router, same problem. If I leave the router for an extended period of time and come back it has usually connected, but takes a while.

Heres some screen shots of log:

gareth41

742 posts

Ultimate Geek

#741346 7-Jan-2013 23:39

It seems no one knows anything about this? Have logged a fault with my ISP so will see what happens.

PenultimateHop

637 posts

Ultimate Geek

Trusted

#741360 8-Jan-2013 05:33

This is something that realistically only your ISP can help troubleshoot, and only if they care to look into it in detail. It could be your router is doing something odd causing an incompatibility between the ISP's BNG and your router in the PPPoE stack (this happens far too often for a 13 year old protocol), a misconfiguration on the RG, UFB network, or ISP network, or a broken router.

What happens if you use a different (non-TPLink) router? Oops - I see you did already. Odd to impact both. Do you have the routers set-up correctly (e.g. tagging on the WAN port, if required)?

gareth41

742 posts

Ultimate Geek

#741765 8-Jan-2013 16:56

PenultimateHop: This is something that realistically only your ISP can help troubleshoot, and only if they care to look into it in detail. It could be your router is doing something odd causing an incompatibility between the ISP's BNG and your router in the PPPoE stack (this happens far too often for a 13 year old protocol), a misconfiguration on the RG, UFB network, or ISP network, or a broken router.

What happens if you use a different (non-TPLink) router? Oops - I see you did already. Odd to impact both. Do you have the routers set-up correctly (e.g. tagging on the WAN port, if required)?

ISP said vlan tagging was off, I havent tried with it on, i'll try it and see.

Also in the last 24 hours it hasnt reconnected at all, sometimes it does if I leave it for a few hours, however ISP is investigating. Ive also tried with a linux box (ubuntu server 12.04.1 LTS) and get the exact same error. But what I just dont get, is why does a pc running windows work? I setup a pppoe connection on my pc (win7 pro) and it connects every single time without any problems, not a single error at all.

bender

220 posts

Master Geek

#742045 9-Jan-2013 08:23

I've seen one similar instance of this before - the ONT for whatever reason just wasn't learning the router's MAC address. It was showing up on the handover circuit but no return traffic was getting back to the router.

Could connect with a laptop plugged in directly etc.

In the end leaving the ONT off for a few minutes resolved it, power cycling it (turning back on immediately) did nothing.

Try turning your ONT off for 10 minutes or so and see if this resolves it, if it does please post back as I suspect it may be a firmware bug in the ONT that needs to be fixed.

gareth41

742 posts

Ultimate Geek

#742437 9-Jan-2013 18:25

bender: I've seen one similar instance of this before - the ONT for whatever reason just wasn't learning the router's MAC address. It was showing up on the handover circuit but no return traffic was getting back to the router.

Could connect with a laptop plugged in directly etc.

In the end leaving the ONT off for a few minutes resolved it, power cycling it (turning back on immediately) did nothing.

Try turning your ONT off for 10 minutes or so and see if this resolves it, if it does please post back as I suspect it may be a firmware bug in the ONT that needs to be fixed.

I left it off for 10min and still same problem, I'll leave it off tonight and see what happens in morning. Have asked ISP to enable dhcp option 82, same as orcon but without the vlan tagging.

gareth41

742 posts

Ultimate Geek

#742542 9-Jan-2013 21:17

I just tried the linksys again, this time with openwrt and it connected about a minute after boot. Configured the iptables for port forwarding then rebooted and its back to square one again, no auth.

When it did connect first time, ifconfig showed ppp0 and the isp assigned ip/gateway etc...

PenultimateHop

637 posts

Ultimate Geek

Trusted

#742734 10-Jan-2013 12:06

Could you get a packet capture of the working Windows PPPoE session and the failing CPE PPPOE session? This would require putting something between the ONT and the PPP client in order to capture packets.

Otherwise, your ISP really needs to debug the PPP session on their BRAS and see what they see.

The only thing that's jumping out to me is if the CPE is putting in some tag in the PADI that the BRAS is rejecting (and choosing not to reply with PADO). A packet capture would be helpful in this regard - although it wouldn't really explain why the first connection works but the subsequent ones don't...

Equinox IT

Cloud spending continues to surge globally, but most organisations haven’t made the changes necessary to maximise the value and cost-efficiency benefits of their cloud investments. Download the whitepaper From Overspend to Advantage now.

gareth41

742 posts

Ultimate Geek

#743502 11-Jan-2013 19:15

PenultimateHop: Could you get a packet capture of the working Windows PPPoE session and the failing CPE PPPOE session? This would require putting something between the ONT and the PPP client in order to capture packets.

Otherwise, your ISP really needs to debug the PPP session on their BRAS and see what they see.

The only thing that's jumping out to me is if the CPE is putting in some tag in the PADI that the BRAS is rejecting (and choosing not to reply with PADO). A packet capture would be helpful in this regard - although it wouldn't really explain why the first connection works but the subsequent ones don't...

What the best device/software to use for that?

Have also just found out from my ISP (kci) that they are forcing a disconnect exactly 24 hours after the initial connection, their responce is they do this for "accounting and customer management". Have asked them to disable it or atleast increase it to a couple weeks. UFB is supposed to be an "always on" connection. Not good when im in the middle of a skype call or downloading files.

gareth41

742 posts

Ultimate Geek

#756672 6-Feb-2013 19:27

Been logging using wireshark, it appears the PADI packet sent from windows is 36bytes, and from the routers its 60bytes. Now im going to look at the source for pppd in openwrt and see if I can change the size of the padi sent.

Microsoft
0000 ff ff ff ff ff ff 00 24 1d 8b 14 c5 88 63 11 09   .......$ .....c..
0010 00 00 00 10 01 01 00 00 01 03 00 08 0d 00 00 00   ........ ........
0020 16 00 00 00                                        ....

TP-Link TL-WR10430ND using openwrt
0000 ff ff ff ff ff ff 90 f6 52 33 22 b2 88 63 11 09   ........ R3"..c..
0010 00 00 00 0c 01 01 00 00 01 03 00 04 00 00 28 5d   ........ ......(]
0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ........ ........
0030 00 00 00 00 00 00 00 00 20 20 20 20               ........

Mikrotik Routerboard RB750N
0000 ff ff ff ff ff ff d4 ca 6d 2f 49 10 88 63 11 09   ........ m/I..c..
0010 00 00 00 0c 01 03 00 04 00 00 00 05 01 01 00 00   ........ ........
0020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ........ ........
0030 00 00 00 00 00 00 00 00 00 00 00 00               ........ ....

Also appears someone elsewhere in the world has the same problem
http://www.vyatta.org/node/6348

ptinson

677 posts

Ultimate Geek

Trusted

#756718 6-Feb-2013 21:10

A packet size of 60 Bytes should not cause an issue.
RFC2516 specifies that a PADI packet "MUST NOT exceed 1484 octets" and 60 Bytes is certainly less.

I would see if your ISP is able to do a packet capture as your modem and then your PC connects.

meat popsicle

gareth41

742 posts

Ultimate Geek

#765857 19-Feb-2013 16:07

ptinson: A packet size of 60 Bytes should not cause an issue.
RFC2516 specifies that a PADI packet "MUST NOT exceed 1484 octets" and 60 Bytes is certainly less.

I would see if your ISP is able to do a packet capture as your modem and then your PC connects.

Ok i've found the problem now, I let my isp know and they're looking into it. Problem is the PADO is being sent but is being corrupted before it gets to the pppoe client, Its always bytes 27 - 32 (6 bytes total) of each PADO that are being corrupted and it doesnt matter what length the PADO is.

Whats very stange is its being corrupted with http headers from somewhere, definitely nothing from my end.

example PADO using Mikrotik router supplied by ISP.
0000 d4 ca 6d 2f 49 14 00 03 32 29 d4 1a 88 63 11 07   ..m/I... 2)...c..

0010 00 00 00 2d 01 03 00 04 00 00 98 67 a9 b1 70 d6   ...-.... ...g..p.

0020 01 02 00 09 4b 43 37 32 30 34 56 58 52 01 04 00   ....KC72 04VXR...

0030 10 e3 cd 03 6e cd 61 ec 27 66 eb f9 86 d0 8f b4   ....n.a. 'f......

0040 d3 9c 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ........ ........

0050 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00   ........ ........

0060 00                                                 .
Bytes in bold are not what they should be and have been copied from some http headers somewhere
Hence in this example the host-uniq is not correct because last two bytes are different, and confirmation of the blank service name has also been lost.

Also happens on laptop running ubuntu linux with nothing else plugged in, so this is definitely not my end causing corrupted packets.

Some more examples, just the lines with different bytes

0010 00 00 00 2d 01 03 00 04 00 00 54 54 50 2f 31 2e   ...-.... ..TTP/1.

0010 00 00 00 2d 01 03 00 04 00 00 70 67 3f 76 3d 35   ...-.... ..pg?v=5

0010 00 00 00 2d 01 03 00 04 00 00 64 62 61 6e 64 6e   ...-.... ..dbandn

0010 00 00 00 2d 01 03 00 04 00 00 31 32 38 3a 20 41   ...-.... ..128: A

0010 00 00 00 2d 01 03 00 04 00 00 54 50 2f 31 2e 31   ...-.... ..TP/1.1

0010 00 00 00 2d 01 03 00 04 00 00 30 31 37 5f 35 30   ...-.... ..017_50

Whats more my ISP tells me they have other users on UFB with no problems at all, im the only one with auth issues.

Have had chorus out, changed the ONT, same prop. Changed me onto a different gpon at the exchange and that too made no difference.

ptinson

677 posts

Ultimate Geek

Trusted

#765862 19-Feb-2013 16:20

That very odd. Insertion of HTTP into a PADO would strike me as very concerning.
I would ask the ISP to explain why the PADO has this data, and provide the pcap for them to investigate, they really should be proving its not being inserted in their network.

If its not then its an even bigger issue.

meat popsicle

gareth41

742 posts

Ultimate Geek

#765868 19-Feb-2013 16:32

ptinson: That very odd. Insertion of HTTP into a PADO would strike me as very concerning.
I would ask the ISP to explain why the PADO has this data, and provide the pcap for them to investigate, they really should be proving its not being inserted in their network.

If its not then its an even bigger issue.

Have already provided them with logs.

Also uploaded a log here I managed to get from the mikrotik
https://mega.co.nz/#!g88XQC6L!Rxp7mnvMOug1xSAuTA9xqEHTeFQawLt1ThBC4n7Hq1s

I left it running over night trying to authenticate. For now im leaving the ONT plugged into my Windows pc, strangely enough this always works first time and a packet log on pc shows no corrupt pado's. Im still at a loss as to why a windows pppoe dialer always works though.