IssueTracker

Please take and attach a full Bugreport next time this happens. Thanks.

Here you go!

I had to zip it due to the size being over 10MB

I have the same issue with my nexus 10.

I think it's a problem with ipv6 (and DNS ?) because when I use a vpn (which 'disables' ipv6) everything works fine.

That would make sense, why LTE isn't affected.

Same issue on OnePlus One CM12 beta, WiFi throughput is perfect but a ton of latency in apps.
LTE with far less throughput is much faster.

That is an accurate description. The WiFi is active and the signal is excellent, unlike some of the other WiFi bugs. However, the amount of latency bogs down any app that accesses the internet.

Found a fix - disabled ipv6 on my router.
ipv6 seems broken on Lollipop, using an ipv6 APN breaks mobile data entirely.

[Comment deleted]

I can confirm this an issue also on my network.

Any networks with an IPv6 AAAA record lags for appoximately 5-10 seconds on wifi before timing out and using IPv4.  The wifi "adapter" shows a self assigned fe:: address which I assume is because Lollipop doesn't support DHCPv6.

Disabling ipv6 seems to help, but this is not a long-term solution, obviously.

Just an update, I set my DHCPv6 server to also allow stateless configuration which fixed the issue.  Pure DHCPv6 causes the lag.

Confirmed disabling ipv6 fixes this issue.

I can not confirm a IP6 issue (Nexus10, 5.01).
If i switch IP6 support on or off at my router (AVM7490) it has no impact on the (long) delays.

Maybe this is a solution for my issue because i use a router with 2.4 and 5 ghz wifi in parallel with WPA2. Also i get better connectivity on 5 ghz !!!
https://code.google.com/p/android/issues/detail?id=87782

I created a new IPv4-only VLAN & WiFi SSID and connected my Nexus 5 to that, and I continue to see the same lag.
Apparently specifying a local IPv6-enabled DNS server causes the problem, too.  As soon as I changed the DHCP server to hand out 8.8.8.8/8.8.4.4 as DNS servers, the lag went away.

I can confirm the behaviour on three different devices after updating to Lollipop.

Nexus 4 (5.0.1)
Nexus 5 (5.0.1)
Nexus 7 2012 (5.0.2)

My home wifi has IPv6 enabled and all devices have a lag before starting to fetch data from a server. Configuring a DNS server outside the network (8.8.8.8/8.8.4.4) on each device solves the problem.

The router is a AVM 7490 and setting the DHCP server to assign the DHCPv6 server, prefix and IPv6 addresses (in contrast to only assigning the DHCPv6 which seems to be the default) also seems to solve the problem.

I do'nt understand how you set the IPV6 on your 7490. i could not finde the proper options in my 7490. can you post screenshots?

I have a Nexus 6 (5.0.1) and have the same issue with extreme wifi latency, but I already use Google's DNS servers and it doesn't make a difference. The DHCP server is configured to hand out 8.8.8.8 and 8.8.4.4, and I've also tried using static settings on the phone, manually entering the DNS servers that way.. no improvement.

I have a nexus10 with 5.02 and a AVM 7490 on a 50MBit VDSL.
if i use 802.11n+g (2.4GHz) with WPA2 i have a slow internet speed.
If i switch to WPA (2.4GHz) or to 811.11n+ac (5GHz) with WPA2 speed is much better!

+1 To this issue. Enabled IPv6 on my pfSense Firewall a few days ago (using a Hurricane Electric tunnel) and started suffering this exact same issue. Would it help if I/We were to provide pcaps for this issue? Or is the cause/issue known?

Same here on my Galaxy S4 International LTE (i9505). Using a pfSense Firewall powered by a HE.net IPv6 Tunnel. Disabling the DHCPv6 and Router Advertisements worked and my WiFi is buttersmooth now. I can definitely confirm it is IPv6 causing issues for me.

Same problem on my network.  HTC One M8.  Same issue on Stock and ARHD.

Native IP6, using a RT-AC66U(in AP-only mode) behind a Sophos UTM firewall.
Sophos is providing IPv6 advertisement.
Laptops, servers, and other non-lollipop Androids work fine.

On Lollipop, DNS lookups take a long time, and often fail.  When I disable IP6 things work fine on the phone. (been using this: https://play.google.com/store/apps/details?id=de.lennartschoch.disableipv6)

The bugreport in #2 is an unexpected configuration. It looks like the network is sending a Router Advertisement that configures IPv6 DNS servers, but not IPv6 addresses.

Because the code that configures the DNS servers is independent from the code that configures the IPv6 addresses, the phone attempts to use the configured DNS servers (2001:4860:4860::8888 and 2001:4860:4860::8844) without having an IPv6 address that can reach them. The DNS lookup code notices the error and retries, but that adds a few seconds to every DNS lookup.

michaeltomes: did you configure the network like this yourself? Or is this behaviour on by default on your wireless router(s)?

I made no adjustments to my router, or phone. The problem simply stopped, even before the Lollipop update. I can't really offer any real suggestions on what caused the initial issue, as I don't know how to reproduce the problem.

michaeltomes: if you can attach the output of "adb shell dumpsys connectivity" (or another bugreport), that would allow us to determine why the problem is gone.

Anyone else seeing this specific problem:

1. While the problem is happening, run "adb shell dumpsys connectivity" and save the output.
2. Disable IPv6 on your wifi router and reboot it. Wifi will disconnect; reconnect if necessary.
3. If things are still slow, that's another bug.
4. If the problem has gone away, attach the output of the command in step #1.

I think I've found part of the issue...

root@htc_m8:/ # getprop | grep net.dns
[net.dns1]: [fda1:51e6:2f7e::1]
[net.dns2]: [2xxx:fxxx:axxx:101::10]
[net.dns3]: [2xxx:fxxx:axxx:101::1]
[net.dns4]: [10.0.0.1]
[net.dns5]: []

For some reason the RADVD supplied DNS entries are playing second fiddle to net.dns1 - which seems to be perma-set to the link-local address with a ::1 at the end.  No idea where that is coming from - all the other machines on my network are getting proper DNSs.

Also, when I try to change that address it doesn't stick:
root@htc_m8:/data # setprop net.dns1 2xxx:fxxx:axxx:101::1
root@htc_m8:/data # getprop net.dns1
fda1:51e6:2f7e::1

Could someone 'adb shell' to their device, and post the output of 'ip a' ? I don't seem to be getting a 'link global' ipv6 address... not sure if this is the issue - my DNS is set for IPv6, but it's not assigned me a valid IPv6 Global IP to the device!

6: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 48:5a:3f:61:32:a0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.12/24 brd 192.168.0.255 scope global wlan0
       valid_lft forever preferred_lft forever
    inet6 fe80::4a5a:3fff:fe61:32a0/64 scope link
       valid_lft forever preferred_lft forever

FWIW; I'm not quite having the issue 36903775 pointed out with the DNS;

[net.dns1]: [2001:4860:4860::8888]
[net.dns2]: [2001:4860:4860::8844]
[net.dns3]: [2001:XXX:XXX:e3f::ff6d]
[net.dns4]: [8.8.8.8]
[net.dns5]: [8.8.4.4]

My pfSense is setup to use '2001:XXX:XXX:e3f::ff6d' as it's primary, and the Google DNS servers as secondary... So it's swapped them around a little.

Arg; above - 'issue 36903775' was referring to comment 27 in this thread. Please ignore the link it generated...

Hi guys!

I have the same problem...
Running pfSense, RA enabled...
Phone: HTC M8 android 5.0.1

When I turn screen on, it sometimes takes from 5-15 seconds for ipv6 to "initialise" and then it`s working great.

So problem is in fact between screen on and those 15 seconds.
If I disable IPv6 on the router, problem is naturally gone.
I tested this in 3 ipv6 enviroments, all 3 using different solutions (pfsense, openwrt and dd-wrt).
My notebook, pc`s and servers are not affected by this so it must be Android or device radio firmware...
If you need logs, please tell me what you need, I will provide.

BR,
G

Building on this some more; I changed my pfSense Router Advertisement config from 'managed' (assignment via DHCPv6) to 'assisted' (DHCPv6 Server assignment combined with Stateless Autoconfig) and I'm receiving Global link IPs correctly, and everything works as expected.
I'm suspecting this may be related to: #32621

So when I do want it to autolink, it doesn't.. https://code.google.com/p/android/issues/detail?id=32621

Further more;
http://en.m.wikipedia.org/wiki/Comparison_of_IPv6_support_in_operating_systems
States:
"Supports ND-RDNSS in Android 5.0 and above. Does not support DHCPv6."

I have it set on assisted.
Otherwise ofcourse does not work... Duh :)

As regards the problems on pfsense: as stated in #23, this is an unexpected configuration. The IPv6 code is built around the assumption that RDNSS would always be used with autoconfiguration. But pfsense advertises DNS servers in RDNSS without enabling stateless address autoconfiguration.

The fix is likely to be not to use IPv6 nameservers unless there are appropriate IPv6 addresses to contact them with. In the meantime, a workaround is to enable stateless address autoconfiguration.

Note that the net.dnsX system properties are only used for legacy purposes. The actual DNS servers used are the ones listed in the output of "adb shell dumpsys connectivity".

For those seeing this that are not using pfsense: can you provide the output of "adb shell dumpsys connectivity" so we can determine what's going on?

Hi!

But I do use stateless autoconfig!
I have it set to assisted, that is DHCPv6 + SLAAC.
And it works just fine with all devices except for android.
And again, after I wake the phone up, I have to wait up to 15 seconds, after 15 seconds all is working just great.
Attached dumpsys file.

gregecslo: what is the preferred / valid lifetime of the IPv6 addresses in the router advertisement? It's possible that if the device is asleep, the device misses enough RA packets that the IPv6 addresses time out.

Thats not the case...
See attached image, lifetimes are huge...

Next steps would be to check the router lifetime is long enough that it doesn't expire, and if that's not what's happening, provide a tcpdump of the packets that the device emits when you turn the screen on and try to use the network.

Hmmm router lifetime was 60 seconds.
I raised it to 7200 seconds.
Will report back, I`m doing testst right now.

A router lifetime that is too low will cause this to happen, yes. That is because the device drops multicast packets to save power, and possibly because of RA deduplication logic in the firmware (device-specific).

Note that an RA lifetime of 60s likely implies that your network sends RAs significantly more frequently than once every 60s, and you should be aware of possible battery life implications before you do something like that.

OK I raised it to 7200 seconds.
Now, in my RA config, I have:

AdvSendAdvert on;
        MinRtrAdvInterval 5;
        MaxRtrAdvInterval 20;
        AdvDefaultLifetime 7200;
        AdvLinkMTU 1500;

This is not OK?

MaxRtrAdvInterval 20 means that the device's CPU has to wake up to receive packets at least once every 20 seconds. That can get expensive when the device is on battery. In a network that doesn't change often, something one packet every 15 minutes (e.g., MinRtrAdvInterval 600, MaxRtrAdvInterval 1200) is more appropriate.

This only applies for SLAAC right?
DHCPv6 and static assigments are unaffected by those packets I believe?

[Comment deleted]

The values in update #43 are safe.  The router sending RAs must
respond to RSes from the client, and the clients sends (at least) 3 at
link up time.

If all 3 RSes are missed (or ignored) then yes, there will necessarily
be a delay in configuring a default route (and any SLAAC'd addresses).
(There are separate efforts underway to make RSes more reliable.)

Hmm I tested with value provided in #43 and the link did not come up.
I removed my prevoius comment because I found out, that sometines it does come up and sometimes it doesn`t...
So safer value is 30/60 or my clients can wait like 600 seconds to max 1200 seconds for ipv6 connectivity. This is not good.

It would be good to understand why the client's Router Solicitations might be getting dropped / ignored.

I did a capture on pfsense, and RS never came to it.
Actually only where router sends RA client gets IP.
So it seems that phone is not even sending RS`es :S

Are you sure? The device not sending RS packets is definitely unexpected. I don't know what HTC code does, but stock Android definitely toggles disable_ipv6 when link comes up, and that causes the kernel to send RS packets.

Not sending RS packets forces networks to send RAs frequently, which is bad for battery life.

Yup, confirmed not sending RS.
Situation:
1. I turn on wifi on SSID AP1
2. Android gets ipv6
3. I move 300meters away and connect to AP2 (which has no ipv6)
4. I move back and connect to AP1
5. No more Ipv6 and no RS being sent from android
6. I toggle wifi OFF and back ON
7. NO RS are being sent at all
8. I toggle wifi OFF
9. I restart RA on pfsense
10. Toggle wifi ON
11. IPv6 address obtained.

lol

Android does not send RSes when roaming on the same network (same SSID, or SSIDs that have been detected to be the same network at layer 3).

AP1 and AP2 are not on the same network and also they use completley different ip ranges.
So when I move from AP-1 to AP-N I loose network connectivity if I do not receive RA packet :)
Android breaks internet....again :()

If the device disconnects and reassociates, then it should send RSes again. If it does not, then that's a bug. I don't think stock Android does that though. Have you tried on a Nexus device?

Before you say things like "Android breaks internet again", please consider the effect of what you are saying on the developers who are taking their time to comment on a public bug tracker like this.

What about us, who test things in OUR FREE time and help you to deliver better product? Devs are paid to do this, we are not. Public bug tracker is more beneficial to you than us, sorry. But OK, I said it in a joke, please do not take it too seriously :)

Back to the topic...
I will try to get some devices different from my M8 and test it.

OK, tried with SGS 4, sam problem also with nexus 7 (2013 - wifi edition).
Tried with 4.4.4 and 5.0.1 and 5.0.2
So this is Android issue and it`s not device specific in my opinion.
Maybe even kernel related...

Can you provide a tcpdump and bugreport taken on the Nexus 7? A good command to use would be "tcpdump -ni any -s0 -U -w /sdcard/icmp6.pcap icmp6"

OK, I can do it tomorrow.
Where can I upload it, or send it via mail?
I don`t want to share it here :)

Hmmm I tried with more android devices (not nexus family) in an enviroment with Cisco router.
Same issue occured.
Cisco is sending RA every 200 seconds and when I moved away from AP and lost signal and then came back, I had to wait 170 seconds to obtain IPv6 address. So it is crucial that Android devices get router expiration around 30 minutes and 1 RA packet every minute or else ipv6 on them is unusable.
I also tested with Windows 8.1 tablet.
When I cam back to AP IPv6 was autoconfigured in about 6 seconds.

And a comment on battery drain...
I come home from work and don`t turn the screen on. I eat my lunch :) and then I turn the screen on and check if I have IPv6 address. No I have not. So my phone is dissmissing all the RAs sent to it therefore no battery drain from that. After 30 seconds it gets RA with default router lifetime of 1800 seconds and I have absoluteley no problems with browsing. RAs are sent between 30 and 60 seconds and phone is not awake bacause of them (at least battery in android settings says so).

You can email the tcpdump to me.

As to why you lose your IPv6 address, I don't understand how that happens. The IPv6 address lifetime is 86400 seconds, so it should stay around for that long.

What happens on some devices is that the wifi chipset is configured the wifi chipset to drop all RAs when the screen is off. They likely do this to avoid battery impact. This is not a good thing to do, because it causes IPv6 TCP connections to get stuck. This in turn causes things like the MCS connection to stop working, so for example, the device won't receive hangouts messages when the screen is off. That's https://code.google.com/p/android/issues/detail?id=32662

Recent Nexus devices do not do this - they drop some of the RAs, but (depending on the RA timers) they should receive enough RAs to maintain a working IPv6 connection.

I actually don`t loose it.
When I come home from work, it connects to my home wifi (which has ipv6+ipv4).
And I do not turn on screen for 30 minutes. When I turn it on, I don`t have ipv6 UNTIL first RA packet arrives. Then it works normally.
So my conclusion is, that android/htc/wifi firmware are dropping every RA packet when screen is off, that equals only ipv4 connectivity (so nothing is broken) and absolutely NO battery drain.
But this is not expected behaviour...

I can`t do tcpdump, nexus is not rooted...
But I made a video that proves, that HTC M8 and Nexus 7 suffer from exactly the same behavior.
Turned wifi off on both.
Set ra packets to 600/1200
Rebooted router
Turned WiFi on on both devices
HTC and Nexus 7 both failed to obtain IPv6 address...

Regards,
Greg

And the video: http://youtu.be/ucajgbY-05M

I captured traffic on my pfsense and NO RS came in from one or other device.

I testet with Nexus7, HTC M8, HTC Desire500 and HTC Desire 610.

Findings:
1. All phones IGNORE RA packets when in sleep mode (display turned off) and I can see lifetime decreasing, when I turn phone on lifetime counter resets as it should.

2. There is NO batter drain from RA packets on all 4 devices, because in sleep RA packets are being discarded (see number 1)

3. When I wake devices up, NONE of them sends Router Solicitation (RS) message to router and if RA send interval is set too high all 4 devices DO NOT OBTAIN IPv6 ADDRESS until they receive RA

4. When RA interval set to min 3 max 10 all 4 devices obtain IPv6 address NO LATER than in 10 seconds


There is no and I mean NO configuration error on my router as windows tablets perform exactly as they should (they send RS, they listen to RA while asleep etc...)

Other people have exactly the same issue see:
https://code.google.com/p/android/issues/detail?id=32662

Not sending RS packet is NOT device manufacturer fault (in sense of radio firmware) but it`s SW fault, either Android or kernel that is used.
Not receiving RA while in sleep IS device manufacturer fault and it`s a radio problem.

But if device would send RS EVERY TIME when it`s woken up, we wouldn`t be discussing in this thread would we :)

So to conclude, Nexus 7 is your device, you are the manufacturer and you have a product that don`t work well in IPv6 enviroment. It is not receiving RA while in sleep mode nor it is sending RS when woken up.

RA packet however can be adjusted.
I set min to 3 and max advert to 10 seconds and all lifetimes (route lifetime, dnssl lifetime, dnssearch lifetime etc) to at least 7200 seconds. This actually make the problem (almost) go away.

I would very much like to hear your comment as dev and appreciate some concrete answer, as this testing took me quite a lot of time :)

Regards,
G

Not sending RS packets when the screen turns on is working as intended. The reason is that sending RS when the screen turns on is not a real fix. When the screen turns on, then either the device already has current, unexpired IPv6 connectivity, or its IPv6 connectivity has expired. If it has connectivity, there's no point sending an RS. If connectivity expired, then apps on the device were already suffering connectivity issues, which is a problem. Instead, the device should receive RAs while asleep.

Here's how it's supposed to work.

- When you connect to a wifi network, the device should send up to 3 RS packets, receive an RA, and configure IPv6.
- When the device is asleep, it should receive RAs occasionally - likely around one RA every 3. These should refresh the timers of the default route and the IPv6 addresses, so the device does not lose IPv6 connectivity.
- When you disassociate, IPv4 and IPv6 configuration is cleared.
- When you reconnect, the device sends RS packets again.

On a device that is behaving like this, receiving RA packets every few seconds *will* impact battery life and is not recommended.


As you say, not receiving multicast RA when the device is asleep is https://code.google.com/p/android/issues/detail?id=32662 . The fix to that is in wifi firmware, so it's device-dependent. Unfortunately I think the Nexus 7 does not have that fix.


On my Nexus 7, I do see RS packets go out when I connect to wifi, as expected, even if I turn on wifi and immediately turn the screen off. What I see is:

$ adb shell /data/tcpdump -ni wlan0 "icmp6 or port 67 or port 68"
...
12:47:30.067462 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from d8:50:e6:88:75:10, length 315
12:47:30.086078 IP 192.168.1.1.67 > 192.168.1.115.68: BOOTP/DHCP, Reply, length 300
12:47:30.327258 IP6 :: > ff02::1:ff88:7510: ICMP6, neighbor solicitation, who has fe80::da50:e6ff:fe88:7510, length 24
12:47:31.327808 IP6 fe80::da50:e6ff:fe88:7510 > ff02::2: ICMP6, router solicitation, length 16
12:47:31.372241 IP6 fe80::225:36ff:fe89:7500 > ff02::1: ICMP6, router advertisement, length 56


The fact that you don't see the RSes is strange. We set accept_ra=2 on all interfaces. When wifi is disconnected, it has disable_ipv6=1, and when it connects, we set disable_ipv6 to 0, which causes the kernel to send an RS.

Have you seen the video?
Nexus 7 is unable to connect to network AFTER wifi is turned on.
It still sometimes loose ipv6 connectivity but I`ve set timers to really high (like 7 days) and route and dns to infinity.

And btw, if so many devices have this bug, isn `t it safer to send RA`s on 10 seconds interval to assure they have connectivity?

I did see the video, but I can't reproduce the problem. My N7 does send RS packets. Without a tcpdump taken on the device it's hard to say conclusively whether a) the device is not sending them, or b) the device is sending them and the network is dropping them before they get to the router.

Before you say that the problem is obviously a) because other OSes don't have that problem, bear in mind that network hardware does sometimes do strange things. For example, my home router responds to all DAD probes that have a traffic class value of 0x00, because it expects the traffic class value to be 0xb8 (presumably because that's what some popular OS sends them with).

Configuring the network to send RA packets every few seconds is definitely not recommended, because it will have substantial battery impact on devices that don't drop RAs. In your case I would suggest:

1. Figure out why the RS packets are getting lost.
2. Send RAs with long lifetimes (e.g., hours). Note that you cannot set an infinite router lifetime; the maximum you can set in the packet is 65535, and the maximum allowed by the standard is 9000 (so implementations might choose to drop values > 9000, or limit them to 9000).

Confirmed that both devices are not sending them.
One plus is sending them for example...

You cannot confirm that RSes are not sent unless you tcpdump on the device.  You can only confirm that any such packets are not received.

I did tcpdump and I see no: ICMP6, router solicitation,

You ran a tcpdump on the device? If so, please attach it or email it to me personally.

Hello!

I have to apologize on blaming Android for IPv6 not being fully functional on my devices.

I found root cause of it and also implemented a fix and now it is working as intended. It took me weekend to test as I have multiple networks in multiple VLANs and different SSIDs.
Main problem was my OpenWRT wireless router, cause and remediation can be found here: https://forum.openwrt.org/viewtopic.php?id=56048

In addition, my HTC M8 was indeed not sending RS packets, I also found out why. It was kernel (not stock) problem. I recompiled kernel, flashed it to M8 and now it is sending RS packets as it should be.

There is still problem with periodic RA packets not being accepted by device while in sleep mode, but this is different story, this is HTC`s problem.

Thanks for everything "lore...@google.com" !

BR,
Greg

Good news!  Thanks for letting us know.

What other open issues remain for this bug?

I would say just device manufacturer specifics.
Like discarding all multicast when in sleep mode, but this is very device specific...
Ye olde Nexus 7 has this problem, newer models don`t. And all the HCT phones I tested have this issue too...
Can Google do something about that? Report it to HTC, as they will simply disregard me as end user :)

Unfortunately, there must be multiple issues.
I also run pfSense, in a multi-VLAN, multi-SSID environment.
I have confirmed, with multiple Nexus devices, that IPv6 "just works" until I flag Lollipop, then it immediately breaks.
Since I'm flashing factory images, I don't think I can run tcpdump from the device.
But there definitely is a regression of some sort in 5.0.

athompso99: have you read the earlier comments around pfsense in this bug, and does the suggested fix work?

Yes, I've read them.
No, I haven't tried them yet, see below.
I have, however, observed the same behavior behind an OpenBSD router, and a
Linux-based router.  I haven't had the opportunity to test IOS or JunOS
with Lollipop and IPv6.

Can you explain yet what client behavior changed as of 5.0, such that on
the same hardware+firmware, merely updating Android causes a loss of IPv6
functionality?  The Nexus 5 in particular is my canonical test case: 4.x
works as expected, 5.x fails (and requires client-specific workarounds).

If 5.0 merely breaks bug-compatibility, due to some new consensus on how
IPv6 works, and multiple open-source projects now need to fix *their*
networking stacks to accommodate some new understanding of IPv6, I'll  open
those bug reports upstream (at least for pfSense/FreeBSD and OpenBSD).

But "tweak settings on your router" is, so far, a pretty bad (and not
always possible) workaround for a broken client, not a fix or resolution.

Yes, I may wind up making those tweaks to accommodate one broken client
(and thank you to everyone here who persisted long enough to find that
workaround!) but as far as I can see there's still a client-side regression
of some sort that *only* affects Android 5.0+.

In related testing by a colleague, adjusting RA timers was confirmed to
have negative results for some other devices.  I'll have to get details,
and find out where I can safely make this adjustment.

As a network engineer, "it must be the network" just doesn't fly when
*nothing changed* except the client, sorry...

-Adam

I don't think anyone said "it must be the network".

As for the question of what changed: if the problem is the same as the problems with pfsense reported by others (which we won't know for sure until more investigation is done), then the explanation is in comment #23: what changed is that Android now supports RDNSS. The device is getting (and using) an IPv6 DNS server via RDNSS without an IPv6 address that it can use to talk to it, and that delays DNS queries.

This is a bug and it will be fixed, but I don't know when that will be. Workarounds are change the network configuration to enable SLAAC (so the device gets an IPv6 address) or change the network configuration to disable RDNSS (or set the lifetime to zero).

Note that this configuration (RDNSS without SLAAC) is not expected to be common. Most IPv6 networks (~all residential ISP deployments, all cellular deployments) have SLAAC enabled and do not hit this issue.

Hi!
Can you post your /var/etc/radvd.conf here from pfSense?

So just to be clear, this issue will be one day resolved? This is affecting all my android devices and is making websurfing nearly impossible on any Comcast devices. There is no option to disable IPV6 address with Comcast, but if I disable IPV6 in my Android device after having to root it and install an app to disable IPV6 temporarily it reaffirms that this bug is a pretty serious issue.  

[Comment deleted]

At least offering an option to disable IPV6 from the network options would help reduce the severity of this issue till IPV6 is fixed in Lollipop. I realize this specific thread has not had a huge influx of reports, but I don't think most people realize their slow Lollipop wifi is related to this bug, this really isn't a minor issue as there are thousands of threads from users complaining of slow wifi, but with no idea of the actual cause. Not everyone has the option to change their gateway to no longer use IPV6 and you really shouldn't expect your users to become network admins to resolve an issue created by an update. I'm not trying to be a dick, but this is a very low priority ticket for a very big issue. Literally every android user with Comcast service has no option to disable IPV6...

[Comment deleted]

I'm using IPv6 on Sonic.net with 6rd on my OpenWRT router. My Windows 8.1 desktop and Linux server configure themselves for IPv6 just fine, as does my Chromebook, and as did my phone on KitKat (ipv6-test.com shows no issues besides DNS6, and there are no connectivity problems).

Once I upgraded to Lollipop on my Verizon LG G3, major connectivity problems began at home. Internet access stopped working over IPv6 which caused some sync problems and delays.

I can ping local IPv6 devices, but not outside ones (cannot ping6 www.google.com, for example; no replies). The WLAN interface picks up an IPv6 address but does not pick up an IPv6 default gateway ("ip -f inet6 route" does not show a default route).

I tried the fix in #74's link, to disable igmp snooping on all my OpenWRT interfaces, to no effect. I don't use pfSense or know what that is. My IPv6 configuration is simple and just works, except with Lollipop...

If you are experiencing IPv6 issues with Lollipop, please post the output of "adb shell dumpsys connectivity", or if possible, a full bugreport.

There is no If about this, if you do a simple google search for this issue, you'll find that this is affecting the majority of Lollipop devices not a minority. I'll do my best to get you such a dump, but I hope you understand its issues like this that cause iOS to continue to dominate the US market and drive customers away from Android. Especially when Devs try to pretend its not an issue caused by the OS, but rather by the network which operates perfectly for all other devices connected iOS included. Once I have a dump for you I'll post it here as an attached file.

Another bug report pointed out that routing is handled through rules and tables, so I posted the output of related commands to route.txt. See bugreport.txt.7zip for the full output of the bugreport command, connectivity.txt for the full output of "dumpsys connectivity", ip.txt for the output of ip addr. No time to redact anything...

Traceroute doesn't make it anywhere. Cannot ping outside of local addresses. Can ping router and nearby devices.

More info: when I was experimenting with this, I left my phone alone for a while and then turned off DHCPv6 on my router. When I happened to test it again, it was able to ping6 www.google.com successfully! (Must have been 5 minutes or more of sitting there.)
... but I was not able to replicate this. I wasn't able to get it to work again at all, DHCPv6 on or off. I *do* know that DHCPv6 is not supported by Android Lollipop, as it is, but my other devices are able to get addresses with or without it. For testing, I tried turning off DHCPv6 on those devices and on the router and they continued to work after restarting their interfaces.

"ping6 -I wlan0 fe80::32b5:c2ff:feb9:819e" (that's the default route) returns "ping: sendmsg: Invalid argument" repeatedly.
Running the same thing on my server, but eth0 instead of wlan0, gets successful replies.
"ping6 fe80::32b5:c2ff:feb9:819e" just returns "connect: Invalid argument" (same as on my server).

I can successfully ping6:
 - my router
 - my desktop
 - my server
...But nothing on the wider Internet.

It seems like a route issue, maybe related to the bizarre way Android handles IPv6 routes (see route.txt).

Just to repeat that this connectivity issue is not present in any other device or Android version I've used. Practically instant IPv6 connectivity on my Chromebook, with or without DHCPv6 turned on at the router.

I haven't grabbed the logs requested yet, but I rooted my device, installed a new rom as well as an app to disable IPV6. Once IPV6 is disabled page now load instantly again instead of taking 5 minutes to load as they do normally since Lollipop. To me this proves conclusively that the issue exists. My major issue here is that it would appear the Devs don't care that they released such a major bug in a public release like this and feel the burden to prove the issue is on the customer. All that tells me is that Android is doomed and we should be looking to other mobile OS's for the future. If major bugs like this can make it to full release and users have to put in the time to find the bug for the devs than there is no way this OS will ever survive the test of time. I never see iOS devs fighting their customers like this to prove an issue exists and get it patched especially when there are dozens of threads online complaining of slow wifi issues with their newly updated devices which were just fine on the exact same networks prior to the update.

I also tested this on 2 other roms, one being a GPE edition based rom and another Sense based rom for my HTC M8 device. All 3 tested roms gave the same results while connected to an IPV6 enabled network. In all testing IPV6 disabling resolved the connection issue.

NetworkFactories for:
  usbnet
  WIFI
  Telephony

Active default network: 101

Current Networks:
  NetworkAgentInfo{ ni{[type: WIFI[], state: CONNECTED/CONNECTED, reason: (unspecified), extra: "FBI_Surveillance_Van-5", roaming: false, failover: false, isAvailable: true, isConnectedToProvisioningNetwork: false]}  network{101}  lp{{InterfaceName: wlan0 LinkAddresses: [192.168.1.149/24,fe80::2ee:bdff:febc:d00f/64,2601:46:c601:6d82:2ee:bdff:febc:d00f/64,2601:46:c601:6d82:505d:d28b:c447:c1ac/64,]  Routes: [fe80::/64 -> :: wlan0,::/0 -> fe80::68ee:96ff:feb3:919a wlan0,2601:46:c601:6d82::/64 -> :: wlan0,192.168.1.0/24 -> 0.0.0.0 wlan0,0.0.0.0/0 -> 192.168.1.1 wlan0,] DnsAddresses: [2001:558:feed::1,2001:558:feed::2,75.75.75.75,75.75.76.76,] Domains: null MTU: 0 TcpBufferSizes: 524288,1048576,2097152,262144,524288,1048576}}  nc{[ Transports: WIFI Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN LinkUpBandwidth>=1048576Kbps LinkDnBandwidth>=1048576Kbps]}  Score{60} validated{true} created{true} explicitlySelected{false} }
    Requests:
      NetworkRequest [ id=1, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN] ]
      NetworkRequest [ id=2, legacyType=-1, [] ]
    Lingered:
    MultiCastFilterRule:

Network Requests:
  Listen from uid/pid:10005/1170 for NetworkRequest [ id=2, legacyType=-1, [] ]
  Request from uid/pid:1000/992 for NetworkRequest [ id=1, legacyType=-1, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VPN] ]
 
mActiveDefaultNetwork: 1 CONNECTED/CONNECTED
mLegacyTypeTracker:
  0 none
  1 NetworkAgentInfo [WIFI () - 101] CONNECTED/CONNECTED
  2 none
  3 none
  4 none
  5 none
  7 none
  10 none
  11 none
  12 none
  13 none
  14 none
  15 none
  51 none
  52 none
  53 none
  54 none
  55 none
  56 none

NetworkTransitionWakeLock is currently not held.
It was last requested for

mUpstreamIfaceTypes:
 1
 7
 9
 4

Tether state:
 wlan0 - InitialState - Available - lastError =0

ConnSrv Change History: 5/140 entries (current 0x71245d ms - Sun May 10 16:21:53 EDT 2015)
  [0xcb8d] [Sun May 10 14:19:10 EDT 2015] (992/1000) 2: , 1 null CONNECTING
  [0xce10] [Sun May 10 14:19:11 EDT 2015] (992/1000) 2: , 1 null CONNECTING
  [0xce29] [Sun May 10 14:19:11 EDT 2015] (992/1000) 2: , 1 CONNECTING DISCONNECTED
  [0xd1d2] [Sun May 10 14:19:12 EDT 2015] (992/1000) 2: , 1 CONNECTING CONNECTED
  [0xd214] [Sun May 10 14:19:12 EDT 2015] (992/1000) 3: , 1 CONNECTED

If you'd like to give me a tutorial on pulling a full bug report for you, I'll be happy to make a go of it, but at this point I'm pretty upset that its requiring this much work on the part of the user to make your OS functional.

I don't think there is any question that the issue exists. I don't have enough info to speculate as to why it is not affecting everyone or if perhaps it IS affecting everyone who uses IPv6.

I don't know if I did it correctly, but I just opened up Terminal Emulator, typed su to become superuser, then: bugreport > /sdcard/bugreport.txt and zipped it up.

twiliterate, thanks for the bugreport. It looks like the device configured IPv6 as expected: it got two global IPv6 address from prefix 2602:243:XXXX:XXXX::, a DNS server from the same prefix, and a link-local route. The router is responding to ND That all looks normal.

"ping6 -I" does not work due to a kernel bug, but I'd expect ping6 without -I to work. traceroute6 should also work, if it's included.

Things to try:

1. ping6 2602:243:XXXX:XXXX::1  # Your DNS server; replace XXXX with real address.
2. ping6 -c 3 2001:4860:4860::8888   # Internet destination (Google DNS)
3. traceroute6 -n 2001:4860:4860::8888
4. time ping -c1 $RANDOM-v4.metric.gstatic.com  # Time a DNS query
5. ip -6 route get 2001:4860:4860::8888  # Check that routing rules are working

Given your earlier statements, I'd expect that the answers will be #1 works, #2 times out, #3 falls into the void, #4 works. #5 should say that packets are sent out through wlan0 to the link-local address of the router.

If so, then the next step would be to run a tcpdump to see what is happening to the packets once they leave the device. "tcpdump -i wlan0 -s0 -U -w /sdcard/wlan0.pcap ip6" on the device would be best, but a tcpdump on the openwrt box might also help.

sublimefly: can you also try the troubleshooting steps above? (Replace the DNS server in step #1 with the Comcast DNS server, 2001:558:feed::1). A bugreport would also help.

I'll try those steps after my son goes to sleep. Attached is a bug report.

sublimefly, that doesn't look like a full bugreport. It should be several MB long, not ~200k. Something like "adb bugreport > bugreport.txt" should work.

Emailed it to you as the max size per comment is 10MB and its around 16MB.

No, there is something about ping6 and the fe80 address that causes a problem. It doesn't work on my server, either, without the -I. The only reason I knew about the -I thing is because I looked it up after I had the problem on my server as well.

---

1. Sonic.net does not currently have an IPv6-addressed DNS server, so none of my devices are configured with one. Could this be related?

---

2. I started a "ping6 www.google.com" on my phone when I got home today and it worked better than expected, since I actually got some success. This sort of on/off connectivity matches my experience with Lollipop a month or two ago, before I went back to KitKat (and now am back to Lollipop again).
icmp_seq 1-4 successful
icmp_seq 38-65 successful
icmp_seq 96-139 successful
icmp_seq 155-199 successful
icmp_seq 215-258 successful
icmp_seq 275-295 successful, then CTRL-C
--- www.google.com ping statistics ---
295 packets transmitted, 186 received, 36% packet loss, time 294451ms

The packet loss actually has continued to improve as time goes on. It's at about 18% loss right now. Still not good, but better than 100%. (Meanwhile, there's 0% loss on my desktop computer and server.)

---

3. When IPv6 is up (aka a ping6 to an outside address is currently succeeding), I get this:
# traceroute6 -n 2001:4860:4860::8888
traceroute to 2001:4860:4860::8888 (2001:4860:4860::8888), 30 hops max, 16 byte packets
1  2602:243:2009:d080::1  1.497 ms  0.787 ms  0.811 ms
2  2602:24b:8179:10::1  26.425 ms  47.872 ms  33.875 ms
3  2001:5a8:5:e::1  29.358 ms  34.561 ms  30.266 ms
4  2001:5a8:5:3::2  33.276 ms  31.301 ms  38.037 ms
5  2001:1868:a200::1  29.293 ms  28.896 ms  28.952 ms
6  2001:504:d::1f  27.919 ms  28.256 ms  27.657 ms
7  2001:4860::1:0:21  28.630 ms  29.058 ms  38.494 ms
8  2001:4860::8:0:6117  54.396 ms  51.418 ms  50.644 ms
9  2001:4860::8:0:61e1  46.880 ms  46.306 ms  48.242 ms
10  2001:4860::8:0:924b  51.021 ms  55.563 ms  57.145 ms
11  2001:4860::2:0:90fe  55.010 ms  53.689 ms  50.248 ms
12  *  *  *
13  2001:4860:4860::8888  55.003 ms  2602:243:2009:d080:a939:7467:cbc2:b197  2991.983 ms !H  3000.079 ms !H

Otherwise, I get this:
traceroute to 2001:4860:4860::8888 (2001:4860:4860::8888), 30 hops max, 16 byte packets
 1  2602:243:2009:d080:a939:7467:cbc2:b197  2991.695 ms !H  2998.979 ms !H  2602:243:2009:d080::1  1.924 ms

---

4. The success of this one is unrelated to whether IPv6 is up or down, which confused me at first, but then I realized this is just an IPv4 ping. It always succeeded. Here's one sample:
# time ping -c1 $RANDOM-v4.metric.gstatic.com
PING 29738-v4.metric.gstatic.com (216.58.192.46) 56(84) bytes of data.
64 bytes from nuq04s30-in-f14.1e100.net (216.58.192.46): icmp_seq=1 ttl=53 time=27.8 ms

--- 29738-v4.metric.gstatic.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 27.857/27.857/27.857/0.000 ms
    0m0.15s real     0m0.00s user     0m0.01s system

---

5. Interesting, here. Results are consistently different depending on whether IPv6 is up or not (ping6 to the Internet is succeeding or not). I ran:
while true; do sleep 1; ip -6 route get 2001:4860:4860::8888; done

Then, I watched the results alongside a ping6 2001:4860:4860::8888 and it was clear that the gaps in the ping corresponded with a change in the route.

When IPv6 is up and working (ping6 packets succeed), I get:
2001:4860:4860::8888 from :: via fe80::32b5:c2ff:feb9:819e dev wlan0  src 2602:243:2009:d080:a939:7467:cbc2:b197  metric 0
    cache

When IPv6 is not working (ping6 packet loss), I get:
2001:4860:4860::8888 from :: via 2001:4860:4860::8888 dev wlan0  src 2602:243:2009:d080:a939:7467:cbc2:b197  metric 0
    cache

---

I don't have the tcpdump PIE binary for Android so I'd need to find that.

The existence of "default dev wlan0  metric 1024" in the wlan0 routing table may be problematic.

Can you delete that route and try again?  In the meantime, I'll have to look to see where in stock Android such a thing might be added.

Here's a pcap taken from my OpenWRT router:
# tcpdump -i br-lan -s0 -U -w /tmp/loss.pcap ip6

I then used Wireshark to export just the packets with the MAC address of my phone based on this filter:
eth.addr == 34:xx:xx:xx:19:cb

The first ~15 seconds, there was no IPv6 connectivity for my phone. The latter ~15 seconds, connectivity had returned. I tried to make it a roughly even split.

pcap is attached.

twiliterate: what ek said, "default dev wlan0" is going to cause problems. The tcpdump confirms that Android believes 2001:4860:4860::8888 is directly connected and is trying to reach it directly, as opposed to via the default router.

I wonder if this is happening due to some strange/invalid router advertisement packet coming from some other device on the network. Is the linux server on the same network as the Android device? If so, can you install rdisc6 on the linux server and attach the output of "rdisc6 eth0"?

I just noticed a typo in my route.txt in #89. I wrote the second command as "ip -6 route show wlan0". The actual command used was "ip -6 route show table wlan0".

Anyway, yes, I just tried removing "default dev wlan0  metric 1024" using:
ip -6 route del table wlan0 default dev wlan0 metric 1024

This does nothing at all the first time, since there seem to be two entries or something. When I try to re-add it, it gives an error that it already existed:
RTNETLINK answers: File exists

This behavior was confirmed by testing again after rebooting the device, and indeed, the first removal results in no change, with the second removal causing the following issues...

I started to get these messages from these processes:
[ ping6 2001:4860:4860::8888 ]
ping: sendmsg: Network is unreachable

[ ip -6 route get 2001:4860:4860::8888 ]
unreachable 2001:4860:4860::8888 from :: dev lo  table 0  proto kernel  src 2600:1010:805f:a34c:e0be:ac76:1b79:f318  metric 4294967295  error -101 hoplimit 255

I add it back using:
ip -6 route add table wlan0 default dev wlan0 metric 1024

...which results in the typical route failure while IPv6 connectivity starts its few-minute journey to being restored:
2001:4860:4860::8888 from :: via 2001:4860:4860::8888 dev wlan0  src 2602:243:2009:d080:a939:7467:cbc2:b197  metric 0
    cache

When it does come back the next time, I only need to remove the default dev wlan0 route *once* to break things. ;-)

Oops, let me correct what I said before! I got a little confused.

When I entered the command:
ip -6 route del table wlan0 default dev wlan0 metric 1024

...and I said there was "no change," that was when IPv6 connectivity was already working properly -- and then running it a second time broke it. But for some reason I didn't try it when it wasn't working, which I'm sure was when you wanted me to try it...

Now, I tried it when IPv6 connectivity was *not* already working (which is when it makes sense to try), and it seemed to resolve the problem! I'm watching it now and verifying this, but I wanted to correct my last statement before anyone thinks too hard about it.

OK, so to correct myself again from earlier, getting this message when trying to add the "default dev wlan0" route back:
RTNETLINK answers: File exists
... was *not* because "default dev wlan0" was already there. It was because another default gateway was already there -- the second (correct) one, seen in my route.txt. I drew the wrong conclusion at first. Running the deletion a second time must have ignored my specificity and deleted the other default gateway as well.

Back to the topic, I can confirm that removing "default dev wlan0" from the wlan0 DOES resolve the problem until Wi-Fi is next reconnected. Yay, progress.

I've attached the results of "rdisc eth0" on the server, which is on the same network as phone.

I took a tcpdump which should give a good picture of what goes on with my phone when it first connects to Wi-Fi, then tries to get on IPv6 (which takes a while), then succeeds and then eventually drops out for a bit and comes back.\

With my OpenWRT router, I captured all traffic involving my phone, starting with Wi-Fi off and then turning it on and using ping6 until I got replies, waited until the replies stopped, and then waited until they started again.

I used this command to take the capture:
tcpdump -i br-lan -s0 -U -w /tmp/etherhost.pcap "ether host 34:xx:xx:xx:19:cb"

I then used Wireshark to remove all TCP packets, and then to remove a couple of EAPOL and work-related DNS packets. Everything else is included.

sublimefly: skimming the bugreport suggests that IPv6 is configured correctly. I did notice that the router lifetime is low (178 seconds). A low lifetime won't break things, but it does suggest that the device might be missing Router Advertisements, and therefore might experience IPv6 connectivity issues when the screen is off.

Can you post the output of the commands posted in #96 and #97?

There's a "fix" floating around which is a script that simply disables IPv6 entirely, and runs at a regular interval (i.e. minutely) by cron. I don't like the idea of turning off IPv6, especially for a new version of Android.

Thanks to your help, for me, to solve the problem without losing IPv6 capability, the script can contain this:
#/system/bin/sh
ip -6 route del table wlan0 default dev wlan0 via :: 2>/dev/null || true

(the "2>/dev/null || true" is just to squelch errors)

Right now, I'm trialing a Tasker profile instead, which runs that command as root every time Wi-Fi is connected.

Again, thanks for all the help so far. I'll try and watch this bug to see if a cause is found or if more info is needed.

twiliterate: if you run "rdisc6 eth0" on the linux server, does the broken route reappear on the device?

In the bugreport I see that the system does try to add a default route on wlan0, but it (correctly) specifies the gateway:

05-10 11:17:10.622 - SND -> {49 network route add 100 wlan0 ::/0 fe80::32b5:c2ff:feb9:819e}

It's possible that somewhere in the code this next hop gets lost and the system ends up asking the kernel to configure a route without a next-hop, resulting in the broken "default dev wlan0  metric 1024" route.

The broken route does not reappear when I run "rdisc6 eth0" on the server -- at least not according to "ip -6 route show table wlan0". I've tried many times in a row, now.

The first time I tried running "rdisc6 eth0" on the server, earlier this evening, the phone's IPv6 connectivity *was* interrupted at just about the same time, but I was not able to replicate that and I still can't, so I figured it was a coincidence.

That's unfortunate. Do you happen to have access to a Nexus device to see if the problem occurs there too? Without access to LG's source code it may not be possible to get to the bottom of this issue, though if you're willing to help we might be able to determine what's going on by running strace on netd.

When the problem is occurring (i.e., IPv6 reachability on/off every few seconds), does "ip monitor" print anything interesting about routes?

I do not have access to a Nexus device. I could try flashing an AOSP ROM with Android 5.0.1...

I don't know if I'd call it interesting, but this stuff shows up just as connectivity disappears. (A portion of the bottom shows up as it returns.):
ff02::16 dev wlan0 lladdr 33:33:00:00:00:16 NOARP
ff02::1:ff38:19cb dev wlan0 lladdr 33:33:ff:38:19:cb NOARP
2602:243:2009:d080:90fe:12d9:10ee:cc88 dev wlan0 lladdr 30:85:a9:95:94:bc STALE
2607:f8b0:4005:803::2004 dev wlan0  FAILED
2607:f8b0:4005:800::1012 dev wlan0  FAILED
2607:f8b0:400e:c01::5f dev wlan0  FAILED
2607:f8b0:4005:803::2004 dev wlan0  FAILED
2607:f8b0:4005:800::1012 dev wlan0  FAILED
2607:f8b0:4005:803::2004 dev wlan0  FAILED
2607:f8b0:4005:800::1012 dev wlan0  FAILED
2607:f8b0:400e:c01::5f dev wlan0  FAILED
2607:f8b0:4005:803::2004 dev wlan0  FAILED
2607:f8b0:4005:800::1012 dev wlan0  FAILED
239.2.0.252 dev wlan0 lladdr 01:00:5e:02:00:fc NOARP
239.2.0.251 dev wlan0 lladdr 01:00:5e:02:00:fb NOARP
239.255.255.250 dev wlan0 lladdr 01:00:5e:7f:ff:fa NOARP
ff02::1:ff4e:b16f dev wlan0 lladdr 33:33:ff:4e:b1:6f NOARP
ff02::1 dev wlan0 lladdr 33:33:00:00:00:01 NOARP
ff02::2 dev wlan0 lladdr 33:33:00:00:00:02 NOARP
ff02::1:ff64:b3a3 dev wlan0 lladdr 33:33:ff:64:b3:a3 NOARP
2000:: dev wlan0
ff02::1:ff00:1 dev wlan0 lladdr 33:33:ff:00:00:01 NOARP
ff02::1:ff00:200e dev wlan0 lladdr 33:33:ff:00:20:0e NOARP
ff02::1:ff00:bc dev wlan0 lladdr 33:33:ff:00:00:bc NOARP
2001:4860:4860::8888 dev wlan0
10.11.12.1 dev wlan0 lladdr 30:b5:c2:b9:81:9e STALE
fe80::32b5:c2ff:feb9:819e dev wlan0 lladdr 30:b5:c2:b9:81:9e router STALE

OK, so this does not occur with ParanoidAndroid: http://forum.xda-developers.com/verizon-lg-g3/development/rom-paranoidandroid-5-t2964416
I tracked down a version from December which still had 5.0.1, but I also tried a version from February 6th which had 5.0.2. Neither exhibits the problem.

I installed Android 5.0.2 x86 in VirtualBox as well, bridging it to the network... but I couldn't get it to start up as of about 5 minutes of booting so that's a no-go for now.

LG does make some of their source available here: http://opensource.lge.com/osSch/list?types=ALL&search=vs985
The LGVS985_G3_Android_L_V23C download would be the one. I downloaded it, and it contains both their Android and Kernel source tarballs within a zip file. I tried extracting the Android source but don't know exactly what I'd be looking for.

lorenzo: If you're able to get me a PIE version of strace that will run on my device or have other ideas, I'd be happy to look into this more later... Otherwise, do you know how to escalate this to LG? Weekend over, so I won't be flooding this issue as much and probably won't have time to mess with compiling things, but I'll check back here when I can.

Also, the VM started and I got a terminal emulator on there. No issue... so this pretty much has to be LG-related somehow. (The faulty route is present in stock Lollipop (VS98523C) but was not present on ParanoidAndroid nor on the x86 version of 5.0.2 from http://www.fixedbyvonnie.com/2015/03/how-to-install-android-5-0-lollipop-in-virtualbox-windows/. In the VM, of course, the routes showed eth0 rather than wlan0, but no problems.)

I'm experiencing the issue on multiple HTC M8 builds, so it's not just an LG issue. Possibly and issue in the source supplied to both companies?

Ssublime, we're both having IPv6 connectivity issues, but we might not be
experiencing the exact same cause.

Also, as of this morning I lost my IPv6 addresses (except an fe80 one) and
I had no IPv6 connectivity. The wlan0 route table lost all of its content
except:
fe80::/64 dev wlan0  proto kernel  metric 256
fe80::/64 dev wlan0  proto static  metric 1024

Running an rdisc6 eth0 on my Linux server did not change the situation. I
toggled Wi-Fi off and on and connectivity was restored, with my Tasker
profile dropping the bad route again.

I noticed the wlan0 table sometimes has a different name, like 1021. Don't
know what that is about.

Anyway, how could this be so broken in so many different ways, I wonder?

I'll be honest, I still have no idea why IPV6 is higher priority on a mobile device than IPV4 for lan connections. I'm not in that field specifically, but it makes little sense to me to push it this hard this early in its adoption.

It's just a bug. Actually, I suspect it is multiple bugs. We're helping to sort it out, here...

I don't want to keep harping on it here, it's just concerning that such major bugs are marked small and than pretty much ignored for more than 6 months post full release.

twiliterate: can you compile strace from external/strace in the AOSP tree? If you pull the AOSP tree for the Android version that LG is using (5.0.2?) and compile for 32-bit architecture (e.g., aosp_hammerhead - see https://source.android.com/source/building-devices.html ), that might work on your device. If so, we can run something like "strace -f -e socket,sendto,close -s 1024 -x $(pidof netd)" and see what netlink commands it is sending the kernel.

I looked at the LG source archive but it doesn't seem to contain the system code, only the kernel and what might be the source of GPL tools.

sublimefly: it definitely looks like you and twiliterate are seeing different problems. Can you provide the debugging steps listed above?

Urgh... Surely someone has an strace binary floating around for Android 5.0.1? I'm trying to figure out compiling it but it's going to take a while.

Here's the strace from my N7 2013 running 5.1.1. Does that work?

The version of strace I found here worked: http://forum.xda-developers.com/showthread.php?t=2516002&page=3
trace1.log: With Wi-Fi off, I started an strace as root using this command, after copying to /data/local and chmod 755 strace (and cd /data/local):
./strace -f -e socket,sendto,close -s 1024 -x -p $(pidof netd) -r -o /data/local/trace.log

Then, I turned Wi-Fi on, left it on long enough for IPv6 to establish connectivity and then to lose it again (and then, if I recall correctly, to gain it back):

--

trace2.log: After doing that and looking at the log, I didn't think it was what you'd need, so I ran a trace again without "-e socket,sendto,close". If it's not in trace1.log, just grep for what you need in trace2.log.

Almost every time I ran the strace command, after turning Wi-Fi back on while strace was running, the Wi-Fi icon would appear briefly and then disappear, only to return about 5-10 seconds later... crash? Whenever this happened, on return, the route table would be correct! (no default dev wlan0 issue).

So, this next log starts with me doing that a few times so that strace could catch netd being bad (rather than the WLAN crashing its way to proper behavior), but otherwise it is the same scheme of enabling WLAN, waiting for IPv6 to come and go and come back.

The password to the zip file is just my username.

My zip utility can't read the file:

$ unzip ../strace.zip
Archive:  ../strace.zip
   skipping: trace1.log              need PK compat. v5.1 (can do v4.6)
   skipping: trace2.log              need PK compat. v5.1 (can do v4.6)

That said, if the wifi icon disconnected when you ran strace, that might indicate that running strace on netd killed it, in which case the results may not be useful.

The strace flags were indeed wrong. Something like this works on my device ("$(pidof netd)" doesn't work because pidof returns all the threads, not just the main process):

$ strace -f -e socket,writev -s 1024 -x -p 198

Then, running "adb shell ndc network route del 104 wlan0 2001:db8::/32" results in:

[pid   623] socket(PF_NETLINK, SOCK_DGRAM|SOCK_CLOEXEC, NETLINK_ROUTE) = 34
[pid   623] writev(34, [{"\x40\x00\x00\x00\x19\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00", 16}, {"\x0a\x20\x00\x00\x00\x04\xfd\x01\x00\x00\x00\x00", 12}, {"\x08\x00\x0f\x00", 4}, {"\xfd\x03\x00\x00", 4}, {"\x14\x00\x01\x00", 4}, {"\x20\x01\x0d\xb8\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", 16}, {"\x08\x00\x04\x00", 4}, {"\x15\x00\x00\x00", 4}, {"", 0}, {"", 0}], 10) = 64

Which you can decode in various ways, but a fairly easy way, if you have the Android source tree, is:

$ cd system/extras/tests/net_test
$ python

RTM_DELROUTE ((RTMsg(family=10, dst_len=32, src_len=0, tos=0, table=0, protocol=4, scope=253, type=1, flags=0), {'RTA_OIF': 21, 'RTA_DST': '2001:db8::', 'RTA_TABLE': 1021}), '')

A plain "ip monitor route" run on the device will also show everything routing-related that's happening on the system, but it can't conclusively prove whether netd or another process (or even the kernel) created the route.

I just wanted to throw in my 2 cents here, but we have been seeing this issue across android devices on our network since we started implementing dual stack a few months ago. Like has been said, it's been on LG, Samsung, Moto, HTC, Nexus and other devices running Lollipop. Our only solution has been to disable dual stack for customer modems. I've tried everything that I can think of on the modem end, both statefull and stateless DHCPv6, but the android devices will not work properly on IPv6. It seems that they will initially connect via IPv6 and work, but if they lose wifi then reconnect, they will no longer work, despite pulling all correct IPv6 information. I've attached a bugreport from a Nexus device that we had with this same issue, hopefully that helps as well.

dustin, this is likely the problem:

2001:48f8:28:859::1 dev wlan0  FAILED

That suggests that the router is not responding to NS packets for its global address, or that the linux kernel doesn't like the replies it gets. The router's link-local address seems fine, however:

fe80::e47:3dff:fe29:77d2 dev wlan0 lladdr 0c:47:3d:29:77:d2 router REACHABLE

Other than DNS not working, in which direction do you see problems? If the device sends packets to the Internet, to they arrive? Can the device receive packets from the Internet?

It's strange though, every other type of device works just fine with this same router, only the android has the issue. IPv4 still works just fine, but since the device thinks that IPv6 is working, it will attempt to use that first until it times out, then fallback to IPv4. I will do some packet captures to check on the direction of the issue, but from what I can tell, IPv6 packets are not reaching their destination.

Wanted to do one more follow-up here. Lorenzo graciously helped me look into my problem further, and it appears to either be a problem with LG's Lollipop firmware or Verizon's variant of it in particular (VS98523C). The problem I've been experiencing with a faulty IPv6 WLAN route (default dev wlan0) is limited to the stock ROM, not any of the alternative ROMs I've tried, and that faulty route is present even when IPv6 is disabled on my router and no other devices are connected to it. The ball is in Verizon/LG's court now. I believe they're aware of the issue at this point. I'll be working around the issue (or using another ROM) for now.

Sorry guys, I've been unable to the testing requested yet. My company forced me to start using iOS for work and had me grab a crap iPhone sadly, which is what I was afraid of from this. I still have my M8 and when I get some more time I'll make sure to run those tests and get back to you guys with the results.

Hey, I found this bug when researching WiFi issues with my Nexus 5.

Hopefully my added information is helpful to resolve the issue. If you require additional details, please let me know and I will happily provide those.

1. Device and software

Nexus 5, Android 5.1.1

2. Issue and symptoms

Since I have upgraded my phone to Android 5.0.2, I am experiencing serious lags in network connectivity when device is connected to my home WiFi. Later versions did not resolve the issue.

When using cellular data services (3G or 4G) there is no problem at all. However, when using the (potentially much faster) WiFi connection, loading a website causes a delay (white screen) of about 5-10 seconds. Similarly, when starting any App that requires internet connectiviy, there is a long delay.

Many friends of mine told me about similar problems with other phones and Android 5.

3. Findings so far

The issue seems to be related to IPv6. My cellular provider (1&1 / Vodafone Germany) does not use IPv6 yet, hence there is no issue when connected to 3G or 4G.

My home network supports IPv4 and also has IPv6 enabled. Android receives configuration via DHCP and I can ping local IPv6 addresses. However, I cannot reach any external IPv6 address from the phone. Using IPv4, I can ping both internal and external (via NAT routing) hosts.

I assume that Android tries to connect via IPv6 first and falls back to IPv4 after some time out, which would explain the lagging.

I setup a secondary WiFi that is IPv4 only and found the connection to work without issues, the same applied to other IPv4 WiFi networks I tried.

Unfortunately I cannot change the IPv6 configuration of my router, as it was supplied by my ISP and runs a branded firmware.

As another test, I disabled IPv6 on my Android phone (echo 1 > /proc/sys/net/ipv6/conf/wlan0/disable_ipv6 - requires root) and found that this also resolved connectivity issues on my home network.

It would be a workaround if IPv6 could be disabled in network settings as also suggested in #171829: https://code.google.com/p/android/issues/detail?id=171829

Still, I only see this as a temporary solution.

3. Service provider and network infrastructure

ISP is Unitymedia Baden-Wuerttemberg(formerly known as KabelBW) - with 2.7 million customers they are the largest provider for cable internet in Germany

My service contract includes 200 MBit/s downlink and 10 MBit/s uplink, so internet connection speed is definetely not the issue ;-)

Router is a provider-supplied Ubee EVW3226 cable router:
http://www.ubeeinteractive.com/products/cable/evw3226

The router runs a Unitymedia-branded firmware which gives me very little control with regards to IPv6 settings. I can neither disable the protocol nor can I influence DHVPv6 settings. As I am forced to use this router, I have to live with that.

For WiFi AP I have tried an Asus RT-N-66U and a Draytek VigorAP 900 (both support 2.4GHz + 5GHz Dual-Band) and found the issue to be identical regardless of WiFi AP.

4. Conclusion

I am certainly not an IPv6 expert, but I think the problem is on the Android side of things here:
- all other devices on the WiFi and Ethernet work flawlessly with both IPv4 and - if supported - IPv6 (e.g. my Windows Desktop, multiple RaspberryPi running Linux, Amazon FireTV Stick, Google Chromecast Stick, iPad, etc.)
- when I was still using Android 4 on my Nexus, I did not have any issue with exactly the same network infrastructure and configuration
- other users (friends of mine) have same issue on their networks

I hope the problem can be identified and resolved. If you need any additional info (logs, config data, etc.) please let me know.

Thanks and regards
Malte

I believe the following issues are related to or duplicates of this one:
https://code.google.com/p/android/issues/detail?id=77892
https://code.google.com/p/android/issues/detail?id=145046
https://code.google.com/p/android/issues/detail?id=171829
https://code.google.com/p/android/issues/detail?id=81298

Malte, can you attach a bugreport taken while experiencing the problem?

Hi Lorenzo,

thanks for getting back on this.

I will send you three bugreport files via PM:

#1 Android just booted up, WiFi connected to home network, IPv6 enabled

#2 Having just loaded a website, suffering the connectivity issues described

#3 Manually disabled IPv6 on iface wlan0 and loaded website without any issues

If you need any additional info or output of shell commands, let me know.

Regards
Malte

Lorenzo,
I'm not sure what additional steps you're wanting me to do. As has been stated before, under IPv6 dual stack implementations, every platform except Android Lollipop works properly through wifi. Why do the customer's have to prove anything? This isn't the network, it's the android wifi. I created a separate thread asking for an option to disable IPv6 for wifi, and it was closed. Why shouldn't it be an option if IPv6 wifi does not properly work like every other O/S on android?

This is why my work forced me to get an iPhone, I'm still following this and hoping it gets fixed so I can try and get back on the android platform. But I have a feeling this is going to hurt androids customer base.

I think Lorenzo is really trying hard to analyze and fix this issue, so let's give him the opportunity to do so before we start our rants - shall we?

I still think that the priority assigned to this issue is way too low.

To me, this is a major problem which renders my Android phone next to unusable. I know at least ten people who suffer exactly the same problem and are obviously really upset about it.

As more and more ISPs are starting to implement IPv6, we will probably see more Dual-Stack configurations prone to this type of problem - which is a total show-stopper.

Let us not discuss the pros and cons of mixing IPv4 and IPv6 on the same network - at least here in Germany the cable internet providers all use these transition mechanisms. Many corporate networks run Dual-Stack nowadays and I guess that DSL providers might eventually also chose this path.
With my provider Unitymedia BW, I cannot even deactivate IPv6 in the supplied router so rooting my phone and disabling that protocol on client side was the only way to temporarily fix the problem.

Bottomline: Users do not have a choice, DS is there and Android should support it.

At the same time, all other devices on this same network just work - I just confirmed that 2 iPads, 3 Windows machines and 2 RaspberryPis happily use IPv6 protocol while a bunch of other clients are still on IPv4 and access the Internet via NAT.

If IPv6 implementation in Android cannot be fixed to support this in the short term, giving users a way to disable IPv6 it as suggested in issue 36949180 would be a temporary solution - still better than switching to mobile data services while at home or at work because the cell network is faster than WiFi :-(

I'm happy to give him more time myself, but this has been a reported issue since November and it constantly feels like the burden is entirely on the customers.

Agreed, as an ISP network engineer who is in charge of implementing IPv6 dual stack on our network, it's frustrating that I've had to create a work around of essentially disabling IPv6 on customer modem's who have android devices with this problem. I agree, this thread's priority should be higher.

Hi!

I've reported this on Oct 2014 ( https://code.google.com/p/android/issues/detail?id=77892 )

I dont know why another user created another issue when it was already created, but please, FIX IT asap. I had to disable IPv6 on all the customer network due to this problem.

Thanks!

Hi Lorenzo,

I have additional information that might be helpful.

I experimented a little more and found that with IPv6 enabled for iface wlan0, ping6 actually works from commandline:

shell@hammerhead:/ $ ip a | grep wlan0 -A10
ip a | grep wlan0 -A10
21: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether c4:9a:02:8b:7b:76 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.60/24 brd 192.168.1.255 scope global wlan0
    inet6 2a02:8070:b288:4700:459a:328b:35c1:94c2/64 scope global temporary dynamic
       valid_lft 604774sec preferred_lft 85774sec
    inet6 2a02:8070:b288:4700:c69a:2ff:fe8b:7b76/64 scope global dynamic
       valid_lft 1027219sec preferred_lft 422419sec
    inet6 fe80::c69a:2ff:fe8b:7b76/64 scope link
       valid_lft forever preferred_lft forever

shell@hammerhead:/ $ ping6 -c4 ipv6.google.com
ping6 -c4 ipv6.google.com
PING ipv6.google.com(fra02s27-in-x04.1e100.net) 56 data bytes
64 bytes from fra02s27-in-x04.1e100.net: icmp_seq=1 ttl=56 time=42.7 ms
64 bytes from fra02s27-in-x04.1e100.net: icmp_seq=2 ttl=56 time=29.6 ms
64 bytes from fra02s27-in-x04.1e100.net: icmp_seq=3 ttl=56 time=30.8 ms
64 bytes from fra02s27-in-x04.1e100.net: icmp_seq=4 ttl=56 time=29.7 ms

--- ipv6.google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3003ms
rtt min/avg/max/mdev = 29.610/33.235/42.703/5.489 ms

However, for any app (including chrome browser) the IPv6 connection apparently fails and fallback to IPv4 occurs after timeout.

I installed the app "IPv6 and More" (https://play.google.com/store/apps/details?id=com.tsts.ipv6MorePro) and tried to ping ipv6.google.com using that tool.

Result: Commandline ping worked BUT "ipv6.google.com is NOT Reachable via API" - see screenshot attached.

I will submit yet another bugreport file taken in that situation to Lorenzo.

So maybe IPv6 connectivity is OK on OS level but gets messed up somewhere in the abstraction layer(s) towards the app API?

Best regards
Malte

Dustin: again, I would suggest that you work with us to determine why Android's IPv6 stack does not work well in your network.

You may be correct that Android is not behaving correctly; without more investigation it's hard to know for sure. What we do know for sure, though, is that globally, well over 5% of Android users have working IPv6 connectivity, and that in major wireline IPv6 deployments worldwide, that percentage is well into the double digits - typically higher than other operating systems on the same network. That's a LOT of users with working IPv6 connectivity, on a lot of networks.

(Malte: the IPv6 statistics for both Nexus 5 and for Android 5.0 in general, on your particular network, also do not indicate a widespread problem.)

Tell me what information and tests I can run for you, and I'm more than happy to do so. The problem is that there are so many different android versions, and I understand that Google only directly supports the Nexus releases, and we're seeing issues with Samsung, LG, Motorola and other "flavors" of android that are modified by OEM's. I'll test what I can for you, just let me know. I've tried doing straight DHCPv6, SLAAC, SLAAC+RDNSS, and haven't had any luck with any of them on our implementation of IPv6 dual stack. I'm sorry about the frustration on my side of things, but I will get you whatever you'd like me to test. Thanks, Lorenzo.

~Dustin