I tried the same test with f35/aarch64 (ami-000ec96ccb51eb679), and firing that instance up, I was able to directly connect to the ipv6 address just fine.

On the off chance that the problem was aarch64, I made an f36 instance using t3a.nano (ami-0f734524c1fd28fc8) and it has the same issue that the aarch64 instance does: no ipv6 connectivity.

I am taking a look. I'll reproduce and work on the changes with the NetworkManager team.

I noticed that nftables isn't present on the default install, so I installed it. "nft list tables" returned nothing. I still couldn't connect. Then I installed/enabled/started firewalld. firewall-cmd --list-services shows the expected "dhcpv6-client mdns ssh". I still couldn't connect.

I've also done a dnf update to the current updates, which did not resolve the problem.

This feels like it could be a "me" problem somehow, but I'm using the exact same security group that I've used internally for a while. I can start up an f35 box using the exact same security group and connect just fine.

I spawned a Fedora 36 instance in us-east-1 and found that I had an IPv6 address configured along with routes, but I couldn't reach anything, including the gateway. I saw net.ipv6.conf.eth0.accept_ra=1 via sysctl, which makes sense. If the device isn't forwarding traffic, a setting of 1 should allow the instance to accept routing advertisements.

For giggles, I set net.ipv6.conf.eth0.accept_ra=2 and IPv6 networking was fully working within a few seconds. I set it back to 1, rebooted, and now IPv6 routing is down again. 🤔

I thought something might be haywire in NetworkManager. I dumped the IPv6 configuration for eth0:

ipv6.method:                            dhcp
ipv6.dns:                               --
ipv6.dns-search:                        --
ipv6.dns-options:                       --
ipv6.dns-priority:                      0
ipv6.addresses:                         --
ipv6.gateway:                           --
ipv6.routes:                            --
ipv6.route-metric:                      -1
ipv6.route-table:                       0 (unspec)
ipv6.routing-rules:                     --
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          no
ipv6.required-timeout:                  -1 (default)
ipv6.ip6-privacy:                       -1 (unknown)
ipv6.addr-gen-mode:                     stable-privacy
ipv6.ra-timeout:                        0 (default)
ipv6.dhcp-duid:                         --
ipv6.dhcp-iaid:                         --
ipv6.dhcp-timeout:                      0 (default)
ipv6.dhcp-send-hostname:                yes
ipv6.dhcp-hostname:                     --
ipv6.dhcp-hostname-flags:               0x0 (none)
ipv6.token:                             --

And, yes, ipv6.method: dhcp makes no sense here. I fixed it with:

nmcli> set ipv6.method auto
nmcli> save
nmcli> activate

About 5 seconds later, it started responding to IPv6 pings. It works after a reboot, too. 🎉

It looks like cloud-init turns on DHCP for IPv6 if it finds any IPv6 addresses in the NIC metadata: https://github.com/canonical/cloud-init/blob/53a995e2f852d043d51ad25c1b9afbbe1edafd57/cloudinit/sources/DataSourceEc2.py#L861-L863

I hopped onto an instance without IPv6 configured and sure enough, cloud-init uses auto there:

# nmcli con show 1dd9a779-d327-56e1-8454-c65e2556c12c |grep method
ipv4.method:                            auto
ipv6.method:                            auto

Upstream bug opened: https://bugs.launchpad.net/cloud-init/+bug/1976526
Opened Fedora BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2092459

I can't tell if any progress has been made on this. I use IPv6 in ec2, and I've used Fedora 35 for stuff, but I can't use Fedora 36 due to this. I'm worried that Fedora 37 might be broken too, and then I won't be able to use Fedora at all in ec2.

Thank you

@kevbo No progress so far. We might need to patch this in Fedora until upstream has a fix.

Thank you for the update

I just created an instance using:
Fedora-Cloud-Base-37-20221004.n.0.x86_64-hvm-us-east-2-standard-0

and it is the same as before: IPv6 isn't working. Which means, in about a month and a half, there likely won't be any Fedora images with working IPv6 available in EC2.

Thanks kevbo, I'll work on this right away.

@kevbo - If this is urgent for you and you're able, give Fedora CoreOS a try. It doesn't use cloud-init so isn't affected by this bug.

PR made to patch this since upstream hasn't moved on it: https://src.fedoraproject.org/rpms/cloud-init/pull-request/29

Thanks for the work on this, Fedora is currently working.

However, there's another issue. IPv6-only subnets don't work. I'm not reporting that as a separate bug because it is very much a cloud-init issue:

https://github.com/canonical/cloud-init/issues/4540

The reason I bring it up is that, deep in the discussion there, it mentions this:

"Back in 2020, the PR #634 was accepted that disables listening for IPv6 router advertisement when a static / static6 address is defined on RHEL and derivatives - see https://github.com/canonical/cloud-init/blob/main/cloudinit/net/sysconfig.py#L523"

They are trying to revisit that commit. Which, if I read all of the steps in this bug, could end up causing this to be changed. I think that commit there may have caused the problem here as well.

I just wanted to bring this to your attention.