Archives

IPV6 SESSION
14 MAY 2015
9:00 A.M.:


JEN LINKOVA: Good morning everyone, I really appreciate your dedication to this technology, 9:00 a.m. So many people, great. Thank you very much.

So, it's IPv6 Working Group. If you prefer to know about DNS, I think it's in the wrong room. And today, we have two sessions today, one is going it be later ‑‑ first of all, thanks to our scribe.

Again, back to agenda. We have four talks today, first two I think are quite short. And I think it would be nice if we put your mobile phone in silence mode so when your phone rings it would not wake up your fellow colleagues in the room.

So, I don't want to waste time and before we start with actual talks, I just want to remind you about one thing ‑‑ about two things actually. First of all, if you prefer for some reason not to listen to talks but to read your e‑mails instead, you can use IPv6 only network, which is available. SSID is IPv6 only EXP, password which is also LSA, I know best effort. If something doesn't work, let me know, I appreciate reports. And a big thanks to RIPE NCC who keeps and Cisco systems who help us with NAT 64 solution, we allow you guys to access v4 only resources in the Internet over IPv6 only network. And if you do listen to talk, as I hope you do, please rate them, because this time you can actually rate not just Plenary talks but Working Group session talks as well.

So, now, Erik...

ERIC VYNCKE: Thank you. So, first session is this morning, is basically for the breakfast session right, if you don't get a croissant, you have a cookie, basically, I found a problem with some content providers that stop deploying IPv6 for a dispute question, we work at RIPE, but in sometimes, failures at bottom layer 7 in the application, and here is why. You remember this http protocol does not have any state, every time you send a new request for a new page, you get a post, a put, whatever, there is no state in it. The server is stateless. The way we keep state like a shipping card for instance, it's simply by using cookies. And how does it go? Quick refresh:

On the left, the customer. On the right the server, and the big blue stuff, it's a big database. So, we start. You log in, you send your user name and your password. The server does some checks, it generates a rand come number, which is the cookie, and it stores in the database with the key, being the cookie, he store your name, the status, like you are authorised and it creates a shopping cart which is of course in the first trust anchors is empty. And it stores your address, because there is attacker that will not describe which is basically called cookie stealing. So now he is sending it back, okay you are authorised, empty shopping cart in the HP P header he sends a cookie. You keep the cookie and for every time you click on the same website this cookie will be sent into the http header. So I want to buy food. Here is my cookie, server checks in the database, a cookie, which line it is, bingo, you are authorised, I am adding this to area cart and you get into your cart which is full now. Next one I am on to order a bar, here is my cookie again, look into the database, found the record, you authorised, you are coming from the right address and basically I can add bar and then set A, indeed bar has been added to the your shopping cart. This is quite common. If you like about session IP, it's everything is there. That's the only way basically to get a long term state management of HTTP.

So far so good. Came the world of no more IPv4, same the world of CGN and of the DNS attack. So what have we start a session with address A, right, we good the cookie C, and then the server uses the cookie with the IP address as a check. And the reason is this: There is no more user name and password sent, if I go ‑‑ we do not send on this slide the user password any more. We simply send the cookie. Anybody that can get the cookie, can fake you, because the cookie is enough to go in the database, find yourself, your user name and so on. Cookie stealing could be able if you sniff the traffic in path, or it can be done by old style malware. It's pretty common by the way. So one way of fixing it is simply checking by the server, where did the cookie is really coming from the right guy, hence the right address. But, as I said, if you do this, and you start with A, and remember, AP eyeball, first transaction is over IPv6. Then v4 is becoming faster than v6, guess what? You switch to Ipv4. You change your IP address. Or, they are CDN, they do not follow this RFC 6888 where you get multi‑public address on the CGN, and the same invited customer, when it's translated for once he use address A, when it comes back to the same CGN, address A has been filled by other customers, it switches to address B, and then maybe after address C.

Here we go: Same thing, right, we start with address A. I'm John Doe, please let me in, yes. Generate the cookie, I remember your address is A and you are authorised. Okay. Now, I got the cookie as before, now the address is changing. Bad CGN, AP eyeball, other ways of changing address. Going back, here is my cookie, but then I cannot find it because you are not coming from the right IP address. So what I say, you are not authorised. It's simply happens because for us in networking, changing the IP address, I mean it's normal, right, that's just a location somewhere in the network. In the application, they use this location as identification, which is wrong. And here we go. Two Belgian websites, Belfuse, connection mistake, and specifically for people here from Holland that should also be in their bed sleeping rather than working here, Rabobank, is a big Dutch company, a bank, it's also existing in Belgium, they say disconnect, time‑out. And the first time it arrives to the end user or to the help desk, can you imagine that's because somewhere a CGN was behaving badly. And shame on my employers, we have one application that does this, so if you using a PI ball to go to Cisco or part it have and you switch before v6, you become disconnected, which is kind of annoying.

So impact: In the small country of Belgium I know two big bankses that basically fallback to IPv4 only, they stopped completely the deployment of the attack, because it's not a single technical change any more, they need to change the change the complete security. So it's slowing down.

In summary, addressees as we know is simply location, don't use it for a notification, for sure we know it, for application, people do not know it. It's common, I talk by AP eyeball, but it's not an AP eyeball, it's CGN if you have a GSM with 3G and wifi, you switch 3G and wifi, bingo, you get the same problem. The only way out of the technical means is multipath TCP but not many people are using multipath TCP besides series, nobody, so back to square one, I'm working with OWASP, which is the security people working on the web to change their web page because they say advice, please check the IP address with the session, which of course is wrong.

That's it, I had ten minutes, I think I'm less than ten minutes, so any questions?

AUDIENCE SPEAKER: Wilhelm Boeddinghaus. I have got one addition, Windows 7 and Windows 8 use a temporary address for outgoing connections, usually the time frame for that address is one day but if you turn that down to one hour, this could happen in the middle of a session so you don't need IPv4 and IPv6 for changing addresses you can do it with IPv6 only.

ERIC VYNCKE: Absolutely. The same thing for IOS device that simply change of IP address every time they wake up, same problem.

AUDIENCE SPEAKER: I think the problem is rather very old, way older than IPv6. We are running a hosting provider, and we have like turned off IP checks about 10, 12 years ago because what we see is big companies have outgoing notifyings or proxies that are doing exactly the same, just switch IP addresses in the middle of the session for any user indeed, the company. So, IP checks are nice, but most of the time they simply don't work and just use HTTP is. And make sure your cookies are safe.

ERIC VYNCKE: But people need to learn about it. The issue of your customer using proxy, the scriber in the enterprise, on banking usually do it at home, this is where it's more of use there, but you are right. Nothing new.

AUDIENCE SPEAKER: When you first started talking about this it seemed like it was CGN problem and I thought why are you bringing this up even in IPv6. It's a problem because even if we say to the people implementing NAT 64 don't do it this way, keep a stable IP address which seems reasonable to me. I think we mentioned the RFC, didn't we? Anyway it can happen for other reasons as well even when I'm on my phone and I walk out the door and I go from wifi to 4G or whatever, so this is unavoidable. As far as I understand, is that the people that make these cookies, they do this to avoid stealing cookies which is a big security problem so then what is the solution?

ERIC VYNCKE: The solution, I mean, we need to get this working ‑‑

AUDIENCE SPEAKER: How?

ERIC VYNCKE: That's the question I'm asking to everyone as well. I'm reporting the problem. Honestly, my point of view on it and I'm a security person, so, we need to protect the cookies itself more now in the browsers, they are much more resilient to cookie stealing, because you can really say never leave, never get access out of my website. Other people can access to it. When you send the cookie you can say JavaScript has no access to so cannot save it. If you are HTTPS on the way it's very enough now to intercept the cookie. Database years ago nothing was encrypted. Cannot steal there and browsers are protecting the cookies inside the browser. So we can, I think, slowly relax the link between the cookie and the IP address. Anyway, we can steal it right, if you are GACGN with 1,000 people you send share the same address so you are losing in this way, not easy though, that's the reason why it's there.

(Applause)

WILHELM BOEDDINGHAUS: Good morning. I want to talk about Extension Headers, should not be a technical talk, more a political talk, so, let's see what discussion comes out of this.

We see that Extension Headers are filtered in the Internet. We have excellent work and presentation being done on this. General Lynn Cova presented at the RIPE 69 meeting in London and there is a lot of work done by Fernando Gont, and we find that Extension Headers are filtered in the network. But, for some reason we have no real idea why, there are many many reasons.

It could be lack of knowledge. Service providers just have no idea. Enterprise at man's have no idea. They have security concerns. Maybe have real security concerns or just a bad feeling. If you have real security concerns you can read RFCs or you can talk to someone but if you have a bad feeling, nothing can help. You might have a business requirement.

You might have slow hardware. If you look in the mailing list, there is always discussion about this veteran router 6500 from Cisco. Some say broken; some say it is really designed; others say bring it out of your network. Well, there is slow hardware, it could be slow hardware path so people are filtering to protect their hardware.

Maybe, RFC 6564 which defines a uniform format header, helps a bit to get better hardware in place. Then we have got all these middle boxes, load balancers, security appliances, firewalls. A few years ago when I still worked at a content provider in Germany, one of the load balancers dropped fragmented packets, it was hard to find and it was hard to convince the manufacture of this box to fix it. And some firewalls don't know about these Extension Headers, we have all seen this.

So, when we think about why are these Extension Headers filtered? I would like to group the Extension Headers into two groups. So, we have the ones with the obvious use cases and the ones with the no so obvious use cases, where nobody really knows what these things are used for.

Start with the obvious ones. Fragmentation, everyone understands the use case, there is one use case, we need the fragmentation, you can be a friend of fragmentation, or maybe you are not a friend of fragmentation. There is valid user fashion, there is invalid user fragmentation if you try to evade RA gout in your local network or anything else. So there are use case, there is one use case and everyone can understand this use case.

We have got the headers that are used for encryption. Fundamentally important, we have this encryption, we need encryption, otherwise we can't install business, business trust, we can't have data protection. We have a trend going inter encryption with HTTPS but there is a lot of encryption in IP Sec and we need these headers and everyone understands why we have these headers.

But, there is not true for the next few headers I want to talk about. We have got the hop by hop header. The hop by hop header makes each and every router on the way to look this packet. It is used for RSVP. One of the legitimate use cases. But I would use it internally in my network, to if somebody from the outside world sends me an RSVP packet, I would drop it. Because, my customers may not reserve any bandwidth in my network. And the customers of my customers as well. So, the providers, if they don't filter this header, have to look at it and they don't want to use it, they could ignore it but maybe this is a special function in their router. So, I would prefer to filter this from the outside.

And the Hop‑by‑Hop header is used for Multicast but just in the local network, this is nothing about filtering, we know that this must not go through the Internet.

So, this Hop‑by‑Hop header is seldomly used. We have got the destination option header. This destination option header is used for mobile IPv6. This is as far as I know the one and only use case we have.

So, transit providers should just forward this packets with these headers because it has nothing to do with it. It is for the destination. Filtering might even be extra work. But if you have a firewall administrator who has a bad feeling about security, has no knowledge about this header, he might say, okay, I'd rather drop it because I don't know what this thing is doing to my network. And if no mobile IPv6 is deployed in the destination network, the destination network should drop this. Because, then it could be a security issue.

Then we have got these fantastic routing header, one of the options that were depricated, option zero, so option 2 again is for mobile IPv6. And if you are not using mobile IPv6, maybe you want to drop this beast. Option 4 is segmentation routing, we heard about this during the panel, it's still under development. I don't know when providers start deploying this today, if I would get a routing header with option 4, I have no real idea what to do with it, lack of knowledge, a bad feeling, I would drop it.

So we have many Extension Headers. Five or six of them. And the way to extent a protocol by extension headers is a very, very good idea. Because we are flexible for the future, but nobody is using it, apart from fragmentation and encryption. So, we use the fragmentation header and this ESP and AH header.

Bandwidth reservation is done internally in my network so I can filter at the outside, and I think it will be done. Multicast is just on the local link. So, nothing to filter there. Mobile IPv6, if I don't use it I have to filter it. I should filter it.

So, please don't expect these headers to be forwarded in the network.

So, have we got more use cases? IPv6 was specified as it is today some 17 years ago, and the development phase was the years before that. So we have got 20 years of Extension Headers, and we have one use case for the destination header, we have got two use cases for the Hop‑by‑Hop header, and I think if we have not more use cases, then everyone will filter. When I prepare training for IPv6, and I prepare the slides for the Extension Headers, I know what to say about the fragmentation header and the security, the encryption headers. But, what do I tell the guys in the audience during the training about the other headers? I have no real idea. I have to tell them well, nobody really knows what to do with it.

So, if cannot provide for use cases, everyone will filter.

So, as a conclusion and that should maybe open a discussion, if we don't use these Extension Headers, there are nearly that technology, and that technology is filtered, because nobody knows what these things do to our network. If they are used internally, well, then any test from the outside will fail, because if you try to send through the network of a service provider, it will be filtered because they just use it internally. Maybe providers filter headers, maybe the destination option header, because they don't know what to do, but they should not, yes, I know, they should not do that. Then we have to educate them.

And we need more use cases, and then the ISPs will learn to use these Extension Headers. We have more and more enterprises behaving like ISPs for the internal customers, they have to be educated as well. But if we have no use cases, this is nearly that technology. And then I think it should be filtered. And I'm curious what you in the audience have as ideas for new use cases that we can have or what you think of a technology that is nearly unused 20 years and today we expect this technology to be in hardware, to be in software and to be in the mind of the system and network administrators.

Thanks so far, let's open the discussion please.

JEN LINKOVA: We have five minutes, I am afraid we might need more than this because we have been discussing it for a long time. But let's try to ‑‑

WILHELM BOEDDINGHAUS: You can't finish up a 20‑year old discussion in five minutes, I know that.

JEN LINKOVA: Questions? Comments?

AUDIENCE SPEAKER: I have. It's not because I am supposed to say something if nobody else does. Benedikt Stockebrand. There's one major problem with the Extension Headers which is that they are not appearing in a fixed order, there is some shoots and whatever in the RFCs, but it's not entirely fixed in what order they should appear and that makes it tremendously painful to sort these things out, yes, I only accidentally, but it shoots and I think it was from a presentation Fernando Gont gave sometime ago, that makes it really hard to deal with them properly in hardware, that's a major screwup from, like, 20 years ago which should probably ‑‑ must probably be cleaned up, if a case must, that makes it really hard, because from an ISP point of view, I'd say I pass any packet people send me unless it actually disturbs my service, but it's completely different at an enterprise level. Eventually, it means yes, yes should ‑‑ or the IETF, so it's their fault of course. They should clean up whatever we have and deprecate whatever we don't need, but as Chris already pointed out, we have to provide operator input to the IETF on that. So, yes, this is really important, and I'm also quite surprised that nobody else wants to talk about it. Apparently... I have talked too much already.

WILHELM BOEDDINGHAUS: I can imagine the cushion at a hardware vendor when the protocol guys come and say hey we have to build new hardware for extension headers, does anyone use it, asks management? No, but they are in the standard, let's invest a few million in ISEC development. So, I think the vendors will not support it properly if there is no use cases.

AUDIENCE SPEAKER: One more thing. That's where the problem is. So especially we take companies like Cisco, which use very, very weak CP, us because they do everything that's performance critical in hardware that opens a sector for the service attacks, that's where things get scary, as far as I'm concerned.

JEN LINKOVA: So, I'd like to comment that we are talking about use cases, right. But I'm pretty sure actually there are more use cases we just don't know about them, because I know that some people are doing interesting stuff in the internal network, so I'm pretty sure they might talk to vendors and ask them for proper support.

Two years ago I had no idea what kind of usage we had and now look we have segment routing. I strongly against the position, nobody using them, let's deprecate the stuff, because we might come with problem err usage, a lot of ideas how can you use Extension Header for different things.

And my last comment is, I look on the TCP protocol there, I wouldn't be surprised if I tried to reach some destination using most of them protocols and they will be filtered, but I do expect them to be filtered at quite close to the destination when network administrator and security people apply their policy, normally by permit what I need on the analysis basis. That's perfectly fine. My host is not using destination headers so I'm filtering it out. I would be very surprised and pleasantly surprised if I see some transit networks doing this. That's why we basically try to do some measurements and see how much of this is actually normal security policy and how much of it is something I have been doing wrong, and from my perspective, first of all ‑‑ my data and I I think how it's close to Fernando Gont ‑‑ is that as increase the size of the header, the drop rate increases up to like from 20% to 70, right. So, I suspect it might be related to the hardware problem and I do hope we're going to fix it over time. Natalie.

AUDIENCE SPEAKER: Hello. Yes, I could not agree more with you, Jen. When I also teach IPv6 courses I spend a serious amount of time explaining to people that when we basically when people designed IPv6, the amount of addresses was because we have no idea how they will be used in the future, and that is a whole new philosophy basically that you give a /56 to a home user or a /48 because we don't know how things will look in the next 15, 20 years. And I think that philosophy was also built in the design of IPv6 with these Extension Headers, let's put them there because we don't know why yet. And that is more a philosophy thing that we have to get used to with IPv6 that we don't know yet how we will use things in the future of the Internet with a whole new possibilities that we will have.

So, although I do see why people are thinking about filtering them now, for security reasons, etc., etc. I'm not sure how bad that necessarily is at this point, and how bad things will be when we find a use case for them to then tell people to not filter them any more, because I think that's where the problem will be eventually, that when we find use cases for them, that by that time so many people have been filtering them for so long that it will be again an issue to ‑‑ a.m. I saying something weird ‑‑ that's where I think the problem will be that in the end we have to reverse people to tell people not to do certain things again.

WILHELM BOEDDINGHAUS: The idea to have these Extension Headers is fantastic and I'm not going to deprecate them or to advise anyone to deprecate them. But if we have newcomers to the protocol. They don't see any use case and they fear these headers and they have no idea what they are doing to their network and at the tend to filter them. And we have many, many newcomers ‑‑ and we want to have newcomers to this protocol because we want to have it widely used. So thanks for the discussion. Thanks for the input. Maybe some more input?

JEN LINKOVA: We have two people and then I'm going to cut the line because we are running out of time.

ERIK BAIS: So, the equipment for instance that we have, it supports the Extension Headers ACL, depth for the Extension Headers is between three or four Extension Headers and after that it just cannot you know, the ACL just doesn't go that far into the back house. So that basically provides an attack servers so you know that whatever they do after that, you know, they can get through the ACLs, which is scary, you know, how would you go about and they say we'll drop anything which is extension header 4, 5, 6 and beyond or... what's going to happen there, you know, what's your view on that, and do we actually need equipment that actually goes into extension header ACL for number 5, number 6, number 7, because we just got the kit, we're really glad that it actually does v6.

WILHELM BOEDDINGHAUS: Maybe we should have a restriction in the protocol that says something like five or sick headers is the most that we can accept. Maybe we should have, or will have clearly defined use cases where we say okay, this use case needs three headers, two for the protocol, for the use case and maybe one for fragmentation, so that we then can define a use case and a security rule for that. But having 15 or more headers, what's possible today, is not okay with the protocol, so that should be filtered.

I have more questions than answers to be honest. Like, most of us, but, if you have hardware that can filter that and can do that in hardware, well that's fine, many those have this hardware. And they filter because they just have no idea what they are doing. I think you know what you're doing there, but most of the newcomers don't know, they have no use cases, they don't see the sense of this and say well, drop them.

JEN LINKOVA: I have very good comment. I I am glad you mentioned this situation because there is a draft which says your firewall should be able to make the distinction between situation it is not protocol X, and I know what kind of could of protocol it is, and it's not protocol X and I have no idea what it is because I could not look deep inside. So if people think it's a good idea to have something like that, yeah, we can work on it. Jan.

JAN ZORZ: So basically Erik stole my question. Just to add to this, I'm really annoyed that people just filter out packets if there is an extension header and don't look really deep into them, look into the values and the numbers in Extension Headers or even the extension header chain if it's valid or invalid, it annoys me that people filter just if there is an extension header drop it. I was running around with mobile IPv6 like, five years ago, and if this would be the case five years ago, the majority of my traffic would be just dropped and I would have no idea if that works or not. So, please, whoever is filtering, just based on if there is extension header or not, stop doing it.

JEN LINKOVA: Okay. I'm afraid we're out of time for this discussion, but I'm sure we can continue in the coffee break. So, thank you very much, Wilhelm, for bringing this topic again.

(Applause)

The next presenter will tell us about OS behaviour.

ENNO REY: Morning everybody, am Enno Rey and this is Christopher Werney, we are going to talk about ‑‑ we have a bulky that the IPv6 behaviour in conflicting environments. We will quickly say what this is about. Still, I'd like to take the opportunity to be being on stage to state that I'm strongly in the camp of Extension Headers should be depricated by the IETF, and whenever you can you can drop them out of your network, they don't add any value, and they really impact security policy of network devices and we have done a number of studies on this, we have all major ID PS Windows by means of extension header combinations so I know what I'm talking about. Thanks for listening. These 30 seconds.

After that, actually we are going to talk about something completely different. The talk is split in three pieces, lay out some fundamentals very quickly, then the main part will be like what have we seen in the lab, and why there might be like surprises and problems. And obviously what are the conclusions of all this that we did in the lab.

There is some related work to that, to the stuff that we did. There is this IETF draft, which was just recently updated to version 4. There is some stuff, some guy discussed actually some of the problem space back in 2003, but his draft was depricated after version 0.

So, it's not only us who see problems in the space. I have only one slide that is called fundamentals which is just to remind you what we are going to talk about is in the space of slack and DHP v6. The text book understanding is there is a main provisions mechanism called slack, that works by router advertisements, those router advertisements have a number of options and flags, the main options, the main option for our purposes is the PIO option, providing prefix information, that in itself can have a stack. If the A flag is set, a device receiving a route advertisement is supposed to use the prefix information to generate an address, or maybe even generate several addresses. Within the router advertisement, there might be an M and an O flag indicating there is other mechanism called DHCP 6 and that can be used to provide additional provisioning information. And the text book understanding is that like DHCP 6 if at all present should be triggered by elements within the router advertisements. The point is, if one looks closely at the relevant specifications, it first turns out well there's so many of them, and second, which adds to the problem space that we are going to lay out, there is different generations of stacks which is particularly annoying when it comes to major core elements of the IPv6, there is different generations, different kind of versions, they are somewhat, well, complementary, somewhat contradictory as we will see. So, keep this in mind when we very quickly go through some of them in the main initial. There is a statement which essentially says if there is conflicting information, if a note we see information from different sources, consider the most recently received information.

Then there is 4861, the core or what is considered in the text book, the core enables discovery RFC, when it comes to the M flag, essentially it says if none of them are set, the HTVP 6 is not supposed to happen. Then we have in 4862, we have a statement which is interesting as it goes like, if a link has no routers, the service to obtain address may you may not this is not a capital may which is even worse, that there is no RFC 2119 same language use, but it may be used ‑‑ it may be used, there may be something, but maybe not. If you read the statement the other way around it could be like okay, assume there is no router advertisements and node may be allowed to use the HTPv6 if it's supposed to be that at all. Probably some you are aware that not all operating systems support HTP v6, Android doesn't. There is a whole debate around this.

Then we have RFC 6106 which says there is might be additional information within the route advertisements, rDNS S, requires server information, part of the router advertisements, this is the standard RFC, Microsoft, are you listening. It says if there is, it should be processed which, as some of you might know, Microsoft operating systems don't, and in this RFC, there is even a statement which says if there is conflicting information of different sources, somewhat scribes a certain way of handling this. This is the specs, this is what the RFC says and already in this space you can note there is a bit of an a mess. Which brings me right now to the problem statement of this talk, which is there is two mechanisms of one attacks that is provision parameter information to nodes.

This is somewhat independent. You can run them on different boxes by different teams, with different types of software. But, not really as we will see, there is some indirect. In most environments, actually most people need both of them as one mechanism lacks certain things, some like RFC 6106 so not so much ‑‑ so if you have both, well you need to both sources of information. Different groups might be responsible for them, which does not necessarily help. And there is different strategies of window support and different amounts of this, which then again adds even more to the mess. And the problem is that even when stuff is discussed in the RFCs, I mean most of you in the room will be aware that we much debate about the approximate, the lack and what was the benefits in IPv6 for, in the last ten years.

Still most of the discussion is about what, how to configure like the cooperation, what is not covered is what happens if there is like conflicting information. If there is two sources which tell different things. And this like situations can occur by misconfiguration, say misguided management processes, by misguided expectations. Like, okay, these boxes are supposed to happen like to work like this as something is in the RFCs, that's why we deploy them in a certain way without actually looking how they ‑‑ what they actually do. There might be all types of like conflicting window defaults as we will see and there might be even be a text in arrows which we are not going to discuss.

But looking at the slides, this raises an interesting question. That is: What is a conflict? In the sense that we mean here. A conflict might be actually both mechanisms are present, there is a network which has a DHCP 6 server, which has a router, or at least one. This router might be clear in the flags it sends, like no M flag, no O flag, which from the text books understanding would mean no DHCP 6 to happen, but still the server might be present. Is this a conflict or not? Like on the configuration implementation level it's clear but on the architecture level it might be not so clear. So it's not even easy to define like what is a conflict, does a conflict really mean conflicting information put out from different sources or just different expectations and levels of implementation? There is a number, number of variants you can come up with what actually conflict might constitute. One more thing to keep in mind before we actually discuss some level results. We did extensive lab testing, there is a white paper out there but we will just lay out some scenarios. One additional thing to keep in mind, is there is an order of things, there is a timeline for the actual behaviour of a node, it might be important if the DHCP was present first and then router advertisement showed up or the other way around, which, sorry for that, even adds more to the mess, so just to give you a very simple example.

This is a Windows system like a fresh Windows system putting up in the network with DHCP is present, but at the moment where the system boots no router advertisements are received which might just be because like in Cisco world, 200 seconds is on many IOS devices the default. What this window system essentially does is, it flows the router solicitations, okay, those should generate a router advertisement as a response. Assume for the moment for whatever reason, it doesn't get a router advertisement. What the system does then is well go and look for a DHCPv6 server, which is from the standard perspective, even that is debatable. No router advertisement, no M flag, no O flac, still go search for a DHCPv6 server, but you could justify this from the main statement in 4862, but so the, what the system does is go look for the DHCPv6 server, perform a full DHCP M flag and get an address. And then, all of a sudden routed advertisement is received and now it's interesting to observe what happens, without showing this trace, I could have put up just like a question to the audience, what do you think would happen?

What actually happens is okay, performance is slack for, actually combined with temporary address, as it's a privacy extension, so Windows default. So get two slack addresses. At that very moment it has, the system has three addresses. And then as we can see here, it releases actually the DHCPv6 received address, because oh know, there is the router, I can talk to that one, I don't need the DHCPv6 information any longer. And this is even another case where we'll see, we will see very different behaviour of operating systems.

So enough of like ranting, and going into it and laying out the problem space. Then let's very quickly discuss some lab results just to give you an idea of what actually happens. If you build a lab and observe how a number of quite common operating systems do behave in reality. Person

CHRISTOPHER WERNEY: So, we set up a small lab, consistenting of a DHCPv6 server based on the ISC server. We had two Cisco routers running the later IOS version, just so we can distribute the rDNS information over the router advertisements and we had a couple of operating systems testing okay how do they behave with this conflicting behaviour. So, the first test case is a pretty basic and simple one, as you can see, neither the old flag or the M flag is cement we have the router sending out the router advertisements and. We have the A flag set. Just to determine okay, how do all these tests that operating systems behave. I won't cover the details, but as you can see on the slides, even in this pretty basic and simple test case, not all operating system tested behaved the same, which is, which could be problematic in some environments.

Then we have another test case. In this scenario we have the M flag set to 1, we have the A flag and the O flag set to 1, and as you can see here, we have a DHCPv6 server and even in this case we have different behaviours of the operating system in in case we wanted to test ‑‑ we have different sources for the same information in this case the DNS server, how do the operating systems behave? What DNS server do they prefer? Do they prefer the DNS server received from the router advertisement or do they prefer the DNS server received from the DHCPv6 server and as you can see on the slides, the results are different for the operating systems which of course may introduce problems, operational problems in your environment.

This is just the results continued. As I said due to time constraints we won't cover them in detail but feel free to approach us after the talk, we will happily discuss any detailed results for them.

This just a summary, maybe a little bit hard to read, but just for reference sake.

ENNO REY: The main point here, is just looking at this, it turns out for all of the test cases, there is pretty much no test case where all of them behaved the same way. So, for like five out of six test cases we have differences when it comes to the behaviour. This is the main message of this one, regardless of the actual results. It's just, well as I said, it's a mess.

CHRISTOPHER WERNEY: Then we had another interesting test case, case number 7, so initially we had just our legitimate router with only a prefix information option set and rDNS information advertised through the router advertisement. But then after a while, we introduced a second router where we set the M flag, so, that the clients may or may not go to the DHCPv6 server to acquire the DHCPv6 address and to the DNS information just to get a feeling okay, after the initial configuration the clients have configured their address of a slack mechanisms and have the DNS server, what are they doing when they receive a route advertisement from the second router where the M flag is set? And in this case, we had an interesting results, I don't want to ‑‑ it's just interesting observation, in this test case, here, for if he Dora and sent OS, firstly they just configured their address simply normal and extracted rDNS S information from the router advertisementment then we introduced the second router, they received the route advertisement from the second router where the M flag is cement they have gone to the DHCPv6 server, got an address, emptied the DNS server information from the DHCPv6 server, but then the interesting thing is when they received the router advertisement from the first router again, they lose all information from the obtained from the DHCPv6 server, so this means the IPv6 address and DD NS server information. And that might be, as written on the slides, a trouble shooting nightmare, but as I said I don't want to bash on those, it was just an interesting observation during our lab testing.

And a couple of more results for Windows 7, window 8.1, and here we have also a summary, just to get a feeling okay how do all these different operating systems behave.

So, this was just a little excerpt from our test cases. As Enno already said we have released a white paper where all is written in detail. So I'll hand back to you for the conclusions.

ENNO REY: So what does all this mean actually? First of all, don't assume anything. Don't assume a certain type of nodes behave in a certain way just because there is something in the RFCs or another O Yours sincerely does it the same way, or you have like a predetermined understanding, oh, in IPv4 it was like this, DHCP had a certain role and DHCP was considered to be exclusive source of information, that's why in IPv6 it's ‑‑ it will be mostly the same, probably we don't have to tell the audience in this room this, but still, we'd like to emphasise don't assume anything. You have actually to test pretty much anything once it comes to like large scale deployment of nodes in a header environment.

Second, I mean this is somewhat a long the same line, well... most of you know this quote, you can really understand certain things and this applies to IPv6 environments more than ever without actually getting your hands dirty. Which brings me to this one.

These are what we think the two most important RFCs ever and the one that I'd like to highlight is 3439, keep the simplest principle in mind when this comes to deploying IPv6 on a large scale. I can't stress this enough.

As for the like an operational implication, of course try to keep whatever type of source of information you might have in your IPv6 network, try to keep the sets of information that those sources provide in sync, which is easier said than done, which is exactly the type of problem with layout, but this is really, really important. Once you have like conflicting information, you will see all types of strange behaviour.

As for the planning, usual there is some question okay, what do you actually recommend how to deploy like the interaction of slack and DHCP? The short answer is it depends. The longer answer is on the slides. In the tend of day it depends on your environment but there is some things to keep in mind, which is supportive of the nodes that you see in your network, if there is androids that you have to go with rDNS in the info. In the route advertisements you can't rely on the DHCP. If it's Windows it's kind of the other way around. If you use DHCPv6 at all, I'd like to stress this one explicitly, be aware that it has a fully different behaviour in the local link. It doesn't have neighbours in the sense. At least in the default way. So you have to operate probably with a clear NDA flag. If you want one provisioned address but leaving the L flag intact, which means at the end of the day means for many environments the best way of going with DHCPv6 is along the lines, now listen carefully, is having a router advertisement with a prefix information in it, which essentially says well, here is a prefix information but you DNA nodes are not supposed to use it for the configuration, so clear the A flag, I still tell you because their no ethernet environment that's why I leave the L flag intact. So I whisper something in you're ear to help get DHCPv6 running smoothly and the way you are used to it from IPv4, that is one DHCPv6 provision system can see another one on the local link, which by default it can't, it will go to the router and wait for the redirect which that's even adds even more to the mess.

For trouble shooting there is some things one has to be know, one has to be able to diagnose the actual behaviour of a node by a certain means on a command line. Obviously it helps the truth is in the packet, as we all know, being able to sniff at any time will be needed to diagnose problems in these types of scenarios, being familiar with this stuff is essential.

So in the end of the day, summarising it, in IPv6 the RFCs, and there is many of them, and there is certain generations of them, don't really provide actually, they don't have too much, as it is called in IETF language, normative value, more like indicative recommendation or advisory value, faceless fact, I won't comment on this right now. I have a strong opinion on like the work the IETF does in this space or misses to do, but ‑‑ well from an operational perspective, expect surprises, get your hands dirty, test your stuff and keep 3439 in mind.

Thank you for architecture attention. We are happy for any type of discussion.

(Applause)

JEN LINKOVA: So, thank you very much, very interesting. I know that slack verse DHCP is a really really hot topic so I expect people to run to the mike.

AUDIENCE SPEAKER: Stephen bawd. Actually, I can kind of understand OS vendors having strange behaviour in connection with in DHCPv6 because I see it working from the other side working with CPEs I see what we get as configuration information from ISPs is not very clean. Some ISPs don't send RAs; they assume you just set up a default ‑‑ which is like broken in so many ways. And in the router space, the M and the O flags have different meanings, meaning you need to get a prefix even if there is no M flag, and you end up just implementing or inventing strange heuristics to try to get a prefix via the IPv6, sometimes this works, but sometimes the ISP also assumes to get a statement address ‑‑ on the IPv6. If you try to get both but it will only give you prefix, this fails you have to restart the process and it's really not very clean and there's really no good way to handle on the clients side either. So, both sides are basically screwed and have to deal with each other in any sensible way, if that's even possible.

AUDIENCE SPEAKER: A mad sad a. I totally agree that this is confusing, and a lot of things will have to be considered. So, for instance, the autoconfiguration, should I take the present Mac address, configure the address or present in the privacy extension or cryptical generated address, then if you go further if you want to consider the security, then in this case I will say also this contradict with the autoconfiguration, because ‑‑ in this case you need to do some configuration for certification, public keys, so it's being ‑‑ so the configuration and automatic process.

But another thing I would like to say that I think the design of IPv6 at the beginning, they thought to get rid of DHCP and they want to configure IP version 6 by autoconfiguration mechanism, so why the impact of the DHCPv6 again. So this is my question.

ENNO REY: Okay, if it was actually a question. We could redirect ‑‑ ask like six men at IETF, but, the point is, I fully agree that my perception is that the DHCPv6 in the IPv6 universe was and is considered like a side thing, which is not really important and you can absorb this in many things not least since 3315, not much has happened, I know 3315 B is being discussed, but like, 20 years have passed, and it still was considered to be a niche but now lie enterprises that deploying widely and Microsoft doesn't support 6106, you can't really work a heterogenous world involving DHCPv6. I'm not a big fan of DHCPv6 anyway, let me state this. But... well, if something went wrong, I would agree with this, but that's not an easy question to answer like from our perspective. There is guys you might ask over in the Hiltons of the world.

AUDIENCE SPEAKER: Iljitsch from Ben a.m. and I was at IETF when we were fighting about DHCPv6 and the autoconfiguration. The one thing I want teen at the previous comment err at the microphone is actually it's not that DHCPv6 was removed from IPv6, because IPv6, when they started working on that, DHCP didn't exist yet. So you have to remember that IPv6 is the successor to the IPv4 of the early 1990s which is a very different IPv4 than the IPv4 of the late 2000s and the 2010s, so that's why it makes sense of course it doesn't help in the current situation, but what I want everybody, I was very happy to see that you don't have any list of things that you want IETF the write up even more RFCs because the problem with that is if people do weird things because they don't read the RFCs very closely, then writing even more RFCs usually doesn't solve the problem. So I'm happy to see that is not what you want.

ENNO REY: Let me just show this one and the implications cases of this one so I fully support like don't add even another generation of RFCs and making the problem space from that angle larger. More something to choose from. This ‑‑ fixing this, would require, like, community would require like Windows and implementations and understanding the problem in the first place, which is exactly what we wanted to contribute to today. Anything else?

JEN LINKOVA: More questions, comments? We are almost on time. So thank you very much.

(Applause)

So, when we have scared people enough about deployment, Aaron is going to tell us positive stories.

AARON HUGHES: Hi, good morning, I'm the CEO of 6connect. And Jen asked me to do a presentation on IPv6 at ISPs and I thought to myself well, we all in these communities generally know quite a bit about dual‑stacking backbones, but ‑‑ we don't really talk a lot about in these communities about what IPv6 for ISPs looks like holistically. And when we think about it, we use a single measuring stick, right, we look at the total traffic that is v6 on the global Internet, by well known measures like Google and then come back and say we have reached X percent and we all go hey, that number is going up to the right. It doesn't give us a sense of the state inside each ISP and where we are with v6 implementations. And I find that the perception is actually different from all different kinds of people that you talk to. Where when you talk to people on the network side they say yep, we're dual stacked, we're done. Backbone is good, peers are done, transits are good, traffic goes through on v6 and v4, that's the end of the story. You talk to people that are in sales or sales engineering, and they will say yes, we have that, it's supported. Or, maybe they don't even know about it.

A lot of DevOps people believe they don't have to do that much. Development Operations staff. So people that write tools, work internally for you know helping with system support, you know, and outside of the minor, we have to change the four integers into understanding what a v6 address S they don't think they have to do all that much. And system support staff generally think things like well I have to change one little bit in the configure file and help a process or restart a service and we're done.

And Cloud providers are really interesting when you talk to them generally if you go to buy a VM instance you can get a v6 address on the VM.

So that's kind of a rough things that I hear. So this is really about exploring the realities. So what I did is I called a bunch of friends who work at ISPs and Cloud providers, small and large, different countries and just got a sense of what, where people were. And trying to figure out a way to measure this is challenging and make I can take something like this and come up with something tangible for a repeat interview or survey format. If this is considered useful to say where are we actually at in terms of v6 implementation in ISPs? So this is a sort of a stab at that.

There will be a couple of examples in here where I have taken interviews and documented what the results were, and then there is some aggregated data at the end of this. So there is three or four examples in hereof people I talked to and their specific answers to these questions.

So this first one was a small Canadian ISP. And they said we had a simple v6 plan, it was phased over time. They were going to start with BGP in the core which made perfect sense and add transit and peers over time. And as part of server refreshes they enabled v6 functionality over time. They were one at an a time or a couple at a time. They found some things had to be depricated and some outlying systems had to be replaced and some network components had to be replaced. And they didn't have a large lab, like many of us do; they had a fairly small lab. So, they made changes in small changes at a time.

And they found a couple of minor bugs, but nothing huge facing customer concerns. So in this case, this is a fairly standard list and this is what I expected to hear from everybody. Was their routers, switches, back end monitoring, DNS, mail, control panel web servers, radius, backup servers, VPN, corportate firewall, the standard list of things you think that's what we're going to dual stack and be done. This is the lessons he gave me along the way.

So, the first one, and this is common I heard, is the big unknown, right, all the servers and routers need to be updated to refresh. That's expected. A mistake for firewalls in v6 not supported in Redhat 5. That was an interesting one. Feature parity I heard over and over again. In this case feature parity. FreeRadius 1 isn't support sending v6 packets packets.

The next lesson he gave me it's the small things. I experienced this myself. In this case fail 2 ban, a firewall script for blocking didn't a v6 support. Lots of custody some scripts written years ago were making the assumption for v6 and IPv4.

This is probably somewhat slang reverse DNS is a BIND, with a smiley face, he meant to say he had challenges with ISC BIND so they migrated to power DNS to simplify IPv6 reverse delegation. This is related to the lack of tools to help this this automation, again that's a DevOps we have nothing to do but the reality is there is a lot more to be done with tools. Another take away from this and obviously we all know this is how to plan for your net block. This person said well a 32 is a lot space for a small ISP unless you are doing something really silly, and they cut these into a 36 per POP or tie down or whatever you want to refer to it and each business got a sparse allocated 48.

They have been working on this for sometime and so now they have got the majority of their services dual ‑stacked and they have run into some problems, but effectively what he told me was that about every week or so, they turn on a service and they start a timer and then as we are going to convert one box, we are going to watch it for 28 days, if it's successful and nothing happens, we'll go onto the next one. And they have been going through this process for sometime and their final step is just enabling v6 for their DSL and fibre customers, which they haven't done today but they are almost there.

So I talked to something slightly larger. This is a medium sized provider in California. So, consider it fairly heavily peered, lots of transit providers, a decent sized network that does co‑load transit, management services, etc.

This was a little more interesting, and fairly common what I heard across the board for ISPs, is that they started peers early on, in this particular case Hurricane Electric was a huge help because they had full v6 tables a long time ago over v6 for peering, which made it really easy if you didn't have access to transit providers and started at the edge and worked into the core. They had trouble with their transit providers. It took sometime. This was about six months to get everybody actually dual stacked and seeing tables in BGP. They had enough redundant to do some more things and not just dependent on being dual stacked with a few providers with full tables. They had sub‑net size challenges and I think early on we had this debate about whether to use 127s or 126s or 64 object BGP links. They had started off with 64s and I can't remember whose presentation it was, but it was sometime ago, I think it Igor from Yahoo, who talked about this TTL bug where he could attack from point to point linking and packets could bounce back and forth on the interface. Then in this particular case, they eventually switched to 126s verse 127 because when they tried to train their operation staff or talk to the customers, nobody was happy with the concept of one link being colon‑colon 0 or just colon‑colon. This is a really interesting one. Out of all the interesting ‑‑ it's different in almost every case. There is a good deal of people that use 126s with: :1 and: :2 and then discard the rest of the 64. It's probably the most common of everybody I talked to, and there are some that are actually incremented 127s out of like single of 4s her region and it looks really nasty and it's very hard to read but people are actually doing this. So, it's an interesting report to see. But I think the overall majority were using 64s with could he and could he and one and could he and could he and 2 on the other end and discarding the rest of the block.

So next in this particular provider. Dual stack was completed. They added SLAAC to a few subnets. It's funny when I was talking to this particular person they were telling me they were apprehensive about adding interfaces for v6 on any of their server environments, because the very first one that he did, he had a VLAN with a bunch of servers on it and he only had a few of them were test and others were production, and in this case a Cisco environment and the first time they added a sub‑net on an another face of course all their boxes ran in UNIX, picked up an address from SLAAC and he inadvertantly had servers that were starting to use v6 for outgoing requests and operators ran into production bugs that they didn't know how to support, so, more carefully started adding v6 to interfaces later, but you know, I just thought it was kind of an interesting take away from the conversation was that nobody was used to SLAAC just being on by default and having to deal with the bug and then weren't prepared for it.

Writing tools like IP M was painful process. It was a different approach for v4, you know a bit like feature parity issues. Internal tools took a long time up to date and training people like steals sand sales engineering with things like line items for v6 blocks and sales force were challenging conversations to have. Another one I didn't expect to see but a lot of feedback I got was just having conversations internally with staff was a real challenge to say hey, we're going to do something different, even if it's like a zero dollar line out and we're adding to invoices to say we are giving you some flavour of resource to use. And this was in preparation to increase prices for the v4 utilisation, but we starting along the way it caused challenges internally.

And I heard this again, selling this story internally to staff was talking about like Y 2k, we are going to need to v6 and getting people to actually react and accept that this was a real serious change was difficult. This was a constant feedback.

So over time, customer entrickle in slowly, staff got to actually do some implementation and started getting some real hands‑on operational experience including customer BGP, INET6 objects, etc. And for this ISP it took another year to get internal services dual ‑‑ stacked, which I think is probably about the range that most of us use. It's over the course of a year it starts to flow from the backbone is done into supporting services internally.

Some of the training issues that came up were interesting, so, teaching people how to deal with PTRs for v6 and I still see this today in overall majority of things like traceroutes where there is just no reverse DNS on things and I don't know if this is because of lack of tools or training, but people for some reason, consistently said it was hard to get people to edit zones with PTRs for v6. And filtering automation is a consistent response, in this particular case they were using IRR PT, which I think I was written by Richard Steinbergen many many years ago which son‑in‑law supports v4, and these guys were using you know rancid ‑‑ W and they said they had no way to deal with v6, they would manually generate these prefix filters, you know and while there are a couple of other solutionings out there because they were used to IRR, we were

They found a lot of hard core references in code. Logging analysis, abuse supporting, just norm atting mail into certain things. Some of their log check scripts were expecting v4 addresses. And they had some challenges with monitoring tools. This one primarily was external related. So internal tools like not use inter map etc., Observium, they all seem to support the idea of monitoring v4 and v6, but the dual stack monitoring services, this is again what I have heard consistently across the board, I don't know of a single external monitoring platform that actually properly supports dual stack, and I heard this over and over again. Everybody that's out there will get you to add an additional host for your v6 stack to monitor those services rather than understanding that a host actually has multiple addresses and you want to do the same thing on both stacks and if there is a failure on either to react to it. There is still no good solution that I found anywhere for dealing with things like name based virtual hosts where you want to have something actually happen a trust anchors on an application stack, look for unexpected return and if there's a failure on either stack, alert. So this is a fairly common one. Debugging on which stack is being used is still a serious support issue. This is a consistent issue, particularly in tickets where people say I'm having problems getting from X to Y, still a big problem in everyone I talk to.

Let's see here, also, consistent, v6 is not engrained in decision making. This is ubiquitous. You know, we all sort of made a commitment over the last few years to walk away and set the expectation that we're not going to work with vendors who don't support v6. Whether it's hardware or software or you know the future viability of your organisation depends on having these things supported in the future and not making mistakes spending money now. Even a consistent reminders to provisioning staff that dual stack must be the default and when you talk to somebody about purchase, this must be included and actually fully supported was a reoccurring issue.


Downstream training is a big deal and this primarily was pushed back from sales. I have heard this consistently, is that you know, people don't really want to have the conversation with their customers about their v6 plan and they have some fear about I think bringing the topic up because your exists customer, your prospect might ask about your v4 inventory and use that a metric to determine whether they should be with you or go with someone else.

Feature parity, I heard this over and over and over again. All kinds of examples of serious feature parity where you know people would expect when they got beyond the edge of the network and wanted to enable the same service on v6 as they had on v4, failed. Primarily in IPS, IDS, firewalls, we heard that, analysis for flows, logs, filtering, DNS, DHCP. And new odd behaviours where they had subnets that were dependent on v4 NAT and added the second stack of v6 and having to figure out how to create some sort of consistent policy across those interfaces.

Internal support star was also a huge issue, people that have 24 by 7 staff and rotating people. The level of education is now a significant difference between the people who understood dual stack and who understood v4 only. We find that a limited number of people to escalate to and all of a sudden if there was a problem on v6 it got escalated very quickly from tier 1 to tier 3 instead of having the same general knowledge on how to debug issues.

Security was a consistent issue. Of course it's entirely different to secure. Duplications of policies don't always work where I have the policy for v4 I'm just going to cop copy to to v6 and it's going to be great. NAT no longer be he demark required a unique policy. I was surprised to hear this, but I heard this a lot that the edge demark change, whether it's a down stream modum or sudden network supporting internal systems really required a lot of thought and changed in internal policy. And that there were a tonne of applications that surprisingly had hard coded BINDings to v4. This was a really funny story actually someone told me about sending a PGP where the application was just listening for the destination port and went do the encryption on the fly and as soon as they dual stacked their SMTP servers, even if they had rules that said encrypt this message it's a must, if the connection outbound hit the SMTP server over v6 it just ignored the application and went on in the plain text. Fantastic. This of course is a feature request, not a bug.

Cloud service providers. This was a great conversation, when you ask a Cloud service provider on a VM to get dual stack, certainly you can get v6 addresses for most of them. When you ask them about the management platforms for the underlying provisions systems and management systems I found absolutely no support for v6. None of them. And even the orchestrators of the VM platforms had no support for v6. It wasn't even on their road map. Nobody seems all that concerned, which I find absolutely amazing. And some of the discussions about how you actually manage these things their typically deployed in multiple instances of overlapping 1918 space and a management layer that does mapping back to things like open stack, UUIDs, the idea of adding v6 to this was unheard of. Even the few I was talk to about what they were doing with management and starting to introduce the idea of open access on very large management VLANs or v‑ X lance, they wouldn't add v6 because as soon as their VM knew about a v6 address it would look up the AAAA and try and use it and wait for the no route to host before it would execute out the default 4 gateway. There is some interesting issues in Cloud service providers as a whole and all their management and supporting systems. And everyone of them said there's just no demand for it and I think that's because they are not asking people to do it.

Eyeball access, this was all over the place, DSL cable mode items have a mixed set of support for v6 and really odd behaviour. Feature parity, as far as I know, is missing from every single vendor out there, being the closest to having feature parity. The rest of them were just everybody said that there was something wrong with feature parity.

Allocation customers sizes vary from all over the place from a 64 to a 48. I think this is interesting, you infer really hear about this except for inside meetings like cable labs, nobody will be completely talks about what they are going to do as consistent policy for their customers or what the future looks like, so right now we're in an interesting state where this is just a blend. And the majority of people that responded to the single 64 as a single connected assignment. I found that interesting. That's pretty low.

So, conclusions:

ISPs have no trouble dual‑stacking their backbones. That's pretty clear. The major public services were mostly easy to deploy. There is still a great deal of work to do with hardware and software vendors. We all know this. Many vendors claim support for v6, but either don't work or are only partially supported. Completely missing in all evaluated Cloud orchestration products. This was a surprised to me but maybe it's not to some you.

Getting to next steps, default for v6 for all customers, services, educating staff same as v4 is going to take a lot of time. Everybody had problems in this area.

And everybody I spoke to was comfortable about talking about these kind of questions, so, I thought that was great. At least it leaves the opportunity to go back and get sort of a state of v6 implementations across ISPs. Although, almost everyone I talked to wanted to be anonymous.

That's it.

(Applause)

JEN LINKOVA: Any ISPs in the room? Any ISPs who is deploying v6 now?

AUDIENCE SPEAKER: Hello, from Telecom France. Thank you for pointing out that things in the IPv6 world are not that bright and shiny that some people would like to believe, and the IPv6 deployment, it's complicated, especially when we get up in the chain of management to CEOs and the people who don't really understand what is it all about, what does it mean, and what are the implications. This is another issue; you didn't really mention it, but it does exist, and it's quite serious sometimes. We are deploying IPv6. We are having a number of issues, mainly people don't understand that you can't just throw in IPv6 and that's over, you no longer need IPv4, everything is nice. They don't understand this. This is something that probably needs some more education.

JEN LINKOVA: Would you be willing to come and share your experience or you prefer to stay anonymous?

AUDIENCE SPEAKER: Some day, yes, I can't answer right now.

JEN LINKOVA: I will be patiently waiting for you.

AARON HUGHES: Maybe one generic question is: Is an updated sort of state of ISP a useful thing to bring to this kind of audience? Perhaps even creating a generic survey we could send out or personally go out and ask people where they are and try and figure out a metric something other than we are X percent of traffic on the global Internet for v6. Is that useful? A couple head nods. Anybody just think it's a waste of time? Okay. Good enough. Jan?

JAN ZORZ: So, you pretty much documented the current state of IPv6 deployment at the providers. But what surprises me is that we are still talking about IPv6 enabled, IPv6 compliant. I thought we cleared that up with RIPE 554, I don't know. Maybe you should share this list of requirements more widely on the other side of the pond.

AARON HUGHES: Yeah... you know, I heard all kind of of stories. One of my favourites was I went out for a printer search a few years ago, and I had a customer that required that they were v6 enabled, they were a Government customer and so I found all of the ones that had v6 compliance on them and bought them all and tested them and my favourite one was it under on a printer that said enabled v6, as soon as I enabled v6, it got a v6 address and turned off v4.

JEN LINKOVA: I really love it. We actually don't not need two protocols. You just need just one, it must be only one.

AUDIENCE SPEAKER: I have a quick comment, and so Peter Hessler, with Hostserver in Germany, and I have seen a few test networks where I set up a dual stack v4 and v6 network, got everything working, turned off v4, and suddenly everything broke because I didn't realise that there was a number of v4 services that were fully dependent and I was not actually utilising all of the v6 ‑‑ and not actually testing all of the v6 services that I thought I was. For people that were implementing the dual stack it's important for them to also test the v6 pure network, maybe not deploy it yet if it doesn't make sense for them, but to keep that in mind that you are likely to run into a lot of problems.

AARON HUGHES: That's a great comment.

JEN LINKOVA: Any more questions, or are people just need coffee so desperately? So I think that concludes the session. Thank you very much.

(Applause)

Last, please rate talks, because we need to know what you'd like to hear about and if you have any suggestions for the next meeting, what you would like to hear about, let us know. Thank you.

(Coffee break)

LIVE CAPTIONING BY MARY McKEON RMR, CRR, CBC

DOYLE COURT REPORTERS LTD, DUBLIN, IRELAND.

WWW.DCR.IE