Archives

Anti‑Abuse Working Group

Wednesday, 13 May, 2015, at 2:00 p.m.:

BRIAN NISBET: Hello. How are we all? Good afternoon. Welcome to the Anti‑Abuse Working Group session for RIPE 70. You are all very welcome. We have quite a reasonable agenda ahead of us today, but with hopefully some very interesting topics and discussion.

I am Brian Nisbet, Tobias is the other co‑chair of the Working Group. Thank you very much to the NCC support on scribe and Jabber, as always to our wonderful stenographers. If, during the meeting, you do ‑‑ you are moved to come to the mic phone, please say who you are and some sort of affiliation, I believe the cool kids are just some random Internet citizen from off the street or something, which has a huge workforce, it seems.

So, the Internet, it employs people. Other administrative issues:

Minutes of RIPE 69 were circulated. There weren't any comments so unless someone wishes to suddenly object now, we will deem them as done and dusted and official.

No? Good.

I have made a couple of very minor changes to the agenda as posted but they are not content changes, they are just order changes, one of which makes a lot of sense. So, I think that is that on administrative stuff.

So, Working Group Chair matters. I am going to sit down now and let Tobias do this bit.

TOBIAS KNECHT: Yes, most of you will know me. At the moment, we have one open chair position, the mailing list was asked if somebody wants to step up and wants to take over the Chair seat of Brian. We haven't heard anything on the mailing list. Is there anything in here that wants to challenge Brian in a way? If there is nobody I think that is an agreement that Brian will take the next two years, if I am not mistaken ‑‑ three years, sorry, and is there anybody that objects? Nobody. Congratulations, Brian.
(Applause)

BRIAN NISBET: Nice and simple. God emperor for life. This was the first time we went through this, hopefully it will always be that painless but please do note that the procedure is there and the process is there. Not only to keep us honest but also to make it a very clear way of other people getting involved in the Working Group, should they so wish.

So, oh, there was one typo in the agreement that you all agreed to on the mailing list, which was that it does say officially it applies to every Working Group in the RIPE community. Given they have not seen fit to grant me that power that is a typo and it only, the procedure only applies to the Anti‑Abuse Working Group.

So, moving on from there. Recent mailing list discussion so, the abuse‑c clean‑up piece which will we will discuss in a moment, the other kind of most recent discussion was in relation to abuse‑c contact methods and people asking about web forums rather than e‑mail and things like that. Now, I believe that was sufficiently discussed on the mailing list but if anyone has any comments they wish to make on that at this point in time now, for, against, otherwise, please you know where the mic is.

AUDIENCE SPEAKER: Co‑chair of the database Working Group. And just to note, I make ‑ dish post an e‑mail to the Euro‑IX holders mailing list which is separate from RIPE NCC mailing list, just for legacy holders and I bring their attention to the abuse‑c discussion which is right now taking place in BoF groups in anti‑abuse and database. Just as a sidenote to the holders' situation.

BRIAN NISBET: Thank you for noting that. Just to talk about this, I suppose, for a moment. The abuse‑c clean‑up, we have ‑‑ my plan certainly, unless the Working Group wishes to do otherwise, is to leave that discussion to the database Working Group and the database mailing list. I think that is the proper place for that discussion. There are other things we will probably need to talk about here but for the moment certainly the ‑‑ that particular piece of work and Denis Walker's pseudoproposal which will turn into a real one, hopefully soon, will be left in database. But obviously we will keep a fairly close eye on it here in regards to that.

So, that was my ‑‑ that was my comment on policies, that is kind of the only one that is currently out there that particularly involves us. There was a comment this morning in Address Policy in regards to people doing regular verification, it happened to be in relation to the use of ASNs and people ‑‑ and the NCC doing regular verification in relation to ASNs. And that is something which I certainly will be keeping an eye on from the point of view of the discussions we have had about regular verification of information. The comment I made in the Address Policy Working Group this morning was that it potentially opens a door, it's not a positive or a negative aspect but if the NCC is suddenly sending everybody a mail once a year to check they are still using their ASNs properly, then what else might somebody suggest or otherwise could be put into that e‑mail. So it's something to be aware of, I think, rather than anything else. And that is I think ‑‑ as I said, unless anybody has, just in relation to the contact methods or otherwise, I think that is a done and dusted on the mailing list, so, policy‑wise we have that, that is being discussed in database.

Again, unless anybody has a deep need for it to be equally discussed here, and of course we can, but I think DB is the right place for it, personally.

So moving swiftly on, this agenda is very much the first 70% of it takes about 10% of the time. Interactions. Obviously, the database Working Group, who we seem to interact with a lot. I think this is a positive thing and abuse‑c is there.

So, the other piece then is the NCC and RIPE community LEA interactions, Marco is going to present on that in just a moment. Just to again say that I was present at the NCC and UK national crime agency's meeting in March, March of this year, and Marco will be talking about it in more detail but I wanted to say the format was changed this year and made more of a discussion format, which I think was substantially most useful for both myself and the other representatives from the RIRs who were present and I think more useful for the LEA representatives as well. We went from almost a chalk and talk situation to a lot more round table and interaction, opening up of the discussion in relation to operators and law enforcement, and it's been great to see certainly the UK LEAs turning up to events like UK network operators forum and trying to engage with the community there, that has been very useful and the right way to go. But I will let Marco talk further on this.

MARCO HOGEWONING: Thank you, Brian. I work for the external relations time in RIPE and as part of my job I try to coordinate our outreach and activities with law enforcement. So the main goal ‑‑ and I have been here before and I promise to keep it short ‑‑ but the main purpose of this outreach is to increase efficiency on both sides. Most data that people are after are available from public data sets so they can just use Whois and they don't have to knock on our door so it's primarily capacity building, what information is available and what is not available, and to a certain level, raise awareness about our policies and procedures.

So the main dialogue at the moment is discuss observations and law enforcement, we in the RIPE NCC regarding some activities, for instance, people trying to break into database accounts and assist each other in finding the right channels and fora to address these concerns and of course, RIPE is one of them and later on you will have a presentation by the national crime agency will talk more in details about some of the stuff I have seen.

As Brian said, law enforcement meeting to increase the level of reporting on this. As Brian said we did another one on March 12th, we change it full day meeting with full cooperation with the National Crime Agency regarding the agenda. ARIN, APNIC had presence there, RIPE NCC Executive Board was present and Brian on behalf of the community, joined our meeting. Law enforcement representation included people from 25 different countries, Europol representatives, and a main part of the day, basically the whole afternoon, was taken up by a discussion exercise led by professor Ian walled on where bailed on some examples of recent investigations things did go wrong, tried to encourage people think out of the box, how can you do that. It might be law, it might also be policy changes or better cooperation.

So the outcomes, as one of the exercises people still largely prefer the current industry multi‑stakeholder open bottom‑up‑led initiatives that govern the Internet. There was indeed agreement that law enforcement need to engage more fishily with all these fora, be more proactive in addressing issues and taking steps to inform you what goes on. Option of IPv6 and the wide spread use of address chairing technology is still a big issue and remains a concern. And was a lot of stress about the need for public private cooperation. We, as responsibility to keep the Internet safe and stable and clean, so we all need to do our part. Operators need to secure their servers and networks and also act swiftly if we do identify compromised systemings. The report is much longer and available on the website, the URL is here.

Other engagement activities, recently there was the global conference on cyber space ‑‑ this is a very big and high level conference with representatives, ministers from over 40 countries. There was a lot of stress on cybercrime and cross‑border law enforcement, again there was a lot of statements made towards public /HR‑RB private cooperation and recognising everybody as their area of responsibility when it comes to Internet governance and when it comes to making the world a better place.

I have also been involved in some capacity building activity with Europol and EC3, again a a lot of using our public data sets and making sure people understand what they are looking at a and bit of tips and tricks to ‑‑ APIs and automation things. Upcoming activities, we are skid /AOULD to have a meeting with Interpol in reign Jan cyber place in at the ran later this month, focus on capacity building and Internet governance again. We still do dedicated law enforcement training courses, helping people to understand who is helping people to work with our tools, planned courses include UK and a day long course in Iran while we are there and of course continue our working with Europol EC3 in particular on outreach, outreach towards our members but also towards the law enforcement community and making sure that everybody understands who we are, what our role is and also to some extent because a lot of people do understand a bit of DNS and do understand domain name side of things, explains where the differences are between the names and the numbers when it comes to, for instance, the registration in blocks V individual registrations, etc.

My final slide then: Is this all useful? Well, we have released our transparency report over 2014, RIPE 633. The number of requests is still declining, seven requests in total, five came in asking for can we get the name and address of this particular user of an IP address. Well, that is information we don't have. So again, we explained what we do. One about the actual procedures regarding LEA requests and we had one request to confirm that the Whois output they had was actually real. So we basically put a stamp ton that say, yes, this is real.

Document is available and of course we are going to keep reporting this.

That is all I have to say. Any questions and any particular ‑‑ did people actually read our law enforcement report about the last meeting? Because there were specific requests from this Working Group? And are you happy, that is my main concern.

BRIAN NISBET: Any questions?

AUDIENCE SPEAKER: Hello. Alexander ‑ private at person. I have two questions. I have read report you published on website and mentioned some presentations were made during this meeting. Is it possible to see these presentations and details or may the reporting so something like this? Because OK, any interaction as I mentioned on previous RIPE meetings and anti‑abuse Working Groups, any interactions with law enforcement must be transparent.

MARCO HOGEWONING: I thought we actually linked to the RIPE NCC presentation. We can of course, publish that because we own it. Unfortunately, I don't have ‑‑ I can't immediately publish the slides, they are represented by the National Crime Agency, but the presentation you are about to see from the NC A is pretty much what was presented at this meeting. So ‑‑ and I think ‑‑

AUDIENCE SPEAKER: For sure ‑‑

MARCO HOGEWONING: I don't know whether we could publish this.

AUDIENCE SPEAKER: Could you ask to make the presentations public. And the second question, I need no answer but I need just to raise it because during this meeting with law enforcement, there was no Russian law enforcement agency in this meeting, but, anyway, having experienced and worked with Russian police and also crime units and something like this, I have one ‑‑ I want ‑‑ you may ‑‑ start working on the following question: Law enforcers during this meetings ‑‑ during education and training courses you are running because getting weapons ‑‑ use it to fight crime but actually they could be abused, so the question is how law enforcement agencies are working with cyber crimes and so on could be controlled by society, by public sector and other things, they ready to answer these questions during this meeting?

MARCO HOGEWONING: So you are referring to the possibility that law enforcement is compromised when it comes to ‑‑

AUDIENCE SPEAKER: How they allow us as a public to control them, control their activities, with this new weapons?

MARCO HOGEWONING: Yeah ‑‑

BRIAN NISBET:

MARCO HOGEWONING: It's a really good question, it comes down to trust and of course, these people, yeah, we have to trust similar that we trust the rest of our opportunity they uphold the law and they report back to their users and their stakeholders just as we report back to our stakeholders and our community.

BRIAN NISBET: I think it's important to note, and I want to say this, is that certainly unless Marco is doing really, really secret things, with the which point we have got to suddenly start wondering about the NCC and I do not wonder about the NCC, the training and the tools that are there, there is no secret access into NCC information or otherwise, these are the public tools which the law enforcement are being taught in the same way as the NCC tries to teach operators how to use them, so there is no secret tools or access, that is all still the private information still requires the court order, etc., that has been discussed numerous times. So while I ‑‑ from a community of view, while I understand your concern in relation to potential forces in certain areas of the world, there is nothing they couldn't do anyway.

MARCO HOGEWONING: We have to realise that part of the capacity building is making sure that they understand because what you might perceive as abuse of tools, is to some extent people not understanding the data that is available and misinterpreting that that could lead to flags putting in the right way or pointing at the right people, we try and see if they do use our data they use it correctly.

AUDIENCE SPEAKER: OK. Let's have more control over them, just on the next meeting ask them how we can control what they are doing. With my experience in Russia, the Russian are being created and the bravest one under criminal case of abusing fighting tools against ‑‑ citizens. So I don't know how it's going in European Union but, OK ‑‑

MARCO HOGEWONING: I have no answer. I know in IGF these things are also addressed from civil society and an ongoing discussion as well so there are fora addressing this from the community perspective.

BRIAN NISBET: Are there any other questions for Marco in relation to any of this? No. Thank you very much.

MARCO HOGEWONING: You can always e‑mail me.
(Applause)

BRIAN NISBET: One thing I added in to let the community know. The /PHO*G, MAAWG mail messages and ‑‑ ‑‑ mobile ‑‑ thank you, Anti‑Abuse Working Group happen to be meeting in Dublin this year, so Tobias would normally go along to that meeting, and the NCC is increasingly going along but I will be attending because it's around the corner and we will be presenting there on the RIPE community, on the Anti‑Abuse Working Group and the NCC and the tools they have, so nothing particularly out of the ordinary but just, you know, it's kind of the attempts we make. It's almost the reciprocal arrangement from the Anti‑Abuse Working Group in Warsaw that the MAAWG group presented at. Just to let you know on that. Grand. So now the presentation mentioned, mapping out cybercrime infrastructure‑a law enforcement approach from Jon Flaherty in the UK.

JON FLAHERTY: Good afternoon. So I am Jon Flaherty. Thanks, Brian and Marco for the introductions before. It's really good for me to be here today from a background of law enforcement in an intelligence and operational and technical and policy role, to put all those areas together in a last 12 years in a cybercrime investigations background. And really try and educate you in why, for the last five or six years, I have been working specifically with Marco. And trying to understand the architecture of the Internet a lot better from an IP perspective, to better my relationship with Internet providers and members such as yourself in the RIPE community.

To trace the who done it and to get to the infrastructure behind the cyber investigation. So when we are looking at a scene of a crime, which in the 21st century is often a computer server, usually we have a footprint in a log files of a serve and it's an IP address and it may lead to a need to develop the IP address, to find out who owns it, maybe who uses it. Associated to that, I need to use certain tools like the Whois tools, to develop that IP address, to find out more about it. I also want to look at the IP block that that IP address relates to, and the ISP that hosts it and maybe some resellers behind it. So that is really why I am here today. That is necessitated an understanding of RIPE policy and process and a really vast interest in tools like RIPE Whois and RIPE Stat which I aim to present today as part of an operational case study and it does mirror the one we did a few weeks ago with RIPE in London.

So just to set the scene. Law enforcement are used to reactive investigation and a fast moving cybercrime environment. We regularly see and react to single strand inquiries where we have got a domain name that is being maybe compromised or set up maliciously, hosted on a server, as a dropper, for want of a better word, for malware, a drive by download, in setting up the future control of BotNet activity.

We are quite used over the years, really, to react to that investigation and target the domain name. We have a prosecution‑led approach. And try and do subscriber text, try and find out certain log files against that domain name, in terms of who owned it, who uploaded content to that site, as well as the victims of crime who have accessed it for abuse purposes. It's very reactive and very ad hoc and it's really difficult to combat that as a threat, when the criminal probably who put it there is in country A and the server is in another country, country B, and the attack, the victim coming in on that is in country C. So that reactive investigation in support of a prosecution doesn't really work for cybercrime investigations when nothing from A to B to C in our own jurisdiction, it creates a problem.

So, the UK national cybercrime unit tries to look at the problem of cybercrime a different way, more a prevent and protect issue rather than a pursue and prosecution‑led investigation. When we start to use a lot of network tools, we find commonalities which I will go through in the next few slides, leading us to a more static and more stable IP infrastructure behind the source domain. And it's that that we seek to know more about and to work with industry on and collaborate with you potentially in a prevent mode to try and take that infrastructure down.

I see it as ‑‑ in my experience, as a re‑seller issue, and I think that RIPE NCC members might be caught up in the middle of this and they might share the same interests as law enforcement in doing something about it collectively.

So we want to really get behind that IP address and that source domain and we want to target the infrastructure and not the incident for greater disruption.

So how does law enforcement begin to map that, and can we disrupt it? If so, in what ways and who can we work with in industry to do that? And for me, that liens on the need for collaboration with you guys, with the RIPE community and service providers.

So, and the operation that we present add few weeks ago is what we call a high priority job, and it's looking at a bulletproof hoster, a provide they are a we believe and is very anti‑law enforcement and anti‑anti‑abuse, really, in that it will pay premium services for the guaranteed up time and connectivity of a range of criminal hosted material, malware, for example, all the way through to child support ‑‑ child abuse exploitation material. The task in this, on an infrastructure side, was to meet the above aim and that is to attribute the suspect ISP and reseller infrastructure to being a bulletproof host. So the key things for me in terms of IP, in terms of governance are attribution and traceability.

To do that, law enforcement need to manage potentially a lot of research and R and D. We used to map in one IP address and one domain name potentially, but we have never mapped an ISP before and we are not up to your level of knowledge in terms of network engineering and the use of some of your network tools to get there. But the main objective for me in this inquiry was to design an ISP mapping strategy, visualised that target infrastructure to try and see if we could attribute complicit criminality to bulletproof hosting, and in terms of disruption put that into an operational strategy. So the crux of this is, starting off with maybe single strand intelligence, one IP address, one domain, things we know on a local basis about this provider, and mapping from there using RIPE tools.

So I came up with a mapping strategy into four parts. Stage one is the collation of all the intelligence we have to date on this hosting provider. What do we know about it, names, phone numbers of people who are associated to it. Have we come across them before in these reactive investigations where we found an IP address on the end of a DDoS attack and, you know, of a network intrusion, some BotNet C2 C activity. Where maybe there has been a commonality where we can see the same IP addresses from the same range or ranges of a reseller or an ISP. Bulletproof hosters tend to advertise their services and their wears in a lot of forums, you might know them as carding forums and criminal forums and they go by nicknames, private message people to offer these premium services and that also, from an OpenSource perspective, is what we can collate and we can add that to the reactive scene of crime IP intelligence to collate intelligence and that is stage one of the inquiry. The real thrust in terms of RIPE tool usage is stage 2. A lot of intelligence we get and receive and look at is IPv4‑led, so we want to look at essentially key‑word searching, stage one intelligence across RIPE data sets. We want to actually map out all those blocks and find out exactly what RIPE hold publically on the Whois and on the wider issue on RIPE Stat via the ISPs who ‑‑ and members who route traffic to them. That brings us into stage 3, where we want to start drilling deeper into target ranges, where we want to map current and historical domain name traffic, pointing to or transiting to a range of different servers, to start to enrich the data and find out maybe which ranges are good and which ranges are bad.

Stage 4 is to map that chart that and use software to visualise that to a decision‑making for lead officers in our investigation.

So in terms of this bulletproof host, stage one, what do we know so far. Not much, really, at this stage. Other than what we call single strand IPv4 intelligence. A few names, some mobile phone numbers, what we believe are business drop addresses, for the people who we suspect are running it, operating under some alias names and at this point in the investigation, all the provider infrastructure looks to be located in one country.

We reviewed that and we collate that to key word search it. So, predominantly, we are looking at drilling a little bit deeper down to Whois records on RIPE Whois. We want to find ISP and re‑seller for IPv4 address ranges, both by searching and querying IP addresses and also names and perhaps those mobile numbers that we have got, all those selecters we want to run through RIPE. Specifically, we are probably looking at results that expand the picture of the bulletproof host by location, type of service and also the different aliases that we need, yes.

RIPE Whois is invaluable to us. I don't see another RIR offering the same Whois services that RIPE do. The full text search the RIPE Whois database is very user‑friendly for law enforcement who may not have the very best of querying tools when searching for IPs and different Whois lookups and different objects. And we make vast use of that to do our OpenSource research so that in terms of stage two mapping, it's an under‑used tool for law enforcement and something the UK wants to really market to the rest of the world to use as best practice.

Following on from RIPE Whois, we will also check the other databases of all the other RIRs, just in case there is infrastructure which often there outside the European and the middle eastern region. We will then move our research on to RIPE Stat, we will be looking at identifying perhaps more IP prefixes announced by certain providers and resellers through RIPE Stat research. Upstream providers, the routing history of them, BG Play is a great widget that we use to map out the peer routes of that, to find out possibly the RIPE members who are associated and providing space to these members, not bad providers, just people who sell space on to resellers which is quite normal.

There is an interesting update for me in RIPE records that came in at the end of last year which is a sponsoring org field, and I don't know how useful that is to you but when we are searching IP related data the sponsor in org field is again invaluable to law enforcement, we can find out the true hierarchy of ‑‑ administration there and it's block so there is lots to be gained there. We have no authority doing some OpenSource research, and expanding the picture of the organised crime group.

What did that give us? So the benefit at the end of stage 2 was a lot more intelligence. We drew a picture at that point on paper at least, of the ISP, an a upstream providers we thought we were looking at. We found lots of consistent hosting providers of reseller space. They are not bad providers, they are probably providers of a cheap virtual hosting, for example, and located near big Internet exchange points, for example. So multiple RIPE NCC members are announcing traffic to them. And just like any organised crime group, in the way that heroin trafficker may split 20 kilos of heroin into ten lots of two, and commit that to going through the port of a country, in ten different lorries, they can guarantee that some will get through and some might get stopped. It looks like this bullet hosting provider is no different. It had a static IP infrastructure, which wasn't moving, which is a bonus but it had diversified a lot of its reseller ranges away from its host country location into much smaller blocks, much smaller consistent/3732 block address ranges and one thing for me is really good is that the RIPE member records on all of those changes were pretty much perfect N terms of the attribution in finding out the hierarchy of that, there is no problem from a law enforcement in the way RIPE members ‑‑ the most granular level update their databases of IP reassignment. So a lot of that reassigned PI and PA space was easy to see and the Whois revealed a lot more provider alias names and handles, gave us a lot more intelligence to increase and enrich the picture of who we thought we were looking at, there is a couple of IP addresses in the US which weren't particularly relevant so the majority of hosting takes place within the RIPE region.

Stage 3 begins to get a little bit more interesting in terms of mapping IPv4 space to what are called DNS enrichment. I want to ask the question there, of what domain traffic and types of service, point to or in the past have pointed to these ranges. Web mail, name of VPN servers, a lot of intelligence from scenes of crime that DDoS attacks and lots of proxies in support of BotNet activity are hanging off VPN points for example. We begin to branch out on that and see a lot more do maim name traffic across these ranges that we have found at stage 2.

We utilise tools, mapping and charting tools like mull teeing go and we will put what we calls transforms into the data set such as domain tools and we will ask questions of those/27 ranges, show me what is pointing to them, we will do the same with far site technology and we will ask passive DNS questions. And ask well, what domain traffic has ever pointed to those IP ranges and has transited them in the past to build that picture. We will also enkitchen the data that we get back through abuse feeds to then filter down even further into the IP ranges to separate maybe the bad ranges from the good. And we will ask questions of abuse feeds in an API format to say, well, against this/27 or this /23 range, show me the bad innocence or the suspected malicious activity or 15 IP addresses have been linked to phishing in the past, two have been associated to C2 C domains in the past and 16 have redirected to exploits so. We are trying to build a picture up to say that it's not a coincidence that this provider potentially keeps cropping up in the same investigation, maybe that single strand intelligence is more common and we are dealing with more of a bulletproof host here.

So at the end of that stage, there was some really interesting results. A lot of the host country ranges, what I would call the front end ranges showed legitimate web traffic and that exists as well as badness on all ISPs and that is a fact. The diversified reseller ranges, some we put in other local jurisdiction, carried ‑‑ I have listed a long sub domain string there that we believe we saw this in abundance on one IP range, that relates to G O Z and DG A, C2 C activity, it was pass DNS source record from end of May 2014, at a time when there was a shift in G O Z to the dot EU ccTLD, so that point I am trying to put the meat on the bones of the IP and corroborate a bit more intelligence to say this particular range may be what we are looking at, it may be bad, may be subject to more abuse than those from end ranges, we are separating the front end from the bad back end here.

Additionally, VPN end point servers and over traffic is seen on some ranges. I have not seen that in an investigation, but that certainly corroborates a lot of threat intelligence that we are getting about using DNS tunneling as a way to ex filtrate data and avoid port 80 access. It's a legitimate tool, potentially there is no way I can prove that at this point, potentially that might add to the picture of what is going on here in terms of VPN end points, it may not do, but it's something more to look at for the operational team. So we are beginning to separate the bad from the good.

Stage four it's pretty self‑explanatory and that is charting the infrastructure. We are managing a lot of data and seeing a lot of domain names and use tools like web C and MaxMind to bulk up Whois data and we can patent match it and look for bad Whois records at domain level, consistent e‑mails being registered to the domain, commonalities between domain and IP records found, cross overs with other investigations, still building that picture.

So against the aim at the start of the presentation, I really think it's worthwhile to take this approach. It was a pilot mapping strategy, we have kind of benchmarked this in terms of a law enforcement technique to do stuff a little bit differently and get to the infrastructure. I think we have found the malicious re‑seller end ranges from the clean front end from the OpenSource research, it's very diversified, a lot of cheap hosting content VPN end points corroborated some of the intelligence we had.

Reseller architecture across more than one member country. So law enforcement always have a problem with evidence being in another jurisdiction. Well, I was putting evidence in my own jurisdiction where I have authority potentially to seize that evidence or look at traffic data against it in the UK so that was worthwhile just on its own to do that. The small IP blocks and the multiple RIPE database handles lead you to more common providers of re‑seller space, around those Internet exchange points.

Attraction to that cheap virtual host and Internet exchange points, you know, criminals don't just work on anonymity, potentially they work on cost as well and they are attracted to that. Some ASN common ants, those sponsoring orgs, we found about four or five members delegating space down to these ‑‑ to this provider. So the use of RIPE NCC tools gave us not just more routes to the best evidence, but it also gave us more opportunities to do something different with the findings. Just going back a few slides. In terms of prevent and protect, I think there is a lot more in the question I have got for the audience, maybe for not an answer today but just something to think about is whether or not we can work together to disrupt the abuse. And promote that needor collaboration and do something called mitigation at scale. The increased mapping has given us a bigger picture and more accurate picture of intelligence that we can action. It's moving us into more proactive what I think is partnership prevention work and outreach to industry. IP re‑seller intelligence can be mapped and shared to members. Past entries can be ‑‑ as a feed to industry. Malicious IP ranges, particularly in terms of those use and the mall war via spam that pushes it, at RIPE to become black house ‑‑ privacy proxy abuse can become an ICANN compliant submission. A lot of the domains we found on the dot EU ccTLD were privy proxy domains, and whilst I respect the right to privacy, I don't think people suspected of crime should have the same level and expectation of privacy and be able to hide behind those types of domains. So maybe an ICANN reveal and relate trigger is something we will action there.

So that is it from me. It's quite a quick overview. It's very near in ‑‑ new in terms of what we did but the whole thrust of this is for me to apply from a position of strength to industry, to tell you what I want on the end of the phone or in a conference like this, and potentially to ask the question, is that what you want, do you want ‑‑ although you probably get these feeds any way in terms of ISP infections on a daily basis do you want law enforcement context on top of that, is that going to get you more involved in take down or potentially reducing the up time of these resources, or are is there another way? Thanks very much.
(Applause)

BRIAN NISBET: So, are there any questions for Jon?

AUDIENCE SPEAKER:

JIM REID: I am just some guy wandered in off the street like Brian told me to same. I think this is a very welcome presentation, I think it's very useful for Internet folks and everybody else in the IRR space to get an understanding where law enforcer is coming from and so I warmly appreciate the fact that you have come here today to explain where you are coming from and the sort of challenge that you guys are facing. If we get more of this dialogue I think it's very helpful to reach an standing of each other's requirements and difficulties and all the rest of it. One thing that struck me you were focusing very much on public data sources and public information, and as, you know, the quality of Whois certainly for the numbering resources, is much better state than you can hope to get for domain names. However, I am wondering about other things that happened with IP addresses, criminals that are perhaps anonymising their use of IP addresses, using ‑‑ 3G access, have they yet become a problem for law enforcement and if so do you have plans to deal with them and is this something you would be comfortable talking about or just say it's in hand?

JON FLAHERTY: Being very open, the VPN for me is a challenge and the next generation tools for law enforcement are things like, without promoting them too much, like passive DNS and ‑‑ pull fixes far side databases so to attribute, it's not about the Whois, it's who was really in terms of getting behind what domain in terms of maybe a VPN domain that pointed to an IP address three weeks ago in an inquiry, it could take us that way further, to stop the law enforcement officer sending you an IP address to subscriber check since moved location. We try and mainstream that via Europol and those new techniques, there is still some work to do.

JIM REID: Yes. You were talking about take down requests and stuff like that. I mean, I think that becomes a bit more problematical, we are dealing with multiple jurisdictions I think it would be very difficult for a one‑size fits all approach for that. It's possible for dialogue and let's see what can be done in this space.

JON FLAHERTY: At the end of this report I will give a number of options, we seem to be not mad on take down but we go through lots of phases, we say take down suspension, block, filter, you know, I actually think Spamhaus is a massive massive player in all of this. I see a lot of BotNets law enforcement coordinated days of action and I think Spamhaus maybe have more of a role to play in that. I think BotNet is part of a spam campaign. It's just a case do ISPs sign up to BotNet blacklists at Spamhaus or is it just spam lists so that might be a more one size fits all rather than an itemised take down per provider.

JIM REID: Thank you.

MARCO HOGEWONING: Clarification, as Jim said, when you talk about like sharing intelligence feeds with ISPs could you do that cross‑border or are you still limited to sharing your intelligence only with UK ISPs or could you potentially share with law enforcement cross‑border ‑‑

JON FLAHERTY: Yes, so the use of third‑party data is something, so the mitigation of scale work is currently looking legally at whether we can do that. That is certainly our interest. We ‑‑ we don't want to be the authors of that data but want to put the context on top of it so I think you have got feed directly from the source, but we want to corroborate that and brand it more to say that this IP is still current, maybe you need to prioritise that because of our layering of further evidence on top of the data holders' feed.

AUDIENCE SPEAKER: Can you share actual result of this research, studying where the domains points, where malicious software is being distributed is interesting but what is the actual results, crimes jailed, crimes stopped, crimes prevented, do you have such results?

JON FLAHERTY: I can give you findings. This is an ongoing operation that is not at that stage yet. We would be quite happy to, we are very open to share in media space the results of our success, which, hopefully, could result in prosecution and lots of disruption. So yes, but not yet.

AUDIENCE SPEAKER: So no actual crime is being prevented yet? Has been prevented, sorry?

JON FLAHERTY: There will be events of hacks and network intrusions that have led us through further research to what we call maybe a criminal enabler of crime and that is the bulletproof hoster. So they will be individually investigated. I can potentially ‑‑ I can find and potentially share some of the off shoots of that with you, but this particular operation into a bulletproof hoster is not there yet.

AUDIENCE SPEAKER: Thank you.

BRIAN NISBET: So, any other questions for Jon? No. In which case, thank you very much.
(Applause)

I would like to say myself again, it has been, I think I have said this a couple of times, over the years of our interaction with law enforcement, as a community, I think that it has gotten Bert and better and the, I think it's very useful for the community to have that level of interaction, so very much hope it will not be the last presentation of this kind that we have at the Working Group. So our second presentation so gentlemen. This is from Ralph Weber and and Bruce Van Nice on DDoS evolving threat. Yes. Please.

BRUCE VAN NICE: Bruce Van Nice with Nominum. One of the things we do is we gather DNS data we get every day several terabytes of data from various ISPs worldwide and one of the things we do with that data is we track DDoS activity. So, this is a graph that we started to create last year, at the beginning of the year we began to see some unusual new queries that had implications for what it did to infrastructure. And as you can see in January it started off pretty modestly, when this first hit the radar in our data set. There was a big uptake and caused a lot of concern because it reached a level where it began to have a fairly visible impact on the DNS infrastructure, specifically on the resolvers that we collect the data from and second article on authoritative serves that were being attacked and third on resource themselves that often times were taken off‑line as a result of the attacks.

So you will see that the summer was pretty exciting, there was some visible attacks that occurred in the summer of 2014. One was a Chinese website called Apple daily was very visibly attacked during the Hong Kong protests. In that activity continued through the summer and then in ‑‑ tend of the year, we saw another really large spike and the stifle the atasks the same, it was using these DNS queries with randomised labels, but we discovered that what was behind that attack had actually changed and so it went from attacks that were using open resolvers out on the Internet to attacks that were using bought malware which we later determined was resident on device like home gateways used for consumer Internet or for other kind of like surveillance cameras that were Internet‑attached. So by the end of the year things are starting to get really exciting. You can see just from our data set there were around 6 billion of these queries. We estimate that we see about 3% of ISP DNS traffic so you can do the math, multiply by by 30 and it starts to reach a level where it's pretty noticeable and it definitely hits the radar and we are sitting around at Christmas going 2015 is going to be exciting there has been this significant uptake in this activity. Here is the data for 2015. And the good news is that the big spikes are gone, it's really interesting, we fully expected we would see this continuing thread and there continue to be this huge spikes in traffic and it literally went away. The obvious thing to think is all is good but it's not the case. What we have discovered is, at least we believe and our data seems to support it, the attackers have gotten a lot smarter so they went and said let's go spike traffic and that will be exciting and fun to being a lot more stealthy and smarter about how they launched their attacks and we are confident they are still achieving their goals in taking out the resources and achieving the goal of stressing the DNS infrastructure. We sell DNS software so that is why we get this data and that is why we are talking about this.

So although it will be ‑‑ might be assumed that all is quiet, it turns out it really isn't. Let me talk about that a little bit.

Just offer some perspective, some really simple statistics N terms of the traffic we see, about 89% is legitimate or normal traffic. And about 11% is this DDoS traffic. This is a couple of things, again it's easy to say well, gee, it's only 11% and the DNS infrastructure is amazingly resilient, it holds up to a lot of different kinds of abuse, but there is a couple of things about these attacks that make that number bigger than what it is, one of the things is that the style of these queries forces DNS servers to do a lot more work they would otherwise do so the implication in terms of processor load is there is a significant increase in processing that is needed to handle these queries and and the other thing that that 11% often occurs over a few hours, you have two things that magnify so 11% actually ends up being a lot and depending on what is being attacked or what infrastructure is being used, again it can succeed in bringing down or otherwise seriously compromiseing that infrastructure.

The tight of this talk was DNS DDoS fast evolving ‑‑ speak into the microphone ‑‑ I can hear myself beautifully. So, I mentioned a couple of different things that we have been seeing. We talked about ‑ I talked about these random attacks that started in early 2014. Those themselves have evolved in a couple of different ways, they went from these attacks that relied on open resolvers that are widely dispersed to attacks that use bought malware and we are seeing variations, DNS amplification, likely you are all familiar with that. That was trendy in 2012 and 13. That is still occur, the way it works is a little bit differential we haven't seen Bots that are doing amplification but open resolvers still do amplification, but you can see that there is been a significant shift to these random sub domain attacks. There is certain advantages to attacking that way and you know attackers tend to be smart and we think that is an obvious explanation as to Y if we look closely at the data surrounding the DNS amplification attacks you can see that there is a lot about I attacks suggest they are being done by amateurs in terms of how they roll out and the style of the attack and lots of other things we can look at. But anyway, that gives us a sense of what the split is between these two kinds of attacks.

I have mentioned open resolvers a couple of times, there is an organisation that really does exceptional work, they have been tracking open resolvers on the Internet very methodically for more than two years now, in fact I don't know how long they have been around, but they hit the radar would years ago with this revelation that at the time when he first started publicising the data there were close to 30 million of these and since that time there has been some progress in terms of getting rid of them or let's just say reducing the impact of them. There are vasious ways that can be done. It's not clear they have been decommissioned, it's possible the activity or the scan actually that enables the identification of these devices has simply been blocked. But what that data shows today there is still about 17 million of these things out there. They are a real nuisance because really handy resource for attackers. We are seeing for these ransom sub domain attacks, the people using them are moderately sophisticated, what they are doing rather than big dramatic spikes in 2014 is they are modulating their traffic, they are sending just enough to take out the resource that they are targeting, so, you know, just an attempt to fly under the radar and do the damage had a they want to do and minimise the impact but perhaps in a sense, we sees these are highly distributed so our data shows in general, whoever doing this for any given attack will use thousands of these open resolvers or ‑‑ there are in any given network, they may have access to thousands and worldwide they have 17 million so there is no shortage of them. We often see that the query rate per IP is very low, in fact some of our data shows it's as low as two tents of a query per second so you can imagine aggregate in any one place, you don't necessarily see a lot of activity, it gets aggregated at resolver so that is where we see the data, even at that level how closely a provider monitors their infrastructure they may not notice unless they are paying attention, it gets monitored as a goes up the hierarchy and the serve her that is being attacked sees the ‑‑ Apple daily were attacked in 2014 and periodically for whatever reason they get attacked again, it is a visible alternative new site in China so there is an obvious motivation why they might be attacked. Talk about the bought based attacks. These we discovered back in November, and they are distinctly different in terms of how they operate. It tends to be fewer IPs and many times attacks only use ten of these device or at the outside we will see a few hundred that are participating actively. We may see a long tail so we mightn't see a couple of thousand all together, but most are not active participants over the duration of an attack, they may only send one query. But anyway, so there is a real obvious difference in terms of behaviour that we see when these different vectors are being used and that is one of the things we use to track this activity, how we can differentiate or distinguish between what is being attacked and how.

In terms of an example of something that was attacked recently a university in US called Rutgers, for some odd reason, maybe unhappy students, they get attacked periodically by these Bots, many others as well but it was attacked last week, actually.

Just to provide a visual, some of you may be familiar with how this works. What this shows what is going on, it's pretty simple, at least for the bought based attacks, this is malware we have confirm as resident on home gateways so I will talk about how it gets loaded. But basically this bought sits and sends these special DNS queries with random labels, those go to the resolver that that home gateway configured to use, IPs resolver. Since the left most label is randomized the query will never be cached so the resolver goes and ask the appropriate resources for an answer and they don't come ‑‑ they come back with no answer and common existent domain. And so this is where the stress comes in, because the resolver as I said has to do a lot more work, the authoritative servers have to answer queries for which also no answer. So, this is what succeeds in taking down the resource. So just a little background on how think infections happen, we know there is scans of the Internet for devices and basically, people scan to see whether or not they can get a login screen and if they can they try default passwords and often times they succeed. I have seen estimates that if I recall, it's a little less than one percent of devices ‑‑ somewhat less than one percent that this this problem, they can gain access, they load their malware and go operate it remotely.

So just to kind of summarise, that was sort of the background. Ralph has some data on the remediation techniques available, this is causing stress on DNS infrastructure, for resolvers since the queries require recrucial there is a lot more work. When authoritative servers that they are trying to use to get answers start to be stressed they slow down and it takes longer to get answers. Some of those authoritative serves fail and that means that the resolver has to go look for another one so that takes even more time so the stress con sense traits as the ‑‑ authoritative serves themselves, these unexpected spikes often exceed the provision limits and so we have data showing what happens to these authorities when they can no longer respond. So it's a problem for the DNS side of every network and that is why people are paying /TAOEPBGS T /RA*FL did some great work. There is a couple of different ways that remediation is being approached. He recently tested some of these approaches so he has some data that he will talk about now.

Ralph Weber: Thanks. So, when the traffic hits the ISP resolver and then gets forwarded there are two kind of points where you can do something to do, to remedy that. The first is, if you, on the ingress, know that the packet or the request is bad, you drop it or on the outbound you can actually, when you know that authoritative server is not responding to you any longer there is no actual sense in stressing it more. However, historically that has been the behaviour that most of the resolvers have done, whenever they get something they didn't know they ask ‑ servers and this has changed recently and I wanted to see how do the different resolvers and methodology work so I set up a small test network, partly in our headquarters and partly a small server I have to measure over the Internet and have some latency introduced into the mix.

I then did ran every major implementation that is out there and first I ran traffic that would characterise normal user traffic, so Google and other stuff, I actually got a snippet from an ISP that I repeatedly ran over again so this is really, really traffic. I did another set for a domain that we then, in the second step of the test, will attack. And when I do that, and the disks log scale pretty much all the major ‑‑ this is live traffic over the Internet, so there are some ServFails, some lost, that is normal behaviour. DNS is UDP based so whenever you lose something you are out of luck. And the set‑up also had the domain set up as you wouldn't do with your domain because it was only one authority of serve and that had a limit, a very low limit on the number of questions he could answer per /SEFPBLGTD it was 100 queries per second that he could answer.

(Second)

When do you that, that is in the normal user traffic pretty much normal behaviour.

The next thing was additional test with the actual domains that we will attack later. And with that again, normal stuff. Some stuff serve failed because the packets were lost, nothing to be concerned about, and on the authoritative server, that is ‑‑ is the amount of traffic we received there. And that of course, I mean, all people like nice graphs so we did nice graphs on how the different implementation showed up on the CPU scale.

Now, when we do and you now look at the, from your side, most right graph, that is actually when we ran the attack traffic so at that point half of the traffic and that is pretty common, half of the traffic was a random supplement attack and if the servers didn't have any protection, they had to do so much recursion that their normal user traffic, so the traffic I go to google.com might end up in a sort of Akamai that short‑lived P T L, when I try to do that and didn't have any resources any more to go out, so I didn't get back an answer for Google or something like that. So in that traffic, I mean, this is a log scale so it's a bit irritating from the percentage but it was around 16% of normal traffic that would fail if this server didn't' have any protections in it.

And on ‑‑ if you run the protected traffic and these protections, I mean, all of these servers in there, by now are in a soon to be available future versions have the outbound protections in so they basically don't try to stress the authoritative server /HOR man the authoritative server can take. And with these protections in, there really is not much difference. If you compare that to the good result and that is the bad result or the attacked traffic result you don't see much difference there. So, the good news is, these outbound protections are working. However, if you look at it from the attack domain perspective, it's a bit different. So, this is the result of the domains that were attacked and from these domains that were attacked, most of the results that came back from ServFail, so it means you couldn't go there. So, in the end, the attacker achieved at least for this ISP or whatever, the attack was the goal to make the domain unavailable at least for the amount of people in that domain. Now, if you use ingress filtering, so basically you decide at the beginning before you actually do something with a packet to drop it, then the resolver still has the possibility to go to the authoritative serve and ask it for something and gets back a good answer. And this is just the ‑‑ again if you flip back and forth, this is the good traffic and that is the attacked traffic so you see that the high bar slip from left to right meaning that the traffic gets lost or doesn't get correctly answered. And when you look at what the actual authoritative servers are getting there, you will see that with an ingress filter you only set the right queries whereas with everything else that has some sort of filtering, you send more traffic than you actually need to but I mean, it's not as much as the authoritative server can't handle so at least the authoritative handle doesn't go down for other domains but for this domain the stuff that you get back is NX domain mostly so you can't go there.

With that, on the authoritative server you see ‑‑ that the good traffic and bad traffic, that is the slide we have. So again you see it shifts from left to right. And again, the CPU is probably more, if you compare that to the initial slide, it's roughly double, so because the attacked traffic was twice as much. The numbers are if you want to download the slides, I have them all here. And the summary is that basically, if you do ‑‑ authoritative you can eliminate some of the bad traffic of the authorities but not all. You correctly answer most of the other clients, so that is good. But you are not protecting the actual domain that is attacked. That only can do ingress filtering. So, if you can do the ingress filtering and we have some capabilities, I think /PO*URS has something and you probably could achieve something with unbound, then do that. If not, at least try to update your servers to the latest versions of your software vendor but have these outbound filterings because otherwise you are stressing the authoritative even longer. And with that, I am done. I am open for questions.

BRIAN NISBET: Thank you very much.
(Applause)

So, are there any questions? Everybody understood all that have and they have already done it on all of their servers. OK. Thank you very much.

And with that, so, is there any other business that anyone wishes to raise? Please.

SPEAKER: Guert again. I would like to raise your attention to this current discussion in the database Working Group meeting about U T F idea implementation in the database, this could either improve the contact details especially for law enforcement agencies or absolutely break anything, it depends on your being optimistic or pessimistic. Thank you.

BRIAN NISBET: So database mailing list take a look.

RUEDIGER VOLK: Deutsche Telekom. I wonder whether anybody else did find that at least one sponsor of this conference is doing spamming.

BRIAN NISBET: I didn't.

RUEDIGER VOLK: OK.

BRIAN NISBET: But that is obviously ‑‑

RUEDIGER VOLK: Kind of, we are nice to our sponsors but I think we do not really endorse this, and the question to help improve the world there is, would this group have resources to kind of guide the sponsors if there is a request, about how to properly behave.

BRIAN NISBET: I certainly think so. My initial reaction to that is that is something I am happy to formally take up with the NCC in the meeting organisation group in regards to what advice is given to sponsors on how to play nice with the other children.

RUEDIGER VOLK: Potentially it would be a good thing to have a resource where one can just point, because there are other conferences, and of course I cannot really tell whether that sponsor took my thing from a RIPE event or from a NANOG event or anything, and I guess there are a couple of events where this community cares that good advice is available.

BRIAN NISBET: Yes, I think whether a central resource might ‑‑ my concern is that other organisations or other events won't look at them. But certainly from the point of view of this event we will confirm that with the NCC and I don't have sight of what they say to the sponsors but I am pretty sure there is something in there and if there isn't we will make sure that there is something in there which talks about proper behaviour with the details there are got from the meeting information.

Blake Willis from eyebrows today a small carrier that is among other things wi‑fi for hotel chains which has interesting traceability and NAT I implications. In general talking about traceability of users, and then maybe also more for the Cooperation Working Group as well but each country in, even speak being Europe, has its own set of requirements and laws and things, and trying to right things like NAT policy and policy and software around each country's legislation, the European Union may solve at some point but I guess in general something that would be appreciated would be more discussion about that, about how, you know, going forwards with things like IPv6 and carrier grade NAT, if there were some sort of more standardised way to handle this information as opposed to each carrier has their own way of logging NAT stuff and each government has their own way of asking about it and at some point in sort of gels and there are legal implications about all this and stuff and trying to keep track of it is a big mess, basically.

BRIAN NISBET: Yes, do you have any suggestions on how to progress that discussion or...?

SPEAKER: I guess it's also for the Cooperation Working Group group as much as anything else but definitely needs to be addressed here as well. There is certainly folks here I can talk to off‑line about that. But in general, this is a subject that would certainly be more interesting for talks and whatnot and I guess that means I need to put up or shut upright.

BRIAN NISBET: There is a certain element of that. I will have a chat with the Cooperation Working Group Chairs as well and see if there is something we could do there. But yeah, if it's suggest wish to discuss, oddly enough the next bit in that slide is agenda for RIPE 71. So any other business? In which case, I will remind people that, you know, it does not have to be the last two months before RIPE meeting when I remind you all you can submit agenda items, please, there is suitable discussion etc., etc., then there is ‑‑ it's never a bad time to say something.

MARCO HOGEWONING: Just to quickly comment on Ruediger, let me restate that we do not share contact details of attendees with our sponsors. There might be other ways for sponsors to try and find out but we do not explicitly share those deattachments

BRIAN NISBET: If one has put one's details into the public systems but I still think the sponsors should be told that minding that data for the purposes of unsolicited e‑mail is not the polite thing to do.

RUEDIGER VOLK: And I think it doesn't matter whether they harvested this conference, if they are not doing proper, say, opt‑in, I think, I think I want to raise a concern.

BRIAN NISBET: I think the concern has been raised.

Mirjam: From the RIPE NCC. I want to say I have note that had Ruediger and will pass it on. I do believe that we have adjusted these sponsorship agreements accordingly recently but we will double‑check, so we don't take that lightly, either.

BRIAN NISBET: OK. I think we are almost done a couple of small points. Just one because I have a horrible moment that I wasn't sufficiently polite. Thank you for your support of me as your co‑chair. In other appointing things to people news, the RIPE Programme Committee elections are ongoing for an inordinately long amount of time until some pint tomorrow, please log into the RIPE meeting pages and you can vote for three candidates with two seats. So please do that. And thank you to the NCC staff, the AV staff, to the stenographers and most importantly, to all of you, we will be in Bucharest in ‑‑ will be the next RIPE meeting and of course the mailing list is always there for discussion and all associated things. So, thank you very much and have a good remainder of the meeting.
(Applause)

LIVE CAPTIONING BY AOIFE DOWNES RPR

DOYLE COURT REPORTERS LTD, DUBLIN IRELAND.

WWW.DCR.IE